Microsoft Robotics Studio and Lego Mindstorms NXT

	In this article, Brian Peek will demonstrate how to use Microsoft Robotics Studio to control the Lego Mindstorms NXT kit.  A simple remote-controlled robot will be built and controlled using C# and the MSRS Visual Programming Language (VPL).
Brian Peek ASPSOFT, Inc. Difficulty: Intermediate Time Required: 2-3 hours Cost: $250 for the NXT kit, $40 for the Xbox 360 Controller Software: Visual Basic or Visual C# Express Editions, Microsoft Robotics Studio v1.5 Hardware: Lego Mindstorms NXT, Xbox 360 Controller for Windows, a Bluetooth adapter and stack installed Download: Download

The Lego Mindstorms NXT kit is an amazing upgrade to the original Lego Mindstorms product.  While the building blocks are not the traditional plastic stud blocks everyone is used to, the new construction parts allow for some very complex designs.  Additionally, Microsoft Robotics Studio supports the Lego NXT kit right out of the box, so it's quite easy to get up and going using MSRS with the Lego kit to start coding a robot in C# or VB.NET.  With that in mind, this article will demonstrate how to use MSRS with the simple TriBot robot that can be easily built using the instructions included in the Lego NXT kit.

LegoNXT TriBot Building

The instructions for building the TriBot are located in the LegoNXT kit.  I am not going to repeat that here as the Lego picture instructions are far better than anything I can come up with.  Really, any 2-wheeled bot will work for this sample, but it is assumed that the TriBot is the one you have constructed.

Plug the wires into brick as follows, looking at the front of the bot so the LCD screen is facing you:

• A - Middle
• B - Left
• C - Right

Additionally, I added the push button sensor to the front of the TriBot which is plugged into port 1 at the bottom.  In the application, when the push button sensor is pressed, the robot will stop in its tracks until the A button is pressed again.  Here is what my TriBot looks like:

LegoNXT and Bluetooth Configuration

Setting up Bluetooth can be troublesome.  I have found that the best way to get things working is with the following procedure.  There are known incompatibilities with LegoNXT brick and certain Bluetooth adapters/stacks, so if this doesn't work for you, you may require a different stack/device.

1. Ensure Bluetooth is enabled on the Lego NXT brick.  This can be done by choosing the Bluetooth option from the main menu, and then selecting On/Off from the Bluetooth menu.  Make sure On is selected.
2. From your PC, search for Bluetooth devices in range.  The PC should fine one named NXT.  Select it, and pair both the PC and the brick with the passkey 1234 (the brick will default to 1234 on its own).
3. When the process is complete, you should have a virtual COM port or two installed.  If more than one is installed, make note of the COM port listed as the "outgoing" port.
4. Open a Command Prompt from the Microsoft Robotics Studio (1.5) program group and run the following command:  dsshost -port:50000 -tcpport:50001 -manifest:samples\Config\LEGO.NXT.Brick.manifest.xml .  This should start a web browser.  If prompted for a username/password, enter your Windows login username and password.
5. The first line of the page in the browser is a text box labeled COM Port.  In this box, enter the COM port the Lego NXT brick was installed on via Bluetooth above and click the Connect button at the bottom of the page.

If all goes to plan, you will hear the brick beep, the screen will display "Microsoft Robotics Studio", and you will be ready to continue.  If this doesn't happen, attempt the procedure above again.  If you still encounter trouble, try a different Bluetooth adapter.

The web page can also be used to configure various things regarding the sensors, motors, get the current battery level, etc.  Feel free to explore these items, but ensure the defaults are used for the rest of this article.

Writing the Code

A note to Vista/XP x64 users:  For the 6 of us on a 64-bit platform, note that you will not be able to use the XInputGamepad service.  Internally this uses XNA which exists in a 32-bit format only.  So, when dssproxy attempts to build the proxy version of your service, it will be loaded as a 64-bit process and will fail to load the 32-bit XNA assemblies.  The same thing happens with dsshost (it will be loaded as a 64-bit process and won't be able to access the 32-bit XNA assemblies).  You will have to use a 32-bit version of Vista/XP or use something like VMWare which supports USB devices so you can connect the controller to the VM.

The code will be written 3 times:  In C#, Visual Basic.NET, and finally in the MSRS Visual Programming Language.  All three versions will perform the same functionality.  It is assumed that you have a fair understanding of Microsoft Robotics Studio.  If you do not, please read through the documentation, specifically the Microsoft Robotics Studio Runtime section.  Additionally, you may wish to read my other MSRS article on Coding4Fun on how to build an R/C car using MSRS which is a bit more introductory.

To begin, we need to create a service to drive our bot.  This will be called LegoTriBot.  Open a Command Prompt from the Microsoft Robotics Studio (1.5) program group and run the following command:

C#:

dssnewservice /service:LegoTriBot

VB

dssnewservice /language:VB /service:LegoTriBot

This will generate a folder with several items, including project and solution files.

Open the generated LegoTriBot.sln file in Microsoft C# 2005 Express Edition or Microsoft Visual Basic 2005 Express Edition. You will see that several files were generated.

This project will use a bumper (the push button sensor), the generic drive service, and the gamepad service so the namespaces will need to be included in our code.  To do this, you will need to set references to the RoboticsCommon.proxy and XInputGamePad.Y2006.M09.proxy assemblies.

Next, open the LegoTriBot.cs/.vb and add the following code to the top of the file:

C#

using bumper = Microsoft.Robotics.Services.ContactSensor.Proxy;
using drive = Microsoft.Robotics.Services.Drive.Proxy;
using gamepad = Microsoft.Robotics.Services.Sample.XInputGamepad.Proxy;

VB

Imports bumper = Microsoft.Robotics.Services.ContactSensor.Proxy
Imports drive = Microsoft.Robotics.Services.Drive.Proxy
Imports gamepad = Microsoft.Robotics.Services.Sample.XInputGamepad.Proxy

Next, ports on which to communicate with these devices need to be defined.  One port per object is created as follows:

C#

// partnerships with bumper, differential drive and gamepad
[Partner("bumper", Contract=bumper.Contract.Identifier, CreationPolicy=PartnerCreationPolicy.UseExisting)]
private bumper.ContactSensorArrayOperations _bumperPort = new bumper.ContactSensorArrayOperations();

[Partner("drive", Contract=drive.Contract.Identifier, CreationPolicy=PartnerCreationPolicy.UseExisting)]
private drive.DriveOperations _drivePort = new drive.DriveOperations();

[Partner("XInputGamepad", Contract=gamepad.Contract.Identifier, CreationPolicy=PartnerCreationPolicy.CreateAlways)]
private gamepad.XInputGamepadOperations _gamepadPort = new gamepad.XInputGamepadOperations();

VB

'' partnerships with bumper, differential drive and gamepad
<Partner("bumper", Contract:=bumper.Contract.Identifier, CreationPolicy:=PartnerCreationPolicy.UseExisting)> _
Private _bumperPort As bumper.ContactSensorArrayOperations = New bumper.ContactSensorArrayOperations()

<Partner("drive", Contract:=drive.Contract.Identifier, CreationPolicy:=PartnerCreationPolicy.UseExisting)> _
Private _drivePort As drive.DriveOperations = New drive.DriveOperations()

<Partner("XInputGamepad", Contract:=gamepad.Contract.Identifier, CreationPolicy:=PartnerCreationPolicy.CreateAlways)> _
Private _gamepadPort As gamepad.XInputGamepadOperations = New gamepad.XInputGamepadOperations()

Now that we have ports to communicate with our services, we must subscribe to our gamepad service and bumper service to receive notifications when either changes.  First, we will subscribe to the gamepad service and setup notifications for thumbstick changes and button changes.  Insert the following code in the Start() method:

C#

// subscribe to button presses and thumbstick changes on the 360 pad
gamepad.XInputGamepadOperations gamepadNotify = new gamepad.XInputGamepadOperations();
_gamepadPort.Subscribe(gamepadNotify);
Activate(Arbiter.Receive<gamepad.ThumbsticksChanged>(true, gamepadNotify, ThumbstickHandler));
Activate(Arbiter.Receive<gamepad.ButtonsChanged>(true, gamepadNotify, ButtonHandler));

VB

'' subscribe to button presses and thumbstick changes on the 360 pad
Dim gamepadNotify As gamepad.XInputGamepadOperations = New gamepad.XInputGamepadOperations()
_gamepadPort.Subscribe(gamepadNotify)
Activate(Arbiter.Receive(Of gamepad.ThumbsticksChanged)(True, gamepadNotify, AddressOf ThumbstickHandler))
Activate(Arbiter.Receive(Of gamepad.ButtonsChanged)(True, gamepadNotify, AddressOf ButtonHandler))

This code subscribes to the gamepad service and sets up two methods to be called when the thumbsticks change or the buttons change.  The thumbsticks are used to drive the bot, and the button is used to re-enable the motors after the bumper switch has been pressed.  We will need to maintain the state of whether or not the motors are enabled, and this can be done by adding a boolean variable to the state object.  To do this, add the following code to the LegoTriBotState object in the LegoTriBotTypes.cs/.vb file:

C#

// maintain whether the motors are enabled
public bool MotorEnabled;

VB

'maintain whether the motors are enabled
Public MotorEnabled As Boolean

Next, we need to implement the actual handler methods for these events.  Add the following code to the class:

C#

private void ThumbstickHandler(gamepad.ThumbsticksChanged msg)
{
    if(_state.MotorEnabled)
    {
        // Left/RightWheelPower expects a value from -1.0f to 1.0f.
        // the Thumbsticks will return a value form -1.0f to 1.0f.  Convenient.

        // create a request
        drive.SetDrivePowerRequest req = new drive.SetDrivePowerRequest();

        // assign the values
        req.LeftWheelPower = msg.Body.LeftY;
        req.RightWheelPower = msg.Body.RightY;

        // post the request
        _drivePort.SetDrivePower(req);
    }
}

private void ButtonHandler(gamepad.ButtonsChanged msg)
{
    if(msg.Body.A)
        _state.MotorEnabled = true;
}

VB

Private Sub ThumbstickHandler(ByVal msg As gamepad.ThumbsticksChanged)
    If (_state.MotorEnabled) Then
        ' Left/RightWheelPower expects a value from -1.0f to 1.0f.
        ' the Thumbsticks will return a value form -1.0f to 1.0f.  Convenient.

        ' create a request
        Dim req As drive.SetDrivePowerRequest = New drive.SetDrivePowerRequest()

        ' assign the values
        req.LeftWheelPower = msg.Body.LeftY
        req.RightWheelPower = msg.Body.RightY

        ' post the request
        _drivePort.SetDrivePower(req)
    End If
End Sub

Private Sub ButtonHandler(ByVal msg As gamepad.ButtonsChanged)
    If (msg.Body.A) Then
        _state.MotorEnabled = True
    End If
End Sub

The above code sets up the controller thumbsticks to drive the TriBot as a tank.  The left stick will control the left motor, and the right stick will control the right motor.  To do this, the value from the LeftY and RightY properties are taken and applied directly to the drive power's LeftWheelPower and RightWheelPower.  This can be done directly since LeftY/RightY return a value between -1.0 and 1.0 and the LeftWheelPower and RightWheelPower properties expect a value between -1.0 and 1.0.  Once the SetDrivePowerRequest object is filled in, it is passed to the _drivePort's SetDrivePower method to turn the wheels.

Finally, we need to setup the bumper functionality.  We must subscribe to the bumper service and receive notifications when the switch is pressed.  In the handler for the switch being pressed, the MotorEnabled property will be set to false so the motors will no longer turn until the user presses the A button on the controller.  First, add the following code to the Start method to subscribe to the bumper:

C#

// subscribe to bumper notifications
bumper.ContactSensorArrayOperations bumperNotify = new bumper.ContactSensorArrayOperations();
_bumperPort.Subscribe(bumperNotify);
Activate(Arbiter.Receive<bumper.Update>(true, bumperNotify, BumperHandler));

VB

' subscribe to bumper notifications
Dim bumperNotify As bumper.ContactSensorArrayOperations = New bumper.ContactSensorArrayOperations()
_bumperPort.Subscribe(bumperNotify)
Activate(Arbiter.Receive(Of bumper.Update)(True, bumperNotify, AddressOf BumperHandler))

Next, implement the BumperHandler method:

C#

private void BumperHandler(bumper.Update msg)
{
    if(msg.Body.Pressed)
    {
        LogInfo("Pressed!");

        _state.MotorEnabled = false;

        // create a request
        drive.SetDrivePowerRequest req = new drive.SetDrivePowerRequest();

        // stop the wheels
        req.LeftWheelPower = 0.0f;
        req.RightWheelPower = 0.0f;

        // post the request
        _drivePort.SetDrivePower(req);
    }
}

VB

Private Sub BumperHandler(ByVal msg As bumper.Update)
    If (msg.Body.Pressed) Then
        LogInfo("Pressed!")

        _state.MotorEnabled = False

        ' create a request
        Dim req As drive.SetDrivePowerRequest = New drive.SetDrivePowerRequest()

        ' stop the wheels
        req.LeftWheelPower = 0.0F
        req.RightWheelPower = 0.0F

        ' post the request
        _drivePort.SetDrivePower(req)
    End If
End Sub

The code above checks to see if the bumper was pressed.  If it was, it creates a SetDrivePowerRequest object, sets the left and right wheel power to 0 and then sends the request off to the drive port to be handled.

Running the Service

To run the service, we need to tell the MSRS runtime which manifest to load that defines the hardware for the generic services we've used (bumper and drive).  To do this, right-click on the project and select Properties.  In the Debug section, add the following argument to the Command line arguments textbox:

-manifest:"samples\config\LEGO.NXT.TriBot.manifest.xml"

Also note that you may need to re-path the location of the project's manifest file to where you have extracted the sample archive if you are building from the sample code.

That's it!  We can now run our service and drive our TriBot around using the gamepad.  Ensure the Xbox 360 Controller is installed and working.  In Visual Studio, press the F5 button to start the service in debug mode.  Be sure to press the A button first to enable the motors for the first time.

Visual Programming Language (VPL)

The LegoTriBot service we just built in code can very easily be built using the Visual Programming Language included in Microsoft Robotics Studio.  Describing the process of dragging and dropping blocks around isn't very easy, so let's start by looking at the finished VPL diagram.

Drag an XInputController block onto the design surface.  First, let's set up the functionality where pressing the A button enables the motors.  Drag an If block over to the design surface.  Then, drag a line from the bottom node of the XInputController block to the If block.  A Connections dialog box will appear.  In the From column, select ButtonsChanged.  In the To column, select Condition.  Finally, click in the text box of the If block and choose A.  We have now set up our conditional.

You'll notice that there are two points on the right side of the If block:  one is for the true statement, the other for the false/else statement.  When our A button is pressed, we want to set the MotorEnabled variable to true.  To do this, drag a Calculate block to the surface.  Set its value to true.  Then, drag a Variable block to the surface.  Click the Add button at the lower right corner of the screen.  Create a new bool variable named MotorEnabled.  Finally, connect the If block to the Calculate block, and connect the Calculate block to the Variable block.  Select SetValue from the dialog box.

Next, let's set up the bumper switch.  This is just like the above, with the addition of setting the drive power to 0 for each wheel.

Drag a GenericContactSensor block to the design surface.  In the Properties window, select Use a manifest from the Configuration drop down, and Import the LEGO.NXT.TriBot.manifest.xml file.  Next, drag over an If block, a Calculate block and a Variable block.  Create and connect them just as you did above, but this time assign the value false to the Calculate block.  Connect the GenericContactSensor block to the If block with the ContactSensorUpdated event.  Finally, drag a GenericDifferentialDrive block to the surface.  As with the GenericContactSensor, assign the same manifest file to the block.  Drag a connection from the If block to this block.  Select the SetDrivePower event in the To column.  In the Properties window, set both the LeftWheelPower and RightWheelPower values to 0.

Finally, we must hook up the thumbsticks.  As with code above, we will want to apply the value of the thumbsticks directly to the drive service if the MotorEnabled variable is true.  So, drag out a Variable block and set it to the MotorEnabled variable which already exists.  Then, drag out an If block.  Connect the XInputController to the Variable block with the From set to ThumbsticksChanged and the To set to GetValue.  Then, connect the Variable to the If block, setting the If block's text box to MotorEnabled.  Now comes the tricky part.  We need to get the thumbstick values to a GenericDifferentialDrive block, however, only if the the MotorEnabled value is true.  To do this, we can use the Join block.  A Join block will join two messages together and pass them along to another block, but only if both messages are received.  We can use this by passing the thumbstick values directly to the Join as one message, and the true path of the If block as the second message.  This way, the messages will only be passed on if there's both a thumbstick value and MotorEnabled is true.

So, drop a Join block onto the design surface.  Next, drag a connection from the XInputController to the Join block as the first message using the ThumbsticksChanged event.  Then, drag a connection from the true fork of the If block to the second message of the Join block.  Now we can set the output to a GenericDifferentialDrive block.  Drag that block over, selecting GenericDifferentialDrive from the Add Activity popup window, and connect the Join to it using the SetDrivePower event.  Click the Edit Values Directly from the DataConnections popup and set the LeftWheelPower target to the msg.LeftY parameter from the Join, and the RightWheelPower target to the msg.RightY parameter from the Join.

That's it!  We now have a VPL-written program that will perform identically to the application we coded in C# and VB above.  To start the project, ensure the Lego NXT brick is connected with MSRS and press F5.  This should sync up with the brick and start the service.

One thing to note is that starting with version 1.5 of MSRS, one can generate a .csproj and associated code by choosing Build->Compile as Service from the VPL menu.  Choose an output location and, after the service is compiled, check the directory for the generated code.  This code can be reviewed, edited, recompiled, etc.  You will find this code int he LegoTriBotVPLCS directory in the sample archive above.  Unfortunately, this code generation process appears to be one-way at the moment, so you cannot import your C# code into the VPL environment.

Conclusion

We have created a robot using Lego NXT and can control it with an Xbox 360 controller via Microsoft Robotics Studio by coding a service in C#, VB or VPL.  As you can see, even a non-experienced coder can develop a MSRS service using the VPL language.  If one requires more power and flexibility, a service can be coded directly in C#, or a VPL service can be converted to a C# project and continued.

Bio

Brian is a Microsoft C# MVP and a recognized .NET expert with over 6 years experience developing .NET solutions, and over 9 years of professional experience architecting and developing solutions using Microsoft technologies and platforms, although he has been "coding for fun" for as long as he can remember.  Outside the world of .NET and business applications, Brian enjoys developing both hardware and software projects in the areas of gaming, robotics, and whatever else strikes his fancy for the next ten minutes. He rarely passes up an opportunity to dive into a C/C++ or assembly language project.  You can reach Brian via his blog at http://www.brianpeek.com/.