Coding4Fun : hardware miscellaneous

WiiEarthVR – A Fully Immersive 3D Experience with Virtual Earth 3D

Coding4Fun — Fri, 14 Nov 2008 11:57:58 GMT

In this article, Brian Peek will demonstrate how to use a Nintendo Wii Remote (Wiimote), a Wii Fit Balance Board, and Vuzix VR920 glasses as input devices for Microsoft Virtual Earth 3D, providing a fully immersive, 3D experience.

Brian Peek

ASPSOFT, Inc.

Difficulty: Intermediate

Time Required: 2-3 hours

Cost: $60 for Wiimote and Nunchuk, $90 for Wii Fit (which includes Balance Board), $400 for Vuzix VR920 glasses

Software: Managed Library for Nintendo's Wiimote, Visual Basic or Visual C# Express Editions

Hardware: Nintendo Wii Remote (Wiimote) with Nunchuk, Wii Fit Balance Board, Vuzix VR920 glasses, a compatible PC Bluetooth adapter and stack

Download: Download

Discussion Forum: Forum

Introduction

Virtual Earth is the 3D interface to Microsoft's Live Maps service. Normally this control is loaded via the web browser and allows interaction with a keyboard, mouse, and Xbox 360 controller. In this article, we will take the Virtual Earth 3D control out of the web browser, use it in a WinForms application, and control it with a Nintendo Wii Remote (Wiimote) and a pair of Vuzix VR920 glasses, while also providing a stereoscopic 3D image to the glasses, creating the illusion of a fully three dimensional environment. Note that use of the Virtual Earth 3D control in this way is undocumented and unsupported at the moment. Because of this, some of the descriptions in this article are educated guesses and may not be 100% accurate…

Originally, this project started as a simple Wiimote interface to Virtual Earth 3D as shown in the video below. Since I wrote that application, I learned of the VR920 glasses and the Wii Fit Balance Board was released, so I’ve decided to create a more immersive experience using all of these controls which was demonstrated at PDC2008, shown here:

Setup

Before we get started, you will need to install the Virtual Earth 3D control. If you haven't done this already, browse to http://maps.live.com/ and click on the 3D link to install the control and supporting software.

Additionally, if you haven't already, please review my Managed Library for Nintendo's Wiimote article on this site. We will be using the library in this article, but I will not repeat the basic information that is located in the original article. You will also need to have the Vuzix VR920 glasses installed and setup according to its own user manual. That will also not be covered here.

Implementation

The Virtual Earth 3D Control

The Virtual Earth 3D (VE3D) control is intended to be used through a well documented JavaScript interface from a web page, however we would not be able to access the Wiimote or VR920 glasses from JavaScript. Therefore, we will be using the VE3D control through its native, but wholly undocumented interface.

Start by creating a new Windows Forms application named WiiEarthVR in C# or VB. As with all controls and 3rd party libraries, a reference needs to be set to the Virtual Earth 3D libraries. Add references to the following items:

Microsoft.MapPoint.Data
Microsoft.MapPoint.Data.VirtualEarthTileDataSource
Microsoft.MapPoint.Geometry
Microsoft.MapPoint.Rendering3D
Microsoft.MapPoint.Rendering3D.Utility

If they do not show up in the .NET references, they can be found by selecting the Browse tab and navigating to C:\Program Files\Virtual Earth 3D\ or C:\Program Files (x86)\Virtual Earth 3D\ . With the references in place, the project file can now be opened and the references will be seen in the References folder in the Solution Explorer as usual.

Creating an instance of the control can be done in code just like any other control. Used in the constructor or load event of the form, the following code will create a VE3D control and add it to the form as fully docked:

     private GlobeControl _globeControl;

    public MainForm()

    {

        InitializeComponent();

        _globeControl = new GlobeControl();

        SuspendLayout();

        _globeControl.Location = new System.Drawing.Point(0, 0);

        _globeControl.Name = "_globeControl";

        _globeControl.Size = ClientSize;

        _globeControl.Anchor = AnchorStyles.Left | AnchorStyles.Right | AnchorStyles.Top | AnchorStyles.Bottom;

        _globeControl.TabIndex = 0;

        _globeControl.SendToBack(); // we want the button to be on top

        pnlGlobe.Controls.Add(_globeControl);

        ResumeLayout(false);

    }

    Private _globeControl As GlobeControl

    Public Sub New()

        InitializeComponent()

        _globeControl = New GlobeControl()

        SuspendLayout()

        _globeControl.Location = New System.Drawing.Point(0, 0)

        _globeControl.Name = "_globeControl"

        _globeControl.Size = ClientSize

        _globeControl.Anchor = AnchorStyles.Left Or AnchorStyles.Right Or AnchorStyles.Top Or AnchorStyles.Bottom

        _globeControl.TabIndex = 0

        _globeControl.SendToBack() ' we want the button to be on top

        pnlGlobe.Controls.Add(_globeControl)

        ResumeLayout(False)

    End Sub

This sets up the VE3D control in its default state. If you were to run an application with only this code, you would see nothing but the earth. The navigation controls and other extras would be missing.

We can start adding items to the VE3D control by listening for the FirstFrameRendered event of the GlobeControl and then setting the appropriate properties. Setting these properties prior to this point can lead to some unexpected results.

In the FirstFrameRendered event handler, if you wish to add the default navigation controls to the screen, the PlugInLoader object is used. The PlugInLoader is created by using the CreateLoader static method, passing in an instance of the GlobeControl's Host object. Then, the NavigationPlugIn can be loaded and activated as shown:

    // load all the spiffy UI navigation goodies

    PlugInLoader loader = PlugInLoader.CreateLoader(this.globeControl.Host);

    loader.LoadPlugIn(typeof(NavigationPlugIn));

    loader.ActivatePlugIn(typeof(NavigationPlugIn).GUID, null);

    ' load all the spiffy UI navigation goodies

    Dim loader As PlugInLoader = PlugInLoader.CreateLoader(Me.globeControl.Host)

    loader.LoadPlugIn(GetType(NavigationPlugIn))

    loader.ActivatePlugIn(GetType(NavigationPlugIn).GUID, Nothing)

The last thing to be added for basic functionality is the data. As it stands, the only data that will appear on the globe is the image of the continents. Zooming in only produces a blurry representation of that base image.

Data layers are created from specially formatted data sources provided by maps.live.com known as content manifests. These are XML files which tell the VE3D control how to load the data required for any view. Content layers can be added by adding them to the DataSources object of the GlobeControl. We can add any of the following layers (note that there may be other content manifests provided by maps.live.com, but these are the only 5 that I am aware of):

URL	DataSourceUsage Type	Description
http://local.live.com/Manifests/HD.xml	ElevationMap	Terrain data
http://local.live.com/Manifests/MO.xml	Model	3D buildings
http://local.live.com/Manifests/AT.xml	TextureMap	Unlabeled aerial
http://local.live.com/Manifests/HT.xml	TextureMap	Labeled aerial
http://local.live.com/Manifests/RT.xml	TextureMap	Roads only

For the best display, add the ElevationMap, Model and Aerial TextureMap layers as shown:

    // set various data sources, here for elevation data, terrain data, and model data.

    _globeControl.Host.DataSources.Add(new DataSourceLayerData("Elevation", "Elevation", @"http://maps.live.com//Manifests/HD.xml", DataSourceUsage.ElevationMap));

    _globeControl.Host.DataSources.Add(new DataSourceLayerData("Texture", "Texture", @"http://maps.live.com//Manifests/AT.xml", DataSourceUsage.TextureMap));

    _globeControl.Host.DataSources.Add(new DataSourceLayerData("Models", "Models", @"http://maps.live.com//Manifests/MO.xml", DataSourceUsage.Model));

    ' set various data sources, here for elevation data, terrain data, and model data.

    _globeControl.Host.DataSources.Add(New DataSourceLayerData("Elevation", "Elevation", "http://maps.live.com//Manifests/HD.xml", DataSourceUsage.ElevationMap))

    _globeControl.Host.DataSources.Add(New DataSourceLayerData("Texture", "Texture", "http://maps.live.com//Manifests/AT.xml", DataSourceUsage.TextureMap))

    _globeControl.Host.DataSources.Add(New DataSourceLayerData("Models", "Models", "http://maps.live.com//Manifests/MO.xml", DataSourceUsage.Model))

By passing the URL of the content manifest, a name for the layer, and what the manifest represents, a new DataSource is created, which is in turn used to create a DataSourceLayerData object which is then given to the VE3D control to consume. This should be done in the FirstFrameRendered event handler as well.

We also need to setup VE3D to turn off any UI elements, turn on the atmosphere effects, and ensure we have a full unobstructed view. Again, in the FirstFrameRendered event handler, we can use the following code to achieve this:

    // turn on the nice atmosphere

    _globeControl.Host.WorldEngine.Environment.AtmosphereDisplay = Microsoft.MapPoint.Rendering3D.Atmospherics.EnvironmentManager.AtmosphereStyle.Scattering;

    // default to all off

    _globeControl.Host.WorldEngine.Display3DCursor = false;

    _globeControl.Host.WorldEngine.SetWindowsCursor(null);

    _globeControl.Host.WorldEngine.ShowNavigationControl = false;

    _globeControl.Host.WorldEngine.ShowCursorLocationInformation = false;

    _globeControl.Host.WorldEngine.ShowScale = false;

    _globeControl.Host.WorldEngine.ShowUI = false;

    _globeControl.Host.WorldEngine.Environment.SunPosition = null;

    _globeControl.Host.WorldEngine.Environment.LocalWeatherEnabled = true;

    _globeControl.Host.WorldEngine.BaseCopyrightText = " "; // workaround for a display issue

    ' turn on the nice atmosphere

    _globeControl.Host.WorldEngine.Environment.AtmosphereDisplay = Microsoft.MapPoint.Rendering3D.Atmospherics.EnvironmentManager.AtmosphereStyle.Scattering

    ' default to all off

    _globeControl.Host.WorldEngine.Display3DCursor = False

    _globeControl.Host.WorldEngine.SetWindowsCursor(Nothing)

    _globeControl.Host.WorldEngine.ShowNavigationControl = False

    _globeControl.Host.WorldEngine.ShowCursorLocationInformation = False

    _globeControl.Host.WorldEngine.ShowScale = False

    _globeControl.Host.WorldEngine.ShowUI = False

    _globeControl.Host.WorldEngine.Environment.SunPosition = Nothing

    _globeControl.Host.WorldEngine.Environment.LocalWeatherEnabled = True

    _globeControl.Host.WorldEngine.BaseCopyrightText = " " ' workaround for a display issue

If you were to run the application at this point, you would see a fully functioning Virtual Earth 3D control with proper data and navigation.

Control Scheme and Bindings

The user will control VE3D with the Wiimote by holding the nunchuk in the left hand, which will move the user forward/back/left/right using the joystick. The C and Z buttons on the front of the nunchuk will be used to raise and lower the altitude of the camera. The Wiimote, held in the right hand, will be used to toggle various things on or off and interact with menus. The user will also stand on the Balance Board which will use their center of gravity to turn them in the environment.

VE3D bindings allow you to change or create new control schemes for VE3D by placing an XML file in a specific directory as follows:

Vista: C:\Users\<username>\AppData\LocalLow\Microsoft\Virtual Earth 3D
XP: C:\Documents and Settings\<username>\Local Settings\Microsoft\Virtual Earth 3D

In this directory you will find a Bindings.xml file. This XML schema defines the default keyboard, mouse, Gamepad and other input device properties. Open the file to see the schema used to define events and parameters.

By default, VE3D will load any file named Bindings*.xml from this directory. For the Wiimote control scheme, create a new file named BindingsWiiEarthVR.xml in this directory. Set the contents of the file to the following:

    <?xml version="1.0" encoding="utf-8" ?>

    <Bindings>

        <BindingSet Name="WiiEarthVRBindings" AutoUse="True" Cursor="Drag">

            <!-- Nunchuk joystick -->

            <Bind Event="Wiimote.NunchukX" Factor="0.5"><Action Name="Strafe"/></Bind>

            <Bind Event="Wiimote.NunchukY" Factor="1"><Action Name="Move"/></Bind>

            <!-- Nunchuk buttons -->

            <Bind Event="Wiimote.NunchukC" Factor="0.20"><Action Name="Ascend"/></Bind>

            <Bind Event="Wiimote.NunchukZ" Factor="-0.20"><Action Name="Ascend"/></Bind>

            <!-- Balance Board -->

            <Bind Event="Wiimote.BalanceBoardX" Factor="-0.0009"><Action Name="Turn"/></Bind>

            <Bind Event="Wiimote.BalanceBoardY" Factor="0.0009"><Action Name="Ascend"/></Bind>

            <!-- Wiimote buttons -->

            <Bind Event="Wiimote.Home"><Action Name="ResetOnCenter"/></Bind>

            <Bind Event="Wiimote.A" Factor="1"><Action Name="Locations, WiiEarthVR, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/></Bind>

            <Bind Event="Wiimote.Left" Factor="-1"><Action Name="Locations, WiiEarthVR, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/></Bind>

            <Bind Event="Wiimote.Up" Factor="-1"><Action Name="LocationsMove, WiiEarthVR, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/></Bind>

            <Bind Event="Wiimote.Down" Factor="1"><Action Name="LocationsMove, WiiEarthVR, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/></Bind>

            <!-- FPS-style keyboard controls in case we don't have a nunchuk -->

            <Bind Event="Key.W" Factor="22"><Action Name="Move" /></Bind>

            <Bind Event="Key.S" Factor="-22"><Action Name="Move" /></Bind>

            <Bind Event="Key.D" Factor="22"><Action Name="Strafe" /></Bind>

            <Bind Event="Key.A" Factor="-22"><Action Name="Strafe" /></Bind>

            <Bind Event="Key.Space" Factor="20"><Action Name="Ascend" /></Bind>

            <Bind Event="Key.C" Factor="-20"><Action Name="Ascend" /></Bind>

            <!-- Other keys -->

            <Bind Event="Key.F1"><Action Name="VR920SetZero, WiiEarthVR, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/></Bind>

            <Bind Event="Key.F2"><Action Name="BalanceBoardSetZero, WiiEarthVR, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/></Bind>

            <Bind Event="Key.F3"><Action Name="ToggleVR920Stereo, WiiEarthVR, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/></Bind>

            <Bind Event="Wiimote.Minus" Factor="-0.1"><Action Name="VR920SetEyeDistance, WiiEarthVR, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/></Bind>

            <Bind Event="Wiimote.Plus" Factor="0.1"><Action Name="VR920SetEyeDistance, WiiEarthVR, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/></Bind>

            <Bind Event="Wiimote.One"><Action Name="ToggleBalanceBoard, WiiEarthVR, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/></Bind>

            <Bind Event="Wiimote.Two"><Action Name="ToggleVR920, WiiEarthVR, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/></Bind>

            <Bind Event="Key.B"><Action Name="ToggleBalanceBoard, WiiEarthVR, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/></Bind>

            <Bind Event="Key.V"><Action Name="ToggleVR920, WiiEarthVR, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/></Bind>

            <Bind Event="Key.F"><Action Name="FullScreen, WiiEarthVR, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/></Bind>

        </BindingSet>

    </Bindings>

The <BindingSet> tags wrap groups of control bindings. It requires a Name and optionally a Cursor. If the binding set is to be used automatically, as it would be in most cases, set the AutoUse parameter to True. Inside of that are <Bind> tags. The tag requires the Event parameter and optionally the Factor parameter. The Event parameter will be used to match the binding to its handler which will be written later. The syntax is <Handler Name>.<Event Name>. The Factor parameter is optional and can be used to scale the data value up or down to increase or decrease sensitivity of the input method. The Action tag inside the Bind tag is used to map the specific binding to a particular method. Once the handler is written, these will make more sense.

The bindings above create the control scheme described above: NunchukX/Y describe what happens when the analog joystick is moved, NunchukC/Z describe what happens with the C/Z buttons are pressed, and so on.

The bindings also allow for several variations. Bindings are defined for both the IR position (IRX, IRY) and accelerometer values (AX, AY). If an IR sensor bar is not available, the accelerometer values of the Wiimote can be used instead. Additionally, keyboard bindings are created in the style of a first person shooter using WASD. These can be used if a Nunchuk is not available.

Note that some bindings append two Events together with a + sign. This allows for button combinations. In this case, for the accelerometer and/or IR sensor, we only want to register the action if a button is pressed down. So, those events which require the button to be held down contain Wiimote.B+ and the event it is combined with.

For those events which require a custom action that will be written separately and not part of the VE3D control, the Action parameter must contain the action name, followed by a comma, and then the full assembly name:

<Bind Event="Wiimote.One">
    <Action Name="ToggleBalanceBoard, WiiEarthVR, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/>
</Bind>

You can change any of these button bindings simply by changing this XML file and deploying to the directory above.

Event Source

An EventSource is needed which will grab data from the Wiimote and pass it along to VE3D as defined by the bindings file above. Create a new class named WiimoteEventSource which derives from Microsoft.MapPoint.Binding.EventSource as follows:

Next, add an enumeration named WiimoteEvent (the name isn't important) which contains all of the Name items from the bindings XML file above. It should look like this:

// all events handled by this event source from XML file
public enum WiimoteEvent
{
    IRX,        // IR X position
    IRY,        // IR Y position
    NunchukX,    // Nunchuk joystick X position
    NunchukY,    // Nunchuk joystick Y position
    NunchukC,    // Nunchuk C button
    NunchukZ,    // Nunchuk Z button
    AX,            // Wiimote accelerometer X
    AY,            // Wiimote accelerometer Y
    Up,            // Dpad up
    Down,        // Dpad down
    Left,        // Dpad left
    Right,        // Dpad right
    A,            // A button
    B,            // B button
    Minus,        // Minus button
    Home,        // Wiimote Home button
    Plus,        // Plus button
    One,        // Wiimote One button
    Two,        // Wiimote Two button
    BalanceBoardX,    // Balance Board COG X
    BalanceBoardY    // Balance Board COG Y
}

' all events handled by this event source from XML file
Public Enum WiimoteEvent
    IRX ' IR X position
    IRY ' IR Y position
    NunchukX ' Nunchuk joystick X position
    NunchukY ' Nunchuk joystick Y position
    NunchukC ' Nunchuk C button
    NunchukZ ' Nunchuk Z button
    AX ' Wiimote accelerometer X
    AY ' Wiimote accelerometer Y
    Up ' Dpad up
    Down ' Dpad down
    Left ' Dpad left
    Right ' Dpad right
    A ' A button
    B ' B button
    Minus ' Minus button
    Home ' Wiimote Home button
    Plus ' Plus button
    One ' Wiimote One button
    Two ' Wiimote Two button
    BalanceBoardX ' Balance Board COG X
    BalanceBoardY ' Balance Board COG Y
End Enum

Next, several methods from the EventSource object need to be overridden: GetEventData, IsModifier, CanModify, TryGetEventId, TryGetEventName, Name. The methods do the following:

Method/Property	Description
GetEventData	Unsure at the moment...does not need to be implemented?
IsModifier	Returns a boolean stating whether the passed in event ID is a modifier (such as the Wiimote.B event above)
CanModify	Returns a boolean stating whether the current event is allowed as a modifier
TryGetEventID	Maps a string event name from the bindings file to the integer value in the enumeration above
TryGetEventName	Maps an integer event ID to the string name in the enumeration above
Name (property)	Returns the name of the handler which must match the name in the XML file above (Wiimote in this case)

The code for these methods is presented below:

    // return out value of the passed enum

    public override bool TryGetEventId(string eventName, out int eventId)

    {

        eventId = (int)Enum.Parse(typeof(WiimoteEvent), eventName);

        return true;

    }

    // return out the string name of the passed in enum value

    public override bool TryGetEventName(int eventId, out string eventName)

    {

        eventName = ((WiimoteEvent)eventId).ToString();

        return true;

    }

    // unknown

    public override EventData GetEventData(int eventId, EventActivateState state)

    {

        throw new NotImplementedException();

    }

    // can the event be used as a modifier?

    public override bool IsModifier(int eventId)

    {

        // yes to all for now

        return true;

    }

    // can the supplied event be used as a modifier?

    public override bool CanModify(int eventId, EventKey other)

    {

        // only if it's from us

        return (other.Source == this);

    }

    // this must match the Source name in the bindings XML file

    public override string Name

    {

        get { return "Wiimote"; }

    }

    ' return out value of the passed enum

    Public Overrides Function TryGetEventId(ByVal eventName As String, <System.Runtime.InteropServices.Out()> ByRef eventId As Integer) As Boolean

        eventId = CInt(Fix(System.Enum.Parse(GetType(WiimoteEvent), eventName)))

        Return True

    End Function

    ' return out the string name of the passed in enum value

    Public Overrides Function TryGetEventName(ByVal eventId As Integer, <System.Runtime.InteropServices.Out()> ByRef eventName As String) As Boolean

        eventName = (CType(eventId, WiimoteEvent)).ToString()

        Return True

    End Function

    ' unknown

    Public Overrides Function GetEventData(ByVal eventId As Integer, ByVal state As EventActivateState) As EventData

        Throw New NotImplementedException()

    End Function

    ' can the event be used as a modifier?

    Public Overrides Function IsModifier(ByVal eventId As Integer) As Boolean

        ' yes to all for now

        Return True

    End Function

    ' can the supplied event be used as a modifier?

    Public Overrides Function CanModify(ByVal eventId As Integer, ByVal other As EventKey) As Boolean

        ' only if it's from us

        Return (other.Source Is Me)

    End Function

    ' this must match the Source name in the bindings XML file

    Public Overrides ReadOnly Property Name() As String

        Get

            Return "Wiimote"

        End Get

    End Property

With that in place, the constructor can be implemented which will call the base constructor and connect to the Wiimote. It is assumed you read the Wiimote article above and know how the library works.

The constructor must take one argument passed from the main from: an instance of the GlobeControl's ActionSystem. This just gets passed directly to the parent object's constructor untouched. The constructor code looks like the following:

public WiimoteEventSource(ActionSystem actionSystem) : base(actionSystem)
{
    // get all connected Wiimotes
    WiimoteCollection wc = new WiimoteCollection();
    wc.FindAllWiimotes();

    // setup wiimotes and event handlers
    foreach(Wiimote wm in wc)
    {
        wm.WiimoteChanged += new EventHandler<WiimoteChangedEventArgs>(OnWiimoteChanged);
        wm.WiimoteExtensionChanged += new EventHandler<WiimoteExtensionChangedEventArgs>(OnWiimoteExtensionChanged);
        wm.Connect();

        // if we don't have an extension, set the report type to IR and accel's only
        if(!wm.WiimoteState.Extension && wm.WiimoteState.ExtensionType != ExtensionType.BalanceBoard)
            wm.SetReportType(InputReport.IRAccel, true);

        if(wm.WiimoteState.ExtensionType == ExtensionType.BalanceBoard)
            _bb = wm;
        else
            _wm = wm;

        // turn off all LEDs
        wm.SetLEDs(0x00);
    }
}

Public Sub New(ByVal actionSystem As ActionSystem)
    MyBase.New(actionSystem)

    ' get all connected Wiimotes
    Dim wc As New WiimoteCollection()
    wc.FindAllWiimotes()

    ' setup wiimotes and event handlers
    For Each wm As Wiimote In wc
        AddHandler wm.WiimoteChanged, AddressOf OnWiimoteChanged
        AddHandler wm.WiimoteExtensionChanged, AddressOf OnWiimoteExtensionChanged
        wm.Connect()

        ' if we don't have an extension, set the report type to IR and accel's only
        If (Not wm.WiimoteState.Extension) AndAlso wm.WiimoteState.ExtensionType <> ExtensionType.BalanceBoard Then
            wm.SetReportType(InputReport.IRAccel, True)
        End If

        If wm.WiimoteState.ExtensionType = ExtensionType.BalanceBoard Then
            _bb = wm
        Else
            _wm = wm
        End If

        ' turn off all LEDs
        wm.SetLEDs(&H00)
    Next wm
End Sub

The OnWiimoteExtensionChanged method simply sets the report mode for the Wiimote based on whether or not a Nunchuk is inserted as shown:

private void OnWiimoteExtensionChanged(object sender, WiimoteExtensionChangedEventArgs args)
{
    if(_wm == null)
        return;

    // if nunchuk inserted, set the report type to return extension data
    if(args.ExtensionType == ExtensionType.Nunchuk && args.Inserted)
        _wm.SetReportType(InputReport.IRExtensionAccel, true);
    else // in all other cases, set it to the default IR and accel's
        _wm.SetReportType(InputReport.IRAccel, true);
}

Private Sub OnWiimoteExtensionChanged(ByVal sender As Object, ByVal args As WiimoteExtensionChangedEventArgs)
    If _wm Is Nothing Then
        Return
    End If

    ' if nunchuk inserted, set the report type to return extension data
    If args.ExtensionType = ExtensionType.Nunchuk AndAlso args.Inserted Then
        _wm.SetReportType(InputReport.IRExtensionAccel, True)
    Else ' in all other cases, set it to the default IR and accel's
        _wm.SetReportType(InputReport.IRAccel, True)
    End If
End Sub

The OnWiimoteChanged event handler is where the Wiimote data is handled and sent off to the VE3D control to reflect the changes. First, we handle the Balance Board. If the Balance Board is the controller reporting data, we take the center of gravity values and pass them to VE3D. This code looks at the appropriate values, determines if they are beyond the specified thresholds for the dead zones, and, if they are, activates the event for that value using the Execute method. Execute is a method in the base EventSource class. This method will activate the event specified from the enumeration (which, remember, is contained in the bindings XML file) with the value associated with that event. An EventData object of some type must be created and passed to the Execute method. There are two EventData types to know about: AxisEventData and ButtonEventData. AxisEventData should be used when an event is activated that will modify the map position in some way. That is, if the map is being turned, elevation is changing, etc. ButtonEventData should be used if the event is a simple toggle like pressing a button down and releasing it.

The Balance Board code can be seen below:

    if(ws.ExtensionType == ExtensionType.BalanceBoard && this.BalanceBoardEnabled)

    {

        float x1 = ws.BalanceBoardState.CenterOfGravity.X;

        if(x1 > Properties.Settings.Default.BBDeadX || x1 < -Properties.Settings.Default.BBDeadX)

            this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.BalanceBoardX), x1 - _zero.X));

        float y1 = ws.BalanceBoardState.CenterOfGravity.Y;

        if(y1 > Properties.Settings.Default.BBDeadY || y1 < -Properties.Settings.Default.BBDeadY)

            this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.BalanceBoardY), y1 - _zero.Y));

    }

    If ws.ExtensionType = ExtensionType.BalanceBoard AndAlso Me.BalanceBoardEnabled Then

        Dim x1 As Single = ws.BalanceBoardState.CenterOfGravity.X

        If x1 > My.Settings.Default.BBDeadX OrElse x1 < -My.Settings.Default.BBDeadX Then

            Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.BalanceBoardX))), x1 - _zero.X))

        End If

        Dim y1 As Single = ws.BalanceBoardState.CenterOfGravity.Y

        If y1 > My.Settings.Default.BBDeadY OrElse y1 < -My.Settings.Default.BBDeadY Then

            Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.BalanceBoardY))), y1 - _zero.Y))

        End If

Next, let's handle the IR and accelerometer data. The IR midpoint of the X and Y axes will be used from the WiimoteState object to activate the IRX and IRY events we defined above in the bindings XML file. The accelerometer X and Y values will be used to activate the AX and AY events.

This snippet assumes that there is a boolean property named UseIR created in the project to determine whether IR or motion values are used. Additionally, it assumes there are property settings created which contain values for the X/Y "dead zones" for the IR and accelerometers. These dead zones are used as a way to only activate the event when the values are pushed beyond the thresholds. This allows there to be a margin where the user's hand will not be read as movement, allowing the user to not have to worry about keeping a steady hand.

The application linked above uses the following default values for dead zones:

NunchukDeadX/Y -> 0.025
WiimoteDeadX/Y -> 0.15
IRDeadX/Y -> 0.1
BBDeadX/Y –> 3

    // if we're using the IR

    if(Properties.Settings.Default.UseIR)

    {

        // and both LEDs are found

        if(ws.IRState.Found1 && ws.IRState.Found2)

        {

            // normalize the midpoints to -0.5 to 0.5 (from 0 to 1.0)

            float x = ws.IRState.MidX - 0.5f;

            float y = ws.IRState.MidY - 0.5f;

            // if we're beyond the thresholds, activate the events

            if(x > Properties.Settings.Default.IRDeadX || x < -Properties.Settings.Default.IRDeadX)

                this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.IRX), x));

            if(y > Properties.Settings.Default.IRDeadY || y < -Properties.Settings.Default.IRDeadY)

                this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.IRY), y));

            // save the last IR settings...these get used if we go beyond the range of the IRs.

            // in that case, the last used positions will be used until the Wiimote comes back in range

            this._lastIRX = x;

            this._lastIRY = y;

        }

        else // one or both LEDs aren't seen

        {

            // activate events based on the last known positions

            if(this._lastIRX > Properties.Settings.Default.IRDeadX || this._lastIRX < -Properties.Settings.Default.IRDeadX)

                this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.IRX), this._lastIRX));

            if(this._lastIRY > Properties.Settings.Default.IRDeadY || this._lastIRY < -Properties.Settings.Default.IRDeadY)

                this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.IRY), this._lastIRY));

        }

    }

    else // we're using motion controls

    {

        // activate the events based on the accelerometer values

        if(ws.AccelState.X > Properties.Settings.Default.WiimoteDeadX || ws.AccelState.X < -Properties.Settings.Default.WiimoteDeadX)

            this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.AX), ws.AccelState.X));

        if(ws.AccelState.Y > Properties.Settings.Default.WiimoteDeadY || ws.AccelState.Y < -Properties.Settings.Default.WiimoteDeadY)

            this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.AY), ws.AccelState.Y));

    }

    ' if we're using the IR

    If My.Settings.Default.UseIR Then

        ' and both LEDs are found

        If ws.IRState.Found1 AndAlso ws.IRState.Found2 Then

            ' normalize the midpoints to -0.5 to 0.5 (from 0 to 1.0)

            Dim x As Single = ws.IRState.MidX - 0.5f

            Dim y As Single = ws.IRState.MidY - 0.5f

            ' if we're beyond the thresholds, activate the events

            If x > My.Settings.Default.IRDeadX OrElse x < -My.Settings.Default.IRDeadX Then

                Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.IRX))), x))

            End If

            If y > My.Settings.Default.IRDeadY OrElse y < -My.Settings.Default.IRDeadY Then

                Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.IRY))), y))

            End If

            ' save the last IR settings...these get used if we go beyond the range of the IRs.

            ' in that case, the last used positions will be used until the Wiimote comes back in range

            Me._lastIRX = x

            Me._lastIRY = y

        Else ' one or both LEDs aren't seen

            ' activate events based on the last known positions

            If Me._lastIRX > My.Settings.Default.IRDeadX OrElse Me._lastIRX < -My.Settings.Default.IRDeadX Then

                Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.IRX))), Me._lastIRX))

            End If

            If Me._lastIRY > My.Settings.Default.IRDeadY OrElse Me._lastIRY < -My.Settings.Default.IRDeadY Then

                Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.IRY))), Me._lastIRY))

            End If

        End If

    Else ' we're using motion controls

        ' activate the events based on the accelerometer values

        If ws.AccelState.X > My.Settings.Default.WiimoteDeadX OrElse ws.AccelState.X < -My.Settings.Default.WiimoteDeadX Then

            Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.AX))), ws.AccelState.X))

        End If

        If ws.AccelState.Y > My.Settings.Default.WiimoteDeadY OrElse ws.AccelState.Y < -My.Settings.Default.WiimoteDeadY Then

            Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.AY))), ws.AccelState.Y))

        End If

    End If

Next, the nunchuk values need to be read and the associated events activated. This is done as follows:

    // if the nunchuk is connected

    if(ws.Extension && ws.ExtensionType == ExtensionType.Nunchuk)

    {

        // activate the nunchuk-based events

        if(ws.NunchukState.X > Properties.Settings.Default.NunchukDeadX || ws.NunchukState.X < -Properties.Settings.Default.NunchukDeadX)

            this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.NunchukX), ws.NunchukState.X));

        if(ws.NunchukState.Y > Properties.Settings.Default.NunchukDeadY || ws.NunchukState.Y < -Properties.Settings.Default.NunchukDeadY)

            this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.NunchukY), ws.NunchukState.Y));

        if(ws.NunchukState.C)

            this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.NunchukC), 1.0f));

        if(ws.NunchukState.Z)

            this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.NunchukZ), 1.0f));

    }

    ' if the nunchuk is connected

    If ws.Extension AndAlso ws.ExtensionType = ExtensionType.Nunchuk Then

        ' activate the nunchuk-based events

        If ws.NunchukState.X > My.Settings.Default.NunchukDeadX OrElse ws.NunchukState.X < -My.Settings.Default.NunchukDeadX Then

            Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.NunchukX))), ws.NunchukState.X))

        End If

        If ws.NunchukState.Y > My.Settings.Default.NunchukDeadY OrElse ws.NunchukState.Y < -My.Settings.Default.NunchukDeadY Then

            Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.NunchukY))), ws.NunchukState.Y))

        End If

        If ws.NunchukState.C Then

            Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.NunchukC))), 1.0f))

        End If

        If ws.NunchukState.Z Then

            Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.NunchukZ))), 1.0f))

        End If

    End If

Finally, the button events need to be activated. A helper method which will check the current button state will be used for determining which button of all the Wiimote buttons is pressed. For those that are, the appropriate event is activated with a call to Execute.

    private void HandleButton(WiimoteEvent we, bool buttonState, bool lastButtonState)

    {

        if(buttonState == lastButtonState)

            return;

        else

        {

            if(buttonState)

                this.Execute(new ButtonEventData(new EventKey(this, (int)we), EventActivateState.Activate));

            else

                this.Execute(new ButtonEventData(new EventKey(this, (int)we), EventActivateState.Deactivate));

        }

    }

    // handle all the Wiimote buttons

    HandleButton(WiimoteEvent.Up, ws.ButtonState.Up, _lastBS.Up);

    HandleButton(WiimoteEvent.Down, ws.ButtonState.Down, _lastBS.Down);

    HandleButton(WiimoteEvent.Left, ws.ButtonState.Left, _lastBS.Left);

    HandleButton(WiimoteEvent.Right, ws.ButtonState.Right, _lastBS.Right);

    HandleButton(WiimoteEvent.A, ws.ButtonState.A, _lastBS.A);

    HandleButton(WiimoteEvent.B, ws.ButtonState.B, _lastBS.B);

    HandleButton(WiimoteEvent.Minus, ws.ButtonState.Minus, _lastBS.Minus);

    HandleButton(WiimoteEvent.Home, ws.ButtonState.Home, _lastBS.Home);

    HandleButton(WiimoteEvent.Plus, ws.ButtonState.Plus, _lastBS.Plus);

    HandleButton(WiimoteEvent.One, ws.ButtonState.One, _lastBS.One);

    HandleButton(WiimoteEvent.Two, ws.ButtonState.Two, _lastBS.Two);

    ...

    // save off the current button state for next time

    _lastBS = ws.ButtonState;

    _lastNunchuk = ws.NunchukState;

    Private Sub HandleButton(ByVal we As WiimoteEvent, ByVal buttonState As Boolean, ByVal lastButtonState As Boolean)

        If buttonState = lastButtonState Then

            Return

        Else

            If buttonState Then

                Me.Execute(New ButtonEventData(New EventKey(Me, CInt(Fix(we))), EventActivateState.Activate))

            Else

                Me.Execute(New ButtonEventData(New EventKey(Me, CInt(Fix(we))), EventActivateState.Deactivate))

            End If

        End If

    End Sub

    ...

    ' handle all the Wiimote buttons

    HandleButton(WiimoteEvent.Up, ws.ButtonState.Up, _lastBS.Up)

    HandleButton(WiimoteEvent.Down, ws.ButtonState.Down, _lastBS.Down)

    HandleButton(WiimoteEvent.Left, ws.ButtonState.Left, _lastBS.Left)

    HandleButton(WiimoteEvent.Right, ws.ButtonState.Right, _lastBS.Right)

    HandleButton(WiimoteEvent.A, ws.ButtonState.A, _lastBS.A)

    HandleButton(WiimoteEvent.B, ws.ButtonState.B, _lastBS.B)

    HandleButton(WiimoteEvent.Minus, ws.ButtonState.Minus, _lastBS.Minus)

    HandleButton(WiimoteEvent.Home, ws.ButtonState.Home, _lastBS.Home)

    HandleButton(WiimoteEvent.Plus, ws.ButtonState.Plus, _lastBS.Plus)

    HandleButton(WiimoteEvent.One, ws.ButtonState.One, _lastBS.One)

    HandleButton(WiimoteEvent.Two, ws.ButtonState.Two, _lastBS.Two)

    ' save off the current button state for next time

    _lastBS = ws.ButtonState

    _lastNunchuk = ws.NunchukState

And, the current button values are stored away to check on the next event so button events are only fired once.

Now that the event source object is written, it needs to be hooked up to the GlobeControl so it can be used. This can be done by creating an instance of the WiimoteEventSource object, passing in the VE3D's ActionSystem from the BindingsManager object. Then, the event source instance is passed to the ActionSystem's EventSourceManager and registered using the RegisterEventSource method. Event sources should re registered before the control is added to the form.

    // wiimote events

    private WiimoteEventSource _wiimoteEventSource;

    ...

    // create a new instance of the Wiimote event handler

    _wiimoteEventSource = new WiimoteEventSource(this.globeControl.Host.BindingsManager.ActionSystem, this);

    // register it in the event source list

    _globeControl.Host.BindingsManager.ActionSystem.EventSourceManager.RegisterEventSource(this._wiimoteEventSource);

    ' wiimote events

    Private _wiimoteEventSource As WiimoteEventSource

    ...

    ' create a new instance of the Wiimote event handler

    _wiimoteEventSource = New WiimoteEventSource(Me.globeControl.Host.BindingsManager.ActionSystem, Me)

    ' register it in the event source list

    _globeControl.Host.BindingsManager.ActionSystem.EventSourceManager.RegisterEventSource(Me._wiimoteEventSource)

Actions and the BindingManager

Our binding list contains several action types that are not defined by the default VE3D actions. These actions and their handlers must be registered with the VE3D control. The actions can be registered as follows in the FirstFrameRendered event handler:

    BindingsSource bs = new BindingsSource(base.GetType());

    _globeControl.Host.BindingsManager.RegisterAction(bs, "Locations", LocationsHandler);

    _globeControl.Host.BindingsManager.RegisterAction(bs, "LocationsMove", LocationsMoveHandler);

    _globeControl.Host.BindingsManager.RegisterAction(bs, "ToggleVR920", ToggleVR920Handler);

    _globeControl.Host.BindingsManager.RegisterAction(bs, "VR920SetZero", VR920SetZero);

    _globeControl.Host.BindingsManager.RegisterAction(bs, "BalanceBoardSetZero", BalanceBoardSetZero);

    _globeControl.Host.BindingsManager.RegisterAction(bs, "ToggleBalanceBoard", ToggleBalanceBoardHandler);

    _globeControl.Host.BindingsManager.RegisterAction(bs, "FullScreen", FullScreenHandler);

    _globeControl.Host.BindingsManager.RegisterAction(bs, "ToggleVR920Stereo", ToggleVR920Stereo);

    _globeControl.Host.BindingsManager.RegisterAction(bs, "VR920SetEyeDistance", VR920SetEyeDistance);

    BindingsSource bs = new BindingsSource(base.GetType());

    _globeControl.Host.BindingsManager.RegisterAction(bs, "Locations", LocationsHandler);

    _globeControl.Host.BindingsManager.RegisterAction(bs, "LocationsMove", LocationsMoveHandler);

    _globeControl.Host.BindingsManager.RegisterAction(bs, "ToggleVR920", ToggleVR920Handler);

    _globeControl.Host.BindingsManager.RegisterAction(bs, "VR920SetZero", VR920SetZero);

    _globeControl.Host.BindingsManager.RegisterAction(bs, "BalanceBoardSetZero", BalanceBoardSetZero);

    _globeControl.Host.BindingsManager.RegisterAction(bs, "ToggleBalanceBoard", ToggleBalanceBoardHandler);

    _globeControl.Host.BindingsManager.RegisterAction(bs, "FullScreen", FullScreenHandler);

    _globeControl.Host.BindingsManager.RegisterAction(bs, "ToggleVR920Stereo", ToggleVR920Stereo);

    _globeControl.Host.BindingsManager.RegisterAction(bs, "VR920SetEyeDistance", VR920SetEyeDistance);

With the actions registered and handlers associated with them, the actual handlers need to be implemented. All event handler methods must be of the following signature:

public bool EventHandler(EventData cause)

Public Function EventHandler(ByVal cause As EventData) As Boolean

Let’s take a look at the FullScreen binding which simply turns the status bar at the bottom of the window on and off:

private bool FullScreenHandler(EventData eventData)
{
    if(eventData.Activate)
        BeginInvoke(new UIEventHandlerDelegate(FullScreen), eventData);
    return true;
}

Private Function FullScreenHandler(ByVal eventData As EventData) As Boolean
    If eventData.Activate Then
        BeginInvoke(New UIEventHandlerDelegate(AddressOf FullScreen), eventData)
    End If
    Return True
End Function

Private Sub FullScreen(ByVal eventData As EventData)
    statusStrip1.Visible = Not statusStrip1.Visible
End Sub

Because we are in the VE3D thread when this handler is called, we need to use BeginInvoke to call the real method on the UI thread.

Be sure to check the source code for the full demo linked above for the location handler methods. I omitted them here since it is just more of the same type of code above.

VR920 Head Tracker

Once the VR920 glasses and driver are installed on your PC, getting the data required from the glasses is quite easy. We are going to create an object named VR920Tracker which will pull data from the device and send it to VE3D.

First we need to setup a few P/Invoke signatures to talk to the glasses as follows:

    [DllImport("IWEARDRV.dll", SetLastError = true, CharSet = CharSet.Auto)]

    private static extern int IWROpenTracker();

    [DllImport("IWEARDRV.dll", SetLastError = true, CharSet = CharSet.Auto)]

    private static extern void IWRZeroSet();

    [DllImport("IWEARDRV.dll", SetLastError = true, CharSet = CharSet.Auto)]

    private static extern int IWRGetTracking(out int yaw, out int pitch, out int roll);

    private const int ERROR_SUCCESS  = 0;

    <DllImport("IWEARDRV.dll", SetLastError := True, CharSet := CharSet.Auto)> _

    Private Shared Function IWROpenTracker() As Integer

    End Function

    <DllImport("IWEARDRV.dll", SetLastError := True, CharSet := CharSet.Auto)> _

    Private Shared Sub IWRZeroSet()

    End Sub

    <DllImport("IWEARDRV.dll", SetLastError := True, CharSet := CharSet.Auto)> _

    Private Shared Function IWRGetTracking(<System.Runtime.InteropServices.Out()> ByRef yaw As Integer, <System.Runtime.InteropServices.Out()> ByRef pitch As Integer, <System.Runtime.InteropServices.Out()> ByRef roll As Integer) As Integer

    End Function

    private const int ERROR_SUCCESS  = 0;

These 3 methods will allows to open access to the glasses, set its zero position, and get the roll, pitch and yaw information from the sensors.

In the constructor for this object, we will open a handle to the glasses and startup a timer that will poll the glasses at a regular interval to get the sensor data as shown below:

    private const int TimerPeriod = 1;

    private MainForm _mainForm;

    private Timer _timer;

    public VR920Tracker(MainForm mainForm)

    {

        _mainForm = mainForm;

        int openResult = IWROpenTracker();

        if(openResult != ERROR_SUCCESS)

            throw new ApplicationException("Could not connect to VR920: " + openResult);

        _timer = new Timer(VR920Poller, null, Timeout.Infinite, TimerPeriod);

    }

    Private Const TimerPeriod As Integer = 1

    Private _mainForm As MainForm

    Private _timer As Timer

    Public Sub New(ByVal mainForm As MainForm)

        _mainForm = mainForm

        Dim openResult As Integer = IWROpenTracker()

        If openResult <> ERROR_SUCCESS Then

            Throw New ApplicationException("Could not connect to VR920: " & openResult)

        End If

        _timer = New Timer(AddressOf VR920Poller, Nothing, Timeout.Infinite, TimerPeriod)

    End Sub

The VR920Poller method referenced above will be called at a specific interval and read the values from the glasses as shown below:

    // collection of values for averaging

    private List<double> _yawValues = new List<double>();

    private List<double> _rollValues = new List<double>();

    private List<double> _pitchValues = new List<double>();

    // last calculated values

    private double _lastYaw;

    private double _lastPitch;

    private double _lastRoll;

    private void VR920Poller(object state)

    {

        lock(this)

        {

            int yaw, pitch, roll;

            int result = IWRGetTracking(out yaw, out pitch, out roll);

            if(result != ERROR_SUCCESS)

                throw new ApplicationException("Could not get VR920 tracking information: " + result);

            _yawValues.Add(VR920ToRadians(yaw));

            _rollValues.Add(VR920ToRadians(roll));

            _pitchValues.Add(VR920ToRadians(pitch));

            if(_yawValues.Count == 5)

            {

                double y = Average(_yawValues, _lastYaw);

                if(Math.Abs(y - _lastYaw) > 0.026)

                    _lastYaw = y;

                double p = Average(_pitchValues, _lastPitch);

                if(Math.Abs(p - _lastPitch) > 0.017)

                    _lastPitch = p;

                _lastRoll = Average(_rollValues, _lastRoll);

                _yawValues.Clear();

                _pitchValues.Clear();

                _rollValues.Clear();

                RollPitchYaw rollPitchYaw = new RollPitchYaw(_lastRoll, _lastPitch, _lastYaw);

                _mainForm.SetRollPitchYaw(rollPitchYaw);

                _mainForm.lblAxes.Text = VR920ToDegrees(yaw) + ", " + VR920ToDegrees(pitch) + ", " + VR920ToDegrees(roll);

            }

        }

    }

    private double Average(List<double> values, double last)

    {

        double total = 0;

        foreach(double value in values)

            total += value;

        total += last;

        return total / (values.Count+1);

    }

    private double VR920ToRadians(int vr920Value)

    {

        return (vr920Value * .00549) * (Math.PI/180);

    }

    private double VR920ToDegrees(int vr920Value)

    {

        return (vr920Value * .00549);

    }

    ' collection of values for averaging

    Private _yawValues As List(Of Double) = New List(Of Double)()

    Private _rollValues As List(Of Double) = New List(Of Double)()

    Private _pitchValues As List(Of Double) = New List(Of Double)()

    ' last calculated values

    Private _lastYaw As Double

    Private _lastPitch As Double

    Private _lastRoll As Double

    Private Sub VR920Poller(ByVal state As Object)

        SyncLock Me

            Dim yaw, pitch, roll As Integer

            Dim result As Integer = IWRGetTracking(yaw, pitch, roll)

            If result <> ERROR_SUCCESS Then

                Throw New ApplicationException("Could not get VR920 tracking information: " & result)

            End If

            _yawValues.Add(VR920ToRadians(yaw))

            _rollValues.Add(VR920ToRadians(roll))

            _pitchValues.Add(VR920ToRadians(pitch))

            If _yawValues.Count = 5 Then

                Dim y As Double = Average(_yawValues, _lastYaw)

                If Math.Abs(y - _lastYaw) > 0.026 Then

                    _lastYaw = y

                End If

                Dim p As Double = Average(_pitchValues, _lastPitch)

                If Math.Abs(p - _lastPitch) > 0.017 Then

                    _lastPitch = p

                End If

                _lastRoll = Average(_rollValues, _lastRoll)

                _yawValues.Clear()

                _pitchValues.Clear()

                _rollValues.Clear()

                Dim rollPitchYaw As New RollPitchYaw(_lastRoll, _lastPitch, _lastYaw)

                _mainForm.SetRollPitchYaw(rollPitchYaw)

                _mainForm.lblAxes.Text = VR920ToDegrees(yaw) & ", " & VR920ToDegrees(pitch) & ", " & VR920ToDegrees(roll)

            End If

        End SyncLock

    End Sub

    Private Function Average(ByVal values As List(Of Double), ByVal last As Double) As Double

        Dim total As Double = 0

        For Each value As Double In values

            total += value

        Next value

        total += last

        Return total / (values.Count+1)

    End Function

    Private Function VR920ToRadians(ByVal vr920Value As Integer) As Double

        Return (vr920Value *.00549) * (Math.PI/180)

    End Function

    Private Function VR920ToDegrees(ByVal vr920Value As Integer) As Double

        Return (vr920Value *.00549)

    End Function

This chunk calls the IWRGetTracking method and tosses the results in 3 lists for roll, pitch and yaw. then, when the list has 5 members in it, the values are averaged and, if that resulting values is past a certain threshold, the values are passed back to the MainForm’s SetRollPitchYaw method as shown below:

    private double _lastYaw;

    public void SetRollPitchYaw(RollPitchYaw rollPitchYaw)

    {

        if(!_initialized || _globeControl.IsDisposed || (_globeControl.Host) == null || _globeControl.Host.CameraControllers.Current == null)

            return;

        double y = ((_globeControl.Host.CameraControllers.Current as ActionCameraController).LastReportedViewpoint.LocalOrientation.Yaw - _lastYaw) + rollPitchYaw.Yaw;

        RollPitchYaw rpw = new RollPitchYaw(rollPitchYaw.Roll, rollPitchYaw.Pitch, y);

        (_globeControl.Host.CameraControllers.Current as ActionCameraController).LastReportedViewpoint.LocalOrientation.RollPitchYaw = rpw;

        _lastYaw = rollPitchYaw.Yaw;

    }

    Private _lastYaw As Double

    Public Sub SetRollPitchYaw(ByVal rollPitchYaw As RollPitchYaw)

        If (Not _initialized) OrElse _globeControl.IsDisposed OrElse (_globeControl.Host) Is Nothing OrElse _globeControl.Host.CameraControllers.Current Is Nothing Then

            Return

        End If

        Dim y As Double = ((TryCast(_globeControl.Host.CameraControllers.Current, ActionCameraController)).LastReportedViewpoint.LocalOrientation.Yaw - _lastYaw) + rollPitchYaw.Yaw

        Dim rpw As New RollPitchYaw(rollPitchYaw.Roll, rollPitchYaw.Pitch, y)

        TryCast(_globeControl.Host.CameraControllers.Current, ActionCameraController).LastReportedViewpoint.LocalOrientation.RollPitchYaw = rpw

        _lastYaw = rollPitchYaw.Yaw

    End Sub

This method takes the roll, pitch and yaw values provided and applies them to the current CameraController used by VE3D (which, by default, is an ActionCameraController).

So, this code ultimately takes the roll, yaw and pitch of the user’s head and translates that directly to the roll, yaw and pitch of the camera in VE3D providing an accurate, real-time view into the VE3D world.

Stereoscopic Images

The final piece of the puzzle is drawing frames to the VR920 glasses in such a way that the final view to the user will be three dimensional. This works similarly to those Magic Eye puzzles you may have seen. Essentially, for every frame, we want to take the current VE3D camera position and move it several units to the left, render that to the left eye of the glasses, and then move the camera several units to the right and render that to the right eye of the glasses. To handle this we, can create an object named VR920StereoStep which derives from the Step class provided by VE3D and add this to the StepManager. When this is done, the VR920StereoStep will be called at the end of every frame drawn by VE3D.

As with the VR920Tracker class, we will need to setup a few P/Invoke method signatures to talk to the stereo driver as shown below:

    [DllImport("iWrstDrv.dll", EntryPoint = "IWRSTEREO_Open", SetLastError=true)]

    public static extern IntPtr OpenStereo();

    [DllImport("iWrstDrv.dll", EntryPoint = "IWRSTEREO_SetStereo")]

    public static extern Boolean SetStereoEnabled(IntPtr handle, Boolean enabled);

    [DllImport("iWrstDrv.dll", EntryPoint = "IWRSTEREO_SetLR")]

    public static extern Boolean SetStereoLR(IntPtr handle, Boolean eye);

    [DllImport("iWrstDrv.dll", EntryPoint = "IWRSTEREO_WaitForAck")]

    public static extern Byte WaitForOpenFrame(IntPtr handle, Boolean eye);

    <DllImport("iWrstDrv.dll", EntryPoint := "IWRSTEREO_Open", SetLastError:=True)> _

    Public Shared Function OpenStereo() As IntPtr

    End Function

    <DllImport("iWrstDrv.dll", EntryPoint := "IWRSTEREO_SetLR")> _

    Public Shared Function SetStereoLR(ByVal handle As IntPtr, ByVal eye As Boolean) As Boolean

    End Function

    <DllImport("iWrstDrv.dll", EntryPoint := "IWRSTEREO_SetStereo")> _

    Public Shared Function SetStereoEnabled(ByVal handle As IntPtr, ByVal enabled As Boolean) As Boolean

    End Function

    <DllImport("iWrstDrv.dll", EntryPoint := "IWRSTEREO_WaitForAck")> _

    Public Shared Function WaitForOpenFrame(ByVal handle As IntPtr, ByVal eye As Boolean) As Byte

    End Function

In the constructor for this object, we will open a handle to the stereo driver and turn on the stereo functionality of the glasses as shown below:

    private static readonly IntPtr INVALID_FILE_HANDLE = (IntPtr)(-1);

    private IntPtr _hStereo = INVALID_FILE_HANDLE;

    private Host _host;

    private bool _stereoEnabled = true;

    public double EyeDistance { get; set; }

    public VR920StereoStep(StepManager manager, Host host) : base(manager)

    {

        EyeDistance = 10;

        _host = host;

        _hStereo = OpenStereo();

        if(_hStereo != INVALID_FILE_HANDLE)

            SetStereoEnabled(_hStereo, true);

        else

            _stereoEnabled = false;

    }

    Private Shared ReadOnly INVALID_FILE_HANDLE As IntPtr = CType(-1, IntPtr)

    Private _hStereo As IntPtr = INVALID_FILE_HANDLE

    Private _host As Host

    Private _stereoEnabled As Boolean = True

    Private privateEyeDistance As Double

    Public Property EyeDistance() As Double

        Get

            Return privateEyeDistance

        End Get

        Set(ByVal value As Double)

            privateEyeDistance = value

        End Set

    End Property

    Public Sub New(ByVal manager As StepManager, ByVal host As Host)

        MyBase.New(manager)

        EyeDistance = 10

        _host = host

        _hStereo = OpenStereo()

        If _hStereo <> INVALID_FILE_HANDLE Then

            SetStereoEnabled(_hStereo, True)

        Else

            _stereoEnabled = False

        End If

    End Sub

Next, we need to override the OnExecute method provided by the base Step class as shown below:

    private bool _rightEye = true;

    private Vector3D _position;

    public override void OnExecute(SceneState state)

    {

        if(_hStereo != INVALID_FILE_HANDLE && _stereoEnabled && (_host.CameraControllers.Current as ActionCameraController).LastReportedViewpoint != null)

        {

            if(!_rightEye)

            {

                _position = (_host.CameraControllers.Current as ActionCameraController).LastReportedViewpoint.Position.Vector;

                (_host.CameraControllers.Current as ActionCameraController).LastReportedViewpoint.Position.Vector = _position + new Vector3D(-EyeDistance, 0, 0);

            }

            else

            {

                _position = (_host.CameraControllers.Current as ActionCameraController).LastReportedViewpoint.Position.Vector;

                (_host.CameraControllers.Current as ActionCameraController).LastReportedViewpoint.Position.Vector = _position + new Vector3D(EyeDistance, 0, 0);;

            }

            SetStereoLR(_hStereo, _rightEye);

            WaitForOpenFrame(_hStereo, _rightEye);

            _rightEye = !_rightEye;

        }

    }

    Private _rightEye As Boolean = True

    Private _position As Vector3D

    Public Overrides Sub OnExecute(ByVal state As SceneState)

        If _hStereo <> INVALID_FILE_HANDLE AndAlso _stereoEnabled AndAlso (TryCast(_host.CameraControllers.Current, ActionCameraController)).LastReportedViewpoint IsNot Nothing Then

            If (Not _rightEye) Then

                _position = (TryCast(_host.CameraControllers.Current, ActionCameraController)).LastReportedViewpoint.Position.Vector

                TryCast(_host.CameraControllers.Current, ActionCameraController).LastReportedViewpoint.Position.Vector = _position + New Vector3D(-EyeDistance, 0, 0)

            Else

                _position = (TryCast(_host.CameraControllers.Current, ActionCameraController)).LastReportedViewpoint.Position.Vector

                TryCast(_host.CameraControllers.Current, ActionCameraController).LastReportedViewpoint.Position.Vector = _position + New Vector3D(EyeDistance, 0, 0)

            End If

            SetStereoLR(_hStereo, _rightEye)

            WaitForOpenFrame(_hStereo, _rightEye)

            _rightEye = Not _rightEye

        End If

    End Sub

OnExecute will be called at the end of each VE3D frame rendered. In this method, we obtain the current position of the VE3D camera and then set its position several units to the left or right (- or + EyeDistance). SetStereoLR from the VR920 API is called, setting the appropriate eye to be active (left = false, right = true). Then WaitForOpenFrame is called which will pause the action long enough for the glasses to draw the entire frame to eye to avoid image tearing. Essentially, this call “sits and spins” at this location until the glasses report back that the entire frame is drawn in the proper eye and that we can continue on drawing the opposite eye for the next frame. Finally, the _rightEye member variable is togged to the opposite value, so the next time through this method the opposite eye is drawn.

With this class in place, back in our MainForm in the FirstFrameRendered event handler, we can instantiate the object and add it to the StepManager as shown below:

    private VR920StereoStep _vr920StereoStep;

    _vr920StereoStep = new VR920StereoStep(_globeControl.Host.RenderEngine.StepManager, _globeControl.Host);

    _globeControl.Host.RenderEngine.StepManager.Add(_vr920StereoStep);

    Private _vr920StereoStep As VR920StereoStep

    _vr920StereoStep = New VR920StereoStep(_globeControl.Host.RenderEngine.StepManager, _globeControl.Host)

    _globeControl.Host.RenderEngine.StepManager.Add(_vr920StereoStep)

Running the Application

To run the demo, do the following:

Copy BindingsWiimote.xml to the appropriate directory listed above
Pair the Wiimote and Balance Board to the computer. See the WiimoteLib article for more information on how to do that
Run the executable
Stand on the Balance Board
Put on the glasses
Zero both the Balance Board and glasses
Toggle everything on or off at will

Controls

F1 – Set VR920 glasses to zero position
F2 – Set Balance Board to zero position
F3 – Toggle stereo mode on and off
Wiimote 1 or B – Toggle Balance Board on or off
Wiimote 2 or V – Toggle VR920 head tracking on or off
F – Toggle full-screen on or off
Nunchuk joystick X/Y – Strafe/Move
Nunchuk C/Z buttons – Raise/lower altitude
Wiimote A – Open location menu/Select location (note that if a new location is selected, the Balance Board is turned off and must be re-enabled after “landing”
Wiimote Dpad Up/Down – Move through List
Balance Board – Lean your body left or right to turn in the VE3D environment

Conclusion

With the above code, we have created a very interesting interface for Virtual Earth 3D. The demo and source code linked above contain a few more features and bindings which enhance the application a bit more. Be sure to give the full demo a try and check out the full source code for a few more implementation details.

Additional Information

Bio

Brian is a Microsoft C# MVP who has been actively developing in .NET since its early betas in 2000, and who has been developing solutions using Microsoft technologies and platforms for even longer. Along with .NET, Brian is particularly skilled in the languages of C, C++ and assembly language for a variety of CPUs. He is also well-versed in a wide variety of technologies including web development, document imaging, GIS, graphics, game development, and hardware interfacing. Brian has a strong background in developing applications for the health-care industry, as well as developing solutions for portable devices, such as tablet PCs and PDAs. Additionally, Brian has co-authored the book "Debugging ASP.NET" published by New Riders, and is currently co-authoring a book titled "Coding4Fun: 10 .NET Programming Projects for Wiimote, YouTube, World of Warcraft, and More" to be published by O'Reilly in December 2008. Brian is also an author for MSDN's Coding4Fun website. You can reach Brian via vis website at http://www.brianpeek.com/ .

Where am I? What am I doing?

Coding4Fun — Sat, 01 Sep 2007 01:17:53 GMT

	Do have a GPS receiver laying around? Do you travel? Do you blog? If you answered yes to these questions, you might be interested in this Windows Live Writer plugin. People who travel and like to blog about it can just click a button to insert their current coordinates along with a Virtual Earth Map.
Arian Kulp Arian's Blog Difficulty: Intermediate Time Required: 1-3 hours Cost: $50 and up (depending on hardware choice) Software: Visual Basic or Visual C# Express Editions, Windows Live Writer Hardware: A GPS receiver that supports the NMEA protocol (they all should...) Download: C# Download VB Download MSI Download (binary only, ready to install!)

Introduction

I've had a yellow Garmin etrex GPS unit in the home for about two years now. My family enjoys geocaching (though we don't go nearly often enough...), and it was a good price. The unit has a small screen, but it lets you enter coordinates, then track your way to them. Really, just having your current coordinates is a pretty cool thing!

I've always enjoyed playing with GPS software, and I've liked the idea of creating my own. Unfortunately, I didn't know how to interface with the hardware, so I was out of luck. After buying a serial/USB cable, I was half-way there, but what to do with the data stream?

In this article, learn how to read the stream of data from (hopefully) any GPS unit, then create a plugin for Windows Live Writer so you can add your current location to any blog entry. Scott Hanselman created a GPS interface in Where the Heck am I? Connecting .NET 2.0 to a GPS back in October of 2006. As always with Scott, it was a great article, but I didn't come across it until I'd made it through a lot of roadblocks already! My result at this point works pretty well so I decided to stick with it, plus it integrates with Windows Live Writer.

Figure 1: The plugin in action

If you want, download the source code from the links above, or just follow along with the article. If you haven't already, download the appropriate version of Visual Studio 2005 Express Edition. The plugin will run on either XP or Vista.

Choosing your Hardware

Though my main GPS receiver is the Garmin etrex, I also recently bought an Ambicom unit with no screen -- just a Bluetooth link. It's the perfect device just for obtaining information for a PDA, laptop, or other Bluetooth-connected device if you don't need display on the device. It uses the SPP profile for data exchange, so it's just another COM port as well.

My other unit, the Garmin, has four metal dots hidden under a rubber flap. A number of sites provide information on the pin-outs, or you can buy a cable (serial). I bought a serial-USB interface to avoid the serial port on the back of the machine. My assumption is that all GPS units end up with a serial connection via virtual or physical COM port. If I'm wrong, you may need to do some extra work to get the data.

Speaking NMEA

Speaking about data... once you have a connection to the unit (4800 bps, 8-N-1), you will have a continuously updated stream of information flowing in. The format of this data is defined by the National Marine Electronics Association (NMEA), and is an older format than most people would expect. Interpreting the information is pretty easy once you understand the format (of course, that's often the case!).

Sample NMEA Data

$GPRMB,V,,,,,,,,,,,,A,N*13

$GPGGA,,,,,,0,00,,,M,,M,,*66

$GPGSA,A,1,,,,,,,,,,,,,,,*1E

$GPGSV,3,1,10,02,60,

$GPRMC,,V,,,,,,,210807,0.2,E,N*36

241,00,04,78,042,00,05,24,314,00,09,20,252,00*7F

$GPGSV,3,2,10,10,07,185,00,12,44,307,00,17,36,089,00,20,08,035,00*7A

$GPGSV,3,3,10,28,05,147,00,30,08,319,00*75

$GPGLL,,,,,,V,N*64

$GPBOD,,T,,M,,*47

$PGRME,,M,,M,,M*00

There are three things to notice at first glance (four if you're really on the ball!). First, every line starts with a dollar sign. Second, that dollar sign is always followed by five upper-case letters (not all begin with "G" though). Third, the data following the capital letters is comma-delimited. If you looked really close, you might have noticed that every line ends with an asterisk, then two more characters that appear to be in hexadecimal format.

The capital letters after the dollar sign represent the message ID. All standard NMEA messages start with "G." In this example, the line beginning with "P" is a Garmin-specific message. It's proprietary, but it's also documented and freely available from their web site.

With each line being comma-delimited, processing is pretty easy at a high-level. From the System.IO.Ports.SerialPort class, you can call ReadLine() to get a line of data as a string. Each line of data is called a "sentence" in the standard. From there, the Split() method can be used to return an array for the "words."

That hexadecimal number is the checksum for the sentence. It's a simple formula too. Take, in order, every character between (not including) the dollar sign and asterisk, and XOR them together. The first character seeds the checksum (in other words, a one-character sentence's checksum would be equal to that one character). The LocationInterface class handles all of the serial I/O and NMEA protocol implementation (the little there is!).

Visual Basic

Public Shared Function GetChecksum(ByVal sentence As String) As String
    ' A sentence can't be shorter than 9 characters ($XXXXX*0x)
    If sentence.Length < 9 Then
        Return ""
    End If

    ' Seed with the second character (remember, skip the $)
    Dim checksum As Integer = Convert.ToByte(sentence(1))
    Dim loc As Integer = 0, [end] As Integer = sentence.Length - 3

    ' Loop through the characters
    For Each c As Char In sentence
        ' XOR every character between the dollar sign and the asterisk
        If loc > 1 AndAlso loc < [end] Then
            checksum = checksum Xor Convert.ToByte(c)
        End If
        loc += 1
    Next

    ' "X2" is the format string for hexadecimal
    Return checksum.ToString("X2")
End Function

Visual C#

// Calculates the checksum for a sentence
public static string GetChecksum(string sentence)
{
    // A sentence can't be shorter than 9 characters ($XXXXX*0x)
    if( sentence.Length < 9 ) return "";

    // Seed with the second character (remember, skip the $)
    int checksum = Convert.ToByte(sentence[1]); // Second character
    int loc = 0, end = sentence.Length - 3;

    // Loop through the characters
    foreach (char c in sentence)
    {
        // XOR every character between the dollar sign and the asterisk
        if (loc > 1 && loc < end)
        {
            c2 = c2 ^ Convert.ToByte(c);
        }
        loc++;
    }
    // "X2" is the format string for hexadecimal
    return checksum.ToString("X2");
}

With this handy method, you can validate every line of data that you have before bothering to process it. If a line's bad, don't make a guess. The nature of NMEA is to repeat data frequently. If you missed it this time, you'll have it a second or so later probably. In my code, when I want to get the next data sentence, I call my ReadNextGoodLine() method. This calls ReadLine() on the SerialPort, and validates the checksum. It will try up to ten times before returning a NULL.

Visual Basic

Private Function ReadNextGoodLine(ByVal port As SerialPort) As String
    ' Must read the next line.  This may not be a complete NMEA line
    ' (or even not an NMEA line at all!)
    Dim line As String = Nothing
    For i As Integer = 0 To 9

        ' Up to ten tries...
        line = port.ReadLine()

        If Not IsSentenceValid(line) Then
            line = Nothing
        Else
            Exit For

        End If
    Next

    Return line
End Function

Visual C#

private string ReadNextGoodLine(SerialPort port)
{
    // Must read the next line.  This may not be a complete NMEA line
    // (or even not an NMEA line at all!)
    string line = null;

    // Up to ten tries...
    for (int i = 0; i < 10; i++)
    {
        line = port.ReadLine();

        if (!IsSentenceValid(line)) line = null;
        else break;
    }

    return line;
}

Now that I can read sentences and validate them, I can also create a basic auto-scan GPS search. From the SerialPort class, I can call GetPortNames(). This returns a string array of port names. I can instantiate each port by name, call open, then call ReadNextGoodLine(). If it returns NULL (couldn't find valid NMEA after ten lines), assume it isn't a GPS unit and keep moving to the next port. The ComPortName property saves the name, and can also be set manually to avoid the search.

Of Latitudes and Longitudes...

One thing I learned from this project, is just how much math is involved with GPS! You take it for granted when the device shows coordinates, but there is so much more going on in there.

Recall that GPS is all about geosynchronous satellites sending out time signals. Based on the time delay from satellites with known geostationary locations, you can locate your position. As any experienced GPS user knows, however, when you first turn on the unit, or if you are in a house, underground, or often even wooded areas, the unit will be searching for a signal and won't display anything. The $GPRMC sentence continues to arrive, but the coordinates may be blank, or indicate the lack of a "fix."

$GPRMC,153538,A,4140.5555,N,09100.0000,W,000.0,246.0,310807,,,A*66

The GPRMC consists of a lot of information, including latitude, longitude, speed, heading, and UTC date/time. If you break up the sentence based on commas, the data words that I concentrate in the sample code are 3 (latitude), 4 (N/S), 5 (longitude), 6 (E/W), and 2 (A means fix, V or blank is no fix).

Visual Basic

Public Function ParseGPRMC(ByVal sentence As String) As CoordinatePair
    Dim loc As CoordinatePair = Nothing
    Dim fix As Boolean = False

    ' Split the sentence into words
    Dim words As String() = sentence.Split(","c)

    ' Not all sentences have data populated
    If words(3) <> "" AndAlso words(4) <> "" AndAlso words(5) <> "" AndAlso words(6) <> "" Then
        ' NMEA coordinates are in DM (Degrees-Minutes) format 
        Dim lat As New Coordinate(Integer.Parse(words(3).Substring(0, 2)), _
            Decimal.Parse(words(3).Substring(2)), _
            DirectCast([Enum].Parse(GetType(HemisphereOptions), words(4)), HemisphereOptions))
        Dim lng As New Coordinate(Integer.Parse(words(5).Substring(0, 3)), _
            Decimal.Parse(words(5).Substring(3)), _
            DirectCast([Enum].Parse(GetType(HemisphereOptions), words(6)), HemisphereOptions))

        loc = New CoordinatePair(lat, lng)

        ' Does the device currently have a satellite fix?
        If words(2) = "A" Then
            ' Fix has been obtained
            loc.Fixed = True
        ElseIf words(2) = "V" Then
            ' Fix has been lost
            loc.Fixed = False
        End If

    End If

    Return loc
End Function

Visual C#

public CoordinatePair ParseGPRMC(string sentence)
{
    CoordinatePair loc = null;

    // Split the sentence into words
    string[] words = sentence.Split(',');

    // Not all sentences have data populated
    if (words[3] != "" && words[4] != "" && words[5] != "" && words[6] != "")
    {
        // NMEA coordinates are in DM (Degrees-Minutes) format 
        Coordinate lat = new Coordinate(int.Parse(words[3].Substring(0, 2)), 
            decimal.Parse(words[3].Substring(2)), 
            (HemisphereOptions)Enum.Parse(typeof(HemisphereOptions), words[4]));
        Coordinate lng = new Coordinate(int.Parse(words[5].Substring(0, 3)), 
            decimal.Parse(words[5].Substring(3)),
            (HemisphereOptions)Enum.Parse(typeof(HemisphereOptions), words[6]));

        loc = new CoordinatePair(lat, lng);
        
        // Does the device currently have a satellite fix?
        if (words[2] == "A") loc.Fixed = true; // Fix has been obtained
        else if (words[2] == "V") loc.Fixed = false; // Fix has been lost
    }

    return loc;
}

The latitude and longitude values in NMEA data are expressed as DM format. DM is short for Degrees-Minutes, and actually the minutes are expressed in decimal form. In the above example, "4140.5555" is meant to be parsed as 41 degrees and 40.5555 minutes. Longitude uses three digits for degrees, so "09100.0000" is 091 degrees and 00.0000 minutes. Degrees-Minutes is just one of three common ways to express coordinates though. Degrees-Minutes-Seconds splits the minutes into minutes and seconds. If you think of it like time, it's easier. If the decimal minutes is .50000, then that's half of 60, or 30 minutes. A number like .55555 would spill into the seconds place as well. You can also use Degrees-Decimal which simply uses one decimal number to express the degrees along with any fractional "extra." Just remember that the source format is Degrees-Minutes, and given any format, you can convert to any other. My Coordinate class provides ToString() methods to return in all three representations, and internally it stores it as int degrees and minutes and decimal seconds. For more information on these numbers, Wikipedia has a great article.

Plugging In

Functioning as a Live Writer plugin requires that you do several things. First of all, you will need to download and install the Windows Live Writer SDK. This gives you access to an assembly to add a reference to in your project. Your entry point class should import the WindowsLive.Writer.Api namespace, and inherit ContentSource. You'll need to add the WriterPlugin attribute with values for a unique GUID (use the Tools | Create GUID menu item in Visual Studio to help with that), and set a name at least.

The EditOptions method must be overridden if you provide an Options dialog (in this sample, it's just a message box. The CreateContent method is the meat of the plugin. This can do anything, then assign a string to the newContent ref parameter. It returns a DialogResult to indicate if it was OK'd or Cancel'd.

The InsertableContentSource attribute lets you set the SideBar text property, and the WriterPlugin attribute has a property for ImagePath to create an image link:

Figure 2: Ready to use

When CreateContent is called, the sample plugin shows the LocationPreviewForm dialog. This in turn triggers the GPSInit method in LocationInterface to find and connect to the GPS unit. If it's found, the COM port and coordinates are shown. All GPS activity is performed in a BackgroundWorker to maintain a responsive user interface. The Title and Description fields are used for a pushpin in a Virtual Earth embedded AJAX map. This is a full-blown instance with the scrolling and zooming.

Figure 3: The plugin dialog

Actually, I would have liked to use a static bitmap, but I'm not sure how to obtain good free mapping images in a small amount of code, other than services like MapPoint which aren't free. Changing this aspect of the code would be very easy in the GetOutput method, and you could remove the image/map altogether and simply show the coordinates or other information. It's not exactly extensible, but it's easy to modify.

Finally, in order to simplify debugging, I created a post-build event to copy the plugin DLL to the Windows Live Writer Plugins directory:

XCOPY /D /Y /R "$(TargetPath)" "C:\Program Files\Windows Live Writer\Plugins\"

I also went into the Debug settings for the project for the project and set the Start Action to Start external program, setting it to the path to the EXE:

C:\Program Files\Windows Live Writer\WindowsLiveWriter.exe

Now, when you press F5, it copies the DLL and attaches the debugger to a new instance of Windows Live Writer for Edit-and-Continue and other debugging. It works well!

One thing to remember, is that Live Writer locks the DLL's when it's running, so you will get an error if you even build while it's running, since the DLL can't be copied.

Next Steps

At this point, I'd like to create more flexibility on what the plugin does with the coordinates. It could locate images through various sources, include altitude, speed, and heading when available, or just pretty up the embedded information. I'd like to work on the GPS classes a bit to make them more robust, handling some of the other messages, and support raising events to be used in other applications. Developing GPS applications is conceptually pretty easy once you have the data. It's just a matter of exposing the data in a good way.

Conclusion

Working with GPS data was much easier than I expected. I had some Bluetooth/USB issues at first, but working with the data was fun and gave me lots of ideas! See what you can do with location information to enhance some existing application, or think up something entirely new. Have fun with the code and as usual, contact me through my blog for comments, complaints, or questions.

Arian Kulp is an independent software developer and writer working in the Midwest. He has been coding since the fifth grade on various platforms, and also enjoys photography, nature, and spending time with his family. Arian can be reached through his web site at http://www.ariankulp.com.

SweetSpot - The Beginnings of a Glucose Meter Downloader

Coding4Fun — Tue, 17 Apr 2007 11:44:09 GMT

In this installment of "Some Assembly Required" column, Scott Hanselman, a Type 1 Diabetic, uses .NET to download blood sugar numbers from a Glucose Meter. Along the way he lays down the rough scaffolding for a plugin system so other meters could be added easily. The application exports to XML or CSV for analysis within Microsoft Excel. Thanks to http://www.sweetspot.dm for their support and the initial idea.

Scott Hanselman

Scott's Blog

Difficulty: Easy

Time Required: Less than 1 hour

Cost: Less Than $50

Software: Visual Basic or Visual C# Express Editions

Hardware: A FreeStyle or FreeStyle Flash Glucose Meter (sample data is included in the code for those who don't have a meter)

Download: Download

Summary: In this installment of "Some Assembly Required" column, Scott Hanselman, a Type 1 Diabetic, uses .NET to download blood sugar numbers from a Glucose Meter. Along the way he lays down the rough scaffolding for a plugin system so other meters could be added easily. The application exports to XML or CSV for analysis within Microsoft Excel. Thanks to http://www.sweetspot.dm for their support and the initial idea. Visit SweetSpot to see a greatly expanded version of this Meter Downloader and take a look at http://www.hanselman.com/fightdiabetes to make a donation to the ADA and Team Hanselman and this year's Walk for Diabetes.

The Idea

I've been diabetic since I was twenty-one. Actually, it happened a just few months before my 21st birthday. Needless to say, it's no fun. Most diabetes prick their fingers at least four times a day, often more. I test my blood sugar at least 10 times a day. This adds up to a lot of data points in a Glucose Meter. Checking your numbers gives you a snapshot of how you're doing, but the really valuable information lies in the analysis - but the data is trapped inside the blood sugar meter.

Most meters have some kind of cable that can be hooked up to your computer, allowing downloads of these hundreds of data points. My friend Adam from SweetSpot and I wanted to get at that raw data, geeks that we are, so we decided to write a program to do it.

Plugins

I'm using the FreeStyle Flash meter, but there's lots of different meters out there. We wanted a simple plugin system so that folks could write their own plugins and expand the application. First, I created this simple interface:

namespace C4FGlucoseMeterDownloader
{
    public interface IMeter
    {
        MeterResponse  GetMeterData(string configuration);
        Image GetMeterPicture();
        string GetDisplayName();
    }
}

It could be made more complex, but this one does the job nicely. The program will call GetMeterData passing in any saved configuration (like USB or Serial details...this was included for future use) when it's time for the plugin to fetch the data from the meter. We'll talk about the "MeterResponse" type later in this article.

We'll also call GetMeterPicture and GetDisplayName while we spin through all the meter types to fill out the details of our main interface as seen at right.

For this sample, we'll be keeping a Dictionary of IMeter instances at the ready. We spin through the currently executing assembly looking for types that implement IMeter. A future version might look for assemblies in a \plugins directory.

Type[] types = Assembly.GetExecutingAssembly().GetTypes();
foreach (Type t in types)
{
    if (typeof(IMeter).IsAssignableFrom(t) && t.IsInterface == false)
    {
        IMeter i = Activator.CreateInstance(t) as IMeter;
        string displayName = i.GetDisplayName();
        this.comboBox1.Items.Add(displayName);
        meterTypes.Add(displayName, i);
    }
}

When using reflection to find types that implement a certain interface, you use the very-not-intuitive method "IsAssignableFrom." It makes sense once you think about it, but it didn't jump out at me. I expected a method like "ImplementsInterface," but perhaps that's just me.

The image of the meter would be stored in the same assembly as the plugins, so we can put it out easily without external dependencies.

public Image GetMeterPicture()
{
    using (Stream sr = Assembly.GetExecutingAssembly().
                 GetManifestResourceStream("
                 C4FGlucoseMeterDownloader.Resources.freestyle_flash_gm.jpg"))
    {
        Image i = Image.FromStream(sr);
        return i;
    }
}

Note the use of the "using" statement. I'm a huge fan of this statement. Be sure to use it anytime you're creating something that is IDisposable and you'll be sure to avoid object leaks.

Meter Communication - Talking Serial

The FreeStyle meter uses a 1/8" headphone jack connecting to a 9-PIN RS-232 serial port. I use a standard USB to 9-PIN adapter to hook it all together. We could ask the user for the Serial Port that the device is connected to, but how would they know? Things like COM Port names are hidden better than ever before. Do I really want my Grandma going into the Device Manager to make an educated guess? Instead, since the FreeStyle has a fantastically simple protocol with easy to spot results, we'll just spin through every COM Port and send the command until one of them works.

There's many ways to tackle Serial Communication with .NET. There's the standard port.Read() way of doing things, but for simple "dump-style" communication, I like the SerialDataReceivedEventHandler. It's an event that's raised by the SerialPort when data shows up. You just have to call port.ReadExisting. It's nice and simple.

string[] portNames = System.IO.Ports.SerialPort.GetPortNames();
foreach (string portName in portNames)
{
    if (found == true)  {  break; }
    try
    {
        Debug.WriteLine("Trying " + portName);
        port = new SerialPort(portName);
        port.BaudRate = 19200;
        port.StopBits = StopBits.One;
        port.Parity = Parity.None;
        port.DataBits = 8;

        port.DataReceived += new SerialDataReceivedEventHandler(PortDataReceived);

        port.ReadTimeout = 500;  port.WriteTimeout = 500;

        port.Open();
        port.Write("mem");

        long timeout = 0;
        while (port.IsOpen)
        {
            System.Threading.Thread.Sleep(500);
            if (++timeout > 10 && found == false)
            {
                break; //give up after 5 seconds..
            }
        }
    }
    //ERROR HANDLING REMOVED FOR BREVITY...SEE THE CODE
    finally
    {
        if (port.IsOpen)
        {
            port.DataReceived -= PortDataReceived;
            port.Close();
        }
        port = null;
    }
}

Here we just keep sending "mem" - the FreeStyle's dump command - to each port we have until one gives us something. The eventHandler sets the "found" flag.

void PortDataReceived(object sender, SerialDataReceivedEventArgs e)
{
    string next = port.ReadExisting();
    found = true;
    System.Diagnostics.Debug.WriteLine("RECEIVED:" + next);
    result += next; //Not using a StringBuilder is inefficient, truly, but this is easier and allows us to watch the the string for "END" without worrying about it being truncated like "EN" and "D"
    if (result.Contains("END"))
    {
        Properties.Settings.Default.LastPortFound = port.PortName;
        port.Close();
    }
}

We keep going until we find the "END" string returned by the Glucose Meter. Again, there's literally a half-dozen ways to have done this. Simply spinning in a while(port.ReadBytes()) loop would do the job as well, but this is a good way to introduce the SerialDataReceivedEventHandler. We just keep appending the received data and the SerialPort class keeps things in order and ensures nothing is dropped.

NOTE: You can run this sample even without a Glucose Meter. Simply uncomment out the line marked "//HACK:" in the MeterFreestyle.cs file and the Meter plugin will return sample data.

Parsing

The data returned from the Meter is quite regular, for example:

128 Oct 09 2006 23:44 26 0x00
122 Oct 09 2006 22:18 26 0x00
261 Oct 09 2006 21:14 26 0x00
070 Oct 09 2006 21:14 26 0x00
0x3B2C END

The first number is the blood sugar value. Ideal numbers are between 80 and 120mg/dl. Diabetics aim for these kinds of values, but often their blood sugar varies. Checking blood sugar often allows us to get back to normal as soon as possible, thereby avoiding side effects of chronically high sugar. Take a look at "Scott's Diabetes Explanation: The Airplane Analogy" for more details on Diabetes, or the Diabetes Section of my blog.

We want to parse this data and turn it into both XML and CSV. I created a simple data structure to get us started, and marked it up for XML Serialization:

namespace C4FGlucoseMeterDownloader
{
    public class MeterResponse
    {
        public MeterResponse() { GlucoseReadings = new List<GlucoseReading>(); }

        [XmlIgnore]
        public List<GlucoseReading> GlucoseReadings;

        [XmlArray("GlucoseReadings")]
        public GlucoseReading[] ArrayGlucoseReadings
        {
            get { return GlucoseReadings.ToArray(); }
            set { GlucoseReadings = new List<GlucoseReading>(value); }
        }
    }

    public class GlucoseReading
    {
        public GlucoseReading() { }

        public GlucoseReading(DateTime date, decimal BG)
        {
            this.BG = BG;
            this.Date = date;
        }

        public decimal BG;
        public DateTime Date;
    }
}

As we parse the raw data, we return GlucoseReadings to build up a MeterResponse.

protected GlucoseReading ParseResultRowToGlucoseReading(string data)
{
    //TODO: Error Handling
    //227  Oct  11 2006 01:38 17 0x00
    string BGString = data.Substring(0,5);
    decimal BGValue = decimal.Parse(BGString.Trim(), 
        System.Globalization.CultureInfo.InvariantCulture);

    string timeString = data.Substring(5,18);
    DateTime recordDateTime = DateTime.Parse(timeString,
        System.Globalization.CultureInfo.InvariantCulture);
    return new GlucoseReading(recordDateTime, BGValue);
}

And the resulting data is then exported automatically into CSV and XML:

public void SerializeAsXml(MeterResponse r, string fileNameSeed)
{
    XmlSerializer x = new XmlSerializer(typeof(MeterResponse));
    using (Stream s = File.OpenWrite(fileNameSeed + ".xml"))
    {
        x.Serialize(s, result);
    }
}

public void SerializeAsCsv(MeterResponse r, string fileNameSeed)
{
    using (StreamWriter sw = new StreamWriter(fileNameSeed + ".csv"))
    {
        sw.WriteLine("Date,Glucose");
        foreach(GlucoseReading gr in r.GlucoseReadings)
        {
            sw.WriteLine(String.Format("{0},{1}",gr.Date,gr.BG.ToString()));
        }
    }
}

Analyzing the Data

The XML could be styled with XSLT, manipulated by another program, or you could store it in a database. The CSV file is easily consumed by Excel as seen in the screenshot at right.

Make sure you use a "scatter plot" in Excel if you want to see all your data. Axes in Line Charts that use Dates as their date type in Excel have all values appearing at midnight. Since Glucose data is time (even minute) sensitive, use a scatter chart to get accuracy.

Don't worry, those aren't my actual blood sugar numbers, it's just generated sample data.

Conclusion

Take a look at http://www.hanselman.com/fightdiabetes to read my personal story and make a donation to the ADA and Team Hanselman and this year's Walk for Diabetes.

There's things that could be extended, added, and improved on with this project. Here are some ideas to get you started:

Add support for more Glucose Meters.
Add embedded charts and graphs.
Create a PDF report and email it to your doctor or family.
Make it a more generic medical application that takes data from blood pressure machines or other medical data-gathering devices.

Be sure to check out http://www.sweetspot.dm and get in on their beta. They've already extended this application significantly with multiple meter support, ClickOnce Deployment, lots of special sauce, and an extensive online analysis system. Thanks again, have fun and have no fear when faced with the words - Some Assembly Required!

Scott Hanselman is the Chief Architect at the Corillian Corporation, an eFinance enabler. He has thirteen years experience developing software in C, C++, VB, COM, and most recently in VB.NET and C#. Scott is proud to be both a Microsoft RD and Architecture MVP. He is co-author of Professional ASP.NET 2.0 with Bill Evjen, available on BookPool.com and Amazon. His thoughts on the Zen of .NET, Programming and Web Services can be found on his blog at http://www.computerzen.com. He thanks his wife and son Zenzo for indulging him in these hobbies!

Hardware Boneyard - Using the CueCat with .NET

Coding4Fun — Tue, 06 Mar 2007 12:26:53 GMT

	In this installment of "Some Assembly Required" column, Scott Hanselman borrows Travis Illig's CueCat BarCode scanner and creates a plugin for Windows Live Writer than lets him blog more easily about books he's reading. We decode the bar code info, change UPCs into ISBNs, call Amazon's Web Service via REST and integrate with Windows Live Writer all in one article. Whew!
Scott Hanselman Scott's Blog Difficulty: Easy Time Required: Less than 1 hour Cost: Less Than $50 Software: Visual Basic or Visual C# Express Editions Hardware: A CueCat - try eBay or your garage Download: Download

Summary: In this installment of "Some Assembly Required" column, Scott Hanselman borrows Travis Illig's CueCat BarCode scanner and creates a plugin for Windows Live Writer than lets him blog more easily about books he's reading. We decode the bar code info, change UPCs into ISBNs, call Amazon's Web Service via REST and integrate with Windows Live Writer all in one article. Whew!

The CueCat BarCode Scanner

Here's what Wikipedia has to say about the CueCat BarCode scanner:

The :CueCat is a cat-themed handheld barcode reader developed in the late 1990s by the now-defunct DigitalConvergence Corporation, which connected to computers using the PS/2 keyboard port and later USB. The :CueCat enabled users to link to an Internet URL by scanning a barcode appearing in an article, catalog or on some other printed matter. In this way a user could be directed to a web page containing related information. The system that supported this functionality is no longer in operation.

Ah, nothing like the smell of obsolete hardware to get me thinking. You can get CueCats all over, there are millions of them, including on ebay. I asked around and my buddy Travis Illig loaned me his. It's ripe for hacking.

Decoding the CueCat Output

The CueCat speaks a funky encoded format that it "types" as if it were a keyboard. For all intents, it IS a keyboard as the PS2 version we're using is installed in series with your existing (if you have one) keyboard.

For example, my copy of Neil Gaiman's excellent "Stardust" produces this output:

.C3nZC3nZC3nZE3r0Chr3CNnY.cGf2.ENr7C3n1C3PWD3rYCxzYChnZ.

Which can be broken up into a series of tokens, some are one character and some are two. The periods are delimiters, so we'll toss them for now.

C3 n Z C3 n Z C3 n Z E3 r 0 Ch r 3 CN n Y EN r 7 C3 n 1 C3 P W D3 r Y Cx z Y Ch n Z

The first part is the serial number of CueCat itself, which freaked a log of privacy folks out in the past. The scanned barcode is actually the last part after the third . delimiter:

EN r 7 C3 n 1 C3 P W D3 r Y Cx z Y Ch n Z

Each of these tokens is a key to a lookup table. The encoding is pretty lame, and very arbitrary, but it confounded folks for a few hours at least. There's dozens of implementations of this decoding to be found on the Web, although none in .NET, which is where we come in. Here's a resource page that includes several implementations in C, Perl and Python, http://www.beau.lib.la.us/~jmorris/linux/cuecat/ but it hasn't been updated in at least six years. There's also some very old but interesting discussion here http://cexx.org/cuecat.htm.

The lookup table looks like this (a C# example):

static Decoder()
{
    decodingTable = new Dictionary<string, int>[3];
    decodingTable[0] = new Dictionary<string, int>();
    decodingTable[1] = new Dictionary<string, int>();
    decodingTable[2] = new Dictionary<string, int>();
 
    decodingTable[0]["Z"] = 0;
    decodingTable[0]["Y"] = 1;
    decodingTable[0]["X"] = 2;
    decodingTable[0]["W"] = 3;
    decodingTable[0]["3"] = 4;
    decodingTable[0]["2"] = 5;
    decodingTable[0]["1"] = 6;
    decodingTable[0]["0"] = 7;
    decodingTable[0]["7"] = 8;
    decodingTable[0]["6"] = 9;
 
    decodingTable[1]["C3"] = 0;
    decodingTable[1]["CN"] = 1;
    decodingTable[1]["Cx"] = 2;
    decodingTable[1]["Ch"] = 3;
    decodingTable[1]["D3"] = 4;
    decodingTable[1]["DN"] = 5;
    decodingTable[1]["Dx"] = 6;
    decodingTable[1]["Dh"] = 7;
    decodingTable[1]["E3"] = 8;
    decodingTable[1]["EN"] = 9;
 
    decodingTable[2]["n"] = 0;
    decodingTable[2]["j"] = 1;
    decodingTable[2]["f"] = 2;
    decodingTable[2]["b"] = 3;
    decodingTable[2]["D"] = 4;
    decodingTable[2]["z"] = 5;
    decodingTable[2]["v"] = 6;
    decodingTable[2]["r"] = 7;
    decodingTable[2]["T"] = 8;
    decodingTable[2]["P"] = 9;
}
 
private static Dictionary<string, int>[] decodingTable;

See how some tokens are two chars and some are one? We'll run through the tokens testing to see if they are in each dictionary and creating the result. For my Gaiman book, we end up with:

978006093471251300

Now what do we do with it?

UPCs and ISBN

Universal Product Codes or UPCs are different from ISBNs. This page at the Book-Scanning Project gave me the algorithm I needed to convert a UPC code into an ISBN that I could use to talk to Amazon.com. Most books have a bar code with 13 digits starting with "978." Turns out that the first 3 digits of a UPC are the country of origin, but "978" is "Bookland." Apparently the UPC number for a book is called the "Bookland EAN."

Here's the Javascript taken from http://isbn.nu.

if (indexisbn.indexOf("978") == 0) {
   isbn = isbn.substr(3,9);
   var xsum = 0;
   var add = 0;
   var i = 0;
   for (i = 0; i < 9; i++) {
        add = isbn.substr(i,1);
        xsum += (10 - i) * add;
   }
   xsum %= 11;
   xsum = 11 - xsum;
   if (xsum == 10) { xsum = "X"; }
   if (xsum == 11) { xsum = "0"; }
   isbn += xsum;
}

Now, converted into C#

public static string ConvertUPCtoISBN(string code)
{
    if (code.StartsWith("978") == false)
    {
        throw new ArgumentOutOfRangeException("UPCs that might be books have length 13 and start with 978");
    }
    code = code.Substring(3,9);
    int xsum = 0;
    for (int i = 0; i < 9; i++)
    {
        xsum += (10 - i) * int.Parse(code[i].ToString());
    }
 
    xsum %= 11;
    xsum = 11 - xsum;
 
    String val = xsum.ToString();
 
    switch (xsum)
    {
        case 10: val = "X"; break;
        case 11: val = "0"; break;
    }
 
    code += val;
    return code; //Now an ISBN
 }

So, my Gaiman UPC of 978006093471251300 turns into an ISBN of 0060934719. This ISBN can be used to talk to Amazon. Note that the source code to this article includes both C# and VB.

Calling Amazon's REST Web Service

Amazon.com has a rich Web Services platform that includes formal SOAP-based Web Services, but for our project, their HTTP GET-based system will work nicely. I visited the Amazon Web Services Home Page and applied for a free Amazon Web Service Developer Key. You'll need to do the same if you want to get the code to work. Amazon has some create documentation on their site that makes this really easy.

I'll make an HTTP GET call to Amazon like this (broken up for readability):

http://webservices.amazon.com/onca/xml?
Service=AWSECommerceService&
AWSAccessKeyId=YOURAWSKEYHERE&
Operation=ItemLookup&
ItemId=0060934719&
IdType=ASIN

This HTTP GET will result in an XML document as seen in this screenshot.

Ah! Just the kind of information I was looking for.

A CueCatConsole

Before we integrate with Windows Live Writer, let's make a little test console to make sure this all works together. I put the CueCatDecoder into a small reusable assembly and called it from the Console Application. I also pull the Amazon Web Services key from the configuration file. Note that VS.NET 2005 creates a strongly typed accessor for the "AWSAccessKeyId" automatically.

I've removed the error handling in this listing for clarity.

CueCatDecoder.Product product = null;
product = CueCatDecoder.Decoder.Decode(input);
Console.WriteLine("Type: {0}.", product.Type.ToString());
string amazonType = "ASIN";
string amazonUrl = String.Format("http://webservices.amazon.com/onca/xml
      ?Service=AWSECommerceService&AWSAccessKeyId={0}
      &Operation=ItemLookup&ItemId={1}
      &IdType={2}
      &SearchIndex=",
            Properties.Settings.Default.AWSAccessKeyId,
            product.ID,
            amazonType);
 
System.Net.WebClient web = new WebClient();
string amazonData = web.DownloadString(amazonUrl);
XmlDocument xmlDocument = new XmlDocument();
XmlNamespaceManager xnm = new XmlNamespaceManager(xmlDocument.NameTable);
xnm.AddNamespace("def", "http://webservices.amazon.com/AWSECommerceService/2005-10-05");
xmlDocument.Load(new System.IO.StringReader(amazonData));
XmlNode itemNode = xmlDocument.SelectSingleNode("/def:ItemLookupResponse/def:Items/def:Item", xnm);
Console.WriteLine(String.Format("FOUND a {0} called {1} by {2}", 
        itemNode.SelectSingleNode("//def:ProductGroup", xnm).InnerText,
        itemNode.SelectSingleNode("//def:Title", xnm).InnerText,
        itemNode.SelectSingleNode("//def:Author", xnm).InnerText));

Notice that since the Amazon XML includes a default namespace, we have to let SelectSingleNode know about that namespace by registering a namespace prefix - in this case "def:" - that we use later in our XPath. There are many ways to query XML within the .NET Base Class Library that are faster, but for a small application like this the XmlDocument is a reasonable choice when performance doesn't matter.

Here's the output from my CueCatConsole:

Coding4Fun: CueCatDecoder by Scott Hanselman @ http://www.hanselman.com
Scan item with CueCat now or press Q then Enter to exit
.C3nZC3nZC3nZE3r0Chr3CNnY.cGen.ENr7C3T7D3f0C3vYDW.
Type: ISBN.
FOUND a Book called The Goal: A Process of Ongoing Improvement by Eliyahu M. Goldratt
Scan item with CueCat now or press Q then Enter to exit
.C3nZC3nZC3nZE3r0Chr3CNnY.cGf2.ENr7CNz6C3z6DxvXChzXD3P6.
Type: ISBN.
FOUND a Book called Programming Sudoku (Technology in Action) by Wei-Meng Lee
Scan item with CueCat now or press Q then Enter to exit
.C3nZC3nZC3nZE3r0Chr3CNnY.cGf2.ENr7C3n1C3PWD3rYCxzYChnZ.
Type: ISBN.
FOUND a Book called Stardust by Neil Gaiman
Scan item with CueCat now or press Q then Enter to exit

Fantastic. It works. Let's integrate it with Windows Live Writer, the new Offline Blogging Editor in BETA (at the time of this writing) from http://www.live.com.

Integrating with Windows Live Writer

Windows Live Writer (download) is a new desktop application that folks with blogs can use to write their blog posts offline. I use DasBlog for my blog at http://www.hanselman.com, and since DasBlog supports the standard MetaWebLog API also supported by Live Writer, I was in business right away.

Windows Live Writer also has an SDK that supports "Content Source Plugins" to extend the capabilities of the application and let folks insert new kinds of data. That's exactly what I want to do! I'd like to click "Blog about Book" and have the book title and author appear linked with an image of the book cover. I'd also like have the Amazon.com URL to include my Amazon Associates ID so I could make a few cents if someone clicks my link and buys the book.

Writing a plugin for Windows Live Writer couldn't be easier. The SDK includes step-by-step instructions. I created a class that derived from WindowsLive.Writer.Api.ContentSource and overrode the CreateContent method, returning my new snippet of HTML in the newContent string variable.

Here's the gist of the source, again with error handling removed for clarity. VB is included in the download.

using System;
using System.Collections.Generic;
using System.Windows.Forms;
using WindowsLive.Writer.Api;
using CueCatDecoder;
 
namespace AmazonLiveWriterPlugin
{
    [WriterPlugin("605EEA63-B74B-4e9d-A290-F5E9E8229FC1", "Amazon Links with CueCat",
        ImagePath = "Images.CueCat.png",
        PublisherUrl = "http://www.hanselman.com",
        Description = "Amazon Links with a CueCat.")]
    [InsertableContentSource("Amazon Links")]
    public class Plugin : ContentSource
    {
        const string AMAZONASSOCIATESID = "amazonassociatesid";
        const string AMAZONWEBSERVICESID = "amazonwebservicesid";
 
        public override System.Windows.Forms.DialogResult CreateContent(System.Windows.Forms.IWin32Window dialogOwner, ref string newContent)
        {
            using(InsertForm form = new InsertForm())
            {
                form.AmazonAssociatesID = this.Options[AMAZONASSOCIATESID];
                form.AmazonWebServicesID = this.Options[AMAZONWEBSERVICESID];
                DialogResult result = form.ShowDialog();
                if (result == DialogResult.OK)
                {
                    this.Options[AMAZONASSOCIATESID] = form.AmazonAssociatesID;
                    this.Options[AMAZONWEBSERVICESID] = form.AmazonWebServicesID;
                    Product p = Decoder.Decode(form.CueCatData);
                    AmazonBook book = AmazonBookPopulator.CreateAmazonProduct(p, form.AmazonWebServicesID);
                    string associatesId = form.AmazonAssociatesID.Trim();
                    string builtAmazonUrl = "<a href=\"http://www.amazon.com/exec/obidos/redirect?link_code=as2&path=ASIN/{0}&tag={1}&camp=1789&creative=9325\">{2} by {3}</a><a href=\"http://www.amazon.com/exec/obidos/redirect?link_code=as2&path=ASIN/{0}&tag={1}&camp=1789&creative=9325\"><img border=\"0\" src=\"http://images.amazon.com/images/P/{0}.01._AA_SCMZZZZZZZ_.jpg\"></a><img src=\"http://www.assoc-amazon.com/e/ir?t={1}&l=as2&o=1&a={0}\" width=\"1\" height=\"1\" border=\"0\" alt=\"\" style=\"border:none !important; margin:0px !important;\" />";
                    newContent = string.Format(builtAmazonUrl, book.ID, associatesId, book.Title, book.Author);
                }
                return result;
            }
        }
    }
}

Notice the crazy Amazon.com URL. Most folks with blogs become Amazon Associates to make a little money to pay for hosting (you won't get rich doing this, believe me). The Amazon Associates site includes link builders that create these complex snippets of HTML. I selected the large image, basic HTML option, then inserted String.Format tokens like {0} in the appropriate places.

Installation is simple, just copy Coding4Fun.AmazonLiveWriterPlugin.dll and Coding4Fun.CueCatDecoder.dll to C:\Program Files\Windows Live Writer\Plugins and they will be automatically detected.

Our plugin appears automatically in the Windows Live Writer sidebar. Notice the text and the icon was picked up from the Plugin class attributes. My Amazon Web Services ID and Associates ID are both stored internally by an Options bag managed by Windows Live Writer, making management of preferences very easy for the plugin developer. Once the CueCat data is scanned, we decode it and make the HTTP GET Web Services call using our external CueCat assembly created earlier.

Conclusion

There's things that could be extended, added, and improved on with this project. Here are some ideas to get you started:

Include support for CDs and other products available on Amazon.
Add support for more web-based databases with UPC codes.
Support different Amazon link styles.
Create a system to print out product tags for a Garage Sale.

Have fun and have no fear when faced with the words - Some Assembly Required!

Don't touch me! - Interfacing with a Fingerprint Reader

Coding4Fun — Tue, 06 Mar 2007 11:07:55 GMT

	In this installment of the "Some Assembly Required" column, Scott Hanselman creates a Family Fingerprint Manager using .NET 2.0 that interfaces with the Microsoft Fingerprint Reader and the GrFinger SDK from Griaule.
Scott Hanselman Scott's Blog Difficulty: Intermediate Time Required: 1-3 hours Cost: $50-$100 Software: GrFinger Hardware: Microsoft FingerPrint Reader Download: Download

Scott Hanselman

Summary: In this installment of the "Some Assembly Required" column, Scott Hanselman creates a Family Fingerprint Manager using .NET 2.0 that interfaces with the Microsoft Fingerprint Reader and the GrFinger SDK from Griaule.

The Microsoft Fingerprint Reader

I love my Microsoft Fingerprint Reader and when I got it I knew I had to ~~hack it~~ write some software for it. Since there's already a lot of great security software out there like the Digital Persona software that Microsoft shipped with it as well as 3rd party software like GrFinger's Desktop Identity, I wanted to write something a little different.

My wife and I made a New Year's Resolution to be more prepared for emergencies and the like in the wake of Hurricane Katrina. I figured, since we've moved all our identification and important papers into the a safety deposit box, why not a copy of our fingerprints as well?

There are do-it-yourself-at-home fingerprinting kits one can buy, but I had this nice Fingerprint Reader lying around....:)

Interfacing with the Reader

Griaule Software out of Brazil has a nice, clean COM API SDK that abstracts away the fingerprint hardware supporting a number of readers including Microsoft's. You can download a trial of their SDK software and use it for 90 days. They also include an open-source driver for the Microsoft Fingerprint Reader so you don't have to use the Digital Persona standard stuff if you don't want to. They've got an excellent sample code section with details on how to interface with a read using VB.NET, C#, Java, VB6, even Excel.

I started by adding references to their COM APIs and generating an interop assembly as seen in the picture at left. The AxGrFinger control library includes a small control that you can drag on to your WinForms design surface that exposes a number of Fingerprint-related events like FingerDown and ImageRequired that make interfacing with the hardware incredibly easy.

I created a PictureBox on my WinForm that would display the fingerprint being acquired along with ten other PictureBoxes, one for each finger. The large picture box will appear only when an image is being acquired then disappear after we copy the large acquired image into one of the smaller "fingertip" images.

The large PictureBox is passed in to a utility class along with a reference to the AxGrFinger control. As the image is acquired a valid "Handle to a Device Context" or HDC (remember those?) is passed into the CapRawImageToHandle method on the AxGrFinger control. When that handle comes back, we just update the large picture box and it displays the newly received image. Once the image has move into the world of WinForms, it's considerably easier to manipulate using the System.Drawing APIs.

    ' Display fingerprint image on screen
    Public Sub PrintBiometricDisplay()
        ' handle to finger image
        Dim handle As System.Drawing.Image = Nothing
        ' screen HDC
        Dim hdc As Integer = GetDC(0)
 
        ' get raw image
        _GrFingerX.CapRawImageToHandle(raw.img, raw.width, raw.height, hdc, handle)
 
        ' draw image on picture box
        If Not (handle Is Nothing) Then
            _pbPic.Image = handle
            _pbPic.Update()
        End If
        ' release screen HDC
        ReleaseDC(0, hdc)
    End Sub

The images from the large picture box are of very high quality and are copied into the smaller fingers one at a time.

Loading and Saving the Fingerprints

There's ten fingerprints (assuming you have ten fingers) to save, but should we save them as one file or ten? I decided that folks would probably want one file per family member using File|Open and File|Save method, but they'd want to reserve the right to export all the fingers as individual (PNG) images.

I created a very simple in-memory representation of a Fingerprint Database like this. It's actually slightly more complex with custom constructors as such in the actual code, but this is the general idea. I'm going to be storing the high-quality image as a byte array rather than a System.Drawing.Bitmap or Image because a byte array is very easily serialized as XML.

    Public Class Fingerprints
        Public FingerImages(10) As FingerImage
        Public Sub New()
            For i As Integer = 0 To 10
                FingerImages(i) = New FingerImage()
            Next i
        End Sub
 
        Public Sub AddFingerImage(ByVal finger As String, ByVal image() As Byte)
            Dim i As Int32 = Int32.Parse(finger)
            FingerImages(i - 1) = New FingerImage(image, finger)
        End Sub
 
        Public Function GetImageFromFinger(ByVal finger As String) As Byte()
            For Each fi As FingerImage In Me.FingerImages
                If (fi.Finger = finger) Then
                    Return fi.Image
                End If
            Next
            Return Nothing
        End Function
 
    End Class
 
    Public Class FingerImage
        Public _image() As Byte
        Public _finger As String
    End Class

Each per-person finger database, like scott.finger for example, will be saved as XML similar to the example below. The byte[] is automatically turned into a BASE64'ed string by the XmlSerializer. This might be considered inelegant by some, but disk space is cheap, and the code was fantastically simple. An easy enhancement would ZIP up the file, but the savings would only be about 30-40%.

<?xml version="1.0"?>
<Fingerprints xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<FingerImages>
    <FingerImage>
      <Image>iVBOR...big opaque BASE64'ed image</Image>
      <Finger>L1</Finger>
    </FingerImage>
</FingerPrints>

Loading the fingers back into the Form is the exact opposite of saving. We use a standard Windows File Dialog with the filter set to show only .finger files. The file is deserialized using the XmlSerializer and the images are loaded back into the appropriate PictureBox.

   Private Sub MenuLoad_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MenuLoad.Click
        Dim OpenFileDialog1 As New OpenFileDialog
 
        '' open "load" dialog
        OpenFileDialog1.Filter = "FINGER files (*.finger)|*.finger|All files (*.*)|*.*"
        OpenFileDialog1.FilterIndex = 1
        OpenFileDialog1.RestoreDirectory = True
        OpenFileDialog1.CheckFileExists = True
 
        '' load image
        If OpenFileDialog1.ShowDialog() = Windows.Forms.DialogResult.OK Then
 
            Dim x As New XmlSerializer(GetType(Fingerprints))
            Dim newdb As Fingerprints
            Using fs As FileStream = File.OpenRead(OpenFileDialog1.FileName)
                newdb = CType(x.Deserialize(fs), Fingerprints)
            End Using
 
            For Each item As Control In Me.Controls
                If IsFingerprintPictureBox(item) Then
                    Dim pic As PictureBox = CType(item, PictureBox)
                    Dim b As Byte() = newdb.GetImageFromFinger(pic.Tag)
                    If (Not b Is Nothing) Then
                        pic.Image = Image.FromStream(New MemoryStream(b))
                    End If
                End If
            Next
        End If
    End Sub

I used the Tag Property of the PictureBox to store the name of the finger. The name is also stored in the .finger file and used to correlate the fingers and PictureBoxes.

The export works similarly, except I save the contents of each PictureBox to the selected directory using the given name as a basefilename. For example, if the user said they wanted to export to Foo.png, I'd create 10 files with names like Foo.l1.png, Foo.r2.png, etc. You can see how easy it is to change image formats using System.Drawing.Image.

            For Each item As Control In Me.Controls
                If IsFingerprintPictureBox(item) Then
                    Dim pic As PictureBox = CType(item, PictureBox)
                    If Not pic.Image Is Nothing Then
                        Dim filename As String = Path.Combine(Path.GetDirectoryName(ExportFileDialog1.FileName), (Path.GetFileNameWithoutExtension(ExportFileDialog1.FileName) & "." & pic.Name & ".png"))
                        Using m As New MemoryStream
                            pic.Image.Save(filename, Imaging.ImageFormat.Png)
                        End Using
                    End If
                End If
            Next

To get this sample to work, you'll need a Microsoft Fingerprint Reader (or other compatible reader) and the Griaule SDK installed. I recommend you uninstall the Digital Persona drivers that came with the Fingerprint Reader otherwise this application may compete for the same hardware if the Digital Persona software is running in the background.

Conclusion

There's a number of fun things that could be extended, added, and improved on with this project. Here are some ideas to get you started:

Adding a PDF export feature using an open-source .NET PDF Library
...or export to Word using Office Automation and the Microsoft Office Interop Assemblies.
Add security by using DPAPI or encrypt the Fingerprint files directly.
Add compression by zipping the .finger files up programmatically.

Have fun and have no fear when faced with the words - Some Assembly Required!

If you do extend this application, be sure to release the source. Thanks again to Griaule for the use of their SDK.

Part 2: Controlling a Microbric Viper Robot with an IR Serial Port using .NET and PowerShell

Coding4Fun — Tue, 20 Feb 2007 04:17:40 GMT

	In this Part Two of "Some Assembly Required", Scott Hanselman extends his original Microbric Viper Robot/Iguanaworks IR solution to script-enable control of the robot with Lee Holme's "PowerShell LOGO." Background
Scott Hanselman Scott's Blog Difficulty: Advanced Time Required: 3-6 hours Cost: $100-$200 Software: Visual Basic or Visual C# Express Editions, Windows PowerShell Hardware: MicroBric Viper, Iguanaworks IR Serial Port Download: C# Download

Summary: In this Part Two of "Some Assembly Required", Scott Hanselman extends his original Microbric Viper Robot/Iguanaworks IR solution to script-enable control of the robot with Lee Holme's "PowerShell LOGO."

Background

The Microbric Viper is a robot construction kit based on a new solderless construction technique. The Viper uses a Basic Atom microcontroller that you program, shockingly, using BASIC. Modules like an Infrared Receiver, motors and switches can be fairly easily controlled as you just screw them directly into the mainboard and address them by number. Once I got my hands on a Microbric, I started trying to talk to the robot using my laptop's Infrared Port. It turns out after a number of failed attempts that Windows doesn't expose standard laptop IR ports as Serial Ports or in any way other than via the IRDA protocol. IRDA was not only overkill for this kind of communication but also a hassle to program to. What I really needed was an IR transmitter that I could talk to using an interface that was clear and clean like the Serial Port interface exposed by System.IO.Ports.

I approached the guys at Iguanaworks as they make an infrared transceiver that is addressable via a standard Serial Port. The Viper comes with a Sony Remote Control so I set off to figure out how to get the Iguanaworks IR transmitter to speak to the Viper using .NET.

There's Video of the whole thing working together posted in the Channel 9 Forums Screencast topic as seen in the screenshot above.

Remote Scripting the Microbric Viper Robot

I used the custom IR Serial Port and created a console application that let me control the Microbric Viper using the keyboard of my computer. Now I'll create an application that'll let me control the Viper with script. What better script to use than Windows PowerShell?

Rather than just writing some PowerShell script that sends IR commands, I thought it'd be more interesting to host PowerShell within a WinForms application and write script that controls both the Viper and an an on screen "turtle," just like the LOGO Programming Language you may have learned in school.

Hosting Windows PowerShell

Hosting PowerShell within your own WinForms application is amazingly easy. Almost embarrassingly easy. Since I wanted a LOGO-like experience, I started by calling Lee Holmes who has a deeply technical blog dedicated to PowerShell. He happily let me extend his existing application to include support for the Microbric Viper.

Lee's application is a simple WinForms project with an Image control and a textbox. There's a .NET object called turtle that you can think of as the "pen." The turtle has methods like Forward, Left, Right, etc.

The turtle can be controlled on the Actions tab via a series of links. Moving the turtle forward 10 pixels is as trivial as:

turtle.Forward(10);

Controlling the turtle via the UI is simple enough, but scripting the turtle is the interesting part.

It's very significant to note that PowerShell is much more than a shell. It's not a shell at all at its heart - It's a UI-non-specific "runspace" for executing commands. PowerShell doesn't need to be seen in order to do its work. You can host PowerShell in any of your applications and instantly script enable them.

The first step is adding a .NET Assembly reference to System.Management.Automation. Within this assembly we'll be focused on the System.Management.Automation.Runspaces namespace.

We create a Runspace where our script will run like this:

runspace = RunspaceFactory.CreateRunspace();
runspace.Open();

Runspaces host Pipelines of commands strung together. We can create a pipeline within our Runspace and pass in the contents of the scriptText textbox from the main WinForm.

Pipeline pipeline = runspace.CreatePipeline(scriptText.Text);

The pipeline can be executed with

pipeline.Invoke()

But we need to let PowerShell have access to an instance of our Turtle class so that it can call methods like turtle.Forward(). You can pass any object from your application into the PowerShell runspace like this:

   1:  runspace = RunspaceFactory.CreateRunspace();

   2:  runspace.Open();

   3:  runspace.SessionStateProxy.SetVariable("turtle", turtle);

   4:  Pipeline pipeline = runspace.CreatePipeline(scriptText.Text);

   5:  pipeline.Invoke();

   6:  runspace.Close();

In this snippet an instance of the turtle class is passed into the SetVariable method along with the string "turtle." The string is the name of the PowerShell variable you want. If we'd passed in "foo",turtle then there'd be an instance of a turtle in a PowerShell variable accessible as "$foo."

Now that I've got access to the turtle from PowerShell, I could draw a square in PowerShell script however I like:

   1:  $turtle.Reset()

2:

   3:  $a = 30

   4:  $turtle.Forward($a)

   5:  $turtle.Right(90)

6:

   7:  $turtle.Forward($a)

   8:  $turtle.Right(90)

9:

  10:  $turtle.Forward($a)

  11:  $turtle.Right(90)

12:

  13:  $turtle.Forward($a)

  14:  $turtle.Right(90)

At this point, I've got full control of the turtle, so I can write whatever script I like. If you're going to host PowerShell in your own application be sure to consider the security ramifications. Someone could certainly include "dir" or "format c:" in the script, so you might consider a whitelist of allowed commands.

Extending the Turtle

In the current version on Lee's website the turtle doesn't update as he draws. Instead the image is updated after the turtle has finished. Changing this won't make our robot integration work any better, but it would certainly be nice to see the robot move in the real world as it draws on screen.

To solve this we need to understand the relationship between our application and the PowerShell runspace we're hosting. When we pass the turtle instance in PowerShell, that instance is being accessed on another thread, in the "PowerShell world."

Since the PowerShell script is going to be accessing the turtle instance, and the turtle instance will be drawing to a Canvas object that we passed in, we'll need a way to refresh the WinForm's image. First we need to invoke the PowerShell pipeline asynchronously. The pipeline will fire off and immediately return control to our application. It will continue executing, accessing the turtle, while a Timer control refreshes the Image control on the WinForm every 1/4 second or so.

The turtle class is changed to a ViperTurtle class and we'll add the IR class from the last article. When the turtle moves, we'll not only draw to the canvas but also send Viper-specific IR codes to move the robot.

Problems with Timing

I had a few timing-related problem with this project.

First, since the IR transmitter is going as fast as I can send signals to it, I had to add a "WaitASec" method that sleeps for a a third of a second. The commands were coming so fast that the robot would get "stuck" as if a real person was pressing buttons too fast on a remote control. If a Forward command is followed by a Left command too quickly, the robot thinks that the Forward command is continuing...as if the button were being held down. If we wait a fraction of a second for the IR beam to shut off, it works much better.

The second issue was that the commands the Viper understands are relative - like "turn left." The longer we "hold down," or continue to send, a command, the longer the robot turns. However, the LOGO-like PowerShell script thinks in absolutes, like "left(90)" where 90 means ninety degrees exactly. If the robot skids or slides on the floor, it can also throw the turns off. I could, and can still, deal with this in a number of ways. First, I could create new IR commands for the Viper to respond to and program its firmware with these custom commands, like "TurnLeft5Degrees" or even at 1 or 2 degree increments. Then I'd need to calculate the circumference of the wheels on the robot and do the math backwards in order to find the correct number of milliseconds to pulse the Viper's onboard motors to turn those wheels. Whew!

Or, I could just build in a "fudge factor" and get it working by trial and error. This is a classic software engineering problem, isn't it? The amount of effort required to get 100% perfect turning weighed against how long it would take to get it working 85-90%.

I came up with these two constant values that worked pretty well for me. You might need to try different values based on the kind of surface you're using, or if you've built your own IR transmitter.

   1:  const double VIPERADJUST = 0.35;

   2:  const double VIPERADJUSTTURN = 0.06;

3:

   4:  public void Left(int degrees)

   5:  {

   6:      direction = (direction + degrees) % 360;

   7:      WaitASec();

   8:      ir.Send(LEFT, (int)(VIPERADJUSTTURN * degrees));

   9:  }

The IR's send method takes a Repeat value as its second parameter. This indicates how many times the IR command should repeat. I found that multiplying the number of degrees I wanted the robot to turn by a value of 0.06 would get me the correct turn. However, I can't send fractional packets, so I'm forced to lose precision as I cast the result of the double calculation down to an int. So a 90 degree turn becomes 5.4, which becomes the left command sent via IR 5 times. In practice it works pretty well, though. If I were to hook a pen up to the Viper in the future and create PenUp and PenDown commands in its firmware, I'd want to get tighter, more accurate turning.

Again, here's Video of the whole thing working together posted in the Channel 9 Forums Screencast topic as seen in the screenshot above.

Purchasing these Parts

The Microbric Viper can be ordered online in North America, check out www.microbric.com for North American distributors. It's only US$89 at Saelig and CAD$99 at RobotShop. They have a number of educational robots that can be assembled by kids of all ages and skill levels. They're great for the classroom, and include projects like Sumo Robots, and a line-following bot, as well as a Spiderbot that climbs rope - all from the same kit.

You can order the IR Transmitter/Receiver from IguanaWorks. The serial version works on Windows or Linux, and there's a Linux USB version. It's not just a Transmitter, but also a learning receiver that works with WinLIRC and turns your computer into a learning remote control and can be used for nearly any project that utilizes IR.

Conclusion

As with all my projects, there's things that could be extended, added, and improved on with this project. Here are some ideas to get you started.

Extend the IR program running inside the Viper to support other commands.
Integrate the Viper with the Microsoft Robotics Toolkit.
Use the IR class to control your home theater and television.
Change this program to control a Roomba, or any IR-controllable robot.

Have fun and have no fear when faced with the words - Some Assembly Required!

Controlling a Microbric Viper Robot with a custom IR Serial Port using .NET

Coding4Fun — Sat, 02 Dec 2006 10:10:00 GMT

Summary: In this installment of "Some Assembly Required", Scott Hanselman teams up with the makers of the Microbric Viper Robot and the guys at Iguanaworks to create a custom IR serial port board to control the Viper Robot using .NET!

Introduction

Programming the Microbric Viper Robot

The IR receiver module is screwed into the Microbric Viper, in my case, into the addressable P14 port on the side of the main body. The Viper includes a main chip called the Basic Atom that is flashed using BASIC programs written in an included BASIC IDE. In my BASIC file a constant is used to address the IR "Data Pin," in this case, P14.

The Basic Atom includes a number of BASIC helper methods that the microprocessor uses to talk to the external world. The "pulsin" method is used to talk to the IR Data Pin and detect a pulse. Infrared receivers use high-speed pulses at a certain frequency (how fast it flashes) and duty cycle (a percentage of how long the LED is on versus off). These pulses of light continue for a set amount of time to indicate ones, zeros or the header.

How the Sony Infrared Protocol works

I'm trying to pulse the IR LED to emulate the Sony Remote Control, and started by looking for some information on how the Sony Infrared Protocol (SIRC) works. I found two excellent writeups.

Excellent explanation about how IR works.
Details of the SIRC (Sony Infrared Control) Protocol.

IR Protocols are fairly sensitive to timing. The pulses need to happen at specific intervals with the correct frequency. Because the timing is so fine-grained, I'm using the new System.Diagnostic.Stopwatch class in .NET (a wrapper for the Win32 QueryPerformanceCounter API) that uses Ticks for its unit of measurement. The Frequency is 2992540000 ticks per second, so I figure that'd be enough resolution.

The Sony remote I'm trying to emulate uses a 40kHz frequency, so it wants to flash the LED one cycle, once, every 1/40000 of a second. That means every 74814 ticks or every 25µs (microseconds are 1/1000000 of a second.)

I'm trying to send a header pulse of 2.4ms in length and I need to cycle the LED once every 25µs. I turn it on for 8µs and turn if off for 17µs. That means it will cycle 96 (2400µs) times for the header, 24 (1200µs) times for a space or zero, and 48 (600µs)times for a one. An image from San Bergmans illustrates:

The Iguanaworks IR serial port board uses DTR (Data Terminal Ready) to turn on the IR LED. When DTR is high, the LED lights up, when it's off, the LED turns off. Using the Stopwatch and some really tight loops I figure I can flash (pulse) the LED fast enough.

Now at this point, you may, dear reader, have already had the thought that perhaps trying to flash the LED via software, rather than hardware, is an astonishingly bad idea. Well, I could have used your insight earlier, my friend. But, on with my tale, as it is the telling that is so enjoyable, right?.

We'll take a look at more details of the internal implementation of the Sony Protocol in a moment.

INTERESTING REMINDER: Remember, you can't see IR (it's infrared, therefore not in our visible spectrum) but you can see it if you point it at a Webcam or digital camera, which is what I've been doing to sanity check the hardware. The picture at left is an image of the LED pointed at my Logitech Webcam.

Infrared Serial Port via Software

Of course, I've started with managed code, because I'm a managed kind of a guy. I started using System.IO.Ports to address the Iguanaworks IR transmitter that is connected to a serial point on my PC. They make both a Serial Port and a USB version, but there are not yet drivers for the USB version so I got a three-foot long serial extension cord and went to work.

ASIDE: Where does the Power come from?

The Iguanaworks IR is a uniquely high-power transmitter that charges up a capacitor in order to provide a range of up to 10-meters. Your mileage may vary. However, it requires a few minutes to charge up the capacitor. Once it is charged up, however, even if you are using it constantly, it'll find a comfortable middle place where the output matches the input. If you use it intermittently, which is more typical, you'll likely get very good range and bright output. In my initial testing, though, while I had no trouble getting output from it using Winlirc (the only officially supported Open Source software package for this transmitter) but when I used my application, the transmitter would peter out and eventually go dim. What the heck was going on?

I fought with it for a while, then decided to RTFS (where "S" is "Schematic). The board layout is here. Notice that the RTS (Serial Port Ready-To-Send) Pin 7 goes straight to VoltageIn. Duh! <slaps forehead>. They are sipping power off the Ready To Send pin and I'm not setting that pin hot via RtsEnable.

   1:  port = new SerialPort(portString);

   2:  port.RtsEnable = true; //needed for power!

   3:  port.BaudRate = 115200;

   4:  port.StopBits = StopBits.One;

   5:  port.Parity = Parity.None;

   6:  port.DataBits = 7;

   7:  port.Handshake = Handshake.None;

So, if you ever find yourself using the High-Power LIRC Transmitter/Receiver in an unsupported way writing your own program, remember to set RTS high or you won't get any power. Heh!

Back to the narrative. I suspected I might be writing more than one serial port class, or possibly using different pieces of hardware, so I defined an IIRSerialPort interface with some basics like On, Open, Off, etc. Since the Iguanaworks IR uses DTR to turn on the LED, it was as easy as setting the port's DtrEnable property to true to get a solid LED output.

Here's part of the first Managed IR Serial Port class.

   1:  public class ManagedIRSerialPort : IIRSerialPort

   2:  {

   3:    SerialPort port = null;

4:

   5:    public ManagedIRSerialPort(string portString)

   6:    {

   7:     port = new SerialPort(portString);

   8:     port.RtsEnable = true; //needed for power!

   9:     port.BaudRate = 115200;

  10:     port.StopBits = StopBits.One;

  11:     port.Parity = Parity.None;

  12:     port.DataBits = 7;

  13:     port.Handshake = Handshake.None;

  14:    }

15:

  16:   public void Open()

  17:   {

  18:     port.Open();

  19:   }

20:

  21:   public void On()

  22:   {

  23:     port.DtrEnable = true;

  24:   }

25:

  26:   public void Off()

  27:   {

  28:     port.DtrEnable = false;

  29:   }

30:

  31:   public void Close()

  32:   {

  33:    port.Close();

  34:   }

  35:  }

This class is just the beginning, as it only turns the LED on and off. Remember I need to "pulse" the LED, turning it on and off 96 times all in the space of 2400µs. I wrote a SendPulse that spun very tightly and used the StopWatch class to manage timing.

   1:  public unsafe void SendPulse(long microSecs)

   2:  {

   3:    long end = LastTimeInTicks + (microSecs * (STOPWATCHFREQ / MILLION));

   4:    int i = 0;

   5:    timer.Reset();

   6:    timer.Start();

   7:    do

   8:    {

   9:      i++;

  10:      port.On();

  11:      MicroWait(pulseWidth);

  12:      port.Off();

  13:      MicroWait(spaceWidth);

  14:    }

  15:    while (LastTimeInTicks < end);

  16:    timer.Stop();

  17:    System.Diagnostics.Debug.WriteLine(i.ToString() + " pulses in " + timer.ElapsedTicks/(double)  req*MILLION+"us");

  18:  }

Even with lots of optimization, I just couldn't get it to cycle fast enough. Remember, I need the header to take 2400µs total. In this screenshot, you can see it's taking an average of 30000µs! That sucks - it's 10 times slower than I need it to be.

So I futzed with this for a while, and then Reflector'd around. I noticed the implementation of set_dtrEnable inside of System.IO.Ports.SerialStream was WAY more complicated than it needed to be for my purposes.

   1:  //Reflector'd Microsoft code from inside System.IO.Ports.Port

   2:  internal bool DtrEnable

   3:  {

   4:   get

   5:   {

   6:     int num1 = this.GetDcbFlag(4);

   7:     return (num1 == 1);

   8:   }

   9:   set

  10:   {

  11:    int num1 = this.GetDcbFlag(4);

  12:    this.SetDcbFlag(4, value ? 1 : 0);

  13:    if (!UnsafeNativeMethods.SetCommState(this._handle, ref this.dcb))

  14:    {

  15:      this.SetDcbFlag(4, num1);

  16:      InternalResources.WinIOError();

  17:    }

  18:    if (!UnsafeNativeMethods.EscapeCommFunction(this._handle, value ? 5 : 6))

  19:    {

  20:      InternalResources.WinIOError();

  21:    }

  22:   }

  23:  }

All I figured I needed to do was call the Win32 API EscapeCommFunction to set the DTR pin high. One thing I learned quickly was that calling EscapeCommFunction was 4 times faster than calling SetCommState for the purposes of raising DTR, so in this code sample those lines are commented out.

I then wrote another implementation of IIRSerialPort, still in managed code, but this one talking to the COM Port using the underlying Win32 APIs.

   1:  public class UnmanagedIRSerialPort : IIRSerialPort

   2:  {

   3:   IntPtr portHandle;

   4:   DCB dcb = new DCB();

   5:   string port = String.Empty;

   6:   public UnmanagedIRSerialPort(string portString)

   7:   {

   8:   port = portString;

   9:   }

10:

  11:   public void Open()

  12:   {

  13:    portHandle = CreateFile("COM1",

  14:    EFileAccess.GenericRead | EFileAccess.GenericWrite,

  15:    EFileShare.None,

  16:    IntPtr.Zero,

  17:    ECreationDisposition.OpenExisting,

  18:    EFileAttributes.Overlapped, IntPtr.Zero);

  19:    GetCommState(portHandle, ref dcb);

  20:    dcb.RtsControl = RtsControl.Enable;

  21:    dcb.DtrControl = DtrControl.Disable;

  22:    dcb.BaudRate = 115200;

  23:    SetCommState(portHandle, ref dcb);

  24:   }

25:

  26:   public void On()

  27:   {

  28:   EscapeCommFunction(portHandle, SETDTR);

  29:   //dcb.DtrControl = DtrControl.Enable;

  30:   //SetCommState(portHandle, ref dcb);

  31:   }

32:

  33:   public void Off()

  34:   {

  35:   EscapeCommFunction(portHandle, CLRDTR);

  36:   //dcb.DtrControl = DtrControl.Disable;

  37:   //SetCommState(portHandle, ref dcb);

  38:   }

39:

  40:   public void Close()

  41:   {

  42:   CloseHandle(portHandle);

  43:   }

44:

  45:   #region Interop Serial Port Stuff

  46:   [DllImport("kernel32.dll")]

  47:   static extern bool GetCommState(IntPtr hFile, ref DCB lpDCB);

48:

  49:   [DllImport("kernel32.dll")]

  50:   static extern bool SetCommState(IntPtr hFile, [In] ref DCB lpDCB);

51:

  52:   [DllImport("kernel32.dll", SetLastError = true)]

  53:   public static extern bool CloseHandle(IntPtr handle);

54:

  55:   [DllImport("kernel32.dll", SetLastError = true)]

  56:   static extern bool EscapeCommFunction(IntPtr hFile, int dwFunc);

  57:  //Snipped so you don't go blind...

  58:  #endregion

  59:  }

Here's the NOT COMPLETE (Remember, it just does DTR) Unmanaged Serial Port class. Thanks PInvoke.NET for the structures to kick start this)!:
File Attachment: UnmanagedIRSerialPort.cs (10 KB)

As you can see I've got it abstracted away with a common interface so I can switch between managed serial and unmanaged serial quickly. I ran the same tests again, this time with MY serial port stuff:

Sweet, almost 10x faster and darn near where I need it to be. However, it's not consistent enough. I need numbers like 2400, 600, 1200. I'm having to boost the process and thread priority just to get here...

   1:  previousThreadPriority = System.Threading.Thread.CurrentThread.Priority;

   2:  System.Threading.Thread.CurrentThread.Priority = System.Threading.ThreadPriority.Highest;

   3:  System.Diagnostics.Process.GetCurrentProcess().PriorityClass = ProcessPriorityClass.RealTime;

...and undo it with...

   1:  System.Threading.Thread.CurrentThread.Priority = previousThreadPriority;

   2:  System.Diagnostics.Process.GetCurrentProcess().PriorityClass = ProcessPriorityClass.Normal;

...and that's just naughty and not getting the job done.

At this point, it's close, but I'm wondering if it's even possible to flash this thing fast enough. I'm at the limit of my understanding of serial ports (Is DTR affected by Baud Rate? Is 115200 the fastest? Would this be faster in C++ (probably not), or is there a faster way to PInvoke?)

More Details of the SIRC Protocol

The Sony IR Protocol uses a header that consists of a pulse 2400µs long, followed by 12 ones or zeros (twelve bits) where a "1" is a pulse 1.2ms long followed by an off state for 600µs long and a "0" is a pulse 600µs long followed by an off state for 600µs long. The whole packet is finalized with an off state 600µs long.

According to the SIRC spec, these are the "1s and 0s" we will be using to control the robot. They are the buttons used for the cursor on Sony remotes. This next code snippet is from the BASIC program that controls the robot, so this is what the receiver on the robot needs to see to take action.

   1:  TestIRData:

   2:  High LED_Pin ;Set LED_Pin high

   3:  if irdata = %0000100100000000 then forward

   4:  if irdata = %1000100100000000 then backward

   5:  if irdata = %0100100100000000 then twistright

   6:  if irdata = %1100100100000000 then twistleft

   7:  if irdata = %0010100100000000 then still

   8:  return

The protocol indicates that it wants the Most Significant Bit on the RIGHT, which is the opposite of how it's done on Intel chips. Note that this doesn't refer to "Endianness," which is byte-by-byte. We're talking about reversing or flipping the entire bit stream.

Here's the constants in C# that correspond (in reverse) to the values the BASIC file. These values are straight from the spec.

   1:  const int FORWARD = 144;

   2:  const int BACKWARD = 2192;

   3:  const int RIGHT = 1168;

   4:  const int LEFT = 3216;

   5:  const int STILL = 656;

I could have stored them already reversed, but in the interest of making the code more reusable for other IR-related projects, I added a reverse function with help from Nicolas Allen and Wesner Moise that takes a long and the number of bits to reverse.

   1:  public long Reverse(long i, int bits)

   2:  {

   3:       i = ((i >> 1) & 0x5555555555555555) | ((i & 0x5555555555555555) << 1);

   4:       i = ((i >> 2) & 0x3333333333333333) | ((i & 0x3333333333333333) << 2);

   5:       i = ((i >> 4) & 0x0F0F0F0F0F0F0F0F) | ((i & 0x0F0F0F0F0F0F0F0F) << 4);

   6:       i = ((i >> 8) & 0x00FF00FF00FF00FF) | ((i & 0x00FF00FF00FF00FF) << 8);

   7:       i = ((i >> 16) & 0x0000FFFF0000FFFF) | ((i & 0x0000FFFF0000FFFF) << 16);

   8:       i = (i >> 32) | (i << 32);

   9:       return i >> (64 - bits);

  10:  }

This makes the SendData() method nice and clean and allows me to send codes straight from the SIRC spec:

   1:  public unsafe void SendData(long data, int bits)

   2:  {

   3:      int i;

4:

   5:      data = Reverse(data,bits);

   6:      for(i=0;i<bits;i++)

   7:      {

   8:          if((data & 1) == 1)

   9:          {

  10:              SendPulse(ONEPULSE);

  11:              SendSpace(ZEROPULSE);

  12:          }

  13:          else

  14:          {

  15:              SendPulse(ZEROPULSE);

  16:              SendSpace(ZEROPULSE);

  17:          }

  18:          data=data>>1;

  19:      }

  20:  }

This method takes each bit at a time and flashes the IR port. But, we've still got the problem of speed. It's just not happening fast enough. The basic design is there, but at this point I'm starting to suspect I need different hardware.

Infrared Serial Port via Custom Hardware

At this point, I went back to the folks at Iguanaworks and explained the problem. They agreed it was time to put the "carrier" onto the hardware. We'd change the original serial port board that turned the LED on and off using the Serial DTR signal so that now it would oscillate when DTR was set hot.

The graphic above shows the original serial port adapter on the right, the prototype in the middle with the "daughter board" attached. We tried a number of resistor values before we got it right. And the completed first-run prototype on the far left. There's ways to make it smaller, but this way gives the user more choices. The frequency can be swapped to emulate different remote controls by changing resistors yourself in the final design, or turning the carrier off completely.

My IR controlling code wouldn't change significantly, it would actually shrink, as it was silly for me to try and do the work that was better handled by hardware. Here's the updated SendPulse() method along with SendSpace().

   1:  public unsafe void SendPulse(long microSecs)

   2:  {

   3:      port.On();

   4:      MicroWait(microSecs);

   5:  }

6:

   7:  public unsafe void SendSpace(long length)

   8:  {

   9:      port.Off();

  10:      MicroWait(length);

  11:  }

It's worth noting also that the guys from Iguanaworks and the guys from Microbric worked very hard, collaborating between Colorado and Australia, along with me chiming in occasionally, to get this to work. To the right you can see an image of the IR pulse on an oscilloscope. You can see the header on the far right, of 2400 microseconds, followed by 0,0,0,0,1,0,0,1,0,0,0,0.

Controlling the Robot from the Console

The next step is hooking it all up to a console app to try moving the robot around with the keyboard. At this point, the hard work is done and we can setup a simple key-checking loop that runs until you press Ctrl-C

   1:  class Program

   2:  {

   3:      const int FORWARD = 144;

   4:      const int BACKWARD = 2192;

   5:      const int RIGHT = 1168;

   6:      const int LEFT = 3216;

   7:      const int STILL = 656;

   8:      const int REPEAT = 4;

9:

  10:      static void Main(string[] args)

  11:      {

  12:          using (IR ir = new IR("COM1"))

  13:          {

  14:              while (true)

  15:              {

  16:                  ConsoleKeyInfo key = Console.ReadKey(true);

  17:                  switch (key.Key)

  18:                  {

  19:                      case ConsoleKey.UpArrow:

  20:                          ir.Send(FORWARD, REPEAT);

  21:                          break;

  22:                      case ConsoleKey.DownArrow:

  23:                          ir.Send(BACKWARD, REPEAT);

  24:                          break;

  25:                      case ConsoleKey.RightArrow:

  26:                          ir.Send(RIGHT, REPEAT);

  27:                          break;

  28:                      case ConsoleKey.LeftArrow:

  29:                          ir.Send(LEFT, REPEAT);

  30:                          break;

  31:                  }

  32:              }

  33:          }

  34:      }

  35:  }

In the next article, we'll attach the Viper and IR classes to PowerShell and script enable them like the LOGO Turtle of years past.

Conclusion

As with all my projects, there's things that could be extended, added, and improved on with this project. Here are some ideas to get you started.

Extend the IR program running inside the Viper to support other commands.
Integrate the Viper with the Microsoft Robotics Toolkit.
Use the IR class to control your home theater and television.
Write a paint program that controls the Viper and draws with a pen what you draw in paint!

Have fun and have no fear when faced with the words - Some Assembly Required!