Coding4Fun : hardware

Build your own Windows 7 Sensors!

Coding4Fun — Thu, 10 Dec 2009 13:00:00 GMT

Tired of having a computer with no sensors? We’ve created a parallax windows 7 sensor driver with source code so you can create your own!

The sensor device we made here is a Passive-Infrared Sensor. So basically it is a motion sensor. The board used in the white paper is the Parallax “BASIC Stamp HomeWork Board”. Sadly these only come in a 10-pack so the fallback would be the BASIC Stamp 2 Module ($49.00) and getting either the BASIC Stamp 2 Carrier Board” ($15.99) or the Board of Education Dev Board (USB) ($41.99).

The code example also has a .Net application that interprets the data from the sensor.

More of an Arduino fan? Coding4Fun is working with Brian Jepson (@bjepson) of MAKE and O'Reilly to create a driver that works with Arduino! I’ll post more when it gets finished.

Coding4Fun at PDC 2009 – Video Montage

Coding4Fun — Fri, 04 Dec 2009 18:25:00 GMT

We’ll be posting one project each month but here is the Coding4Fun booth at PDC 2009! You can even view the Iron Bartender challenge at GeekFest as well!

Overall Recap of each project

Iron Bartender!

Coding4Fun at PDC!

Coding4Fun — Tue, 17 Nov 2009 18:09:56 GMT

Coding4Fun is in attendance at the Microsoft Professional Developer Conference! We are demoing 5 projects that will have full how-to’s and will be open-sourced as well. Drinktendr, Mind Blaster, Laser Graffiti, Augmented Reality, and Wi-Fi Warthogs are powered by .Net, XNA, WPF, Ling2Sql, WiiMote, MSMQ, Power Wheels, a Mind Set and a giant freaking laser!

Drinktendr

Ever get tired of pouring your own drink? Have someone get upset at you since it was too strong or too weak? Well, why not have a machine pour you the perfect beverage every time! Drinktendr uses Linq2Sql and SQL Server to know what drinks it can make with what is on hand. Combine this with WPF; the end user is given a top end experience.

Mind Blaster

With aliens threatening to invade Earth, it’s your job to travel into outer space and destroy the alien threat…with your mind! Wearing a brainwave detecting headset, coupled with head tracking hardware using a wiimote, use only your brain to destroy alien ships before they destroy you and the rest of humanity.

Laser Graffiti

Putting graffiti on something is not a very nice thing to do but what if with a flick of a switch, it was gone. With a projector, an off the shelf webcam and a laser pointer, you can do just that. Aim the laser at the building and start drawing. Turn off the laser and it all goes away! Using WPF and XNA, we can apply a variety of effects including fire, paint, and various particles. This project was influenced by the Graffiti Research Lab.

Augmented Reality

We’ll be showing off two AR games. ARroller and AR Domino Knockdown. ARroller is a single-player “marble game” experienced through a 3DOF-tracked video–see-through head-worn display. The player holds a tracked board on which a virtual marble must be manipulated through a maze of obstacles by tilting and moving the board. AR Domino Knockdown is a two-player first-person-shooter. Players hold 6DOF-tracked UMPCs through which they fire virtual balls at a configuration of virtual dominos on a shared table.

Wi-Fi Warthogs

Go big or go home. After seeing Halo, we decided we had to make some remote control wart hogs to play laser tag with. Using Xbox controllers, a wireless network, and Power Wheel cars, we’ve made a game of laser tag like no other.

Remote Controlled lunaring with Arduino

Coding4Fun — Tue, 18 Aug 2009 22:01:00 GMT

Nothing puts the fun back into coding like watching your very own creations come to life in the physical world. Luckily, with the current glut of high-tech toys and electronic gadgets on the market, it’s never been easier to experience the thrill of designing your own hardware by modifying what’s already available. A great way to get into the hobby is to wander the isles of your local toy store or browse the pages of an online novelty shop and just let your imagination run wild. I try to go through the toy department with a certain child-like wonderment. I look at everything on every shelf and try to imagine what might be done to improve upon the design, to make it more interesting, or just plain more fun.

In this article, we’ll take a quick look at this hobby by exploring one of my most well received projects to date. We’ll take a simple night-light that cycles through the phases of the moon based on input from an infrared remote control and modify it so that it displays the phases of the moon based off the actual calendar date instead.

Run it now: Download Installer
Source Code: Download
Difficulty: Intermediate
Time Required: 8 hours
Cost: $90
Software Needed: Arduino Development Environment, Lunarupdate arduino sketch, Visual Basic or Visual C# Express
Hardware: ThinkGeek R/C Moon Phase light - $29.99, Sparkfun DS1307 RTC module - $19.99, Arduino Duemilanove - $34.95, 0-100k Potentiometer - $0.95, 9V DC power adapter (optional) - $7.00 Required only if you plan to run the moon from mains power instead of battery.

Gathering the materials

Let’s take a moment to think about what is needed in order for our project to work.

Obviously, we need some way to keep track of the current date, and preferably retain that date even when the batteries run out. We also need some means of controlling the LEDs on the device and to calculate the current lunar phase based on that date. In addition, it would be nice to have some user-friendly method by which we could update the stored date, should it start to drift over time.

As it happens, everything we need to get our prototype off the ground is readily available – and for a reasonable price.

Arduino: The Arduino is an open source electronics prototyping platform designed to help artists and designers rapidly prototype their ideas; it allows designers to write their programs in a highly abstracted form of the C programming language that hides a lot of the technical underpinnings of microcontroller programming. Because it is open source, the Arduino comes in many varieties. You can even build one yourself. A pre-built Arduino typically runs around $30 or less, but building one yourself can be significantly cheaper.

DS1307 Real Time Clock Module: While the Arduino offers a powerful and flexible platform on which to build our project, we still need some way to keep track of the date. For this, we will use a DS1307 real time clock module. I opted to go with the offering from SparkFun Electronics, a Colorado-based company that specialies in providing drop-in solutions for hobbyists and prototypes. This DS1307 module comes pre-configured with an oscillator, coin cell battery backup, and ready to communicate with the Arduino using a standard I2C two-wire bus.

I also chose to add a standard 100k potentiometer (variable resistor) in-line with the existing Light-dependent resistor (LDR) so that we can easily tweak the moon’s sensitivity to light in different environments.

Finally, in order to update the DS1307 clock, I wrote a very simple C# application that simply queries the system date of your PC and passes that to the Arduino which in turn uses the I2C bus to set the date on the DS1307.

Getting Started

If you’ve ever peered into the mysterious inner workings of an electronic device, then you’ll appreciate how extremely straightforward this moon really is.

The video below will give you a basic overview of the hardware that we’re using.

Essentially, all we’ve done here is to remove the existing controller from the moon and replace it with our Arduino controller, which is in-turn connected to the DS1307 RTC.

If you’re unfamiliar with how to read electronic schematic diagrams, the table below explains how the parts are interconnected.

Arduino pin:	Connected to:
Digital pin 3	Center leg of potentiometer
Digital pins 7-12	LED lights (Red 7 … Purple 12)
Analog pin 5	DS1307 SCL pin
Analog pin 4	DS1307 SDA pin
GND	GND of DS1307
GND	(-) Negative battery Terminal
GND	Outside leg of potentiometer
GND	GND (Brown) wire from LED panel
5V	One leg of LDR
5V	5V of DS1307
VIN	Center leg of power switch.

Also connect:

Unused leg of LDR	Unused leg of potentiometer
Outside leg of power switch	(+) Positive battery terminal

If it all sounds too complicated. It isn’t. Once you have the components in your hands, you can easily identify the parts listed above and put everything together by simply using the table above as a checklist.

The Code

The code for interfacing the Arduino with the DS1307 module is available in the Arduino playground. The Arduino playground is a fantastic resource of community-contributed code samples that you can use in your own projects to interact with a remarkable array of hardware devices and software platforms.

Once we retrieve the date from the DS1307, we can calculate the phase of the moon in our Arduino program:

// Adapted from Stephen R. Schmitt’s Lunar phase computation program.
// Originally written for the Zeno programming language.
// http://home.att.net/~srschmitt/lunarphasecalc.html

byte getPhase(int Y, int M, int D) {
  double AG, IP;
  byte phase;
  long YY, MM, K1, K2, K3, JD;
  YY = Y - floor((12 - M) / 10);
  MM = M + 9;
  if(MM >= 12)
    MM = MM - 12;
  
  K1 = floor(365.25 * (YY + 4712));
  K2 = floor(30.6 * MM + 0.5);
  K3 = floor(floor((YY / 100) + 49) * 0.75) - 38;

  JD = K1 + K2 + D + 59;
  if(JD > 2299160)
    JD = JD -K3;
  
  IP = normalize((JD - 2451550.1) / 29.530588853);
  AG = IP*29.53;
  
  if(AG < 1.20369)
    phase = B00000000;
  else if(AG < 3.61108)
    phase = B00000001;
  else if(AG < 6.01846)
    phase = B00000011;
  else if(AG < 8.42595)
    phase = B00000111;
  else if(AG < 10.83323)
    phase = B00001111;
  else if(AG < 13.24062)
    phase = B00011111;
  else if(AG < 15.64800)
    phase = B00111111;
  else if(AG < 18.05539)
    phase = B00111110;
  else if(AG < 20.46277)
    phase = B00111100;
  else if(AG < 22.87016)
    phase = B00111000;
  else if(AG < 25.27754)
    phase = B00110000;
  else if(AG < 27.68493)
    phase = B00100000;
  else
    phase = 0;
  
  return phase;    
}

Finally, the Arduino is capable of communicating with a PC via standard serial communication thanks to the FTDI USB-to-serial controller that comes standard with many Arduino boards.

C#:

// Use a button Click event to send data to the Arduino.
// The following example is intended to be a very basic example
// of writing the date to the serial port. A more robust application
// would use a protocol and CRC to ensure that the data being transmitted
// was valid.
private void updateButton_Click(object sender, EventArgs e)
{
    byte[] result = new byte[7];

    DateTime currTime = DateTime.Now;

    serialPort.PortName = “COM4”;
    serialPort.BaudRate = 19200;
    serialPort.DataBits = 8;
    serialPort.Parity = Parity.None;
    serialPort.StopBits = StopBits.One;

    try
    {
        serialPort.Open();
    }
    catch (IOException ioe)
    {
         // Handle your exceptions.
    }

    // Some Arduino models reboot automatically when the serial
    // port is open. Allow a small delay to account for this.
    Thread.Sleep(10000);

    result[0] = BitConverter.GetBytes(currTime.Second)[0];
    result[1] = BitConverter.GetBytes(currTime.Minute)[0];
    result[2] = BitConverter.GetBytes(currTime.Hour)[0];
    result[3] = BitConverter.GetBytes(1)[0];
    result[4] = BitConverter.GetBytes(currTime.Day)[0];
    result[5] = BitConverter.GetBytes(currTime.Month)[0];
    result[6] = BitConverter.GetBytes((currTime.Year % 2000))[0];
    
    try
    {
         serialPort.Write(result, 0, result.Length);
         serialPort.Close();
    }
    catch (InvalidOperationException ioe)
    {
         // Handle more exceptions.
    }
    catch (TimeoutException te)
    {
         // Handle more exceptions.
    }
}

Arduino

All that’s left is for the Arduino to read and process the data. Using our extremely simple example above, let’s take a look at some Arduino code.

void loop() {
  while(Serial.available()) {
    reader = Serial.read();
    if(reader == 255) {
      // do nothing - dummy byte
    }
    else {
        // populate our array with values
      needsUpdate = true;
      update[addr] = reader;
      addr += 1;
    }
  }
  
    // Once we have all the values, update the clock.
  if((needsUpdate) && (addr == 7)) { 
    setDateDs1307(update[0],
          update[1],
          update[2],
          update[3],
          update[4],
          update[5],
          update[6]);
      // Store the last two digits of the year in EEPROM.
    EEPROM.write(0, update[6]);
  }
  delay(100);
  needsUpdate = false;
  addr = 0;
}

Conclusion

That’s really all there is to it. Feel free to check out the complete code samples at imagitronics.org, or strike out on your own. Take part in the true spirit of the hobby by extending or modifying the moon to meet your own ends: implement a communication protocol, create a more robust desktop application, find ways to make the device cheaper using a USB HID controller and allowing the moon to run from USB power.

My intent in writing this article was to get people excited about the prospect hardware design and prototyping and to remove any preconceived notions about the difficulty of such projects.

Most importantly, just remember my motto: Dream it, build it, break it, repeat.

Special Thanks

John Vaughters’s DS1307 RTC functions
SparkFun Electronics’ EAGLE CAD library
Rob Faludi’s Arduino EAGLE CAD library
Stephen Schmitt’s Zeno programming language lunar phase sample

About the author

Tim Farley is an Iraq war veteran, Systems Integrator for the Department of Defense (DoD), military spouse, and sole maintainer of imagitronics.org; a site devoted to his personal projects and hobbies. Although he has been working in and around government agencies for over 12 years, he hopes to someday break into world of toy/game research and design.

GPS tools

Coding4Fun — Tue, 11 Aug 2009 22:01:00 GMT

GeoFrameworks and Jon Person released their GPS Framework for .NET on CodePlex! The really nice thing about this framework is it works with both the full blown .NET Framework and the Compact Framework. If you’re developing for a Windows Mobile, embedded, laptop or desktop computer, it should handle all your needs.

Just a few of the features are:

Automatic detection of serial GPS devices (or devices found via a virtual serial port).

Automatic detection of Bluetooth devices (when using the Microsoft Bluetooth stack.)

Automatic baud rate detection.

Automatic recovery of lost connections.

Advanced GPS precision via Kalman filtering.

Support for desktops and mobile devices.

Support for real-time GPS data without relying on Microsoft's GPS API.

Support for control and monitoring of precision.

A set of animated gauge controls for desktops and mobile devices (Altimeter, Compass, Speedometer, SatelliteViewer, SatelliteSignalBar).

Update: Forgot to put in the link! http://gps3.codeplex.com

Building Multiplayer Texas Holdem Poker For The Zune

Coding4Fun — Tue, 05 May 2009 22:06:00 GMT

Have you ever been interested in creating a game that harnesses XNA’s powerful network library to create multiplayer experiences for the Zune device?

The Zune firmware version 3.1 brought us a professionally built incarnation of Texas Holdem that supports network play. Understanding how to send and receive data with the Zune can be a little daunting at first, but once you understand the pattern, it’s easy.

To build out the entire game, you probably need about a week, but you can build some simpler examples in far less time.

This is an earlier project from before the release of my Zune game development book. Accordingly, some of the code samples you see in this article may be inconsistent with what you find in the download. The code in the article is the “correct” way to do things. The code in the download is still a work in progress.

Source Code: Download
Difficulty Intermediate|Advanced
Time Required: 40 hours
Hardware: 2x Zune devices ($170 ea)
Software Needed: Visual C# Express, XNA Game Studio 3.0

The Workflow

Developing multiplayer games for the Zune is interesting because you have to deploy to each device individually. Once you have a build that works for you, it’s helpful to run-deploy (Control+F5) to one device, leave it running there, and then plug in the other Zune and debug-deploy (F5) to it. This way you have one debuggable instance of the game running. Make sure to set the appropriate Zune device as the default in the XNA Game Studio Device Center (accessible from the Tools menu).

Starting A Network Session

Because the Zunes connect over an ad-hoc, peer-to-peer connection rather than through an access point, you will have to designate one Zune as the host device. The host is usually determined to be the one that creates a new game. Therefore, all Zunes that join the host’s network session are simply peers. The difference between the host and the peers is that the host usually maintains the game state on top of executing the game as well, because the game data has to be centralized somewhere. Keep that in mind, because if one Zune is doing substantially more processing, it can lag behind and mess up your network session. Also, remember that all Zunes are running the exact same copy of the game, so the game must support both host and peer scenarios.

Create / Join / Lobby Model

Most peer-to-peer connected games allow a user to create or join a game. After doing so, the player is funneled into an area called the Lobby where they can specify their readiness. When all players are ready, the host can start the game. Some of this functionality is provided directly by the XNA Framework’s Net and GamerServices libraries.

I normally create three separate screens based on the Network Game State Management sample from the Creators Club website. The first is the Create screen, which looks exactly like a lobby. It starts up a network session and waits for players to join and become ready. The code to create a network session looks like this (note that I have employed some abstraction to make my code a little more cohesive):

public void CreateZuneSession(int maxNetworkPlayers)
{
    KillSession();

    try
    {
        Session = NetworkSession.Create(NetworkSessionType.SystemLink, 1,      
            maxNetworkPlayers); 
        Me = Session.LocalGamers[0];
    }
    catch (NetworkNotAvailableException)
    {
        throw new NetworkNotAvailableException("Zune wireless is not 
            enabled.");
    }
    catch (NetworkException ne)
    {
        throw ne;
    }

    if (Session == null)
        throw new NetworkException("The network session could not be 
            created.");
}

All Zune network sessions are of the type SystemLink, much like LAN-networked Xbox consoles. The ‘1’ parameter specifies the number of local players – of course, on a Zune, there can only be one.

This line of code sets the Session property to a newly created network session. The Join screen, running on another Zune, will find this session asynchronously as an available network session and attempt to join it. The first step is to enumerate available network sessions:

/// <summary>
/// Asynchronous: Begins to discover network sessions.
/// </summary>
public void BeginGetAvailableSessions()
{
    // Destroy any existing connections
    KillSession();

    NetworkSession.BeginFind(NetworkSessionType.SystemLink, 1, null, new 
        AsyncCallback(SessionsFoundCallback), null);
}

This code looks for SystemLink sessions and calls the callback method SessionsFoundCallback when the operation completes (successfully or unsuccessfully). If sessions are found, an event is fired. Other screens can subscribe to this event and transition to other screens or do other processing with the network session.

/// <summary>
/// Called when network sessions are found.
/// </summary>
/// <param name="result"></param>
public void SessionsFoundCallback(IAsyncResult result)
{
    AvailableNetworkSessionCollection availableSessions = null;
    availableSessions = NetworkSession.EndFind(result);

    if (NetworkSessionsFound != null)
        NetworkSessionsFound(availableSessions);
}

When all players are ready and the host presses the middle button, the network session’s StartGame() method is called, which will cause all connected peers to receive a GameStarted event. This loads up the playing screen.

if (allPlayersReady && atLeastTwoPlayers)
{
    if (ScreenManager.Network.Session != null)
        ScreenManager.Network.Session.StartGame();
}

That’s basically how to connect up two Zunes in a network session. See my book for a much more detailed explanation.

Dealing With Cards

As Poker is a card game, you might want to develop a testable, standalone library that you can use not only to house your objects, but also to write out all the intense logic required for a game like poker. The task of determining what a player’s best hand is out of seven cards (and whether it beats another player’s hand) is more in-depth than you might expect.

An easy way to go about this is to create a Zune game library. I called mine CardLib. CardLib has objects such as these:

Card which has a suit, a value, some comparers and some utility methods)
Deck, which has a collection of cards and methods like Shuffle, ResetDeck, etc.
Dealer, which has a deck, a list of the five community cards, and methods like Shuffle, DealCard, Burn, DealFlop, DealTurn, etc.
HoldemHand, which contains all of the logic for determining what a hand is (always 7 cards)
BestHand, which takes a HoldemHand and determines what the best possible hand is within that hand

Shuffling Cards

Shuffling cards is surprisingly easy. I use the Knuth-Fisher-Yates shuffling algorithm, which takes every card in the deck and randomly swaps it with another.

/// <summary>
/// Knuth-Fisher-Yates shuffling algorithm:
/// http://www.codinghorror.com/blog/archives/001015.html
/// </summary>
public void Shuffle()
{
    Random rand = new Random();
    for (int cardIndex = _cards.Count - 1; cardIndex > 0; cardIndex--)
    {
        int randomIndex = rand.Next(cardIndex + 1);
        SwapCardsByIndex(cardIndex, randomIndex);
    }
    _dealIndex = 0;
}

Poker Logic

An example method you’ll see in this library is this snippet to check if the hand is a royal flush:

public bool IsRoyalFlush(out List<Card> winningCards)
{
    List<Card> straightFlushCards;

    if (IsStraightFlush(out straightFlushCards))
    {                
        // Check to make sure the straight flush cards are: 10 J Q K A
        straightFlushCards.Sort();

        if (straightFlushCards[0].Value == 10 &&
            straightFlushCards[1].Value == Card.Jack &&
            straightFlushCards[2].Value == Card.Queen &&
            straightFlushCards[3].Value == Card.King &&
            straightFlushCards[4].Value == Card.Ace)
        {
            winningCards = straightFlushCards;
            return true;
        }

        else
        {
            winningCards = null;
            return false;
        }
    }
    else
    {
        winningCards = null;
        return false;
    }
}

There is a ton of similar and even more disgusting logic to peruse in the HoldemHand.cs file in the region entitled “Poker Logic Fun Times.” This logic covers every possible poker hand and could probably be refactored to be way more elegant.

When it comes time to evaluate the winner, each player’s best hand is determined and they are evaluated against each other to determine the winner. This code returns a list of WinnerInfo objects, which returns the player info and a list of the cards they won with.

public List<WinnerInfo> DetermineWinners()
{
    BestHand overallBestHand = null;
    HoldemPlayer winner = null;

    List<WinnerInfo> winners = new List<WinnerInfo>();

    foreach (HoldemPlayer player in _players)
    {
        if (player.Status != PlayerStatus.Folded)
        {
            HoldemHand tempHand = new HoldemHand(player.Pocket, _communityCards);
            BestHand bestHand = tempHand.GetBestHand();

            if (overallBestHand == null) // if there is no best hand yet
            {
                overallBestHand = bestHand;
                winner = player;
                winners.Add(new WinnerInfo(overallBestHand, winner, Me));
            }
            else
            {
                // if this hand beats the current best hand
                if (BestHand.Beats(bestHand, overallBestHand))
                {
                    winners.Clear();
                    overallBestHand = bestHand;
                    winner = player;
                    winners.Add(new WinnerInfo(overallBestHand, winner, Me));
                }
                else
                {
                    if (BestHand.IsEquivalentTo(bestHand, overallBestHand))
                    {
                        winners.Add(new WinnerInfo(bestHand, player, Me));
                    }
                }
            }
        }
    }

    _winners = winners;
    return winners;
}

Drawing Cards On The Screen

Rather than have 52 different sprites that represent each possible card, I just made my own sheet of Zune-sized playing cards. Horizontally, they are ordered by value (1-13) and vertically they are ordered by suit (both alphabetically and corresponding to the numeric value of the Suit enumeration in the project, 1-4).

When a specific card is requested to be drawn, a formula is used to calculate the source rectangle from this larger image. This is very similar to how fonts worked before spritefonts were introduced into XNA. This allows us to strip out the graphic of a specific card and draw it onscreen at some other location.

private void DrawCard(Card card, Vector2 position)
{
    if (card != Card.Undefined && card.Suit != Suit.Unassigned)
        ScreenManager.SpriteBatch.Draw(_texDeck, position, GetCardSourceRect(card), Color.White);
}

private Rectangle GetCardSourceRect(Card card)
{
    if (card.Suit == Suit.Unassigned)
        throw new ArgumentException("Unassigned cards cannot be drawn.");

    int cardColumn = card.Value - 1;
    int cardRow = (int)card.Suit - 1;

    int x = cardColumn * MainScreenElements.CARD_WIDTH;
    int y = cardRow * MainScreenElements.CARD_HEIGHT;

    return new Rectangle(x, y, MainScreenElements.CARD_WIDTH, MainScreenElements.CARD_HEIGHT);
}

Managing Network Data

One thing that increases the complexity of this game (in no small way) is the management of network data. First of all, the host’s actions are a superset of the peer’s actions (a host is also a peer, but it also has to manage the game and network state). For example, when a peer decides to bet, it has to send a message to the host saying “I would like to bet,” and then the host will process that message and relay it to the other peers. The host is also responsible for managing whose turn it is and determining who the winner is. Theoretically, you could do a lot of this with every peer acting equally but it feels safer to me to have the host responsible for important activities like dealing cards. In fact, the host has to be the only one who can deal cards, because if each client maintained its own deck, it would be randomized for every peer when the deck is shuffled.

How Data Is Sent And Received

I will usually create a static class called NetworkMessageSender that is responsible for sending various messages. I keep an enumeration of type byte that holds the possible network messages.

I always send the byte indicating the message type first, so that when the peer receives a byte, it knows how to respond. For example, if the peer receives a CardDealt message, it can pop off a string and a card from the incoming packets. If the card is not intended for the peer, it can simply discard the message.

Data you want to send is written to a packet writer object. When the data is ready to be sent, you call the SendData method of the LocalGamer object. Depending on how important packet receipt is, you can specify the type of transmission. I use ReliableInOrder during poker because network data is exchanged relatively infrequently. Although there is only one local network gamer, you can use a foreach loop to ensure this code will work on other platforms.

private static void SendData()
{
    foreach (LocalNetworkGamer gamer in NetworkSessionManager.Session.LocalGamers)
        gamer.SendData(NetworkSessionManager.PacketWriter, SendDataOptions.ReliableInOrder);
}

Specific chunks of data are then sent using various static methods. Each piece of information is written sequentially. The card is serialized into a string format before being sent. The following chunk of code is used to send a card to a player.

public static void DealCards(HoldemPlayer player)
{
    NetworkSessionManager.PacketWriter.Write((byte)NetworkMessageType.Deal);
    NetworkSessionManager.PacketWriter.Write(player.Name);
    NetworkSessionManager.PacketWriter.Write(player.Pocket.Card1.Serialize());
    NetworkSessionManager.PacketWriter.Write(player.Pocket.Card2.Serialize());
    SendData();
}

On the other side of that, whenever the screen updates, the game is constantly checking for new network data. The following code is part of a method that is called from the game screen’s Update loop.

private void UpdateNetworkSession()
{
    NetworkSessionManager.Update();

    foreach (LocalNetworkGamer localGamer in 
        NetworkSessionManager.Session.LocalGamers)
    {
        while (localGamer.IsDataAvailable)
        {
            NetworkGamer sender;
            localGamer.ReceiveData(NetworkSessionManager.PacketReader, 
                out sender);

            // Interpret the first piece of information, the message.
            NetworkMessageType message = (NetworkMessageType)NetworkSessionManager.PacketReader.ReadByte();

            // Determine what to read and what to do based on this message type
            switch (message)
            {
                // major snippage
                case NetworkMessageType.Deal: // Happens when a card is dealt
                {
                    string playerName = NetworkSessionManager.PacketReader.ReadString();
                    string card1 = NetworkSessionManager.PacketReader.ReadString();
                    string card2 = NetworkSessionManager.PacketReader.ReadString();

                 _gameplayManager.Players[playerName].Pocket.Set(PlayerPocket.FIRST_CARD_INDEX, new 
                    Card(card1));
                 _gameplayManager.Players[playerName].Pocket.Set(PlayerPocket.SECOND_CARD_INDEX, new 
                    Card(card2));
                }
                break;
            }
        }
    }
}

There is a whole bunch of similar code that underlies how the game executes. When a message needs to be sent, the NetworkMessageSender class is used. To determine what happens when a message is received, we look at that ginormous switch statement that gets called in the screen update loop.

Remember that the host is also a peer and it will receive the very same messages it sends (unless you specify otherwise). Be careful not to double-process messages if they are sent by the host. In some cases you will need to check that the current device is or is not the host. In the sample download, this is achieved just by checking a Boolean value that is set early on in the game.

Conclusion

Building Poker for the Zune, from the ground up, is no small feat! Networked Zune games are far simpler when you have a much more limited set of possible data and messages that could be sent. For example, networked Pong, Battleship or Tetris would be pretty easy compared to poker. Turn-based games on the Zune also provide an interesting challenge in terms of what happens when a Zune drops from the network session.

This is just a small dip into networked Zune game development. For a deep dive, check out my book, Zune Game Development using XNA 3.0, also available on Apress.com as an eBook. The final chapter is a sprawling 120 pages covering how to build Crazy Eights for the Zune from the ground up.

About The Author

Dan Waters is an Academic Developer Evangelist at Microsoft, based in Tampa, FL. When he’s not out showing the latest and greatest MS technology to students and faculty, he’s spending time with his wife and young daughter or rocking out on one of his (far too numerous) guitars. Follow Dan on Twitter or check out his blog.

Wiimote Virtual Reality Desktop

Coding4Fun — Thu, 12 Feb 2009 00:15:00 GMT

In this article I want to introduce a Virtual Reality system that can be build with two Nintendo Wii Remotes (Wiimotes) and just a little extra hardware that costs less than $10 USD.

Timo Fleisch

Difficulty: Expert

Time Required: 3-4 hours

Cost: : $80 for two Wiimotes, ~$20 for IR-LEDs, stripboards, battery holder

Software: Visual C# Express Edition 2008, XNA Games Studio 3.0, Managed Library for Nintendo's Wiimote, OpenCV Computer Vision Library and EmguCV C# Wrapper

Hardware: Two Nintendo Wii Remote (Wiimote), Anaglyph 3D Glasses (Red/Green Glasses) or Zalman 3D Monitor, compatible PC Bluetooth adapter and stack

Download Binary: Download

Download Source: Download

Introduction

The term Virtual Reality (VR) has been around many years by now but is still associated with very special and expensive hardware that is only available for research laboratories or big companies. Most people might think of a setup where the user wears a head-mounted-display and a data glove. Technically speaking Virtual Reality is defined by two things:

Immersion and
Interactivity

By immersion is meant, that the user gets the impression of being inside a computer generated world. Usually VR systems use stereoscopic output devices to give the user a real 3D impression of the virtual world. But besides visual immersion, haptic and acoustic can also play an important role.

3D images can for example be experienced in IMAX 3D cinemas – but this is still no Virtual Reality. What is missing is the interactivity. When the user is able to interact with the virtual world, by navigation and manipulation of the objects then the definition for VR is complete.

In this article I want to introduce a Virtual Reality system that can be build with two Nintendo Wii Remotes (Wiimotes) and just a little extra hardware that costs less than 10 USD. By that VR should be affordable for everyone. The demo software and source can be downloaded from my website:
http://www.vrhome.de.

Virtual Reality Desktop

A VR Desktop setup uses a monitor as 3D output display and is sometimes referred to as fishtank Virtual Reality. I use a stereo monitor that needs a pair of polarization glasses to display stereo images. If you don't own a stereo monitor you can still construct the VR Desktop by using the anaglyph or red/green stereo method. For that you just need a pair of red/green glasses and you can use any monitor. Even though you loose color information the stereo effect is still very good.

For interactivity the two Wiimotes come into play. In this setup there are two kinds of interaction possible. First we have a kind of navigation support. Navigation inside a rendered 3D scenes means the movement of the camera, as the camera represents the eyes of the user. For the Desktop setup this means we have to track the users' eyes or head position to move the 3D scene camera correctly – therefore this is also called head tracking. For the head tracking I mount one Wiimote at the top of the monitor to point at the user's face.

The second Wiimote is mounted from the top in vertical down position facing the area in front of the monitor. This will be used to interact with the 3D scene by manipulating the transformation of a 3D object. As you use the hand for this interaction I call this hand tracking. The Virtual Reality Desktop setup is shown in Picture 1.

Wiimote 6DOF Tracking

The core of this VR setup is using the Nintendo Wiimote Controller as a tracking camera. As one might already know the Wiimote is equipped with an infra-red camera that is able to recognize up to four infra-red lights. The main idea now is to build a device with four infra-red LEDs that can be recognized by the Wiimote. This device I call IR-LED beacon. Using the values of the 4 LEDs as recognized by the Wiimote I use an algorithm to reconstruct the original position and orientation of the IR-LED beacon. Getting all 3 axis translations and all 3 axis rotation angles of the device is called 6 degree of freedom tracking – or short 6DOF tracking.

IR-LED Beacon

Picture 1: Wiimote Desktop Virtual Reality Setup

Because this setup uses head tracking as well as a tracking for a hand manipulation two beacon have to be built. In pictures 2 and 3 the beacons that I built for my setup are shown.

Picture 2: Head Tracking IR-LED Beacon

Picture 3: Hand Tracking IR-LED Beacon

As can bee seen the beacons just consist of 4 IR-LEDs, a battery holder and wiring. For easy construction and less wiring I use a stripboard. When choosing the IR-LEDs it is important to look out for a very wide angle of radiation. Typical LED just have a small angle, try to find LEDs with angles greater than 120°, a good source might be digikey.com. Three of the four LEDs are aligned in a line with only slightly different height. The fourth LED is mounted above the line with more height. This special order of the lights is needed by the algorithm to be able to assign the IR-points recognized by the Wiimote to the original LEDs of the beacon. It is also important that the fourth LED has not the same height, so that the LEDs are not so planar. Please see picture 4 for a schematic layout of the beacon. For power supply I just us on AAA battery and connect all LEDs in parallel to the battery poles. For easy handling I use a battery holder which are also available at electronic components supply stores.

After soldering everything together the exact positions of the LEDs have to be measured in the correct order. It is very important to measure the 3D position of the LED lights very accurately. If the data for these coordinates is not accurate the tracking results will be poor. The values have to be in millimeters.

Picture 4: Schematic of IR-LED Beacon

For example, my LEDs have the following measurements:

<point3d value="1, 0, 8.5"/>
<point3d value="29, 0, 11"/>
<point3d value="56, 0, 6"/>
<point3d value="29, 45, 21"/>

Configuring the Software

After position the Wiimotes as shown and constructing and measuring the beacons you have to configure the software using two configuration files. You will find the files in the application's directory.

Settings.xml

<Settings> 
    <Stereo
        eyeDistance = "0.02"
        switchLeftRight = "False"
        fieldOfView = "60"
        antiAlias = "False"
        stereoMode = "lineInterlaced"
        fullscreen = "False"
        resolution = "1280,1024"
        displayDevice = "Screen"
        anaglyph = "True"
        windowPosition = "0,0" />
</Settings>

If you own a line-interlaced 3D Monitor from Zalman, you can change the anaglyph to False. If you have difficulties adjusting your eyes to the stereo image you can try changing the value for the eyeDistance. Reducing the value makes it easier to adjust but might reduce the 3D effect. The other parameters should work as you might expect from the name.

Tracker.xml

In this file the configuration for the tracking is stored. It defines the Wiimotes and its positions and the IR-LED beacons. Furthermore it defines filtering parameters. Here I will point to the places where you need to adjust the values for your setup.

First you will need to specify the exact position of your Wiimotes relative to the center of the screen. You need to measure this distance for each Wiimote in millimeter. Probably the values will be similar to the default values:

<TrackerCam 
    id="2" cameramodel="Wiimote" 
    translation="0,200,50" 
    rotation="0,0,0" scale="0.001" 
    xAxis="x" yAxis="y" zAxis="z"> 
</TrackerCam>
<TrackerCam
    id="1" cameramodel="Wiimote"
    translation="0,350,350" rotation="0,0,0" scale="0.001"
    xAxis="x" yAxis="z" zAxis="y">
</TrackerCam>

The Wiimote with id=2 is the one mounted at the top of the monitor to do the head tracking. In the translation field enter the distance of the front tip of the Wiimote to the center of the screen. In the default settings the Wiimote is 200mm above (y-value) and 50mm in front (z-value) of the center. Do the same for the second Wiimote in the following entry with id=1. You can see, that here the y-axis is assigned to the z-axis and the z-axis is assigned to the y-axis. This is necessary because the Wiimote is mounted in vertical position. If the software recognizes the Wiimotes in wrong order you can exchange the order of the definition blocks.

The second part where you need to make changes is the definition of the IR-LED-Beacons:

<MarkerBody>
    <WiiMarkerBody id="0" name="WiiMote Head Beacon" nearClip="20"
        farClip="1500" translation="0,0,0" rotation="0,0,0">
        <point3d value="0, 4, 7"/>
        <point3d value="40, 4.5, 10.5"/>
        <point3d value="83.5, 5, 7.5"/>
        <point3d value="38, 45, 18"/>
    </WiiMarkerBody>
    <WiiMarkerBody id="4" name="WiiMote Hand Beacon" nearClip="20" 
        farClip="1500" translation="0,0,0" rotation="0,0,0">
        <point3d value="1, 0, 8.5"/>
        <point3d value="29, 0, 11"/>
        <point3d value="56, 0, 6"/>
        <point3d value="29, 45, 21"/>
    </WiiMarkerBody>
</MarkerBody>

Here you need to change the values for the beacon points in millimeter according to your measurements. Please note that the correct order of the points is necessary. They have to be from LED1 to LED4 according to the schematic in Picture 4.

Furthermore you will find a description for the tracked device:

<TrackedDevice id="4" type="WiiMote" rotation="True" translation="True">
    <MarkerBodyId>0</MarkerBodyId>
    <TrackerCamId>2</TrackerCamId>
    <ReverseTranslation>False,True,True</ReverseTranslation>
    <ReverseRotation>False,False,False</ReverseRotation>
    <LocalTranslation>-45,25,0</LocalTranslation>
    <LocalRotation>0,0,0</LocalRotation>
    <WorldTranslation>0,0,0</WorldTranslation>
    <WorldRotation>0,0,0 </WorldRotation>
    <RotationFilterId>4</RotationFilterId>
    <TranslationFilterId>4</TranslationFilterId>
</TrackedDevice>

Here you assign which Wiimote (TrackedCamId) to use with which IR-LED-Beacon (MarkerBodyId). The only value that you might want to adjust from these settings is the LocalTranslation. The translation and rotation values that are calculated need to have their reference point on the beacon. By default the first LED will be this reference. Usually you want to have a different reference point, like the center of the beacon. Therefore measure the distance from the first LED to your preferred reference point in the same metric as the beacon values and write it in the LocalTranslation.

There are lot of more parameters you can tweak inside this configuration file, but to make the VR-Desktop run this shouldn't be necessary.

Run the Software

Before running the software it is necessary to connect the Wiimotes with the computer. For that the computer needs to be equipped with a compatible Bluetooth adapter. For more detailed description on how to do that please refer to Brian Peek's Wiimote Library article:

http://blogs.msdn.com/coding4fun/archive/2007/03/14/1879033.aspx

After putting in your values in the configuration files and placement and connection of the two Wiimotes you can run the binary installation by clicking on the VRDesktoDemo in the start menu. If you want to run from the source code you have to copy the OpenCV dlls from (VRDesktopSrc)\ExtLibs\OpenCV\opencvlib to the binary destination directory of the compiled project, e.g. (VRDesktopSrc)\VRDesktopDemo\bin\x86\Release before you can start the application.

Using the Library

Using the VRDesktop in your own XNA application is very easy. Here I will point out the relevant steps from the VRDesktopDemo application.

To start from scratch you will have to create a new XNA Windows Game Project. First include the references to the two libraries Tgex and Tvrx. Then open the created Game class.

At the top the namespaces for libraries have to be added:

using Tgex.Graphics;
using Tgex;
using Tvrx;

Then the parent class has to be changed from Game to VRGame:

public class VRDesktop : VRGame

VRGame is part of the Tgex library and adds the support for the stereo display. It creates a stereo camera and the application window according to the settings file. You can use the class more or less similar to the original game class. The main difference is, that for the drawing you must not override the Draw(GameTime time) function but the new DrawScene(GameTime time) because the Draw function of the VRGame class takes care about the stereo rendering.

For storing the transformation matrix of the hand tracker we define a variable:

Matrix modelTransform = Matrix.Identity;

And in this simple example we define a variable for the model:

Model model;

In the Initialize function the initialization of the TrackerManager needs to be done:

protected override void Initialize()
{
    // The tracker manager is a singleton but needs to be initialized once.
    TrackerManager.Instance.Initialize();

    // initialize base clase.
    base.Initialize();
}

In the LoadContent function the model is loaded and the tracking is started:

protected override void LoadContent()
{
    // for this demo just load the coordinate cross
    model = Content.Load<Model>("coordinate");
    modelTransforms = new Matrix[model.Bones.Count];

    // start tracking now
    TrackerManager.Instance.StartTracking();
}

The Update function is for the main game logic. First we allow the user to exit the game and the tracker to stop regularly:

// Allows the default game to exit on Xbox 360 and Windows
if ((GamePad.GetState(PlayerIndex.One).Buttons.Back == ButtonState.Pressed)
    || (Keyboard.GetState().IsKeyDown(Keys.Escape)))
{
    TrackerManager.Instance.StopTracking();
    this.Exit();
}

Before getting the latest tracking data we have to call an update on the TrackerManager:

TrackerManager.Instance.Update();

To get the transformation data we have to call the GetProxyTransform(indexNumber) from the manager. The proxies are defined in the tracking.xml file. In the example we call:

// tracking proxy with id 1 is the hand tracker
modelTransform = TrackerManager.Instance.GetProxyTransform(1);

// tracking proxy with id 0 is the head tracker - change eye position.
m_camera.EyePosition = 
    TrackerManager.Instance.GetProxyTransform(0).Translation;

The m_camera is defined in the VRGame parent class. The camera class will also make the necessary camera transformation adjustments for screen projected head-tracking by creating a perspective of center projection matrix.

Finally in the DrawScene function the model mesh is drawn. Here we need to pass the model transformation matrix as well as the camera matrices to the effect:

//Draw the model, a model can have multiple meshes, so loop
foreach (ModelMesh mesh in model.Meshes)
{
    //This is where the mesh orientation is set, as well 
    // as our camera and projection
    foreach (BasicEffect effect in mesh.Effects)
    {
        effect.EnableDefaultLighting();
        effect.World = modelTransforms[mesh.ParentBone.Index]
                       * Matrix.CreateScale(0.01f) // coordinate scale
                       * modelTransform;
        effect.View = m_camera.ViewMatrix;
        effect.Projection = m_camera.ProjectionMatrix;
    }

    //Draw the mesh, will use the effects set above.
    mesh.Draw();
}

That’s all what has to be done to make use of the Wiimote Virtual Reality Desktop in your own application.

How does it work

For the interested reader I will explain now in more detail how the Wiimote tracking actually works. Anyhow, I will not go into the mathematical details of the core algorithm but provide you with the necessary references. I will just focus on the Wiimote tracking part and will not go into details of neither the tracking library Tvrx nor the game library Tgex.

Pose Estimation

Technically speaking, the Wiimote tracking is an optical marker based tracking. It is optical because we use the Wiimote camera and it is marker based, because we don't use the whole camera image. Instead the Wiimote returns the position of up to four infrared LEDs which represent our markers.
The four LED positions are given in 2D coordinates which represent the markers as seen by the camera. Technically this is a 2D projection of the real LEDs according to the focus of the camera lens, this projection plane is called image plane - the 2D coordinated are therefore called image points.
The main task now is to calculate the position and rotation of the IR-LED Beacon in real space according to the image points of the LEDs measured by the Wiimote. This task is called Pose Estimation and has been investigated by scientists since many years. One main application of pose estimation is computer vision for robotics. The pose estimation algorithm used for the Wiimote tracking has already been published in 1995 and still performs nice for our purpose. If you are interested in the exact details about how the pose estimation algorithm works, you can study the original paper, please see the references.
To be able to perform the pose estimation calculation you need the exact information about the image plane. Actually, the plane is nothing else then the image sensor of the camera. The picture is focused on the image sensor by the camera lens. For the calculation of the projection we need a mathematical model for the camera, in computer vision usually a pinhole camera model is used. The pinhole is the projection point origin. The image plane is at a certain distance from the origin. This distance is called focus length.
Now, what we need for the calculation is the focal length and the size of image plane, the camera sensor. These values are also called the intrinsic camera values. Unfortunately Nintendo does not publish these values for the Wiimote, so the values I use are assumptions:
focal length in pixel = 1380

// assume 1/4" cd sensor (even though it probably is not)
pixel size in mm = 0.0035
chip resolution = 1024x768

// Wiimote center (approx.)
principal point = 512x384

The resolution of the values returned by the Wiimote are 1024x768. Obviously this is not the physical resolution because cameras with this resolution would cost more than 1000 USD. The Wiimote has a PixArt Imaging Inc. (http://www.pixart.com.tw) sensor and probably has a resolution of 352x288 or 164x124. However, trying to guess the real values with the help of the PixArt Sensor data sheets did not work out satisfactory so I decided to fix the pixel size and resolution on the above values and estimate the focal length. Even thought the values are not correct, they only need to be correct relative to each other to make the pose estimation work. The principal point is the actual origin of the image plane. Ideally this value should be measured, here I just assume it is the middle of the sensor chip.

Overview of the tracking algorithm

The overall tracking algorithm can be divided in the following steps:

Retrieve the image point values of the infrared LEDs from the Wiimote.
Assign the image points to the LED beacon lights.
Run the pose estimation to calculate the rotation and translation of the LED beacon.
Filter the resulting rotation and translation values.
Build a transformation matrix and transform the result according to configuration file.
(Wiimote position and orientation and local LED beacon transformation)

In the following I will describe the steps in more detail and with code examples.

Retrieve image points

The connection and data retrieval with the Wiimote is done in the WiiMoteTracker class. This class implements the interface IMarkerTracker, that defines the interface for an optical marker based tracker. The Wiimote is initialized in the Initialize() function and connected when StartTracking() is called :

public void Initialize()
{
    // test static variable for first time call
    if (m_wiimoteCount == 0)
        m_wiimoteCollection.FindAllWiimotes();

    if (m_wiimoteCollection.Count <= m_wiimoteCount)
    {
        ErrorHandler.Report("Invalid WiimoteTracker count, only "
        + m_wiimoteCollection.Count.ToString() + " Wiimotes found");
        return;
    }

    wm = m_wiimoteCollection.ElementAt(m_wiimoteCount);
    m_wiimoteId = m_wiimoteCount;
    m_wiimoteCount++;

    // setup the event to handle state changes
    wm.WiimoteChanged += wm_WiimoteChanged;

    // setup the event to handle insertion/removal of extensions
    wm.WiimoteExtensionChanged += wm_WiimoteExtensionChanged;

    // create filter for accelerator values
    AverageFilterDesc filterDesc = new AverageFilterDesc();
    filterDesc.numOfValues = 1000;

    for (int i = 0; i < 3; i++)
    {
        m_acceleratorFilter[i] = new AverageFilter();
        m_acceleratorFilter[i].SetFilterDesc(filterDesc);
    }

    // create filter for image points
    filterDesc = new AverageFilterDesc();
    filterDesc.numOfValues = 5;

    for (int i = 0; i < 8; i++)
    {
        m_imagePointsFilter[i] = new AverageFilter();
        m_imagePointsFilter[i].SetFilterDesc(filterDesc);
    }
}

public void StartTracking()
{
    if (!m_isTracking)
    {
        m_isTracking = true;

        // connect to the Wiimote
        try
        {
            wm.Connect();

            // set the report type to return the IR sensor and accelerometer data 
            // (buttons always come back)
            wm.SetReportType(InputReport.IRAccel, true);
            wm.SetLEDs(m_wiimoteId);
        }
        catch
        {
            ErrorHandler.Report("Cannot connect to Wiimote");
            m_isTracking = false;
        }
    }
}

To receive the Wiimote data the wm_WiimoteChanged Callback has been registered. This function is then called whenever the Wiimote has updated values. Inside this function the infrared LED values are read and the LED beacon light assignment is done.

For that, first a list with with Vector2 for the image points is created:

// put in list
irList.Clear();

for (int i = 0; i < 4; i++)
    irList.Add(new Vector2((float)
           (ws.IRState.IRSensors[i].Position.X * m_resolution.X) - m_principalPoint.X,
           (float)(ws.IRState.IRSensors[i].Position.Y * m_resolution.Y) - m_principalPoint.Y));

Assign image points

Then the values have to be assigned to the IR-LEDs by putting them into the right order. This is done by a simple geometric pattern recognition. The idea is to have a geometric pattern that is invariant to the projection from 3D to 2D. As can be seen in picture 4, the 3 LEDs of the LED beacon are arranged in more or less a line and the 4^th LED is above of the line. In the 2D image data of the Wiimote the 3 LEDs also form more or less a line. Therefore, the first step in the assignment algorithm is to find the 3 image points which are most close to form a line.The line test is done in the following function:

void TestPoints(Vector2 lineStartPoint, Vector2 lineEndPoint,
                Vector2 onLinePoint, Vector2 freePoint)
{
    float lambda;
    float dist = onLinePoint.DistanceToLine(lineStartPoint, lineEndPoint, out lambda);

    // check if projected point is between line end and start point
    if ((lambda > 0) && (lambda < 1))
    {
        // if distance is short, make this combination the result
        if (dist < m_pointLineDist)
        {
            m_pointLineDist = dist;
            m_lineStartPoint = lineStartPoint;
            m_lineEndPoint = lineEndPoint;
            m_onLinePoint = onLinePoint;
            m_freePoint = freePoint;
        }
    }
}

The function is called with the four image points as input. It assumes the first point to be the line start point and the second point to be the line end point. Then the distance of the third point to the line is calculated. This is done by using a C# 3.0 extension function of Vector2:

public static float DistanceToLine(this Vector2 point,
                                   Vector2 startLinePoint,
                                   Vector2 endLinePoint,
                                   out float lambda)
{
    Vector2 rv = endLinePoint - startLinePoint;
    Vector2 p_ap = point - startLinePoint;
    float dot_rv = Vector2.Dot(rv, rv);
    lambda = Vector2.Dot(p_ap, (rv / dot_rv));
    Vector2 distVec = point - (startLinePoint + lambda * rv);
    return distVec.Length();
}

The line distance test is a typical algorithm as described on: http://mathenexus.zum.de/html/geometrie/abstaende/AbstandPG.htm

It returns the distance to the line and an lambda value, which defines the position of the line projected point. If the projection point is outside the line start and line end point the lambda will be below 0 or greater than 1.

If the line was valid the distance of the third point is compared to the formerly smallest distance, and if it is smaller, then this order of image points will be saved as best solution.

To find the right order of the image points this function has to be called with all possible combinations of the 4 LED image points. In my code I do all the calls explicitly after initializing the minimum distance with float max value:

m_pointLineDist = float.MaxValue;

// write all test cases explicite
// in the end the three point line should be found
TestPoints(irList[0], irList[1], irList[2], irList[3]);
TestPoints(irList[0], irList[1], irList[3], irList[2]);
TestPoints(irList[0], irList[2], irList[1], irList[3]);
TestPoints(irList[0], irList[2], irList[3], irList[1]);
TestPoints(irList[0], irList[3], irList[1], irList[2]);
TestPoints(irList[0], irList[3], irList[2], irList[1]);
TestPoints(irList[1], irList[2], irList[0], irList[3]);
TestPoints(irList[1], irList[2], irList[3], irList[0]);
TestPoints(irList[1], irList[3], irList[0], irList[2]);
TestPoints(irList[1], irList[3], irList[2], irList[0]);
TestPoints(irList[2], irList[3], irList[0], irList[1]);
TestPoints(irList[2], irList[3], irList[1], irList[0]);

Now that we have the right order of the three points that form the line and the 4^th point, it is still necessary to determine the right direction of the line. In our LED beacon the 4^th LED is above the line. If the start and end point of the line would be interchanged then the 4^th LED would be below the line. Mathematically we check if the order of the points is clockwise or counterclockwise:

// only remaing test is check if line start and 
// end point is in right order
// check start and end line point with free point
// if clockwise direction then ok, if counterclockwise then exchange start 
// and end point

Vector2 E1 = m_lineStartPoint - m_freePoint; // P1-P2 
Vector2 E2 = m_lineEndPoint - m_freePoint; // P3-P2 
bool clockwise;
if ((E1.X * E2.Y - E1.Y * E2.X) >= 0)
    clockwise = true;
else
    clockwise = false;

if (!clockwise)
{
    Vector2 tmp = m_lineEndPoint;
    m_lineEndPoint = m_lineStartPoint;
    m_lineStartPoint = tmp;
}

The algorithm for the clockwise check is taken from

http://www.geocities.com/siliconvalley/2151/math2d.html

Then the correctly ordered points are slightly filtered with a simple average filter over the last 5 values and multiplied with the pixel size the change from pixel metric to millimeter metric. Finally the points are passed to the pose estimation class.

// now write resulting order to image points
m_imagePoints[0].X = m_imagePointsFilter[0].Filter(m_lineStartPoint.X);
m_imagePoints[0].Y = m_imagePointsFilter[1].Filter(m_lineStartPoint.Y);
m_imagePoints[1].X = m_imagePointsFilter[2].Filter(m_onLinePoint.X);
m_imagePoints[1].Y = m_imagePointsFilter[3].Filter(m_onLinePoint.Y);
m_imagePoints[2].X = m_imagePointsFilter[4].Filter(m_lineEndPoint.X);
m_imagePoints[2].Y = m_imagePointsFilter[5].Filter(m_lineEndPoint.Y);
m_imagePoints[3].X = m_imagePointsFilter[6].Filter(m_freePoint.X);
m_imagePoints[3].Y = m_imagePointsFilter[7].Filter(m_freePoint.Y);

for (int i = 0; i < 4; i++)
{
    m_imagePoints[i] *= m_pixelSize;
}

// send points to estimation
m_poseEstimate.UpdateImagePoints(m_imagePoints);

Run pose estimation

The pose estimation is done by the class Posit. This class implements the IPoseEstimate interface:

public interface IPoseEstimate
{
    void InitializeCameraParameter(double focalLengthMM, bool flipImage,
                        float scale, int[] assignAxis, int[] assignAxisSign);
    void InitializeMarkerBody(Vector3[] markerPoints);
    void UpdateImagePoints(Vector2[] imagePoints);
    void GetTransform(out Vector3 position, out Vector3 rotation);
    void StartEstimation();
    void StopEstimation();
}

The pose estimation has to be initialized with the focal length of the tracking camera. The 3D positions of the real device – in our case the LEDs of the LED beacon – are passed in the InitializateMarkerBody function. The measured image points are passed with the UpdateImagePoints call and the calculated result can be taken from the GetTransform function. Because the pose estimation itself runs in an own thread asynchronously it has to be started and stopped by StartEstimation and StopEstimation. By using the interface it is easy to plug in different pose estimation algorithms.

As mentioned before the pose estimation algorithm used here is the PosIt algorithm published by D. DeMenthon. I use the implementation from the OpenCV computer vision library. As this library is C code it has to wrapped to manage code. I use the freely available wrapper EmguCV. Before the pose estimation can be done a pose estimation object has to be created. This is done when the 3D positions of the markers are passed:

public void InitializeMarkerBody(Vector3[] markerPoints)
{
    m_numOfMarker = markerPoints.Length;
    MCvPoint3D32f[] worldMarker = new MCvPoint3D32f[m_numOfMarker];
    for (int i = 0; i < m_numOfMarker; i++)
    {
        worldMarker[i].x = markerPoints[i].X;
        worldMarker[i].y = markerPoints[i].Y;
        worldMarker[i].z = markerPoints[i].Z;
    }
    m_positObject = CvInvoke.cvCreatePOSITObject(
    worldMarker, m_numOfMarker);
    m_imagePoints = new MCvPoint2D32f[m_numOfMarker];
    m_imagePointsBuffer = new Vector2[m_numOfMarker];
}

MCvPoint32f is a managed structure for the OpenCV CvPoint32f and similar to a Vector3f. The CvInvoke class of the EmguCV wrapper is a collection of static functions to invoke the original OpenCV functions. Because the pose estimation algorithm was not included in the class I had to insert the following functions:

/// <summary>
/// Create pose estimation object
/// </summary>
[DllImport(CV_LIBRARY)]
public static extern IntPtr cvCreatePOSITObject(MCvPoint3D32f[] points, 
                                                int point_count);
/// <summary>
/// Do pose estimation
/// </summary>
[DllImport(CV_LIBRARY)]
public static extern void cvPOSIT(IntPtr posit_object, 
                                  MCvPoint2D32f[] image_points,
                                  double focal_length,
                                  MCvTermCriteria criteria,
                                  float[] rotation_matrix,
                                  float[] translation_vector);
/// <summary>
/// Release pose esitmation object
/// </summary>
[DllImport(CV_LIBRARY)]
public static extern void cvReleasePOSITObject(IntPtr posit_object);

The returned object of the CvInvoke.cvCreatePOSITObject function call is a simple IntPtr and is used later for the pose estimation function.

The pose estimation itself is done in an own thread in the PoseEstimate() function. First the new image points are fetched. If no update is available we wait for new values. This is done with the Monitor.Wait and Monitor.Pulse mechanism:

// copy image points
Monitor.Enter(m_imagePointsLock);
if (!m_imagePointsUpdate)
    Monitor.Wait(m_imagePointsLock);

for (int i = 0; i < m_numOfMarker; i++)
{
    m_imagePoints[i].x = m_imagePointsBuffer[i].X;
    m_imagePoints[i].y = m_imagePointsBuffer[i].Y;
}
m_imagePointsUpdate = false;
Monitor.Exit(m_imagePointsLock);

After getting the new image values the cvPOSIT function is invoked:

MCvTermCriteria criteria;
criteria.type = TERMCRIT.CV_TERMCRIT_EPS | TERMCRIT.CV_TERMCRIT_ITER;
criteria.epsilon = 0.00001;
criteria.max_iter = 500;
CvInvoke.cvPOSIT(m_positObject,m_imagePoints, m_focalLengthMM,
                 criteria, POSITRot, POSITTrans);

Because the algorithm is an iterative algorithm the MCvTermCriteria defines when the algorithm should terminate. Here I defined that it should either terminate when either 500 iteration steps have been reached or when the difference of the values from the former iteration are smaller than 0.00001. You can play around with these value to see how the tracking accuracy reacts. Besides the termination criteria you have to pass the cvPOSIT function the IntPtr to the posit object, the image points and the camera focal length in millimeter. As result you get a 9 float array for the rotation matrix and a 3 float array for the translation result.

Because the rotation values should later be filtered they are translated to euler angels in the EulerAngles function. Euler angles define the rotation by giving the rotation about each coordinate axis. Before storing the final values there is some axis swapping and scaling according to the settings in the tracker.xml.

Filter the estimation results

Because the resolution of the Wiimote camera is not very hight and optical tracking always to some degree noisy the transformation results have quite strong jitters. To reduce the jittering the result values have to be filtered. As a side effect of strong filtering in tracking the virtual object seems not to follow the tracked objects movements directly and feels like swimming behind. A good compromise between jitter reduction and direct response is using Kalman filters. Kalman filters use a mathematical model to predict the change of the values and then uses the measured data to correct its prediction. A good introduction to Kalman Filters is the Siggraph 2001 Course from Greg Welch – see References. Anyhow, determine the best parameters for the filter for non-mathematicians is difficult. A good reference how to apply the filter in the tracking domain is the dissertation of Ronald Azuma “Predictive Tracking for Augmented Reality”. Please refer to that document if you want to learn the meaning of the parameters. For the Wiimote VR-Desktop the Kalman parameters are defined in the tracking.xml file:

<Kalman class="KalmanFilter" id="4" A="1, 0.005, 0, 1" 
    measurement_noise_cov="1.0" process_noise_cov_1="0.0000001"
    process_noise_cov_2="0.0000001"/>

The Kalman implementation used is again part of the OpenCV library. The EmguCV wrapper comes with a complete wrapper for this function. In my implementation there is an interface for data filers, the IDataFilter. The KalmanFilter class implements this interface. Beside the Kalman filter there is also a simple AvarageFilter in the library. The initialization of the Kalman filter is like this:

public void SetFilterDesc(DataFilterDesc desc)
{
    m_kalman = new Kalman(2, 1, 0);
    filterDesc = (KalmanFilterDesc)desc;
    // set A - second parameter is frames per second
    m_kalman.TransitionMatrix.Data.SetValue(filterDesc.A[0], 0, 0);
    m_kalman.TransitionMatrix.Data.SetValue(filterDesc.A[1], 0, 1);
    m_kalman.TransitionMatrix.Data.SetValue(filterDesc.A[2], 1, 0);
    m_kalman.TransitionMatrix.Data.SetValue(filterDesc.A[3], 1, 1);
    // set H
    m_kalman.MeasurementMatrix.Data.SetValue(1.0f, 0, 0);
    m_kalman.MeasurementMatrix.Data.SetValue(0.0f, 0, 1);
    // set Q
    CvInvoke.cvSetIdentity(m_kalman.ProcessNoiseCovariance.Ptr, new MCvScalar(1));
    m_kalman.ProcessNoiseCovariance.Data.SetValue(filterDesc.process_noise_cov_1, 0, 0);
    m_kalman.ProcessNoiseCovariance.Data.SetValue(filterDesc.process_noise_cov_2, 1, 0);
    // set R
    CvInvoke.cvSetIdentity(m_kalman.MeasurementNoiseCovariance.Ptr, new MCvScalar(1e-5));
    m_kalman.MeasurementNoiseCovariance.Data.SetValue(filterDesc.measurement_noise_cov, 0, 0);
    CvInvoke.cvSetIdentity(m_kalman.ErrorCovariancePost.Ptr, new 
    MCvScalar(500));
    m_kalman.ErrorCovariancePost.Data.SetValue(2, 0, 0);
}

After initializing a float value can be simply filter in the Filter function:

public float Filter(float inData)
{
    // Z measurement
    data.Data[0,0] = inData;
    m_kalman.Predict(predict);
    m_kalman.Correct(data);
    return m_kalman.CorrectedState[0, 0];
}

Because in the resulting transformation of the pose estimation there are 3 float values for the translation and 3 float values for the rotation altogether 6 separate instances of the filter are needed. In the Tvrx library the filtering is done inside the TrackedDevice class, which is a parent class for tracked devices and is derived to the TrackedWiimote:

public virtual void Filter() 
{
    m_rawTranslation.X = m_translationFilter[0].Filter(m_rawTranslation.X);
    m_rawTranslation.Y = m_translationFilter[1].Filter(m_rawTranslation.Y);
    m_rawTranslation.Z = m_translationFilter[2].Filter(m_rawTranslation.Z);
    m_rawRotation.X = m_rotationFilter[0].Filter(m_rawRotation.X);
    m_rawRotation.Y = m_rotationFilter[1].Filter(m_rawRotation.Y);
    m_rawRotation.Z = m_rotationFilter[2].Filter(m_rawRotation.Z);
}

Final transformations

Also in the TrackedDevice class the transformation of the tracking values from the camera world space coordinate system to the actual in game virtual space coordinate system takes place:

public virtual void TransformToVirtualSpace() 
{
    Matrix bodyTransformMatrix =
    Matrix.CreateFromYawPitchRoll(m_rawRotation.Y, m_rawRotation.X, m_rawRotation.Z)
                                  * Matrix.CreateTranslation(m_rawTranslation);
    Matrix result = m_TrackerWorldTransform * m_DeviceWorldTransform;
    result = bodyTransformMatrix * result;
    result = m_DeviceLocalTransform * result;
    Vector3 scale;
    result.Decompose(out scale, out m_Rotation, out m_Translation);
}

First a transformation matrix of the Euler angels and the translation vector is composed to the Body-Transform. From the tracking.xml configuration file we also have matrices for Tracker-World-Transform, Device-World-Transform and Device-Local-Transform.

In addition to the Tracker-World-Transform translation from the tracker.xml I calculate the rotation angles of the Wiimotes using the acceleration sensors. By that it is possible to rotate the Wiimotes around the x and z axis to better focus the area you want to track and still get correct tracking results automatically.

Finally, to get the correct final transformation the matrices have to be multiplied in the correct order:

Device-Local-Transform * Body-Transform * Tracker-World-Transform * Device-World-Transform

Now the pose estimation transformation result is ready to be read by the TrackerManager.

Conclusion and Limitation

As I have shown it is possible to create a low-cost Desktop Virtual Reality Setup using two Wiimotes and anaglyph stereo glasses. But because of the low resolution of the Wiimote camera the quality is not comparable to professional monocular tracking systems. However, the quality of the presented system could still be improved by correctly measuring the intrinsic parameters of the Wiimote camera. There are various known algorithms how to measure these parameters by a set of sample data of an object with regular known geometry. For standard cameras usually a checkerboard pattern is used. An algorithm for that purpose is also integrated in OpenCV, so applying it to the Wiimote shouldn't be to hard.

Another issue with intrinsic parameters that I completely ignored so far is the lens distortion. All camera lenses do distort the image to a certain degree. By assuming a circular distortion the algorithms that measure the intrinsic parameters also calculate distortion parameters. If those parameters would be measured the Wiimote image points could be easily undistorted and the pose estimation results would be improved.

References

D. DeMenthon and L.S. Davis, "Model-Based Object Pose in 25 Lines of Code", International Journal of Computer Vision, 15, pp. 123-141, June 1995.
Paper about the used pose estimation algorithm by Daniel DeMenthon. You can find this brilliant paper on his homepage.
OpenCV: Open Source computer vision library. Includes implementation of the pose estimation algorithm and Kalman filter.
EmguCV: C# wrapper for OpenCV
WiimoteLib: Managed Library for Nintendo's Wiimote by Brian Peek
XNAnimation: Very nice library for animations in XNA. I didn't use it for the actual Wiimote tracking, but used his animation demo application in the demonstration video.
XNA: Game development library in C# that I use as base for my applications.
An Introduction to the Kalman Filter. Greg Welch and Gary Bishop. Siggraph 2001 Course 8.
Predictive Tracking for Augmented Reality. Ronald Tadao Azuma. Dissertation. University of North Carolina. February 1995.

Controlling Your Festive Lights with the .NET Micro Framework

Coding4Fun — Fri, 28 Nov 2008 07:58:46 GMT

Author:	Rob Miles: www.robmiles.com
Download:	http://coding4fun.net/source/festivelights1.0.zip
Software:	Visual Studio 2008 Express Edition or better, .NET Micro Framework 3.0
Hardware:	Digi Connect-ME GHI Electronics Embedded Master Devices Solutions Tahoe II
Time Required:	3 hours
Cost:	30 dollars for lights plus the .NET Micro Framework device

The Micro Framework is one of the newest kids on the .NET block, but it does something really rather wonderful. It brings embedded development within the reach of any C# programmer. If you know C# and love Visual Studio, you can now get started building hardware and controlling it with your software. Moreover, it lets developers achieve one of their most cherished dreams, to control their festive lights using programs that they have written. This project shows you how to do just that and adds an extra magical feature, in that you can make all your festive lights flash red whenever I, Rob Miles, make a new post on that most famous of blogs, www.robmiles.com.

Actually, you can modify the code so that you can make your lights do most anything in response to an event that happens on the web. You could signal home that you are running late, send the weather forecast to your Christmas tree or explore any number of communication options. You might decide that this is so useful that you leave your decorations up all year round.

If you just want to play with the .NET Micro Framework and get a feel for how easy it is to create software for tiny devices you don't actually need to use any extra hardware at all. The project comes with a complete emulation of the lights display so that you can run the whole thing on your computer and learn how hardware and software can be made to work together without burning your fingers with a soldering iron.

However building the hardware will give you an understanding of how some simple electronic components can be controlled from C# and even how serial and parallel data transfer works. It is also great fun.

To get started you will need some hardware and some software. Let's take each in turn.

Hardware

Processor Hardware

The .NET Micro Framework lets you run C# programs on tiny embedded devices. There are a number of these available today, and they are getting progressively cheaper. You can base this project on any.NET Micro Framework device that has a network port and three or more output ports. The ones I'd recommend are:

Digi Connect-ME: http://www.digi.com/products/embeddedsolutions/digiconnectme.jsp
GHI Electronics Embedded Master: http://www.ghielectronics.com/embeddedmaster.php
Devices Solutions Tahoe II: http://devicesolutions.net/Products/Tahoe.aspx

I built the project for Digi Connect board but the code can be customised for any of the above platforms. The beauty of the .NET Micro Framework is that you hardly need to change your program even if the underlying hardware is completely replaced. The only thing you will have to do is adjust the code to target different output pins. I'll flag this part of the program up when we get to it.

The project as supplied runs on a special emulator that runs on the PC and behaves like a .NET Micro Framework device with lights connected, so you can get started exploring the code right away.

Lights

I'm keeping things deliberately low voltage for this project. This means that kids of all ages can have a go at building the hardware without messing around with mains. The lights that I used were supplied as lines of twenty leds wired up as four strands of five leds each. All the leds in each strand were the same colour. The lights were fitted with a little battery box which held three AA batteries and a tiny controller. To get my display I simply removed the battery box and connected the strands to my hardware.

Figure 1: My Battery Powered Lights

You can do the same with any low power lighting kit that you wish to use. The drivers that I'm using can handle around 500 milliamps of continuous current and so you can connect a fair number of lights to each strand.

Buying the Lights

The lights I used can be bought in the UK from Lights4Fun: www.lights4fun.co.uk and are called "C-LED-4.5-M 20 Multi Coloured Battery Operated LED Fairy Lights". They are supplied with a battery box and controller that you can remove to connect to the Darlington drivers. I used an old 5 volt mobile phone charger to power the lights. If you search eBay for "led christmas lights battery" you should find plenty of suppliers.

Driver Hardware

We can't connect a .NET Micro Framework device directly to our lights. There are two reasons for this:

The Micro Framework device will not be able to switch the amount of current that we need to drive the lights themselves.
The Micro Framework device will not have enough outputs to control all the lights that we want to use.

This means that we need to make some hardware that goes between the processor and our lights. I actually don't see this as a problem. Making hardware is great fun, seeing something that you've built spring into life is very nice. The hardware we are going to use will provide a means by which we can control many hundreds of lights from a single Micro Framework board if we wish. It also has the ability to switch reasonable amounts of current, so you can use quite large numbers of lights if you wish. We are going to use two semiconductor components, a CD4094 Shift Register/Latch and a ULN2803 Octal Darlington Driver. A pair of these chips will allow us to control 8 outputs. If you want to control more outputs you simply get more chips and chain them together. For my version of the project I just used one of each chip, you can use as many as you like.The shift registers and Darlington amplifiers can be obtained in the UK from Maplin: www.maplin.co.uk. The chip numbers and part numbers as are as follows: QW54J 4094 Shift Register, QY79L ULN2803A Darlington Driver. In the US you can obtain the components and breadboard from Digi-Key: www.digikey.com.

Serial and Parallel Data

You might be wondering how we can use just three output lines to control lots of lights. We are going to do this by using the three output lines to provide a serial data stream which is converted by our hardware into parallel data that can be used to control our lights. This is a fundamental principle of digital electronics and is how, amongst many other things, computer networks transfer data.

We are going to use three signals which are called clock, data and latch. Each of these can be set high (a voltage is present) or low (no voltage is present) by the .NET Micro Framework device under the control of our software. The signals are connected to the clock, data and latch inputs of our CD4094 shift register so that the program can talk to it.

The clock line triggers the shift register to do two things:

Shift all the bits along to make room.
Sample the value of the data input and store this value in the space that was created.

To get a feel for how this works, take a look at Figure 2 below. It shows a shift register with a pattern of bits in it. The pattern is 01100001. Note that although this represents a number; it can also be regarded as a pattern of 0s and 1s in the shift register itself. The value 0 means 0 volts and the value 1 means some volts. These are the signals that will be used to control our lights. I'm going to call them 0 and 1 from now on. The Shift Register has Clock, Data and Latch signals connected and they are all set to 1. We can ignore the latch part of the chip for now.

Figure 2: A shift register and latch with some data in it

When the clock signal changes from 1 to 0 this causes the shift register to perform the two steps described above. First the data is shifted along to the right. Note that this means that there is an "empty" location at the start of the register, and that the right most bit in the register "falls off" the register and disappears. Figure 3 shows how this works.

Figure 3: Shifting along the values in the shift register

Once the shift has finished the shift register can copy the data signal into the empty bit at the left as shown in Figure 4 below.

Figure 4: Storing the new data bit

By repeating this process 8 times a program can load a new pattern of 8 bits into the shift register. Then it is time to latch the new value to control the lights. This is the point at which the lights will appear to change. When the latch value is changed from 0 to 1 this causes the chip to copy the value in the Shift Register into the latch, as shown in Figure 5 below.

Figure 5: Copying the shift register pattern into the latch.

The latch is required so that the lights will not flicker as the new patterns are shifted into position. Each of the bits in the latch is connected to an output pin on the CD4094 which is used to switch a particular light colour on or off.

We need to create some C# that will provide the appropriate sequence of signals. It turns out that using the .NET Micro Framework to achieve this is actually very easy. The displayByte method below sends an 8 bit value into a shift register and then latches it into the output. If you read through the code you can see how the clock, data and latch values are all set to true (high) or false (low) to first clock the data out and then trigger the latch to display the pattern on the lights. The input is an 8 bit byte value and the program uses a mask to pick out the value of each bit in turn and set the data output accordingly.

private static void displayByte(byte value)
{
    latchPort.Write(false);
    clockPort.Write(false);

    byte mask = 1;

    for (int i = 0; i < 8; i++)
    {
        if ((value & mask) > 0)
        {
            dataPort.Write(true);
        }
        else
        {
            dataPort.Write(false);
        }
        clockPort.Write(true);
        clockPort.Write(false);
        mask <<= 1;
    }
    latchPort.Write(true);
    latchPort.Write(false);
}

The dataPort, clockPort and latchPort variables are instances of the .NET Micro Framework class OutputPort, which provides a method called Write which can be used to control the state of the output signal. We will consider how these are created a little later in the article.

The sample code for this project comes with a software emulator of the CD4094 which shows how it works. Figure 6 below shows that a new pattern is in the process of being shifted into the shift register, while the lights retain the previous one in the latch. The Clock and Data signals are high and the next statement will drop the Clock signal to add the next bit into the new pattern.

Figure 6: The light emulator

This figure also shows that I have two bits in the pattern controlling a strand of each colour. You can single step through the displayByte method above and watch it perform this output.

You can use this technique every time you want to control a large number of outputs using a small number of output pins. The CD4094 has "daisy chain" inputs and outputs so that the devices can be connected in sequence. If I used two devices I could control 16 bits, with three I could control 24 and so on. This would require only minimal changes to the software.

Output Driver

The CD4094 device will produce a signal output, but it is not really powerful enough to drive things like lights. To do this we need an amplifier and the ULN2803 Octal Darlington Driver is perfect for this. It is packaged as a single chip which contains 8 pairs of transistors. Each transistor pair is wired in a "Darlington" configuration and can be used as a switch which is controlled by on output from the CD4094. When the transistors are turned on they allow current to pass through them and this will cause the lamps to light. The lights that I bought used a "pull down" arrangement to make them light up. All of the light emitting diode (LED) lamps had one end wired to a common line that was connected to the positive supply. To make the a chain of LEDs light the other end that controls that chain needed to be pulled down to the ground level. This is a common arrangement with lights like these. The ULN2803 driver has the transistors wired in an arrangement that allows it to pull signals low in this way. Figure 7 shows how this arrangement works. The resistor shown is actually wired into each LED in the set of lights that I used.

Figure 7: Lighting the LEDs

Complete Circuit

The complete circuit shown in Figure 8 simply links each output of the CD4094 shift register to an input on the ULN2803 Darlington Driver. Note that there is no reason to link any particular bit with any other, the diagram shown is one which will translate most easily to a prototype "breadboard".

Figure 8: The Complete Circuit

The diagram just shows four lights connected to OUT1 from pin 18 of the ULN2803, the other 7 channels are connected in exactly the same way. Some of the pins can be left unconnected. Pins 9 and 10 of the CD4094 are only required if you are connecting multiple shift registers together and the common connection on pin 10 of the ULN2803 is not required. The Latch, Data and Clock signals on the CD4094 are connected to the output signals from the Micro Framework device.

Figure 9 shows the completed circuit built up on a prototype breadboard. The chip on the left is the CD4094 and the one on the right is the ULN2803.

Figure 9: The Completed Circuit

The red wires carry the positive voltage from the power supply, which is connected to the top left of the circuit. The green wires are ground. Other coloured wires are used for signals. The Clock, Latch and Data signals are brought out to a connector which will fit a Digi-ME prototyping board.

Selecting Lights

Each bit in the value sent to the displayByte method and then into the shift register will be mapped to a particular chain of lights. The mapping of these is not particularly important, since you can use program constants to represent particular values. I wired the red signals to output pins 4 and 8. To light up just the red lights I used the value 0x88 which is the appropriate bit pattern. I then set up constants for all the other colors:

const byte GREEN = 0x11;
const byte BLUE = 0x22;
const byte YELLOW = 0x44;
const byte RED = 0x88;

Setting Up the Hardware

The .NET Micro Framework provides a set of classes that can be used to represent the hardware in a system. The program uses instances of the OutputPort class to represent the output pins. These are created in the method that sets up the hardware:

const Cpu.Pin clockPin = Cpu.Pin.GPIO_Pin0;
const Cpu.Pin dataPin = Cpu.Pin.GPIO_Pin1;
const Cpu.Pin latchPin = Cpu.Pin.GPIO_Pin2;

static OutputPort clockPort;
static OutputPort dataPort;
static OutputPort latchPort;

private static void setupOutputs()
{
    clockPort = new OutputPort(clockPin, false);
    dataPort = new OutputPort(dataPin, false);
    latchPort = new OutputPort(latchPin, false);
}

In this version of the hardware I have connected pin 0 of the processor to the clock, pin 1 to the data and pin 2 to the latch. If you use different pins you can change the settings above.

Software

Now that we have working hardware we can consider how the software is to work. Note that this version of the program works correctly but lacks exception handlers that would make it truly robust. I've left these out to simplify the explanation. To make sense of this description you will need to have a copy of the program itself available for reference.

You can develop the code using Visual Studio 2008 Express edition, which you can download from http://www.microsoft.com/express/

The software is written for the .NET Micro Framework 3.0 which you can download from http://www.microsoft.com/downloads/details.aspx?FamilyID=9356ed6f-f1f0-43ef-b21a-4644dd089b4a&displaylang=en

Reading the Blog

The program reads the RSS feed from a blog and looks for the <pubDate> value. This contains the date when the feed was last updated. Whenever this date changes the lights must flash red for a few seconds before resuming a random display. Users of the full .NET Framework can use the HTTPRequest class to build a GET command to be sent to a server. Unfortunately the .NET Micro Framework does not support this, so we have to access the web feed using socket based communication. This part of the program is heavily based on the SocketClient example supplied with the .NET Micro Framework.

// This method requests a page from the specified server.
private static String GetWebPage(String server, string webPage)
{
    const Int32 c_httpPort = 80;
    const Int32 c_microsecondsPerSecond = 1000000;

    // Create a socket connection to the specified server and port.
    using (Socket serverSocket = ConnectSocket(server, c_httpPort))
    {
        // Send request to the server.
        String request = "GET "+ webPage + 
             " HTTP/1.1\r\nHost: " + server + 
             "\r\nConnection: Close\r\n\r\n";
        Byte[] bytesToSend = Encoding.UTF8.GetBytes(request);
        serverSocket.Send(bytesToSend, bytesToSend.Length, 0);

        // Allocate a buffer to receive HTML chunks
        Byte[] buffer = new Byte[1024];

        // 'page' refers to the HTML data as it is built up.
        String page = String.Empty; 

        // Wait up to 30 seconds for initial data 
        // Will throw exception if connection closed
        DateTime timeoutAt = DateTime.Now.AddSeconds(30);
        while (serverSocket.Available == 0 && 
               DateTime.Now < timeoutAt)
        {
            System.Threading.Thread.Sleep(100);
        }

        // Poll for data until 30 second time out
        // Returns true for data and connection closed
        while (serverSocket.Poll(30 * c_microsecondsPerSecond,
                                 SelectMode.SelectRead))
        {
            // Zero all bytes in the re-usable buffer
            Array.Clear(buffer, 0, buffer.Length);

            // Read a buffer-sized HTML chunk
            Int32 bytesRead = serverSocket.Receive(buffer);

            // If 0 bytes in buffer, then connection is closed, 
            // or we have timed out
            if (bytesRead == 0)
                break;

            // Append the chunk to the string
            page += new String(Encoding.UTF8.GetChars(buffer));
        }

        return page;   // Return the complete string
    }
}

This method is called to fetch the journal RSS feed from my blog:

string address = "www.robmiles.com";
string name = "/journal/rss.xml";

string html = GetWebPage(address, name);

You can use it to download from any RSS feed or page on the web. It will throw an exception if the page cannot be read. This version of my program does not perform exception handling however.

Getting the Published Date

The date is held in the form:

<pubDate>Wed, 05 Nov 2008 22:38:52 +0000</pubDate>

It would be enough just to record this content and check for changes in the text, but I decided that I might want to use the date information in a later version of the program and so I created some small helper methods to read numbers from the input string and a larger method to read the publish date itself:

private static DateTime getRecentPubDate(string html, string startTag)
{
    int index = html.IndexOf(startTag) ;

    if (index < 0) 
        throw new Exception("Missing tag " + startTag);

    index += startTag.Length;

    // spin past the name of the day
    while (index < html.Length && html[index] != ',') index++;
    if (index == html.Length) 
        throw new Exception("Short publish date");

    int dayValue = getInt(html, ref index, ' ');

    string monthName = getString(html, ref index, ' ');
    int monthValue = getMonth(monthName);

    int yearValue = getInt(html, ref index, ' ');
    int hourValue = getInt(html, ref index, ':');
    int minuteValue = getInt(html, ref index, ':');
    int secondValue = getInt(html, ref index, ' ');

    return new DateTime(yearValue, monthValue, dayValue, 
                        hourValue, minuteValue, secondValue);
}

The main body of the program uses this method to extract the date out of the RSS feed. When a new date is found it is time to update the lights.

Lights and Threads

My first version of the program flashed the lights for a while and then checked to see if a new blog post had been made. This worked OK, but the process of loading the RSS feed from the server and checking the dates can take a few seconds, which meant that the flashing lights would freeze every now and then. This did not look very good, and so I decided to use two threads instead. One is in charge of flashing the lights in a random pattern and the other loads the RSS feed from my blog and checks the date of the most recent publication.

Note that the threading I am using is exactly the same as threading in the full .NET Framework.

Thread Communication

The two threads communicate by means of a single boolean variable which is set to true when the blog alert is to take place. The display thread reads this flag and flashes the lights red if it is time to alert.

public static bool alert = false;

public static void flasher()
{
    setupOutputs();

    while (true)
    {
        randomDisplay(400, 10);
        if (alert)
        {
            alert = false;
            flashRed(600, 20);
        }
    }
}

The methods randomDisplay and flashRed do exactly what you would expect. Each of them gets two numbers to control the lights. The first number gives the delay in milliseconds between flashes (a few hundred milliseconds giving best results). The second gives the number of times that the lights should be flashed before the method finishes. RandomDisplay displays random colors, whereas flashRed is simply flashes the red lights. The alert flag is cleared before the red lights are flashed so that the system will resume normal display after the alert.

The second thread in the system performs the blog download and date test behaviour. This all happens inside the Main method:

public static void Main()
{
    flashThread = new System.Threading.Thread(flasher);

    flashThread.Start();

    string address = "www.robmiles.com";
    string name = "/journal/rss.xml";

    String html = GetWebPage(address, name);

    DateTime lastUpdate = getRecentPubDate(html, "<pubDate>");

    Debug.Print("Initial Update Value : " + lastUpdate.ToString());

    while (true)
    {
        System.Threading.Thread.Sleep(10000);

        html = GetWebPage(address, name);

        DateTime blogUpdate = getRecentPubDate(html, "<pubDate>");

        if (!blogUpdate.Equals(lastUpdate))
        {
            Debug.Print("Updated at : " + blogUpdate.ToString());
            lastUpdate = blogUpdate;
            alert = true;
        }
    }
}

To reduce the load on the network the system only checks the blog feed every 10 seconds.

Future Work

The program itself works fine, but the error handling is not wonderful. Although some of the methods used throw exceptions these are not caught anywhere, causing the blog reading thread to terminate. This does not stop the lights flashing, but it does mean that there will be no more alerts. However, it is not too hard to create a version which contains proper error handling and even flashes the lights different colors to indicate the alarm conditions. I have created a version that flashes the lights yellow every now and then if the network connection fails. There is also considerable scope for reading other web based sources and changing the output accordingly. Feel free to do all these things and make sure that you have fun.

Thanks

Thanks go to Ian Mitchell of Ormston Technology (http://www.ormtec.co.uk/) for doing such a great job of hardware design and build.

WiiEarthVR – A Fully Immersive 3D Experience with Virtual Earth 3D

Coding4Fun — Fri, 14 Nov 2008 11:57:58 GMT

In this article, Brian Peek will demonstrate how to use a Nintendo Wii Remote (Wiimote), a Wii Fit Balance Board, and Vuzix VR920 glasses as input devices for Microsoft Virtual Earth 3D, providing a fully immersive, 3D experience.

Brian Peek

ASPSOFT, Inc.

Difficulty: Intermediate

Time Required: 2-3 hours

Cost: $60 for Wiimote and Nunchuk, $90 for Wii Fit (which includes Balance Board), $400 for Vuzix VR920 glasses

Software: Managed Library for Nintendo's Wiimote, Visual Basic or Visual C# Express Editions

Hardware: Nintendo Wii Remote (Wiimote) with Nunchuk, Wii Fit Balance Board, Vuzix VR920 glasses, a compatible PC Bluetooth adapter and stack

Download: Download

Discussion Forum: Forum

Introduction

Virtual Earth is the 3D interface to Microsoft's Live Maps service. Normally this control is loaded via the web browser and allows interaction with a keyboard, mouse, and Xbox 360 controller. In this article, we will take the Virtual Earth 3D control out of the web browser, use it in a WinForms application, and control it with a Nintendo Wii Remote (Wiimote) and a pair of Vuzix VR920 glasses, while also providing a stereoscopic 3D image to the glasses, creating the illusion of a fully three dimensional environment. Note that use of the Virtual Earth 3D control in this way is undocumented and unsupported at the moment. Because of this, some of the descriptions in this article are educated guesses and may not be 100% accurate…

Originally, this project started as a simple Wiimote interface to Virtual Earth 3D as shown in the video below. Since I wrote that application, I learned of the VR920 glasses and the Wii Fit Balance Board was released, so I’ve decided to create a more immersive experience using all of these controls which was demonstrated at PDC2008, shown here:

Setup

Before we get started, you will need to install the Virtual Earth 3D control. If you haven't done this already, browse to http://maps.live.com/ and click on the 3D link to install the control and supporting software.

Additionally, if you haven't already, please review my Managed Library for Nintendo's Wiimote article on this site. We will be using the library in this article, but I will not repeat the basic information that is located in the original article. You will also need to have the Vuzix VR920 glasses installed and setup according to its own user manual. That will also not be covered here.

Implementation

The Virtual Earth 3D Control

The Virtual Earth 3D (VE3D) control is intended to be used through a well documented JavaScript interface from a web page, however we would not be able to access the Wiimote or VR920 glasses from JavaScript. Therefore, we will be using the VE3D control through its native, but wholly undocumented interface.

Start by creating a new Windows Forms application named WiiEarthVR in C# or VB. As with all controls and 3rd party libraries, a reference needs to be set to the Virtual Earth 3D libraries. Add references to the following items:

Microsoft.MapPoint.Data
Microsoft.MapPoint.Data.VirtualEarthTileDataSource
Microsoft.MapPoint.Geometry
Microsoft.MapPoint.Rendering3D
Microsoft.MapPoint.Rendering3D.Utility

If they do not show up in the .NET references, they can be found by selecting the Browse tab and navigating to C:\Program Files\Virtual Earth 3D\ or C:\Program Files (x86)\Virtual Earth 3D\ . With the references in place, the project file can now be opened and the references will be seen in the References folder in the Solution Explorer as usual.

Creating an instance of the control can be done in code just like any other control. Used in the constructor or load event of the form, the following code will create a VE3D control and add it to the form as fully docked:

     private GlobeControl _globeControl;

    public MainForm()

    {

        InitializeComponent();

        _globeControl = new GlobeControl();

        SuspendLayout();

        _globeControl.Location = new System.Drawing.Point(0, 0);

        _globeControl.Name = "_globeControl";

        _globeControl.Size = ClientSize;

        _globeControl.Anchor = AnchorStyles.Left | AnchorStyles.Right | AnchorStyles.Top | AnchorStyles.Bottom;

        _globeControl.TabIndex = 0;

        _globeControl.SendToBack(); // we want the button to be on top

        pnlGlobe.Controls.Add(_globeControl);

        ResumeLayout(false);

    }

    Private _globeControl As GlobeControl

    Public Sub New()

        InitializeComponent()

        _globeControl = New GlobeControl()

        SuspendLayout()

        _globeControl.Location = New System.Drawing.Point(0, 0)

        _globeControl.Name = "_globeControl"

        _globeControl.Size = ClientSize

        _globeControl.Anchor = AnchorStyles.Left Or AnchorStyles.Right Or AnchorStyles.Top Or AnchorStyles.Bottom

        _globeControl.TabIndex = 0

        _globeControl.SendToBack() ' we want the button to be on top

        pnlGlobe.Controls.Add(_globeControl)

        ResumeLayout(False)

    End Sub

We can start adding items to the VE3D control by listening for the FirstFrameRendered event of the GlobeControl and then setting the appropriate properties. Setting these properties prior to this point can lead to some unexpected results.

In the FirstFrameRendered event handler, if you wish to add the default navigation controls to the screen, the PlugInLoader object is used. The PlugInLoader is created by using the CreateLoader static method, passing in an instance of the GlobeControl's Host object. Then, the NavigationPlugIn can be loaded and activated as shown:

    // load all the spiffy UI navigation goodies

    PlugInLoader loader = PlugInLoader.CreateLoader(this.globeControl.Host);

    loader.LoadPlugIn(typeof(NavigationPlugIn));

    loader.ActivatePlugIn(typeof(NavigationPlugIn).GUID, null);

    ' load all the spiffy UI navigation goodies

    Dim loader As PlugInLoader = PlugInLoader.CreateLoader(Me.globeControl.Host)

    loader.LoadPlugIn(GetType(NavigationPlugIn))

    loader.ActivatePlugIn(GetType(NavigationPlugIn).GUID, Nothing)

The last thing to be added for basic functionality is the data. As it stands, the only data that will appear on the globe is the image of the continents. Zooming in only produces a blurry representation of that base image.

Data layers are created from specially formatted data sources provided by maps.live.com known as content manifests. These are XML files which tell the VE3D control how to load the data required for any view. Content layers can be added by adding them to the DataSources object of the GlobeControl. We can add any of the following layers (note that there may be other content manifests provided by maps.live.com, but these are the only 5 that I am aware of):

URL	DataSourceUsage Type	Description
http://local.live.com/Manifests/HD.xml	ElevationMap	Terrain data
http://local.live.com/Manifests/MO.xml	Model	3D buildings
http://local.live.com/Manifests/AT.xml	TextureMap	Unlabeled aerial
http://local.live.com/Manifests/HT.xml	TextureMap	Labeled aerial
http://local.live.com/Manifests/RT.xml	TextureMap	Roads only

For the best display, add the ElevationMap, Model and Aerial TextureMap layers as shown:

    // set various data sources, here for elevation data, terrain data, and model data.

    _globeControl.Host.DataSources.Add(new DataSourceLayerData("Elevation", "Elevation", @"http://maps.live.com//Manifests/HD.xml", DataSourceUsage.ElevationMap));

    _globeControl.Host.DataSources.Add(new DataSourceLayerData("Texture", "Texture", @"http://maps.live.com//Manifests/AT.xml", DataSourceUsage.TextureMap));

    _globeControl.Host.DataSources.Add(new DataSourceLayerData("Models", "Models", @"http://maps.live.com//Manifests/MO.xml", DataSourceUsage.Model));

    ' set various data sources, here for elevation data, terrain data, and model data.

    _globeControl.Host.DataSources.Add(New DataSourceLayerData("Elevation", "Elevation", "http://maps.live.com//Manifests/HD.xml", DataSourceUsage.ElevationMap))

    _globeControl.Host.DataSources.Add(New DataSourceLayerData("Texture", "Texture", "http://maps.live.com//Manifests/AT.xml", DataSourceUsage.TextureMap))

    _globeControl.Host.DataSources.Add(New DataSourceLayerData("Models", "Models", "http://maps.live.com//Manifests/MO.xml", DataSourceUsage.Model))

By passing the URL of the content manifest, a name for the layer, and what the manifest represents, a new DataSource is created, which is in turn used to create a DataSourceLayerData object which is then given to the VE3D control to consume. This should be done in the FirstFrameRendered event handler as well.

We also need to setup VE3D to turn off any UI elements, turn on the atmosphere effects, and ensure we have a full unobstructed view. Again, in the FirstFrameRendered event handler, we can use the following code to achieve this:

    // turn on the nice atmosphere

    _globeControl.Host.WorldEngine.Environment.AtmosphereDisplay = Microsoft.MapPoint.Rendering3D.Atmospherics.EnvironmentManager.AtmosphereStyle.Scattering;

    // default to all off

    _globeControl.Host.WorldEngine.Display3DCursor = false;

    _globeControl.Host.WorldEngine.SetWindowsCursor(null);

    _globeControl.Host.WorldEngine.ShowNavigationControl = false;

    _globeControl.Host.WorldEngine.ShowCursorLocationInformation = false;

    _globeControl.Host.WorldEngine.ShowScale = false;

    _globeControl.Host.WorldEngine.ShowUI = false;

    _globeControl.Host.WorldEngine.Environment.SunPosition = null;

    _globeControl.Host.WorldEngine.Environment.LocalWeatherEnabled = true;

    _globeControl.Host.WorldEngine.BaseCopyrightText = " "; // workaround for a display issue

    ' turn on the nice atmosphere

    _globeControl.Host.WorldEngine.Environment.AtmosphereDisplay = Microsoft.MapPoint.Rendering3D.Atmospherics.EnvironmentManager.AtmosphereStyle.Scattering

    ' default to all off

    _globeControl.Host.WorldEngine.Display3DCursor = False

    _globeControl.Host.WorldEngine.SetWindowsCursor(Nothing)

    _globeControl.Host.WorldEngine.ShowNavigationControl = False

    _globeControl.Host.WorldEngine.ShowCursorLocationInformation = False

    _globeControl.Host.WorldEngine.ShowScale = False

    _globeControl.Host.WorldEngine.ShowUI = False

    _globeControl.Host.WorldEngine.Environment.SunPosition = Nothing

    _globeControl.Host.WorldEngine.Environment.LocalWeatherEnabled = True

    _globeControl.Host.WorldEngine.BaseCopyrightText = " " ' workaround for a display issue

If you were to run the application at this point, you would see a fully functioning Virtual Earth 3D control with proper data and navigation.

Control Scheme and Bindings

The user will control VE3D with the Wiimote by holding the nunchuk in the left hand, which will move the user forward/back/left/right using the joystick. The C and Z buttons on the front of the nunchuk will be used to raise and lower the altitude of the camera. The Wiimote, held in the right hand, will be used to toggle various things on or off and interact with menus. The user will also stand on the Balance Board which will use their center of gravity to turn them in the environment.

VE3D bindings allow you to change or create new control schemes for VE3D by placing an XML file in a specific directory as follows:

Vista: C:\Users\<username>\AppData\LocalLow\Microsoft\Virtual Earth 3D
XP: C:\Documents and Settings\<username>\Local Settings\Microsoft\Virtual Earth 3D

In this directory you will find a Bindings.xml file. This XML schema defines the default keyboard, mouse, Gamepad and other input device properties. Open the file to see the schema used to define events and parameters.

By default, VE3D will load any file named Bindings*.xml from this directory. For the Wiimote control scheme, create a new file named BindingsWiiEarthVR.xml in this directory. Set the contents of the file to the following:

    <?xml version="1.0" encoding="utf-8" ?>

    <Bindings>

        <BindingSet Name="WiiEarthVRBindings" AutoUse="True" Cursor="Drag">

            <!-- Nunchuk joystick -->

            <Bind Event="Wiimote.NunchukX" Factor="0.5"><Action Name="Strafe"/></Bind>

            <Bind Event="Wiimote.NunchukY" Factor="1"><Action Name="Move"/></Bind>

            <!-- Nunchuk buttons -->

            <Bind Event="Wiimote.NunchukC" Factor="0.20"><Action Name="Ascend"/></Bind>

            <Bind Event="Wiimote.NunchukZ" Factor="-0.20"><Action Name="Ascend"/></Bind>

            <!-- Balance Board -->

            <Bind Event="Wiimote.BalanceBoardX" Factor="-0.0009"><Action Name="Turn"/></Bind>

            <Bind Event="Wiimote.BalanceBoardY" Factor="0.0009"><Action Name="Ascend"/></Bind>

            <!-- Wiimote buttons -->

            <Bind Event="Wiimote.Home"><Action Name="ResetOnCenter"/></Bind>

            <Bind Event="Wiimote.A" Factor="1"><Action Name="Locations, WiiEarthVR, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/></Bind>

            <Bind Event="Wiimote.Left" Factor="-1"><Action Name="Locations, WiiEarthVR, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/></Bind>

            <Bind Event="Wiimote.Up" Factor="-1"><Action Name="LocationsMove, WiiEarthVR, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/></Bind>

            <Bind Event="Wiimote.Down" Factor="1"><Action Name="LocationsMove, WiiEarthVR, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/></Bind>

            <!-- FPS-style keyboard controls in case we don't have a nunchuk -->

            <Bind Event="Key.W" Factor="22"><Action Name="Move" /></Bind>

            <Bind Event="Key.S" Factor="-22"><Action Name="Move" /></Bind>

            <Bind Event="Key.D" Factor="22"><Action Name="Strafe" /></Bind>

            <Bind Event="Key.A" Factor="-22"><Action Name="Strafe" /></Bind>

            <Bind Event="Key.Space" Factor="20"><Action Name="Ascend" /></Bind>

            <Bind Event="Key.C" Factor="-20"><Action Name="Ascend" /></Bind>

            <!-- Other keys -->

            <Bind Event="Key.F1"><Action Name="VR920SetZero, WiiEarthVR, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/></Bind>

            <Bind Event="Key.F2"><Action Name="BalanceBoardSetZero, WiiEarthVR, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/></Bind>

            <Bind Event="Key.F3"><Action Name="ToggleVR920Stereo, WiiEarthVR, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/></Bind>

            <Bind Event="Wiimote.Minus" Factor="-0.1"><Action Name="VR920SetEyeDistance, WiiEarthVR, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/></Bind>

            <Bind Event="Wiimote.Plus" Factor="0.1"><Action Name="VR920SetEyeDistance, WiiEarthVR, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/></Bind>

            <Bind Event="Wiimote.One"><Action Name="ToggleBalanceBoard, WiiEarthVR, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/></Bind>

            <Bind Event="Wiimote.Two"><Action Name="ToggleVR920, WiiEarthVR, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/></Bind>

            <Bind Event="Key.B"><Action Name="ToggleBalanceBoard, WiiEarthVR, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/></Bind>

            <Bind Event="Key.V"><Action Name="ToggleVR920, WiiEarthVR, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/></Bind>

            <Bind Event="Key.F"><Action Name="FullScreen, WiiEarthVR, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/></Bind>

        </BindingSet>

    </Bindings>

The <BindingSet> tags wrap groups of control bindings. It requires a Name and optionally a Cursor. If the binding set is to be used automatically, as it would be in most cases, set the AutoUse parameter to True. Inside of that are <Bind> tags. The tag requires the Event parameter and optionally the Factor parameter. The Event parameter will be used to match the binding to its handler which will be written later. The syntax is <Handler Name>.<Event Name>. The Factor parameter is optional and can be used to scale the data value up or down to increase or decrease sensitivity of the input method. The Action tag inside the Bind tag is used to map the specific binding to a particular method. Once the handler is written, these will make more sense.

The bindings above create the control scheme described above: NunchukX/Y describe what happens when the analog joystick is moved, NunchukC/Z describe what happens with the C/Z buttons are pressed, and so on.

The bindings also allow for several variations. Bindings are defined for both the IR position (IRX, IRY) and accelerometer values (AX, AY). If an IR sensor bar is not available, the accelerometer values of the Wiimote can be used instead. Additionally, keyboard bindings are created in the style of a first person shooter using WASD. These can be used if a Nunchuk is not available.

For those events which require a custom action that will be written separately and not part of the VE3D control, the Action parameter must contain the action name, followed by a comma, and then the full assembly name:

<Bind Event="Wiimote.One">
    <Action Name="ToggleBalanceBoard, WiiEarthVR, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/>
</Bind>

You can change any of these button bindings simply by changing this XML file and deploying to the directory above.

Event Source

An EventSource is needed which will grab data from the Wiimote and pass it along to VE3D as defined by the bindings file above. Create a new class named WiimoteEventSource which derives from Microsoft.MapPoint.Binding.EventSource as follows:

Next, add an enumeration named WiimoteEvent (the name isn't important) which contains all of the Name items from the bindings XML file above. It should look like this:

// all events handled by this event source from XML file
public enum WiimoteEvent
{
    IRX,        // IR X position
    IRY,        // IR Y position
    NunchukX,    // Nunchuk joystick X position
    NunchukY,    // Nunchuk joystick Y position
    NunchukC,    // Nunchuk C button
    NunchukZ,    // Nunchuk Z button
    AX,            // Wiimote accelerometer X
    AY,            // Wiimote accelerometer Y
    Up,            // Dpad up
    Down,        // Dpad down
    Left,        // Dpad left
    Right,        // Dpad right
    A,            // A button
    B,            // B button
    Minus,        // Minus button
    Home,        // Wiimote Home button
    Plus,        // Plus button
    One,        // Wiimote One button
    Two,        // Wiimote Two button
    BalanceBoardX,    // Balance Board COG X
    BalanceBoardY    // Balance Board COG Y
}

' all events handled by this event source from XML file
Public Enum WiimoteEvent
    IRX ' IR X position
    IRY ' IR Y position
    NunchukX ' Nunchuk joystick X position
    NunchukY ' Nunchuk joystick Y position
    NunchukC ' Nunchuk C button
    NunchukZ ' Nunchuk Z button
    AX ' Wiimote accelerometer X
    AY ' Wiimote accelerometer Y
    Up ' Dpad up
    Down ' Dpad down
    Left ' Dpad left
    Right ' Dpad right
    A ' A button
    B ' B button
    Minus ' Minus button
    Home ' Wiimote Home button
    Plus ' Plus button
    One ' Wiimote One button
    Two ' Wiimote Two button
    BalanceBoardX ' Balance Board COG X
    BalanceBoardY ' Balance Board COG Y
End Enum

Next, several methods from the EventSource object need to be overridden: GetEventData, IsModifier, CanModify, TryGetEventId, TryGetEventName, Name. The methods do the following:

Method/Property	Description
GetEventData	Unsure at the moment...does not need to be implemented?
IsModifier	Returns a boolean stating whether the passed in event ID is a modifier (such as the Wiimote.B event above)
CanModify	Returns a boolean stating whether the current event is allowed as a modifier
TryGetEventID	Maps a string event name from the bindings file to the integer value in the enumeration above
TryGetEventName	Maps an integer event ID to the string name in the enumeration above
Name (property)	Returns the name of the handler which must match the name in the XML file above (Wiimote in this case)

The code for these methods is presented below:

    // return out value of the passed enum

    public override bool TryGetEventId(string eventName, out int eventId)

    {

        eventId = (int)Enum.Parse(typeof(WiimoteEvent), eventName);

        return true;

    }

    // return out the string name of the passed in enum value

    public override bool TryGetEventName(int eventId, out string eventName)

    {

        eventName = ((WiimoteEvent)eventId).ToString();

        return true;

    }

    // unknown

    public override EventData GetEventData(int eventId, EventActivateState state)

    {

        throw new NotImplementedException();

    }

    // can the event be used as a modifier?

    public override bool IsModifier(int eventId)

    {

        // yes to all for now

        return true;

    }

    // can the supplied event be used as a modifier?

    public override bool CanModify(int eventId, EventKey other)

    {

        // only if it's from us

        return (other.Source == this);

    }

    // this must match the Source name in the bindings XML file

    public override string Name

    {

        get { return "Wiimote"; }

    }

    ' return out value of the passed enum

    Public Overrides Function TryGetEventId(ByVal eventName As String, <System.Runtime.InteropServices.Out()> ByRef eventId As Integer) As Boolean

        eventId = CInt(Fix(System.Enum.Parse(GetType(WiimoteEvent), eventName)))

        Return True

    End Function

    ' return out the string name of the passed in enum value

    Public Overrides Function TryGetEventName(ByVal eventId As Integer, <System.Runtime.InteropServices.Out()> ByRef eventName As String) As Boolean

        eventName = (CType(eventId, WiimoteEvent)).ToString()

        Return True

    End Function

    ' unknown

    Public Overrides Function GetEventData(ByVal eventId As Integer, ByVal state As EventActivateState) As EventData

        Throw New NotImplementedException()

    End Function

    ' can the event be used as a modifier?

    Public Overrides Function IsModifier(ByVal eventId As Integer) As Boolean

        ' yes to all for now

        Return True

    End Function

    ' can the supplied event be used as a modifier?

    Public Overrides Function CanModify(ByVal eventId As Integer, ByVal other As EventKey) As Boolean

        ' only if it's from us

        Return (other.Source Is Me)

    End Function

    ' this must match the Source name in the bindings XML file

    Public Overrides ReadOnly Property Name() As String

        Get

            Return "Wiimote"

        End Get

    End Property

With that in place, the constructor can be implemented which will call the base constructor and connect to the Wiimote. It is assumed you read the Wiimote article above and know how the library works.

The constructor must take one argument passed from the main from: an instance of the GlobeControl's ActionSystem. This just gets passed directly to the parent object's constructor untouched. The constructor code looks like the following:

public WiimoteEventSource(ActionSystem actionSystem) : base(actionSystem)
{
    // get all connected Wiimotes
    WiimoteCollection wc = new WiimoteCollection();
    wc.FindAllWiimotes();

    // setup wiimotes and event handlers
    foreach(Wiimote wm in wc)
    {
        wm.WiimoteChanged += new EventHandler<WiimoteChangedEventArgs>(OnWiimoteChanged);
        wm.WiimoteExtensionChanged += new EventHandler<WiimoteExtensionChangedEventArgs>(OnWiimoteExtensionChanged);
        wm.Connect();

        // if we don't have an extension, set the report type to IR and accel's only
        if(!wm.WiimoteState.Extension && wm.WiimoteState.ExtensionType != ExtensionType.BalanceBoard)
            wm.SetReportType(InputReport.IRAccel, true);

        if(wm.WiimoteState.ExtensionType == ExtensionType.BalanceBoard)
            _bb = wm;
        else
            _wm = wm;

        // turn off all LEDs
        wm.SetLEDs(0x00);
    }
}

Public Sub New(ByVal actionSystem As ActionSystem)
    MyBase.New(actionSystem)

    ' get all connected Wiimotes
    Dim wc As New WiimoteCollection()
    wc.FindAllWiimotes()

    ' setup wiimotes and event handlers
    For Each wm As Wiimote In wc
        AddHandler wm.WiimoteChanged, AddressOf OnWiimoteChanged
        AddHandler wm.WiimoteExtensionChanged, AddressOf OnWiimoteExtensionChanged
        wm.Connect()

        ' if we don't have an extension, set the report type to IR and accel's only
        If (Not wm.WiimoteState.Extension) AndAlso wm.WiimoteState.ExtensionType <> ExtensionType.BalanceBoard Then
            wm.SetReportType(InputReport.IRAccel, True)
        End If

        If wm.WiimoteState.ExtensionType = ExtensionType.BalanceBoard Then
            _bb = wm
        Else
            _wm = wm
        End If

        ' turn off all LEDs
        wm.SetLEDs(&H00)
    Next wm
End Sub

The OnWiimoteExtensionChanged method simply sets the report mode for the Wiimote based on whether or not a Nunchuk is inserted as shown:

private void OnWiimoteExtensionChanged(object sender, WiimoteExtensionChangedEventArgs args)
{
    if(_wm == null)
        return;

    // if nunchuk inserted, set the report type to return extension data
    if(args.ExtensionType == ExtensionType.Nunchuk && args.Inserted)
        _wm.SetReportType(InputReport.IRExtensionAccel, true);
    else // in all other cases, set it to the default IR and accel's
        _wm.SetReportType(InputReport.IRAccel, true);
}

Private Sub OnWiimoteExtensionChanged(ByVal sender As Object, ByVal args As WiimoteExtensionChangedEventArgs)
    If _wm Is Nothing Then
        Return
    End If

    ' if nunchuk inserted, set the report type to return extension data
    If args.ExtensionType = ExtensionType.Nunchuk AndAlso args.Inserted Then
        _wm.SetReportType(InputReport.IRExtensionAccel, True)
    Else ' in all other cases, set it to the default IR and accel's
        _wm.SetReportType(InputReport.IRAccel, True)
    End If
End Sub

The OnWiimoteChanged event handler is where the Wiimote data is handled and sent off to the VE3D control to reflect the changes. First, we handle the Balance Board. If the Balance Board is the controller reporting data, we take the center of gravity values and pass them to VE3D. This code looks at the appropriate values, determines if they are beyond the specified thresholds for the dead zones, and, if they are, activates the event for that value using the Execute method. Execute is a method in the base EventSource class. This method will activate the event specified from the enumeration (which, remember, is contained in the bindings XML file) with the value associated with that event. An EventData object of some type must be created and passed to the Execute method. There are two EventData types to know about: AxisEventData and ButtonEventData. AxisEventData should be used when an event is activated that will modify the map position in some way. That is, if the map is being turned, elevation is changing, etc. ButtonEventData should be used if the event is a simple toggle like pressing a button down and releasing it.

The Balance Board code can be seen below:

    if(ws.ExtensionType == ExtensionType.BalanceBoard && this.BalanceBoardEnabled)

    {

        float x1 = ws.BalanceBoardState.CenterOfGravity.X;

        if(x1 > Properties.Settings.Default.BBDeadX || x1 < -Properties.Settings.Default.BBDeadX)

            this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.BalanceBoardX), x1 - _zero.X));

        float y1 = ws.BalanceBoardState.CenterOfGravity.Y;

        if(y1 > Properties.Settings.Default.BBDeadY || y1 < -Properties.Settings.Default.BBDeadY)

            this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.BalanceBoardY), y1 - _zero.Y));

    }

    If ws.ExtensionType = ExtensionType.BalanceBoard AndAlso Me.BalanceBoardEnabled Then

        Dim x1 As Single = ws.BalanceBoardState.CenterOfGravity.X

        If x1 > My.Settings.Default.BBDeadX OrElse x1 < -My.Settings.Default.BBDeadX Then

            Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.BalanceBoardX))), x1 - _zero.X))

        End If

        Dim y1 As Single = ws.BalanceBoardState.CenterOfGravity.Y

        If y1 > My.Settings.Default.BBDeadY OrElse y1 < -My.Settings.Default.BBDeadY Then

            Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.BalanceBoardY))), y1 - _zero.Y))

        End If

Next, let's handle the IR and accelerometer data. The IR midpoint of the X and Y axes will be used from the WiimoteState object to activate the IRX and IRY events we defined above in the bindings XML file. The accelerometer X and Y values will be used to activate the AX and AY events.

This snippet assumes that there is a boolean property named UseIR created in the project to determine whether IR or motion values are used. Additionally, it assumes there are property settings created which contain values for the X/Y "dead zones" for the IR and accelerometers. These dead zones are used as a way to only activate the event when the values are pushed beyond the thresholds. This allows there to be a margin where the user's hand will not be read as movement, allowing the user to not have to worry about keeping a steady hand.

The application linked above uses the following default values for dead zones:

NunchukDeadX/Y -> 0.025
WiimoteDeadX/Y -> 0.15
IRDeadX/Y -> 0.1
BBDeadX/Y –> 3

    // if we're using the IR

    if(Properties.Settings.Default.UseIR)

    {

        // and both LEDs are found

        if(ws.IRState.Found1 && ws.IRState.Found2)

        {

            // normalize the midpoints to -0.5 to 0.5 (from 0 to 1.0)

            float x = ws.IRState.MidX - 0.5f;

            float y = ws.IRState.MidY - 0.5f;

            // if we're beyond the thresholds, activate the events

            if(x > Properties.Settings.Default.IRDeadX || x < -Properties.Settings.Default.IRDeadX)

                this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.IRX), x));

            if(y > Properties.Settings.Default.IRDeadY || y < -Properties.Settings.Default.IRDeadY)

                this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.IRY), y));

            // save the last IR settings...these get used if we go beyond the range of the IRs.

            // in that case, the last used positions will be used until the Wiimote comes back in range

            this._lastIRX = x;

            this._lastIRY = y;

        }

        else // one or both LEDs aren't seen

        {

            // activate events based on the last known positions

            if(this._lastIRX > Properties.Settings.Default.IRDeadX || this._lastIRX < -Properties.Settings.Default.IRDeadX)

                this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.IRX), this._lastIRX));

            if(this._lastIRY > Properties.Settings.Default.IRDeadY || this._lastIRY < -Properties.Settings.Default.IRDeadY)

                this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.IRY), this._lastIRY));

        }

    }

    else // we're using motion controls

    {

        // activate the events based on the accelerometer values

        if(ws.AccelState.X > Properties.Settings.Default.WiimoteDeadX || ws.AccelState.X < -Properties.Settings.Default.WiimoteDeadX)

            this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.AX), ws.AccelState.X));

        if(ws.AccelState.Y > Properties.Settings.Default.WiimoteDeadY || ws.AccelState.Y < -Properties.Settings.Default.WiimoteDeadY)

            this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.AY), ws.AccelState.Y));

    }

    ' if we're using the IR

    If My.Settings.Default.UseIR Then

        ' and both LEDs are found

        If ws.IRState.Found1 AndAlso ws.IRState.Found2 Then

            ' normalize the midpoints to -0.5 to 0.5 (from 0 to 1.0)

            Dim x As Single = ws.IRState.MidX - 0.5f

            Dim y As Single = ws.IRState.MidY - 0.5f

            ' if we're beyond the thresholds, activate the events

            If x > My.Settings.Default.IRDeadX OrElse x < -My.Settings.Default.IRDeadX Then

                Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.IRX))), x))

            End If

            If y > My.Settings.Default.IRDeadY OrElse y < -My.Settings.Default.IRDeadY Then

                Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.IRY))), y))

            End If

            ' save the last IR settings...these get used if we go beyond the range of the IRs.

            ' in that case, the last used positions will be used until the Wiimote comes back in range

            Me._lastIRX = x

            Me._lastIRY = y

        Else ' one or both LEDs aren't seen

            ' activate events based on the last known positions

            If Me._lastIRX > My.Settings.Default.IRDeadX OrElse Me._lastIRX < -My.Settings.Default.IRDeadX Then

                Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.IRX))), Me._lastIRX))

            End If

            If Me._lastIRY > My.Settings.Default.IRDeadY OrElse Me._lastIRY < -My.Settings.Default.IRDeadY Then

                Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.IRY))), Me._lastIRY))

            End If

        End If

    Else ' we're using motion controls

        ' activate the events based on the accelerometer values

        If ws.AccelState.X > My.Settings.Default.WiimoteDeadX OrElse ws.AccelState.X < -My.Settings.Default.WiimoteDeadX Then

            Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.AX))), ws.AccelState.X))

        End If

        If ws.AccelState.Y > My.Settings.Default.WiimoteDeadY OrElse ws.AccelState.Y < -My.Settings.Default.WiimoteDeadY Then

            Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.AY))), ws.AccelState.Y))

        End If

    End If

Next, the nunchuk values need to be read and the associated events activated. This is done as follows:

    // if the nunchuk is connected

    if(ws.Extension && ws.ExtensionType == ExtensionType.Nunchuk)

    {

        // activate the nunchuk-based events

        if(ws.NunchukState.X > Properties.Settings.Default.NunchukDeadX || ws.NunchukState.X < -Properties.Settings.Default.NunchukDeadX)

            this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.NunchukX), ws.NunchukState.X));

        if(ws.NunchukState.Y > Properties.Settings.Default.NunchukDeadY || ws.NunchukState.Y < -Properties.Settings.Default.NunchukDeadY)

            this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.NunchukY), ws.NunchukState.Y));

        if(ws.NunchukState.C)

            this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.NunchukC), 1.0f));

        if(ws.NunchukState.Z)

            this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.NunchukZ), 1.0f));

    }

    ' if the nunchuk is connected

    If ws.Extension AndAlso ws.ExtensionType = ExtensionType.Nunchuk Then

        ' activate the nunchuk-based events

        If ws.NunchukState.X > My.Settings.Default.NunchukDeadX OrElse ws.NunchukState.X < -My.Settings.Default.NunchukDeadX Then

            Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.NunchukX))), ws.NunchukState.X))

        End If

        If ws.NunchukState.Y > My.Settings.Default.NunchukDeadY OrElse ws.NunchukState.Y < -My.Settings.Default.NunchukDeadY Then

            Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.NunchukY))), ws.NunchukState.Y))

        End If

        If ws.NunchukState.C Then

            Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.NunchukC))), 1.0f))

        End If

        If ws.NunchukState.Z Then

            Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.NunchukZ))), 1.0f))

        End If

    End If

Finally, the button events need to be activated. A helper method which will check the current button state will be used for determining which button of all the Wiimote buttons is pressed. For those that are, the appropriate event is activated with a call to Execute.

    private void HandleButton(WiimoteEvent we, bool buttonState, bool lastButtonState)

    {

        if(buttonState == lastButtonState)

            return;

        else

        {

            if(buttonState)

                this.Execute(new ButtonEventData(new EventKey(this, (int)we), EventActivateState.Activate));

            else

                this.Execute(new ButtonEventData(new EventKey(this, (int)we), EventActivateState.Deactivate));

        }

    }

    // handle all the Wiimote buttons

    HandleButton(WiimoteEvent.Up, ws.ButtonState.Up, _lastBS.Up);

    HandleButton(WiimoteEvent.Down, ws.ButtonState.Down, _lastBS.Down);

    HandleButton(WiimoteEvent.Left, ws.ButtonState.Left, _lastBS.Left);

    HandleButton(WiimoteEvent.Right, ws.ButtonState.Right, _lastBS.Right);

    HandleButton(WiimoteEvent.A, ws.ButtonState.A, _lastBS.A);

    HandleButton(WiimoteEvent.B, ws.ButtonState.B, _lastBS.B);

    HandleButton(WiimoteEvent.Minus, ws.ButtonState.Minus, _lastBS.Minus);

    HandleButton(WiimoteEvent.Home, ws.ButtonState.Home, _lastBS.Home);

    HandleButton(WiimoteEvent.Plus, ws.ButtonState.Plus, _lastBS.Plus);

    HandleButton(WiimoteEvent.One, ws.ButtonState.One, _lastBS.One);

    HandleButton(WiimoteEvent.Two, ws.ButtonState.Two, _lastBS.Two);

    ...

    // save off the current button state for next time

    _lastBS = ws.ButtonState;

    _lastNunchuk = ws.NunchukState;

    Private Sub HandleButton(ByVal we As WiimoteEvent, ByVal buttonState As Boolean, ByVal lastButtonState As Boolean)

        If buttonState = lastButtonState Then

            Return

        Else

            If buttonState Then

                Me.Execute(New ButtonEventData(New EventKey(Me, CInt(Fix(we))), EventActivateState.Activate))

            Else

                Me.Execute(New ButtonEventData(New EventKey(Me, CInt(Fix(we))), EventActivateState.Deactivate))

            End If

        End If

    End Sub

    ...

    ' handle all the Wiimote buttons

    HandleButton(WiimoteEvent.Up, ws.ButtonState.Up, _lastBS.Up)

    HandleButton(WiimoteEvent.Down, ws.ButtonState.Down, _lastBS.Down)

    HandleButton(WiimoteEvent.Left, ws.ButtonState.Left, _lastBS.Left)

    HandleButton(WiimoteEvent.Right, ws.ButtonState.Right, _lastBS.Right)

    HandleButton(WiimoteEvent.A, ws.ButtonState.A, _lastBS.A)

    HandleButton(WiimoteEvent.B, ws.ButtonState.B, _lastBS.B)

    HandleButton(WiimoteEvent.Minus, ws.ButtonState.Minus, _lastBS.Minus)

    HandleButton(WiimoteEvent.Home, ws.ButtonState.Home, _lastBS.Home)

    HandleButton(WiimoteEvent.Plus, ws.ButtonState.Plus, _lastBS.Plus)

    HandleButton(WiimoteEvent.One, ws.ButtonState.One, _lastBS.One)

    HandleButton(WiimoteEvent.Two, ws.ButtonState.Two, _lastBS.Two)

    ' save off the current button state for next time

    _lastBS = ws.ButtonState

    _lastNunchuk = ws.NunchukState

And, the current button values are stored away to check on the next event so button events are only fired once.

Now that the event source object is written, it needs to be hooked up to the GlobeControl so it can be used. This can be done by creating an instance of the WiimoteEventSource object, passing in the VE3D's ActionSystem from the BindingsManager object. Then, the event source instance is passed to the ActionSystem's EventSourceManager and registered using the RegisterEventSource method. Event sources should re registered before the control is added to the form.

    // wiimote events

    private WiimoteEventSource _wiimoteEventSource;

    ...

    // create a new instance of the Wiimote event handler

    _wiimoteEventSource = new WiimoteEventSource(this.globeControl.Host.BindingsManager.ActionSystem, this);

    // register it in the event source list

    _globeControl.Host.BindingsManager.ActionSystem.EventSourceManager.RegisterEventSource(this._wiimoteEventSource);

    ' wiimote events

    Private _wiimoteEventSource As WiimoteEventSource

    ...

    ' create a new instance of the Wiimote event handler

    _wiimoteEventSource = New WiimoteEventSource(Me.globeControl.Host.BindingsManager.ActionSystem, Me)

    ' register it in the event source list

    _globeControl.Host.BindingsManager.ActionSystem.EventSourceManager.RegisterEventSource(Me._wiimoteEventSource)

Actions and the BindingManager

Our binding list contains several action types that are not defined by the default VE3D actions. These actions and their handlers must be registered with the VE3D control. The actions can be registered as follows in the FirstFrameRendered event handler:

    BindingsSource bs = new BindingsSource(base.GetType());

    _globeControl.Host.BindingsManager.RegisterAction(bs, "Locations", LocationsHandler);

    _globeControl.Host.BindingsManager.RegisterAction(bs, "LocationsMove", LocationsMoveHandler);

    _globeControl.Host.BindingsManager.RegisterAction(bs, "ToggleVR920", ToggleVR920Handler);

    _globeControl.Host.BindingsManager.RegisterAction(bs, "VR920SetZero", VR920SetZero);

    _globeControl.Host.BindingsManager.RegisterAction(bs, "BalanceBoardSetZero", BalanceBoardSetZero);

    _globeControl.Host.BindingsManager.RegisterAction(bs, "ToggleBalanceBoard", ToggleBalanceBoardHandler);

    _globeControl.Host.BindingsManager.RegisterAction(bs, "FullScreen", FullScreenHandler);

    _globeControl.Host.BindingsManager.RegisterAction(bs, "ToggleVR920Stereo", ToggleVR920Stereo);

    _globeControl.Host.BindingsManager.RegisterAction(bs, "VR920SetEyeDistance", VR920SetEyeDistance);

    BindingsSource bs = new BindingsSource(base.GetType());

    _globeControl.Host.BindingsManager.RegisterAction(bs, "Locations", LocationsHandler);

    _globeControl.Host.BindingsManager.RegisterAction(bs, "LocationsMove", LocationsMoveHandler);

    _globeControl.Host.BindingsManager.RegisterAction(bs, "ToggleVR920", ToggleVR920Handler);

    _globeControl.Host.BindingsManager.RegisterAction(bs, "VR920SetZero", VR920SetZero);

    _globeControl.Host.BindingsManager.RegisterAction(bs, "BalanceBoardSetZero", BalanceBoardSetZero);

    _globeControl.Host.BindingsManager.RegisterAction(bs, "ToggleBalanceBoard", ToggleBalanceBoardHandler);

    _globeControl.Host.BindingsManager.RegisterAction(bs, "FullScreen", FullScreenHandler);

    _globeControl.Host.BindingsManager.RegisterAction(bs, "ToggleVR920Stereo", ToggleVR920Stereo);

    _globeControl.Host.BindingsManager.RegisterAction(bs, "VR920SetEyeDistance", VR920SetEyeDistance);

With the actions registered and handlers associated with them, the actual handlers need to be implemented. All event handler methods must be of the following signature:

public bool EventHandler(EventData cause)

Public Function EventHandler(ByVal cause As EventData) As Boolean

Let’s take a look at the FullScreen binding which simply turns the status bar at the bottom of the window on and off:

private bool FullScreenHandler(EventData eventData)
{
    if(eventData.Activate)
        BeginInvoke(new UIEventHandlerDelegate(FullScreen), eventData);
    return true;
}

Private Function FullScreenHandler(ByVal eventData As EventData) As Boolean
    If eventData.Activate Then
        BeginInvoke(New UIEventHandlerDelegate(AddressOf FullScreen), eventData)
    End If
    Return True
End Function

Private Sub FullScreen(ByVal eventData As EventData)
    statusStrip1.Visible = Not statusStrip1.Visible
End Sub

Because we are in the VE3D thread when this handler is called, we need to use BeginInvoke to call the real method on the UI thread.

Be sure to check the source code for the full demo linked above for the location handler methods. I omitted them here since it is just more of the same type of code above.

VR920 Head Tracker

Once the VR920 glasses and driver are installed on your PC, getting the data required from the glasses is quite easy. We are going to create an object named VR920Tracker which will pull data from the device and send it to VE3D.

First we need to setup a few P/Invoke signatures to talk to the glasses as follows:

    [DllImport("IWEARDRV.dll", SetLastError = true, CharSet = CharSet.Auto)]

    private static extern int IWROpenTracker();

    [DllImport("IWEARDRV.dll", SetLastError = true, CharSet = CharSet.Auto)]

    private static extern void IWRZeroSet();

    [DllImport("IWEARDRV.dll", SetLastError = true, CharSet = CharSet.Auto)]

    private static extern int IWRGetTracking(out int yaw, out int pitch, out int roll);

    private const int ERROR_SUCCESS  = 0;

    <DllImport("IWEARDRV.dll", SetLastError := True, CharSet := CharSet.Auto)> _

    Private Shared Function IWROpenTracker() As Integer

    End Function

    <DllImport("IWEARDRV.dll", SetLastError := True, CharSet := CharSet.Auto)> _

    Private Shared Sub IWRZeroSet()

    End Sub

    <DllImport("IWEARDRV.dll", SetLastError := True, CharSet := CharSet.Auto)> _

    Private Shared Function IWRGetTracking(<System.Runtime.InteropServices.Out()> ByRef yaw As Integer, <System.Runtime.InteropServices.Out()> ByRef pitch As Integer, <System.Runtime.InteropServices.Out()> ByRef roll As Integer) As Integer

    End Function

    private const int ERROR_SUCCESS  = 0;

These 3 methods will allows to open access to the glasses, set its zero position, and get the roll, pitch and yaw information from the sensors.

In the constructor for this object, we will open a handle to the glasses and startup a timer that will poll the glasses at a regular interval to get the sensor data as shown below:

    private const int TimerPeriod = 1;

    private MainForm _mainForm;

    private Timer _timer;

    public VR920Tracker(MainForm mainForm)

    {

        _mainForm = mainForm;

        int openResult = IWROpenTracker();

        if(openResult != ERROR_SUCCESS)

            throw new ApplicationException("Could not connect to VR920: " + openResult);

        _timer = new Timer(VR920Poller, null, Timeout.Infinite, TimerPeriod);

    }

    Private Const TimerPeriod As Integer = 1

    Private _mainForm As MainForm

    Private _timer As Timer

    Public Sub New(ByVal mainForm As MainForm)

        _mainForm = mainForm

        Dim openResult As Integer = IWROpenTracker()

        If openResult <> ERROR_SUCCESS Then

            Throw New ApplicationException("Could not connect to VR920: " & openResult)

        End If

        _timer = New Timer(AddressOf VR920Poller, Nothing, Timeout.Infinite, TimerPeriod)

    End Sub

The VR920Poller method referenced above will be called at a specific interval and read the values from the glasses as shown below:

    // collection of values for averaging

    private List<double> _yawValues = new List<double>();

    private List<double> _rollValues = new List<double>();

    private List<double> _pitchValues = new List<double>();

    // last calculated values

    private double _lastYaw;

    private double _lastPitch;

    private double _lastRoll;

    private void VR920Poller(object state)

    {

        lock(this)

        {

            int yaw, pitch, roll;

            int result = IWRGetTracking(out yaw, out pitch, out roll);

            if(result != ERROR_SUCCESS)

                throw new ApplicationException("Could not get VR920 tracking information: " + result);

            _yawValues.Add(VR920ToRadians(yaw));

            _rollValues.Add(VR920ToRadians(roll));

            _pitchValues.Add(VR920ToRadians(pitch));

            if(_yawValues.Count == 5)

            {

                double y = Average(_yawValues, _lastYaw);

                if(Math.Abs(y - _lastYaw) > 0.026)

                    _lastYaw = y;

                double p = Average(_pitchValues, _lastPitch);

                if(Math.Abs(p - _lastPitch) > 0.017)

                    _lastPitch = p;

                _lastRoll = Average(_rollValues, _lastRoll);

                _yawValues.Clear();

                _pitchValues.Clear();

                _rollValues.Clear();

                RollPitchYaw rollPitchYaw = new RollPitchYaw(_lastRoll, _lastPitch, _lastYaw);

                _mainForm.SetRollPitchYaw(rollPitchYaw);

                _mainForm.lblAxes.Text = VR920ToDegrees(yaw) + ", " + VR920ToDegrees(pitch) + ", " + VR920ToDegrees(roll);

            }

        }

    }

    private double Average(List<double> values, double last)

    {

        double total = 0;

        foreach(double value in values)

            total += value;

        total += last;

        return total / (values.Count+1);

    }

    private double VR920ToRadians(int vr920Value)

    {

        return (vr920Value * .00549) * (Math.PI/180);

    }

    private double VR920ToDegrees(int vr920Value)

    {

        return (vr920Value * .00549);

    }

    ' collection of values for averaging

    Private _yawValues As List(Of Double) = New List(Of Double)()

    Private _rollValues As List(Of Double) = New List(Of Double)()

    Private _pitchValues As List(Of Double) = New List(Of Double)()

    ' last calculated values

    Private _lastYaw As Double

    Private _lastPitch As Double

    Private _lastRoll As Double

    Private Sub VR920Poller(ByVal state As Object)

        SyncLock Me

            Dim yaw, pitch, roll As Integer

            Dim result As Integer = IWRGetTracking(yaw, pitch, roll)

            If result <> ERROR_SUCCESS Then

                Throw New ApplicationException("Could not get VR920 tracking information: " & result)

            End If

            _yawValues.Add(VR920ToRadians(yaw))

            _rollValues.Add(VR920ToRadians(roll))

            _pitchValues.Add(VR920ToRadians(pitch))

            If _yawValues.Count = 5 Then

                Dim y As Double = Average(_yawValues, _lastYaw)

                If Math.Abs(y - _lastYaw) > 0.026 Then

                    _lastYaw = y

                End If

                Dim p As Double = Average(_pitchValues, _lastPitch)

                If Math.Abs(p - _lastPitch) > 0.017 Then

                    _lastPitch = p

                End If

                _lastRoll = Average(_rollValues, _lastRoll)

                _yawValues.Clear()

                _pitchValues.Clear()

                _rollValues.Clear()

                Dim rollPitchYaw As New RollPitchYaw(_lastRoll, _lastPitch, _lastYaw)

                _mainForm.SetRollPitchYaw(rollPitchYaw)

                _mainForm.lblAxes.Text = VR920ToDegrees(yaw) & ", " & VR920ToDegrees(pitch) & ", " & VR920ToDegrees(roll)

            End If

        End SyncLock

    End Sub

    Private Function Average(ByVal values As List(Of Double), ByVal last As Double) As Double

        Dim total As Double = 0

        For Each value As Double In values

            total += value

        Next value

        total += last

        Return total / (values.Count+1)

    End Function

    Private Function VR920ToRadians(ByVal vr920Value As Integer) As Double

        Return (vr920Value *.00549) * (Math.PI/180)

    End Function

    Private Function VR920ToDegrees(ByVal vr920Value As Integer) As Double

        Return (vr920Value *.00549)

    End Function

This chunk calls the IWRGetTracking method and tosses the results in 3 lists for roll, pitch and yaw. then, when the list has 5 members in it, the values are averaged and, if that resulting values is past a certain threshold, the values are passed back to the MainForm’s SetRollPitchYaw method as shown below:

    private double _lastYaw;

    public void SetRollPitchYaw(RollPitchYaw rollPitchYaw)

    {

        if(!_initialized || _globeControl.IsDisposed || (_globeControl.Host) == null || _globeControl.Host.CameraControllers.Current == null)

            return;

        double y = ((_globeControl.Host.CameraControllers.Current as ActionCameraController).LastReportedViewpoint.LocalOrientation.Yaw - _lastYaw) + rollPitchYaw.Yaw;

        RollPitchYaw rpw = new RollPitchYaw(rollPitchYaw.Roll, rollPitchYaw.Pitch, y);

        (_globeControl.Host.CameraControllers.Current as ActionCameraController).LastReportedViewpoint.LocalOrientation.RollPitchYaw = rpw;

        _lastYaw = rollPitchYaw.Yaw;

    }

    Private _lastYaw As Double

    Public Sub SetRollPitchYaw(ByVal rollPitchYaw As RollPitchYaw)

        If (Not _initialized) OrElse _globeControl.IsDisposed OrElse (_globeControl.Host) Is Nothing OrElse _globeControl.Host.CameraControllers.Current Is Nothing Then

            Return

        End If

        Dim y As Double = ((TryCast(_globeControl.Host.CameraControllers.Current, ActionCameraController)).LastReportedViewpoint.LocalOrientation.Yaw - _lastYaw) + rollPitchYaw.Yaw

        Dim rpw As New RollPitchYaw(rollPitchYaw.Roll, rollPitchYaw.Pitch, y)

        TryCast(_globeControl.Host.CameraControllers.Current, ActionCameraController).LastReportedViewpoint.LocalOrientation.RollPitchYaw = rpw

        _lastYaw = rollPitchYaw.Yaw

    End Sub

This method takes the roll, pitch and yaw values provided and applies them to the current CameraController used by VE3D (which, by default, is an ActionCameraController).

So, this code ultimately takes the roll, yaw and pitch of the user’s head and translates that directly to the roll, yaw and pitch of the camera in VE3D providing an accurate, real-time view into the VE3D world.

Stereoscopic Images

The final piece of the puzzle is drawing frames to the VR920 glasses in such a way that the final view to the user will be three dimensional. This works similarly to those Magic Eye puzzles you may have seen. Essentially, for every frame, we want to take the current VE3D camera position and move it several units to the left, render that to the left eye of the glasses, and then move the camera several units to the right and render that to the right eye of the glasses. To handle this we, can create an object named VR920StereoStep which derives from the Step class provided by VE3D and add this to the StepManager. When this is done, the VR920StereoStep will be called at the end of every frame drawn by VE3D.

As with the VR920Tracker class, we will need to setup a few P/Invoke method signatures to talk to the stereo driver as shown below:

    [DllImport("iWrstDrv.dll", EntryPoint = "IWRSTEREO_Open", SetLastError=true)]

    public static extern IntPtr OpenStereo();

    [DllImport("iWrstDrv.dll", EntryPoint = "IWRSTEREO_SetStereo")]

    public static extern Boolean SetStereoEnabled(IntPtr handle, Boolean enabled);

    [DllImport("iWrstDrv.dll", EntryPoint = "IWRSTEREO_SetLR")]

    public static extern Boolean SetStereoLR(IntPtr handle, Boolean eye);

    [DllImport("iWrstDrv.dll", EntryPoint = "IWRSTEREO_WaitForAck")]

    public static extern Byte WaitForOpenFrame(IntPtr handle, Boolean eye);

    <DllImport("iWrstDrv.dll", EntryPoint := "IWRSTEREO_Open", SetLastError:=True)> _

    Public Shared Function OpenStereo() As IntPtr

    End Function

    <DllImport("iWrstDrv.dll", EntryPoint := "IWRSTEREO_SetLR")> _

    Public Shared Function SetStereoLR(ByVal handle As IntPtr, ByVal eye As Boolean) As Boolean

    End Function

    <DllImport("iWrstDrv.dll", EntryPoint := "IWRSTEREO_SetStereo")> _

    Public Shared Function SetStereoEnabled(ByVal handle As IntPtr, ByVal enabled As Boolean) As Boolean

    End Function

    <DllImport("iWrstDrv.dll", EntryPoint := "IWRSTEREO_WaitForAck")> _

    Public Shared Function WaitForOpenFrame(ByVal handle As IntPtr, ByVal eye As Boolean) As Byte

    End Function

In the constructor for this object, we will open a handle to the stereo driver and turn on the stereo functionality of the glasses as shown below:

    private static readonly IntPtr INVALID_FILE_HANDLE = (IntPtr)(-1);

    private IntPtr _hStereo = INVALID_FILE_HANDLE;

    private Host _host;

    private bool _stereoEnabled = true;

    public double EyeDistance { get; set; }

    public VR920StereoStep(StepManager manager, Host host) : base(manager)

    {

        EyeDistance = 10;

        _host = host;

        _hStereo = OpenStereo();

        if(_hStereo != INVALID_FILE_HANDLE)

            SetStereoEnabled(_hStereo, true);

        else

            _stereoEnabled = false;

    }

    Private Shared ReadOnly INVALID_FILE_HANDLE As IntPtr = CType(-1, IntPtr)

    Private _hStereo As IntPtr = INVALID_FILE_HANDLE

    Private _host As Host

    Private _stereoEnabled As Boolean = True

    Private privateEyeDistance As Double

    Public Property EyeDistance() As Double

        Get

            Return privateEyeDistance

        End Get

        Set(ByVal value As Double)

            privateEyeDistance = value

        End Set

    End Property

    Public Sub New(ByVal manager As StepManager, ByVal host As Host)

        MyBase.New(manager)

        EyeDistance = 10

        _host = host

        _hStereo = OpenStereo()

        If _hStereo <> INVALID_FILE_HANDLE Then

            SetStereoEnabled(_hStereo, True)

        Else

            _stereoEnabled = False

        End If

    End Sub

Next, we need to override the OnExecute method provided by the base Step class as shown below:

    private bool _rightEye = true;

    private Vector3D _position;

    public override void OnExecute(SceneState state)

    {

        if(_hStereo != INVALID_FILE_HANDLE && _stereoEnabled && (_host.CameraControllers.Current as ActionCameraController).LastReportedViewpoint != null)

        {

            if(!_rightEye)

            {

                _position = (_host.CameraControllers.Current as ActionCameraController).LastReportedViewpoint.Position.Vector;

                (_host.CameraControllers.Current as ActionCameraController).LastReportedViewpoint.Position.Vector = _position + new Vector3D(-EyeDistance, 0, 0);

            }

            else

            {

                _position = (_host.CameraControllers.Current as ActionCameraController).LastReportedViewpoint.Position.Vector;

                (_host.CameraControllers.Current as ActionCameraController).LastReportedViewpoint.Position.Vector = _position + new Vector3D(EyeDistance, 0, 0);;

            }

            SetStereoLR(_hStereo, _rightEye);

            WaitForOpenFrame(_hStereo, _rightEye);

            _rightEye = !_rightEye;

        }

    }

    Private _rightEye As Boolean = True

    Private _position As Vector3D

    Public Overrides Sub OnExecute(ByVal state As SceneState)

        If _hStereo <> INVALID_FILE_HANDLE AndAlso _stereoEnabled AndAlso (TryCast(_host.CameraControllers.Current, ActionCameraController)).LastReportedViewpoint IsNot Nothing Then

            If (Not _rightEye) Then

                _position = (TryCast(_host.CameraControllers.Current, ActionCameraController)).LastReportedViewpoint.Position.Vector

                TryCast(_host.CameraControllers.Current, ActionCameraController).LastReportedViewpoint.Position.Vector = _position + New Vector3D(-EyeDistance, 0, 0)

            Else

                _position = (TryCast(_host.CameraControllers.Current, ActionCameraController)).LastReportedViewpoint.Position.Vector

                TryCast(_host.CameraControllers.Current, ActionCameraController).LastReportedViewpoint.Position.Vector = _position + New Vector3D(EyeDistance, 0, 0)

            End If

            SetStereoLR(_hStereo, _rightEye)

            WaitForOpenFrame(_hStereo, _rightEye)

            _rightEye = Not _rightEye

        End If

    End Sub

OnExecute will be called at the end of each VE3D frame rendered. In this method, we obtain the current position of the VE3D camera and then set its position several units to the left or right (- or + EyeDistance). SetStereoLR from the VR920 API is called, setting the appropriate eye to be active (left = false, right = true). Then WaitForOpenFrame is called which will pause the action long enough for the glasses to draw the entire frame to eye to avoid image tearing. Essentially, this call “sits and spins” at this location until the glasses report back that the entire frame is drawn in the proper eye and that we can continue on drawing the opposite eye for the next frame. Finally, the _rightEye member variable is togged to the opposite value, so the next time through this method the opposite eye is drawn.

With this class in place, back in our MainForm in the FirstFrameRendered event handler, we can instantiate the object and add it to the StepManager as shown below:

    private VR920StereoStep _vr920StereoStep;

    _vr920StereoStep = new VR920StereoStep(_globeControl.Host.RenderEngine.StepManager, _globeControl.Host);

    _globeControl.Host.RenderEngine.StepManager.Add(_vr920StereoStep);

    Private _vr920StereoStep As VR920StereoStep

    _vr920StereoStep = New VR920StereoStep(_globeControl.Host.RenderEngine.StepManager, _globeControl.Host)

    _globeControl.Host.RenderEngine.StepManager.Add(_vr920StereoStep)

Running the Application

To run the demo, do the following:

Copy BindingsWiimote.xml to the appropriate directory listed above
Pair the Wiimote and Balance Board to the computer. See the WiimoteLib article for more information on how to do that
Run the executable
Stand on the Balance Board
Put on the glasses
Zero both the Balance Board and glasses
Toggle everything on or off at will

Controls

F1 – Set VR920 glasses to zero position
F2 – Set Balance Board to zero position
F3 – Toggle stereo mode on and off
Wiimote 1 or B – Toggle Balance Board on or off
Wiimote 2 or V – Toggle VR920 head tracking on or off
F – Toggle full-screen on or off
Nunchuk joystick X/Y – Strafe/Move
Nunchuk C/Z buttons – Raise/lower altitude
Wiimote A – Open location menu/Select location (note that if a new location is selected, the Balance Board is turned off and must be re-enabled after “landing”
Wiimote Dpad Up/Down – Move through List
Balance Board – Lean your body left or right to turn in the VE3D environment

Conclusion

With the above code, we have created a very interesting interface for Virtual Earth 3D. The demo and source code linked above contain a few more features and bindings which enhance the application a bit more. Be sure to give the full demo a try and check out the full source code for a few more implementation details.

Additional Information

Bio

Brian is a Microsoft C# MVP who has been actively developing in .NET since its early betas in 2000, and who has been developing solutions using Microsoft technologies and platforms for even longer. Along with .NET, Brian is particularly skilled in the languages of C, C++ and assembly language for a variety of CPUs. He is also well-versed in a wide variety of technologies including web development, document imaging, GIS, graphics, game development, and hardware interfacing. Brian has a strong background in developing applications for the health-care industry, as well as developing solutions for portable devices, such as tablet PCs and PDAs. Additionally, Brian has co-authored the book "Debugging ASP.NET" published by New Riders, and is currently co-authoring a book titled "Coding4Fun: 10 .NET Programming Projects for Wiimote, YouTube, World of Warcraft, and More" to be published by O'Reilly in December 2008. Brian is also an author for MSDN's Coding4Fun website. You can reach Brian via vis website at http://www.brianpeek.com/ .

Want more cowbell? Yes please!

Coding4Fun — Tue, 11 Nov 2008 07:49:00 GMT

Author:	Clint Rutkas: twitter, blog
Download:	http://peacelovecode.com/code/cowbell/cowbell.zip
Software:	Visual Studio 2008 Standard or better (Trial Download) Windows Mobile 6.0 SDK Windows Mobile 6.1 Emulator Images (Optional)
Hardware:	Windows Mobile Pro (Optional)
Time Required:	2 hours
Cost:	Free-ish

After playing way too much guitar hero, bought a guitar only to quickly realize I’m horrible. Asking my friends, I decided to adopt the world’s hardest instrument to play, that’s right, the cow bell. It takes years to master and I decide to turn my cell phone, an HTC Touch Diamond into this magical instrument too.

This is a fairly straight forward application. We’ll touch on altering the volume of a cell phone along with interacting with a touch screen and how to do a deploy for Windows Mobile so you can have a cowbell band of your own. The application will also properly resize an image to match the screen proportions, however you’ll have to download the source code for that.

You’ll see while getting your system up and working with the emulator may take a few additional steps, it will code mostly like a typical windows application for this instance.

The source code shown isn’t everything you need to get the application running, please download the source code to get the rest. The link is at the top of the article.

Getting up and running with baby steps

After you get Visual Studio 2008 up and running, if you don’t have the Windows Mobile 6.0 SDK, please download it and install it. The link is at the top of this article. Visual Studio 2008 only comes with the Windows mobile 5.0 SDK.

On the right is a picture of the emulator in one of styling forms. On top of just being able to test on an emulator, you can do a direct deploy to your cell phone. Now a Windows Mobile Professional phone can do this directly, however a Windows Mobile Standard (non-touch screen) does require a few extra steps. I blogged about these steps over on my blog. After your phone is properly configured, all you need to do is change your target deployment.

I highlighted the device deploy instead of the emulator.

Smart Device Creation

To create a new solution we go to File –> New –> Project. We’ll select the language of our choice (c# for me) then go to “Smart Device”

Easy so far, right? Next comes selecting the framework along with your target platform. Select “Windows Mobile 6 Professional SDK” then click OK and you’re good to start.

Altering Volume

To alter your volume currently in the compact framework, you have to do some pinvokes to call native API calls. This is scary for some but don’t fret. There is a site called pinvoke.net that makes doing these calls simple as pie. We’ll create a new class called Audio to encapsulate all audio functions. This includes playing a file and altering the volume. First we’ll do the simple ability to play an audio file. Since we’ll be grabbing our data from the resources, we’ll use a MemoryStream object convert the bytes to a stream so we can read it in. We’ll need to include the System.Media namespace to be able to play audio. To do the pinvokes, we’ll include the System.Runtime.InteropServices.

private readonly SoundPlayer sp;

    public Audio(byte[] AudioResource)
    {
        sp = new SoundPlayer(new MemoryStream(AudioResource));
        sp.Load();
    }

    public void Play()
    {
        sp.Play();
    }

    public void PlayLoop()
    {
        sp.PlayLooping();
    }

    public void Stop()
    {
        sp.Stop();
    }

    Private ReadOnly sp As SoundPlayer

    Public Sub New(ByVal AudioResource As Byte())
        sp = New SoundPlayer(New MemoryStream(AudioResource))
        sp.Load()
    End Sub

    Public Sub Play()
        sp.Play()
    End Sub

    Public Sub PlayLoop()
        sp.PlayLooping()
    End Sub

    Public Sub [Stop]()
        sp.[Stop]()
    End Sub

And in the application, we’ll call the audio file like the following:

    private readonly Audio audio = new Audio(Properties.Resources.CowBellAudio);

    Private ReadOnly audio As New Audio(My.Resources.CowBellAudio)

Next we’ll add in the ability to alter the volume with the pinvokes. Sadly, I’m not sure why the set volume numbers act as they do. After searching the internet, these numbers kept appearing. In a perfect world, I would have liked to see 0 to 100 or something like that but it isn’t so. We’ll create get / sets in the wrapper to make a more seamless wrapper object to hide the pinvoke calls.

    [DllImport("coredll.dll")]
    private static extern int waveOutGetVolume(IntPtr hwo, out uint dwVolume);

    [DllImport("coredll.dll")]
    private static extern int waveOutSetVolume(IntPtr hwo, uint dwVolume);

    public static uint Volume
    {
        get
        {
            uint _volume;
            waveOutGetVolume(IntPtr.Zero, out _volume);
            return _volume;
        }

        set
        {
            waveOutSetVolume(IntPtr.Zero, value);
        }
    }

    public enum Volumes
    {
        OFF = 0,
        LOW = 858993459,
        NORMAL = 1717986918,
        MEDIUM = -1717986919,
        HIGH = -858993460,
        VERY_HIGH = -1
    }

    Declare Function waveOutGetVolume Lib "coredll.dll" (ByVal uDeviceID As Integer, ByRef lpdwVolume As Integer) As Integer
    Declare Function waveOutSetVolume Lib "coredll.dll" (ByVal device As IntPtr, ByVal volume As Integer) As Integer

    Public Shared Property AudioVolume() As Integer
        Get
            Dim _volume As Integer
            waveOutGetVolume(IntPtr.Zero, _volume)
            Return _volume
        End Get

        Set(ByVal value As Integer)
            waveOutSetVolume(IntPtr.Zero, value)
        End Set
    End Property

    Public Enum Volumes
        OFF = 0
        LOW = 858993459
        NORMAL = 1717986918
        MEDIUM = -1717986919
        HIGH = -858993460
        VERY_HIGH = -1
    End Enum

Creating the Cowbell

Pretty Pictures and sounds

This is a cowbell application so we need an actual cowbell on it. After a quick image search, I found the cowbell image on the internet and the X is from the Famfamfam Silk icon collection.

We’ll want to embed these images to make deployment easier and have everything self-contained so we’ll use the resource in the project’s property tab. To reach this, you go to Project->Properties or right click on your project and at the bottom, there is the Properties menu option.

In here, we’ll want to use the Images section along with the audio section.

When you select the section you want, click the Add Resource button and track down the file you need. Referencing these resources is pretty straight forward now but each language has a slightly different way to do it.

    Properties.Resources.cowBellImage;

    My.Resources.cowBellImage

Easy no?

Items on the form

We’ll want to turn off some elements on the form to maximize the cowbell. We’ll set the WindowState to Maximum, TopMost to True, MinimizeBox to False and FormBorder to None.

We’ll drag on two picture boxes on the form naming them picCowbell and picClose. We’ll set the background to both elements to be transparent. We’ll also want to double click on each picture boxes to create a Click event. For the picClose element, we’ll add in the Close() method to shut down the application.

We’ll also create a MouseDown event on the picCowbell. Here we’ll determine how loud to play then play the sound. We’ll create the Audio object during the form creation so it is already loaded into memory and will only be created once.

private readonly Audio audio = new Audio(Properties.Resources.CowBellAudio);
private void picCowbell_MouseDown(object sender, MouseEventArgs e)
{
    Audio.Volumes value;
    switch ((int)Math.Round((e.Y / (Height * 1.0)) * 5))
    {
        //case 0: // implied with default;
        //case 1:
        default:
            value = Audio.Volumes.LOW;
            break;
        case 2:
            value = Audio.Volumes.NORMAL;
            break;
        case 3:
            value = Audio.Volumes.MEDIUM;
            break;
        case 4:
            value = Audio.Volumes.HIGH;
            break;
        case 5:
            value = Audio.Volumes.VERY_HIGH;
            break;
    }

    // no need to take the performance hit of changing the volume
    // if the values aren't different
    if( oldValue != value)
    {
        Audio.Volume = (uint) value;
        oldValue = value;
    }
    audio.Play();
}

Private ReadOnly audio As New Audio(My.Resources.CowBellAudio)
Private Sub picCowbell_MouseDown(ByVal sender As System.Object, ByVal e As System.Windows.Forms.MouseEventArgs)
    Dim value As Audio.Volumes
    Select Case Convert.ToInt32(Math.Round((e.Y / (Height * 1)) * 5))
        Case 2
            value = audio.Volumes.NORMAL
            Exit Select
        Case 3
            value = audio.Volumes.MEDIUM
            Exit Select
        Case 4
            value = audio.Volumes.HIGH
            Exit Select
        Case 5
            value = audio.Volumes.VERY_HIGH
            Exit Select
        Case Else
            'case 0: // implied with default;
            'case 1:
            value = audio.Volumes.LOW
            Exit Select
    End Select

    ' no need to take the performance hit of changing the volume
    ' if the values aren't different
    If oldValue <> value Then
        audio.AudioVolume = Convert.ToInt32(value)
        oldValue = value
    End If
    audio.Play()
End Sub

Hailing a CAB for deployment

We’ll right click on the solution in the solution explorer or we can go to File->New->Project. If you do the file menu route, you’ll have a screen that is slightly different than mine. At the bottom you’ll have a drop down menu where you’ll want to change it from “Create a new solution” to “Add to Solution”.

Since I want a CAB install, I’ll go to the “Other Project Types” then to “Setup and Deployment” and select a Smart Device CAB Project”.

From there, right click on “Application Folder” and go to “Add”. Select “Primary Output” and hit OK.

So with the output added in now, we’ll need to do some more magic. In the “File System” window, we’ll right-click on “File System on Target Machine” and go “Add Special Folder” then to “Start Menu Folder”.

Now that we have our start menu, we’ll go back to the “Application Menu” and right click on the “Primary output from Cowbell”. Select “Create Shortcut” and drag that shortcut to your start menu folder. I suggest you rename it too.

Then you can build it, drag the CAB over and install on your device.

Rocking out

To download the application’s full source code, please download it from http://peacelovecode.com/code/cowbell/cowbell.zip

About the Author

Clint Rutkas works for Microsoft and has created a few cool projects in the past like a Disco Dance Floor, an automated bartender system and a self-balancing skateboard. His blog is http://BetterThanEveryone.com where he posts about what crazy application idea he is working on next.

BlueBoss - Bluetooth Proximity Detection

Coding4Fun — Fri, 27 Jun 2008 01:09:42 GMT

Summary:

Bluetooth is now found in a variety of devices and enable the user to use wireless accessories. The Bluetooth protocol allows a user to “discover” any device that is in proximity to your Bluetooth radio. Why not see who is in proximity to you? Why not have the presence of a device execute programs or alert you?

Andy Konkol – http://copyandwaste.com

Hardware:

Bluetooth radio (USB dongle)
SMA Female Jack
SMA male to N-male pigtail
2.4 GHz antenna (with N-female connector)

Software:

Download:

Source / Binary Download

Bluetooth and Hardware:

Bluetooth was designed for devices to communicate wirelessly over short distances. However, with a very simple hardware modification you can extend the range of your Bluetooth radio with standard 2.4ghz antennas used in wireless networking (802.11 a/b/g).

Modifying Bluetooth dongles to accept external antennas is documented all over the Internet. In principle it is very easy: find the antenna lead and solder on a connector/antenna. I purchased a very cheap Bluetooth USB dongle on eBay and opened the casing. After finding the antenna trace on the circuit board I soldered on a SMA Female connector to it. After soldering the antenna jack in place I slipped a 3 inch chunk of heatshrink and heated it to cover the exposed circuit board. Now I had a Bluetooth radio that accepts external antennas. Adding an antenna simply increases the range of your radio, allowing you to “see” devices from a farther distance.

To connect an external antenna to the dongle I needed to use a connector converter called a pigtail. I used a SMA male to N-male pigtail. I connected one end of the pigtail to my dongle and the other to an omni-directional 9dbi panel antenna that had an N-female connector.

Software:

To take advantage of my newly modified hardware I needed to download the Coding4Fun Toolkit. Included in the toolkit is an API for Bluetooth devices. This API allows you to do a wide variety of things with your Bluetooth radio but I focused on two methods from the ServiceAndDeviceDiscovery library: DiscoverAllDevices and DiscoverDeviceByName.

DiscoverAllDevices allows you to “scan” the airwaves on the 2.4 GHz band and report back what devices your radio sees.

DiscoverDeviceByName allows you to scan for a particular device with a specified name and report back if it is present or not.

private bool DevicePresent()
{
    BluetoothDeviceServicesManager workerBTMgr = new BluetoothDeviceServicesManager();
    Device workerDevice = workerBTMgr.DiscoverDeviceByName(_watchItem.DeviceName);

    return (workerDevice != null);
}

public void Run(String Operation)
{
    switch (Operation)
    {
        case "SingleDevice":
            if (DevicePresent())
            {
                _parentForm.Invoke(_parentForm.AddToDeviceSeenList, new object[] { _watchItem });
            }
            else
            {
                _parentForm.Invoke(_parentForm.RemoveFromDeviceSeenList, new object[] { _watchItem });
            }
            break;
        case "AllDevices":
            BluetoothDeviceServicesManager workerBTMgr = new BluetoothDeviceServicesManager();
            List<Device> Devices = workerBTMgr.DiscoverAllDevices();
            _parentForm.Invoke(_parentForm.ThreadUpdateDiscoverBox, new object[] { Devices });
            break;
    }
}

Both of these methods require that the device you are scanning for is in discoverable mode (which most manufacturers enable by default). Using the two methods described above I was able to tell if a device is in proximity. And ultimately enabling me create alerts and execute programs based on what device is present.

To perform device discovery and not have my UI lag I had to create two worker threads. One worker thread to discover all devices and display it under my devices listbox, and another to discover devices by name specified in the “watchlist.” I am not an expert at multi-threaded programs but I managed to implement them without any major headaches.

User Interface:

Since the “Add to watchlist” and “Edit” buttons essentially do the same thing, I decided to overload a windows form. I also wanted to keep track of the parent form and disable it when the WatchItemForm was shown.

The second overload allows me to fill in the form control's text based on the data that is already set for the WatchItem that has been selected (the Edit button).

public WatchItemForm(Form1 f, String DeviceName)
{
    InitializeComponent();
    this._parentForm = f;
    lblDeviceName.Text = DeviceName;
}

public WatchItemForm(Form1 f, WatchItem item)
{
    InitializeComponent();
    this._parentForm = f;

    lblDeviceName.Text = item.DeviceName;
    tbxPicturePath.Text = item.ImagePath;
    tbxProgramPath.Text = item.ProgramPath;
    this._parentForm.Enabled = true;
}

I wanted to have a pop up notify window similar to outlook and was able to find Robert Misiak's NotifyWindow. This is a very simple library which allows you to create pop up notify windows very easily. I edited NotifyWindow to include a “picturepath” variable as well as picturebox on the form. As you can see creating a NotifyWindow is quite easy:

public void NotifyDeviceWindow(WatchItem x)
{
    NotifyWindow nw;
    nw = new NotifyWindow();
    //validate Alert Message
    if (x.AlertMessage == null)
    {
        nw.Text = x.DeviceName;
    }
    else
    {
        nw.Text = x.AlertMessage;
    }
    //validate picture
    if (x.ImagePath != null)
    {
        FileInfo imgfile = new FileInfo(x.ImagePath);
        if (imgfile.Exists)
        {
            nw.PicturePath = x.ImagePath;
        }
        else
        {
            MessageBox.Show("Image does not exist");
        }
    }
    nw.Notify();

    if (x.ProgramPath != null)
    {
        RunProcess(x.ProgramPath);
    }
}

Process Flow/Software Operation:

User clicks the “Discover” button, a thread is spawned which enumerates all Bluetooth devices in proximity
Thread finishes and updates the Devices listbox
User selects a discovered device and clicks “Add to watchlist” A watch item form is spawned and asks you for an alert message, a picture path, and an executable path
User clicks save , a WatchItem object is created and added to the watchList object
A timer starts and every 10 seconds a thread is spawned to discover that device by name
If the device is discovered, it is added to the “deviceSeenList” and a notify alert is sent and the executable is executed.
If that device is still present after the next timer click, no notification is sent, If that device is not present it is removed from the “deviceSeenList”

Screenshots:

Home Automation with Microsoft Robotics Developer Studio 2008

Coding4Fun — Sat, 24 May 2008 09:56:55 GMT

Typically we think of robots as machines from science fiction or as industrial robots such as those that build and paint cars. In the world of Microsoft Robotics Developer Studio, anything that has sensors and/or actuators can be considered a robot. In this article, we look at an automated house as a robot and apply the Decentralized Software Services model of Microsoft Robotics Studio to implementing some home automation tasks.

Charles Stacy Harris III

Difficulty: Intermediate

Time Required: 6-10 hours

Cost: $100-$200 for hardware (Optional, $0 if using only simulation)

Software: Visual Studio or Visual Studio Express, Microsoft Robotics Developer Studio 2008 CTP April, ControlThink Z-Wave PC SDK

Hardware: (All Optional) Elk M1G/M1EZ Security and Automation Panel, Z-Wave Dimmer Switches, Z-Wave Controller such as the ControlThink ThinkStick or equivalent.

Download: Download

House, Robot

Those of you who know me or who have read my previous Coding4Fun article Home Automation with Windows Workflow, know that I’m nuts about home automation. I’m even more passionate about robotics and I’ve wanted to try out some things with Microsoft Robotics Studio since the very first version. Recently I had a long weekend, so I spent a few hours putting together some simple home automation examples using Microsoft Robotics Developer Studio 2008 CTP April. Check out the Microsoft Robotics Developer Center for details on how to get the CTP.

The examples in this article duplicate what I did using Windows Workflow in my previous article, but instead use the Decentralized Software Services (DSS) approach of Microsoft Robotics, and Microsoft Visual Programming Language (VPL) for a simple way to write automation tasks. The Microsoft Robotics Developer Center contains a lot of detailed information about DSS, VPL, and the other technologies found in Microsoft Robotics Developer Studio. For now, let’s just cover some basics. To get further along with the code in this article, you’ll want to check out some of the excellent tutorials presented by the Microsoft Robotics team.

Services, Messages, and Ports

A DSS Service is the basic component upon which Microsoft Robotics applications are built. In fact, DSS services are a generic construct that can and have been used outside of the context of robotics. DSS services contain state and the service state is manipulated via messages sent to the service on a service port.

Messages sent to a service are structured .NET classes that may contain a message payload that determines how or even if state will be modified, or what part of a service state should be retrieved. There are also message that do not directly manipulate state, but may have some other side-effects. Services respond to messages such as CREATE, LOOKUP, UPDATE, etc. In addition, you can define messages that extend the semantics of the core messages.

Ports are the mechanism through which services communicate. Ports accept a set of message types that are defined by the service itself. In addition, ports are also used for outbound communication in situations such as subscribing to event notification from a service.

Again, there are excellent online resources for learning the details of Microsoft Robotics, DSS, and the underlying technologies for Microsoft Robotics developer Studio. You’ll probably want basic familiarity with the Microsoft Robotics architecture including the Concurrency and Coordination Runtime (CCR) along with the notion of Arbiters and iterators.

Robo-Moose

The security and automation system in my house is centered on an Elk M1G alarm panel from Elk Products Incorporated . This panel allows for up to 208 input zones in the form of contact switches, motion sensors, and so on. It also allows for up to 208 outputs, on-board task scripting and many other features. One of the key features for my use is the capability of being able to monitor and control the panel via an Ethernet adaptor. So, my first order of business was to write a DSS service – the ElkService – to communicate with the Elk M1G via sockets. This ElkService exchanges messages with the Elk M1G via an ASCII protocol that is documented here.

The code that the ElkService uses to read from the Elk M1G hardware is shown below. Note that the code looks sequential, but is actually asynchronous. The StreamAdapter.Read method sets up a task to do an asynchronous IO operation and the line “yield return (Choice)ioResultPort;” returns this task to the runtime which is iterating over all tasks returned by ElkReader.

/// <summary>
/// ElkReader opens a socket connection to the hardware panel. It then
/// enters a loop that
/// </summary>
/// <returns></returns>
public IEnumerator<ITask> ElkReader()
{
    Connect();

    // Send a request to the Elk panel get a report of the entire zone status.
    SendElkMessage(RequestStrings.ZoneStatus);

    byte[] buffer = new byte[256];

    int bytesRead = -1;
    Exception ex = null;

     do
    {
        var ioResultPort = StreamAdapter.Read(networkStream, buffer, 0, buffer.Length);

        yield return (Choice)ioResultPort;

        ex = ioResultPort;
        if (ex != null)
            throw ex;

        bytesRead = ioResultPort;
        if (bytesRead != 0)
            ProcessRawElkMessage(Encoding.ASCII.GetString(buffer, 0, bytesRead));

    } while (bytesRead != 0);
}

When the ElkService receives a packet from the hardware panel, it converts the packet from a string to a custom message containing the raw sensor data and posts the message to the main port of the ElkService itself. For example, when the service receives a “ZC” message from the hardware, it converts this to an UpdateRawZone message that contains the raw data from the security panel. This happens in the method ProcessRawElkMessage. Here is a fragment of that code:

public void ProcessRawElkMessage(string message)
{
    string messageType = message.Substring(2, 2);

    switch (messageType)
    {
        case "ZC": // Zone status change
            var zoneState = new UpdateRawZoneRequest
            {
                Id = byte.Parse(message.Substring(4, 3)),
                State = byte.Parse(message.Substring(7, 1),
                              NumberStyles.HexNumber)
            };

            var updateZoneMessage = new UpdateRawZone();
            updateZoneMessage.Body = zoneState;

            _mainPort.Post(updateZoneMessage);
            break;
…

UpdateRawZoneRequest is the message payload and is used to send data about a zone change event to a subscriber. UpdateRawZone is the actual message type that is transmitted on the port for the ElkService.

[DataContract]
[Description("UpdateRawZone Request Message Payload")]
public class UpdateRawZoneRequest
{
    [DataMember]
    public byte Id { get; set; }

    [DataMember]
    public byte State { get; set; }
}


/// <summary>
/// Update Elk zone status
/// </summary>
[Description("UpdateZone request message")]
public class UpdateRawZone : Update<UpdateRawZoneRequest,
        PortSet<DefaultUpdateResponseType, Fault>>
{
}

The message processing code can also be written a bit more compactly as:

public void ProcessRawElkMessage(string message)
{
    string messageType = message.Substring(2, 2);

    switch (messageType)
    {
        case "ZC": // Zone status change
            var updateZoneMessage = new UpdateRawZone
            {
                Body = new UpdateRawZoneRequest
                {
                    Id = byte.Parse(message.Substring(4, 3)),
                    State = byte.Parse(message.Substring(7, 1),
                          NumberStyles.HexNumber)
                }
            };

            _mainPort.Post(updateZoneMessage);
            break;
…

UpdateRawZoneHandler is a message receiver that is active on the main port and which responds to the update message by updating internal state, and notifying subscribers of the change in state.

[ServiceHandler(ServiceHandlerBehavior.Exclusive)]
public IEnumerator<ITask> UpdateRawZoneHandler(UpdateRawZone message)
{
    _state.ZoneStates[message.Body.Id - 1] = message.Body.State;

    SendNotification(_submgrPort, message);

    message.ResponsePort.Post(DefaultUpdateResponseType.Instance);

    yield break;
}

In the Zone

The ElkService represents the core service that communicates with the Elk M1G hardware. The next service – ElkZoneSensor – represents a higher level “sensor array” that is attached to the Elk hardware. ElkZoneSensor “partners” with the ElkService. This means that the ElkZoneSensor relies on the ElkService to function. The DSS infrastructure will ensure that the ElkService is started if needed when starting the ElkZoneSensor service.

The ElkZoneSensor subscribes to updates from the ElkService for changes in the hardware zones. When these changes occur, the ElkService notifies the ElkZoneSensor which updates its state and notifies any of its subscribers. The ElkZoneSensor uses a higher level message – UpdateZone – that contains a sensor ID and an enumeration that describes the state of the sensor.

[DataContract]
public class Zone
{
    [DataMember]
    [Description("The Elk hardware zone id of the sensor")]
    [DataMemberConstructor]
    public byte Id { get; set; }

    [DataMember]
    [Description("The Elk defined state of the sensor")]
    public ZoneStatus Status { get; set; }
}

Candlepower

The next services that we need for the robotic house are services to control the lights. In this case, I started with a generic contract for the DSS service. A generic contract in DSS terms just defines the set of messages to which a service will respond and defines the port type on which those messages will be sent. The generic contract does not contain code to actually implement the service behavior. One of the benefits of a generic service contract is that you can specify the contract, write algorithms that use the contract, and later connect the generic service contracts to code that actually implements the behavior. This is very much like programming to an abstract interface in the object oriented world, and having multiple classes that actually implement the interface.

We start off with a generic service called the GenericDimmer that represents a dimmer switch and we implement two concrete services: the SimulatedDimmer and the ZWaveDimmer. SimulatedDimmer just logs a message to represent its state change. The ZWaveDimmer actually controls the lights via a hardware Z-Wave controller.

Here’s what the GenericDimmer contract looks like:

namespace Robotics.GenericHouseControls.Dimmer
{


    /// <summary>
    /// GenericDimmer Contract class
    /// </summary>
    public sealed class Contract
    {
        /// <summary>
        /// The Dss Service contract
        /// </summary>
        [DataMember]
        public const String Identifier = "http://schemas.tempuri.org/2008/04/generichousecontrolsdimmer.html";
    }

    /// <summary>
    /// The GenericDimmer State
    /// </summary>
    [DataContract]
    public class GenericDimmerState
    {
        [DataMember]
        [DataMemberConstructor(Order = 1)]
        public int Id;

        [DataMember]
        public int Level;
    }

    /// <summary>
    /// GenericDimmer Main Operations Port
    /// </summary>
    [ServicePort(AllowMultipleInstances = true)]
    public class GenericDimmerOperations : PortSet<DsspDefaultLookup, DsspDefaultDrop, Get, HttpGet, On, Off, SetLevel>
    {
    }

    /// <summary>
    /// GenericDimmer Get Operation
    /// </summary>
    public class Get : Get<GetRequestType, PortSet<GenericDimmerState, Fault>>
    {

        /// <summary>
        /// GenericDimmer Get Operation
        /// </summary>
        public Get()
        {
        }

        /// <summary>
        /// GenericDimmer Get Operation
        /// </summary>
        public Get(Microsoft.Dss.ServiceModel.Dssp.GetRequestType body) :
            base(body)
        {
        }

        /// <summary>
        /// GenericDimmer Get Operation
        /// </summary>
        public Get(Microsoft.Dss.ServiceModel.Dssp.GetRequestType body, Microsoft.Ccr.Core.PortSet<GenericDimmerState, W3C.Soap.Fault> responsePort) :
            base(body, responsePort)
        {
        }
    }

    [DataContract]
    public class OnRequest
    {
    }

    [Description("Turn the switch on")]
    public class On : Update<OnRequest, PortSet<DefaultUpdateResponseType, Fault>>
    {
    }

    [DataContract]
    public class OffRequest
    {
    }

    [Description("Turn the switch off")]
    public class Off : Update<OffRequest, PortSet<DefaultUpdateResponseType, Fault>>
    {
    }

    [DataContract]
    public class SetLevelRequest
    {
        [DataMember]
        [DataMemberConstructor]
        public int Level { get; set; }
    }

    [Description("Set the dim level")]
    public class SetLevel : Update<SetLevelRequest, PortSet<DefaultUpdateResponseType, Fault>>
    {
    }
}

This code defines the contract only. Now to implement that contract we can run the DssNewService utility with the /implement switch and generate a service based on this contract. For example:

dssnewservice /s:SimulatedDimmer /n:Robotics.HouseControls.SimulatedDimmer /i:GenericHouseControls.Y2008.M04.Proxy.dll

will generate a new service called SimulatedDimmer in the .NET namespace Robotics.HouseControls.SimulatedDimmer. The service will have a stubbed out implementation of the GenericDimmer contract including service startup code, state, and stubbed out message handlers.

Here’s what the main service file looks like for the simulated dimmer after I added logging to the generated methods. I’ve also deleted some of the using statements for easier reading.

using pxdimmer = Robotics.GenericHouseControls.Dimmer.Proxy;

namespace Robotics.HouseControls.SimulatedDimmer
{


    /// <summary>
    /// House Controls Service
    /// </summary>
    [DisplayName("Simulated Dimmer")]
    [Description("The Simulated Dimmer Switch Service")]
    [Contract(Contract.Identifier)]
    [AlternateContract(pxdimmer.Contract.Identifier)]
    public class SimulatedDimmerService : DsspServiceBase
    {

        /// <summary>
        /// _state
        /// </summary>
        [ServiceState]
        [InitialStatePartner(Optional = true)]
        private pxdimmer.GenericDimmerState _state = new pxdimmer.GenericDimmerState();

        /// <summary>
        /// _main Port
        /// </summary>
        [ServicePort("/simulateddimmer", AllowMultipleInstances = true)]
        private pxdimmer.GenericDimmerOperations _mainPort = new pxdimmer.GenericDimmerOperations();


        /// <summary>
        /// Default Service Constructor
        /// </summary>
        public SimulatedDimmerService(DsspServiceCreationPort creationPort) :
            base(creationPort)
        {
        }

        /// <summary>
        /// Service Start
        /// </summary>
        protected override void Start()
        {
            base.Start();

            // Add service specific initialization here.
            LogInfo(string.Format("Dimmer State => Id:{0} Level:{1}", _state.Id, _state.Level));
        }

        [ServiceHandler(ServiceHandlerBehavior.Concurrent)]
        public IEnumerator<ITask> OnHandler(pxdimmer.On update)
        {
            LogInfo(string.Format("Simulated Dimmer => Id:{0} On", _state.Id));

            update.ResponsePort.Post(DefaultUpdateResponseType.Instance);
            yield break;
        }

        [ServiceHandler(ServiceHandlerBehavior.Concurrent)]
        public IEnumerator<ITask> OffHandler(pxdimmer.Off update)
        {
            LogInfo(string.Format("Simulated Dimmer => Id:{0} Off", _state.Id));

            update.ResponsePort.Post(DefaultUpdateResponseType.Instance);
            yield break;
        }

        [ServiceHandler(ServiceHandlerBehavior.Exclusive)]
        public IEnumerator<ITask> SetLevelHandler(pxdimmer.SetLevel update)
        {           
            _state.Level = update.Body.Level;
            LogInfo(string.Format("Simulated Dimmer => Id:{0} Level:{1}", _state.Id, update.Body.Level));

            update.ResponsePort.Post(DefaultUpdateResponseType.Instance);            
            yield break;
        }
    }
}

Check the code download for the implementation of the ZWaveDimmer. You’ll find that it’s pretty straightforward. It should be simple to write similar code if you use a different technology for lighting control. You just write a service that matches the GenericDimmer contract, and you should be able to configure the GenericDimmer using a manifest to point to the actual dimmer service. Check out the documentation in Microsoft Robotics Developer Studio for more information on manifests and how to use the manifest editor.

Bulb On, Bulb Off!

The “Hello, World” application for any home automation software that I’ve tested in my house has always been the automating of my pantry light. The pantry has a Z-Wave light switch inside and the pantry door has a contact switch that is connected to the Elk M1G. When the door is open, we want the light to turn on, and when the door is closed we want the light to turn off. Now that we have all the pieces for this, here’s what the VPL diagram for this looks like:

As you can see, it was also a relatively easy matter to add text-to-speech to announce the opening and closing of the pantry door. You may find out quickly that this is an annoyance to those in the testing areas – for me that was my kitchen – so it’s nice to know that two quick hits of the delete key will also remove the text-to-speech. This is part of the beauty of the Microsoft Robotics programming model!

The second example shows how you can use a motion detector – in this case connected to zone 19 on the Elk – to trigger a light. The light comes on at a level that you control from the VPL code, and stays on for a duration that you can also control from the code. For the example, I set the timer to a pretty low level, but you can customize it. You could also add additional rules to only turn on the light at certain times, or better still connect an ambient light sensor to decide whether you should turn the light on.

In this diagram, if motion is detected – that is zone 19 is triggered with a status of ViolatedOpen – the light is turned on at a low dim level and a timer is set. When the timer completes, the light is turned off. If, however, there is more motion before the timer completes, the SetTimer message will cancel the old timeout and restart the timer.

There are more VPL examples included in the download including an extra service that you can use to send email from your VPL code. I’m experimenting with using the email service to send me a text message whenever my alarm is armed or disarmed.

Miscellanea

Since I haven’t had a lot of time to study the testing framework and simulation environments used by Microsoft Robotics, I included a service called the ElkTestDataPlayback service that allows me to replay the contents of a text file to simulate the ElkService getting actual data. This works pretty well for testing. The contents of the file are merely raw data captures that I made from the data sent by the Elk M1G panel. To use this service in VPL, you just drop it onto your diagram, configure it’s TestDataFileName via “Set initial configuration” to point to a data file, and send it a message to start playback.

Important: If you have the Elk hardware, you will want to uncomment the SpawnIterator(ElkReader) line in the ElkService.Start method. I know this is a hack for now. It would probably be better to write a SimulatedElk service based on a generic service contract. If I have time, I will do that in the future. That would involve a lot more work than I had for this weekend project!

TODO List

Since this was my first foray into Microsoft Robotics Developer Studio and I had limited time to really explore the features in depth, there are a few things that I’ve left unimplemented. Here are some of the things that I would like to expand on in the future:

1. Support for more of the Elk features: Currently, I only support Zone Change notifications. Fuller support of the Elk hardware would not be difficult and would be beneficial to anyone wanting to use this on an ongoing basis. There’s some stub code for a few other notifications if you want to add to the service.

2. Generic contracts for the Elk M1G: I didn’t base the Elk services on generic contract because I’m not familiar with enough other types of security and automation panels to make the needed generalizations. It would be interesting to try to come up with something more generic. I thought about using the Generic Contact Sensor array, but I’m not sure that it models what we need. For example, a door contact is considered violated when it’s open, but a water sensor is considered violated when it’s closed.

3. Additional Z-Wave features: I’ve left out notification for lighting level changes, support for scenes and a host of other features. There’s so much more that’s possible with full Z-Wave support.

4. Refined Generic Contracts: Generic contracts allow developers to write code that is independent of the final hardware. That is one of the major benefits of the model used by Microsoft Robotics. You could for example have a generic light switch for simulation and testing, and based on configuration you could plug in a Z-Wave switch or an Insteon switch when it’s time to deploy the actual solution.

5. Simulation Support: One of the major benefits of Microsoft Robotics Developer Studio is the simulation environment. For example, if we had a motion detector in the simulation environment, we could test some of the algorithms that I presented above by having a simulated mobile robot “violate” the motion detector’s zone and trigger the corresponding VPL code.

6. Exception Handling: As I build more on the code in this article, I’ll add exception handling and retry logic. Microsoft Robotics has some extremely robust features such as Causalities that make handling exceptions in a concurrent environment much easier.

7. HttpHandlers: I have a start at code for handling HTTP requests in the ZWaveController service. Handling HTTP requests is handy for building client applications that access your services without themselves having to be DSS services.

Other experiments that might be fun to try include integrating your Microsoft Robotics controlled home with remote control applications that are based on WPF or Silverlight, or perhaps integrating control from a Windows Media Center PC or Windows Home Server.

I hope you enjoy Microsoft Robotics Developer Studio as much as I have. I look forward to seeing what other people build with this fantastic technology. If people are interested, I can continue to share my home automation experiments. I think Microsoft Robotics Developer Studio will be the major tool in my home automation toolbox for quite some time.

Project Settings Note:

You will have to modify the project settings for the projects included in the House.Robot solution since your installation of Microsoft Robotics Developer Studio will be in a different directory than mine. The DssProjectMigration.exe command line tool will make short work of this. Just run it with a parameter that specifies the directory you want it to search, and it will find all of the project files in that directory and subdirectories and convert them to be buildable with your installation settings.

If you use the ElkTestDataPlayback service, in the examples, you’ll need to set the path to the location of the test data file either in the initial configuration for the component, or in a manifest that configures the component. Make sure that you do not put quotes around the path name. I included a test data file from my system.

Animated Musical Holiday Light Show - Version 2.0

Coding4Fun — Mon, 24 Dec 2007 01:25:00 GMT

	Create an animated holiday light show for indoor or outdoor use using some Phidget Interface Kits, extension cords, and .NET.
Brian Peek ASPSOFT, Inc. Difficulty: Intermediate Time Required: 3-6 hours Cost: $100-$200 Software: Visual Basic or Visual C# 2008 Express Editions, Phidgets .NET libraries Hardware: Phidget Interface Kit 0/0/4, extension cords, wire nuts, spare wire Download: Download Discussion Forum: Forum

Welcome to version 2.0 of the software and article! I have rewritten the key points of this article to address the new features in the latest version of the software. Read on for a full explanation...

New Features for 2.0

MIDI support
- Load a MIDI file and a sequence will be automatically created based on the data contained in the file. You can automatically create your own show with barely any effort at all!
Playlists
- Load up several sequences and have them play in order to create a multi-song show!
More accurate timing
- The timing algorithm used in the last version was very imprecise...it worked, but it wasn't perfect. The new version uses a method which gives millisecond precision timing
Cut/Copy/Paste
- One can now cut/copy/paste cells on the grid...makes it easy to repeat chunks of sequences without having to tap out the rhythm by hand over and over again
Edit existing sequences
- After loading a sequence created by someone else (or yourself), you can now re-assign Phidget boards, serial numbers, output ports, etc. to get them to run on your setup.

Note that any sequences created with the old version will continue to play just fine, but editing them could be troublesome due to the new timing method.

And now back to the article with updates. The hardware section remains the same, but the software discussion below is updated with new information.

I'm sure by now everyone with a computer has seen the videos of holiday light shows timed to holiday music such as this one. For this holiday season, I decided to create my own indoor show using some off-the-shelf components and .NET.

WARNING: The hardware portion of this project uses standard 120V AC current. As you are likely aware, this is enough voltage to seriously hurt or kill you. Please be careful and follow the instructions closely.

What You Will Need

Phidget Interface Kit 0/0/4 – a USB-controlled board with 4 relays. I used 2 kits for a total of 8 channels. Any number can be used.
5 6' (or longer/shorter for your needs) indoor or outdoor 2-prong extension cords for each Phidget board used. I was able to find these at the local home improvement store for about $1 each. If you will be creating an outdoor show, be sure to purchase extension cords that are meant for outdoor use.

Some 14-16 gauge wire. If in doubt, simply buy one additional extension cord to use for the additional wire.
2 wire nuts that can handle the wire gauge selected above for each Phidget board used.

A project box large enough to hold one or many Phidget boards and associated wiring. I chose a 7"L x 5"W x 3"H enclosure from Radio Shack which provides ample room to hold 2 Phidget Interface Kits and wiring.
Some holiday lights
External speakers (or possibly an FM transmitter for an outdoor show)
Phidgets .NET libraries
Microsoft Visual Basic or Visual C# Express Editions

Building the Hardware

The hardware we are going to build will allow for one Phidget board to be plugged into a single AC outlet and provide 4 output outlets that can be switched off and on by the Phidget board's relays. By building two of these and placing them in a project box, I have a neat and tidy control box with 2 USB inputs, 2 AC male plugs for the wall, and 8 AC female plugs for my lights. The following description will be for building a single unit.

Let's start by preparing the extension cords. Cut the female end off of one extension cord. Split the cord up the center and strip the insulation off each of the wires to expose the ends. Twist the ends with your fingers to create a neat, twisted wire as shown:

Next, take the remaining four extension cords and cut the male ends off each. As before, split the cord up the center a small bit, strip off some insulation, and twist the exposed ends as shown:

Now, cut 4 equal lengths of your 14-16 gauge wire. These should be no longer than 2-4 inches in length. Strip some insulation off each end and twist up any loose wires.

For the next part, you will need to pay close attention. Each extension cord should have two different types of insulation around the wires. One side should have a ribbed edge, and one side should have a smooth edge shown below:

The ribbed side should be in line with the "fat" prong/receptacle (the neutral side) and the smooth side should be in line with the smaller prong/receptacle (the active/"hot" side). It is important that the next steps be followed carefully, noting which wires I am referring to. Using the wrong wire can lead to a short, blown fuses, kicked circuit breakers, or even worse things.

Take the 4 short wires you cut earlier and twist them all together along with the ribbed/neutral wire from the extension cord with the male end still attached.

Twist a wire nut over the exposed ends to keep them together and covered.

Next, twist together the smooth/"hot" wire from the four extension cords with the female ends still attached along with the smooth/"hot" wire from the extension cord with the male end attached (i.e. the other wire from the male cord used above).

Again, twist on a wire nut to keep things covered and safe.

You should now be left with the opposite ends of the 4 cut wires exposed, and the 4 ribbed/neutral wires of the female extension exposed. These will all be put into the screw terminals of the Phidget Interface Kit.

The Phidget Interface Kit board has 4 groups of screw terminals. Each group contains 3 items: NO, XC, and NC, where X is the relay number in question. These stand for "Normally Open", "Common", and "Normally Closed". For this project, the lights should normally be off and switched on via the software, so the NO (Normally Open) and XC (Common) ports will be used.

Place one wire from each exposed bundle into each NO and XC port. Note that it does not matter which wire you plug into which terminal of each group, just that each group has only one short wire and only one extension cord wire.

In the end, there will be one each of the 4 short wires in each group on the board, and one each of the extension cord wires in each group on the board as pictured:

Below is a very simple schematic of the wiring of a single board:

I decided to keep things neat and tidy and mount the Phidget boards into a project box. Since I have 2 boards to manage, I placed both inside a single, large box. I drilled holes in the short sides to expose the boards' USB ports. I then notched out some spaces on the top edge and lid for the extension cords to pass through. I mounted the boards inside the box with some carefully placed two-sided foam tape.

The finished product can be seen below:

And that's it! If this box will be placed outside, take the time to properly weatherproof the box so the elements cannot damage anything inside.

A Word on Amperages

Your individual light strings will be plugged into each extension cord outlet. An average strand of mini-lights draws about .3 amps. An average string of larger bulbs will draw 1-2 amps. The relays on the Phidget board are rated at 10 amps. Additionally, a standard house circuit will allow up to 15-20 amps before overloading. Check the circuit breaker in your home on which the outlet you'll be using lives for the allowed amperage. Also, keep in mind that any other devices that are plugged into that circuit elsewhere in the house will be drawing power, so you may not be able to draw a full 15 amps from it. So, be sure to not draw more than 10A per channel, nor more than 15-20A in total, including all additional devices plugged into that circuit. Keep this in mind as you string your lights together on each channel.

Writing the Software

Ensure the Phidgets libraries for .NET are installed on the development machine. To compile the source code, Visual Basic and/or Visual C# Express 2005 will also be need to be installed and working.

The Light Sequencer application uses a grid-style interface to show the list of channels and when each channel is switched on and off.

Squares can be toggled on or off by highlighting them, right-clicking with the mouse and selecting On or Off from the context menu. They can also be toggled by pressing the O (for on) or F (for off) keys on the keyboard.

Additionally, sequences can be "recorded" by pressing the Record button in the toolbar. This will start the music and allow the user to tap out the rhythm for each channel by pressing the number key on the keyboard corresponding to the channel.

Sequences can be saved at any time and reloaded for editing or play with the Phidgets devices connected.

When I first started the software for this project, the first problem I ran into was the fact that the DataGridView control does no support multiple headers. As is shown in the screenshot above, the grid is broken down into seconds and then milliseconds. For display purposes. it is easier to break the header down into two segments, one showing the labeled second markers, and one showing the subdivisions per second.

To accomplish this, I created two DataGridViews: one for the header, which contains no data and is only as tall as the header row, and one for the sub-header and the data below it. This works great except for scrolling. To accomplish this, I simply listen for the Scroll event on the main grid and apply the scrolling offset to the header grid:

Visual C#

private void dgvMain_Scroll(object sender, ScrollEventArgs e)
{
    dgvHeader.HorizontalScrollingOffset = e.NewValue;
}

Visual Basic

Private Sub dgvMain_Scroll(ByVal sender As Object, ByVal e As ScrollEventArgs) Handles dgvMain.Scroll
    dgvHeader.HorizontalScrollingOffset = e.NewValue
End Sub

Additionally, I ran into some performance issues drawing the grid. At first, drawing a grid with so many columns was quite slow. By setting the grid's Visible property to false before adding the rows and columns and then returning the Visible property to true, the grid now draws quite quickly.

The next issue tackled was starting and stopping a music file. In version 2, the software supports sampled music (MP3, WAV, etc.) as well as MIDI files. As I was attempting to write a MIDI file parser and player (so I could have access to the internal data for auto-generating a sequence) I found a fantastic MIDI library written by Leslie Sanford. This library is used by the Light Sequencer application.

With two playback libraries in place, I created an interface which contains Start, Stop, Load, etc. methods so that the front-end could use any playback engine interchangeably. The MCIPlayback engine uses standard MCI commands for playing sampled music. Commands are executed by passing them to the mciSendString function exported by winmm.dll. In order to use this function from .NET, we must import the method and setup its signature as follows:

Visual C#

[DllImport("winmm.dll")]
static extern Int32 mciSendString(String command, StringBuilder buffer, Int32 bufferSize, IntPtr hwndCallback);

Visual Basic

Declare Function mciSendString Lib "winmm.dll" Alias "mciSendStringA" (ByVal command As String, _
   ByVal buffer As StringBuilder, ByVal bufferSize As Int32, _
   ByVal hwndCallback As IntPtr) As Int32

To open a music file, the open command is used as follows:

open "<path to file>" type mpegvideo alias MediaFile

This opens the file and creates an alias named MediaFile which can be used to refer to the file for all future commands. Send the above command using the mciSendString method would look as follows:

Visual C#

string cmd = "open \"" + _musicFile + "\" type mpegvideo alias MediaFile";
mciSendString(cmd, null, 0, IntPtr.Zero);

Visual Basic

Dim cmd As String = "open """ + file + """ type mpegvideo alias MediaFile"
mciSendString(cmd, Nothing, 0, IntPtr.Zero)

The remaining commands we will need are:

play MediaFile from 0 -- Play loaded file from beginning
stop MediaFile -- Stop playing current file
close MediaFile -- Close the file

Next, the Phidget boards behavior needed to be implemented. Talking to the Phidget board is very easy. After creating an instance of the InterfaceKit object, a specific device can be opened by calling the open method, passing in the serial number of the device to be opened.

The serial numbers for each attached Phidget device can be determined by creating an instance of the Phidgets.Manager class, setting up the Attach event handler, and listening for the attach events as follows:

Visual C#

...
Phidgets.Manager phidgetsManager = new Phidgets.Manager();
phidgetsManager.Attach += new AttachEventHandler(phidgetsManager_Attach);
phidgetsManager.open();
...

void phidgetsManager_Attach(object sender, AttachEventArgs e)
{
    Debug.WriteLine(e.Device.Name + " - " + e.Device.SerialNumber)
}

Visual Basic

...
Dim phidgetsManager as New Phidgets.Manager
AddHandler phidgetsManager.Attach, AddressOf Me.phidgetsManager_Attach
phidgetsManager.open()
...

Private Sub phidgetsManager_Attach(ByVal sender As Object, ByVal e As AttachEventArgs)
    Debug.WriteLine(e.Device.Name & " - " & e.device.SerialNumber)
End Sub

Setting the relay state is as easy as indexing into the outputs array of the InterfaceKit object and setting the indexed output to true or false.

In code, all of this would look like:

Visual C#

InterfaceKit ik = new InterfaceKit();
ik.open(1234);
ik.outputs[0] = true;

Visual Basic

Dim ik as New InterfaceKit
ik.open(1234)
ik.outputs(0) = True

In order to maintain precise timing, the Stopwatch class from the System.Diagnostics namespace is used. This internally uses the QueryPerformanceCounter Win32 API method to give extremely precise time values.

When it is time to playback a sequence, a thread is started which starts the music using the appropriate playback engine, and then sits in a loop, waiting for the number of milliseconds to pass specified in the sequence (50ms as default). When that amount of time has elapsed, we send the channel states of the current tick to the relays connected.

Recording a sequence with the keyboard works in a similar fashion. A thread is started and the music is played. While the music is playing, KeyDown and KeyUp events are listened for. After translating the KeyCode of the pressed key to the channel number, an internal array of which keys are "on and off" are maintained. When the number of milliseconds elapsed hits the appropriate mark, every channel is updated with the current value of that array. That is, which keys are up and down.

When the user stops playback, or the song ends, the channel data is returned to the main form and displayed in the main grid.

If a MIDI file is selected, sequence data is automatically generated based on the MIDI data. A MIDI file contains a series of tracks or channels with a series of commands. These commands tell the MIDI hardware what note to turn on, when, and for how long (among other things). When the MIDI file is loaded, every command from every channel is enumerated and its time values are converted into milliseconds. Once all the commands are gathered and organized, they are placed into this application's channel structure and displayed on the grid. This allows the lights to flash in precise time to the MIDI file being played.

Using the Software

Ensure that the Phidget devices you will be using are attached to the PC. Start by creating a new sequence from the File menu or by clicking the New Sequence button. In the dialog that appears, locate a music file to play back and enter the length of time that the sequence should run. Be sure to note which Phidget devices are attached and which channels they map to on the grid. Click OK when complete.

The screen will redraw and present the grid interface for the length of time specified. At this point, cells can be turned on and off by highlighting a cell and right-clicking, or by pressing the "O" key to turn the cell On, and the "F" key to turn the cell off. Multiple cells can be selected and changed at once.

To use the recording interface, click the Record Sequence button or choose Record Sequence from the Sequence menu. Be sure to select the correct choice of "Overwrite channel data" or "Append channel data." As you record additional channels, you will almost always want to append and not overwrite.

Click the start button and a brief countdown will begin. When the countdown reaches 0, the music will begin. A channel can be recorded by pressing the keyboard key of the channel number. For example, to tap out the rhythm of channel 1, press the 1 key at the appropriate times.

When complete, press the Escape key, or click the Stop button. When the Record window is closed, the main grid will be updated with the sequence recorded.

Creating a sequence is certainly a time consuming task since each channel needs to be recorded. While the rhythm interface allows one to record many channels simultaneously, I think it would be impossible for anyone to type out an entire sequence for all channels in one go. In my opinion, it is easiest to record one or two channels at a time and append the data as you go. In the end, you can use the grid interface to tweak the values and clean up any mistakes.

The sequence can be played back at any time. Simply press the Play button and watch your holiday lights play back to the timing you created. Press the Stop button to end the current playback.

Sequences can be saved at any time by selecting Save from the file menu.

To test the channels by hand, select "Test Channels" from the "Tools" menu. As with the recording screen, press the number keys associated with the channel to turn on or off to test that channel.

I found it was much easier to also have the lights plugged into the appropriate channels as I created my sequence. That way I could see the results of my recordings immediately.

To create a playlist of many sequences, select New Playlist from the File menu. Add your existing sequence files, order them as you wish, and save the playlist. From this screen you can also play the playlist, set it up to repeat, advance tracks, etc.

To edit data on an existing sequence (Phidget serial numbers, mapped MIDI channels, etc.) select Edit Sequence Properties from the Sequence menu. This will display the New Sequence dialog box and allow you to edit the existing setup.

Setting It All Up

So now that you have hardware, music and an animated sequence, it's time to hook it all up! If you are doing an indoor show, a set of external speakers should be more than ample for playing the music for your light show. For an outdoor show, you may wish to purchase an FM transmitter to output the music over a very low-powered FM frequency so that visitors can listen to the music on their car radios. I personally have not gone this route, however you can find a variety of FM transmitters for sale around the 'net. In a quick search, I found Ramsey Electronics which sells a variety of FM transmission hardware that is more than appropriate depending on your budget. I am certain there are plenty of other devices that fit the bill as well.

To fire up the show, just plug in your USB Phidget devices, plug the lights into the appropriate channels, load the Light Sequencer application, and press the Play button!

Conclusion

And there we have it! Holiday lights timed to your favorite holiday song. Take your time in creating a sequence and show us what you've created!

I plan on maintaining and updating this article as we get closer and closer to the holidays, so please check back often for updates. I will note updates at the top of the article. Additionally, please send me any and all feedback, bug reports, feature requests, or anything else you have to say! You can find my contact info in the readme.txt file located in the source code download linked above, or visit my website.

Thanks

Special thanks to Michelle Leavitt for help setting up lights and sequence ideas, and my dad for advice on wiring up the relays.

Thanks to Leslie Sanford for the incredible MIDI Toolkit.

And, a big thank you to the beta testers for version 2: Allen Leno, Steve Runion, Steve Trueman, Corey Emmert.

Bio

Brian is a Microsoft C# MVP and a recognized .NET expert with over 6 years experience developing .NET solutions, and over 9 years of professional experience architecting and developing solutions using Microsoft technologies and platforms, although he has been "coding for fun" for as long as he can remember. Outside the world of .NET and business applications, Brian enjoys developing both hardware and software projects in the areas of gaming, robotics, and whatever else strikes his fancy for the next ten minutes. He rarely passes up an opportunity to dive into a C/C++ or assembly language project. You can reach Brian via his blog at http://www.brianpeek.com/.

WiiEarth -- Wiimote Interface for Virtual Earth

Coding4Fun — Thu, 18 Oct 2007 12:30:00 GMT

	In this article, Brian Peek will demonstrate how to use a Nintendo Wii Remote (Wiimote) as a controller for Microsoft Virtual Earth 3D.
Brian Peek ASPSOFT, Inc. Difficulty: Intermediate Time Required: 2-3 hours Cost: $60 for Wiimote and Nunchuk Software: Managed Library for Nintendo's Wiimote, Visual Basic or Visual C# Express Editions Hardware: Nintendo Wii Remote (Wiimote) with Nunchuk, a compatible PC Bluetooth adapter and stack, optional IR sensor bar

Note: As of April 18, 2008 this code no longer works due to API changes in VE 3D. We will keep this article up on Coding4Fun.

Introduction

Virtual Earth is the 3D interface to Microsoft's Live Maps service. Normally this control is loaded via the web browser and allows interaction with a keyboard, mouse, and Xbox 360 controller. In this article, we will take the Virtual Earth control out of the web browser, use it in a WinForms application, and control it with a Nintendo Wii Remote (Wiimote). Note that use of the Virtual Earth 3D control in this way is undocumented and unsupported at the moment. Because of this, some of the descriptions in this article are educated guesses and may not be 100% accurate.

<\/embed><\/div>";" alt="">

Setup

Before we get started, you will need to install the Virtual Earth 3D control. If you haven't done this already, browse to http://maps.live.com/ and click on the Install 3D link to install the control and supporting software.

Additionally, if you haven't already, please review my Managed Library for Nintendo's Wiimote article on this site. We will be using the library in this article, but I will not repeat the basic information that is located in the original article. Note that this application uses a newer version of the Wiimote library which is not yet uploaded. It will be available in a few days.

Implementation

The Virtual Earth 3D Control

The Virtual Earth 3D (VE3D) control is intended to be used through a well documented JavaScript interface from a web page, however we would not be able to access the Wiimote from a web browser. Therefore, we will be using the VE3D control through its native, but wholly undocumented interface. Note that on 10/15/07 a new version of VE3D was released that changed the API drastically from the previous version. This article reflects the newer version.

Start by creating a new Windows Forms application named WiiEarth in C# or VB. As with all controls and 3rd party libraries, a reference needs to be set to the Virtual Earth 3D libraries. Unfortunately this cannot be done from the Visual Studio IDE because of the way the control is installed to the Global Assembly Cache. So, to set reference to the necessary assemblies, ensure the project is not open in Visual Studio and open the .csproj/.vbproj file in notepad. Add the following XML in the <ItemGroup> which contains the base references, such as System and System.Data:

<Reference Include="Microsoft.MapPoint.Data">
  <Private>False</Private>
</Reference>
<Reference Include="Microsoft.MapPoint.Data.CompactMapFile">
  <Private>False</Private>
</Reference>
<Reference Include="Microsoft.MapPoint.Data.VirtualEarthTileDataSource">
  <Private>False</Private>
</Reference>
<Reference Include="Microsoft.MapPoint.Geometry">
  <Private>False</Private>
</Reference>
<Reference Include="Microsoft.MapPoint.Graphics3D">
  <Private>False</Private>
</Reference>
<Reference Include="Microsoft.MapPoint.GraphicsAPI">
  <Private>False</Private>
</Reference>
<Reference Include="Microsoft.MapPoint.Rendering3D">
  <Private>False</Private>
</Reference>
<Reference Include="Microsoft.MapPoint.Utility">
  <Private>False</Private>
</Reference>
<Reference Include="Microsoft.MapPoint.Rendering3D.Utility">
  <Private>False</Private>
</Reference>
<Reference Include="Microsoft.MapPoint.Rendering3D.WorldMemoryDataSource">
  <Private>False</Private>
</Reference>

With the references in place, the project file can now be opened and the references will be seen in the References folder in the Solution Explorer as usual.

// the Virtual Earth 3D control
private GlobeControl globeControl;

private void MainForm_Load(object sender, EventArgs e)
{
    // create a new instance of the VE control
    this.globeControl = new GlobeControl();

    // setup the globeControl to fill the window
    this.globeControl.Dock = DockStyle.Fill;
    this.globeControl.Location = new System.Drawing.Point(0, 0);
    this.globeControl.Name = "Globe";
    this.pnlGlobe.Controls.Add(globeControl);
}

' the Virtual Earth 3D control
Private globeControl As GlobeControl

Private Sub MainForm_Load(ByVal sender As Object, ByVal e As EventArgs) Handles MyBase.Load
    ' create a new instance of the VE control
    Me.globeControl = New GlobeControl()

    ' setup the globeControl to fill the window
    Me.globeControl.Dock = DockStyle.Fill
    Me.globeControl.Location = New System.Drawing.Point(0, 0)
    Me.globeControl.Name = "Globe"
    Me.pnlGlobe.Controls.Add(globeControl)
End Sub

This sets up the VE3D control in its default state. If you were to run an application with only this code, you would see nothing but the earth. The navigation controls and other extras would be missing. If you wish to add the default navigation controls to the screen, the PlugInLoader object is used. The PlugInLoader is created by using the CreateLoader static method, passing in an instance of the GlobeControl's Host object. Then, the NavigationPlugIn can be loaded and activated as shown:

// load all the spiffy UI navigation goodies
PlugInLoader loader = PlugInLoader.CreateLoader(this.globeControl.Host);
loader.LoadPlugIn(typeof(NavigationPlugIn));
loader.ActivatePlugIn(typeof(NavigationPlugIn).GUID, null);

' load all the spiffy UI navigation goodies
Dim loader As PlugInLoader = PlugInLoader.CreateLoader(Me.globeControl.Host)
loader.LoadPlugIn(GetType(NavigationPlugIn))
loader.ActivatePlugIn(GetType(NavigationPlugIn).GUID, Nothing)

Data layers are created from specially formatted data sources provided by local.live.com known as content manifests. These are XML files which tell the VE3D control how to load the data required for any view. The following helper method can be used to easily load data layers:

private void AddFeederSource(string uri, string layerId, string id, DataSourceUsage usage)
{
    // create a data source from a URL
    DataSource dataSource = VirtualEarthTileDataSource.ConnectTo(
        new Uri(uri),
        this.globeControl.Host.WebProxy,
        WorldMemoryDataSource.DataPath);

    // create a layer from that data source
    DataSourceLayerData layer = new DataSourceLayerData(layerId, id, dataSource, usage);

    // add it to the globe
    this.globeControl.Host.DataSources.Add(layer);
}

Private Sub AddFeederSource(ByVal uri As String, ByVal layerId As String, ByVal id As String, ByVal usage As DataSourceUsage)
    ' create a data source from a URL
    Dim dataSource As DataSource = VirtualEarthTileDataSource.ConnectTo(New Uri(uri), Me.globeControl.Host.WebProxy, WorldMemoryDataSource.DataPath)

    ' create a layer from that data source
    Dim layer As DataSourceLayerData = New DataSourceLayerData(layerId, id, dataSource, usage)

    ' add it to the globe
    Me.globeControl.Host.DataSources.Add(layer)
End Sub

With this helper method in place, we can add any of the following layers (note that there may be other content manifests provided by local.live.com, but these are the only 5 that I am aware of):

URL	DataSourceUsage Type	Description
http://local.live.com/Manifests/HD.xml	ElevationMap	Terrain data
http://local.live.com/Manifests/MO.xml	Model	3D buildings
http://local.live.com/Manifests/AT.xml	TextureMap	Unlabeled aerial
http://local.live.com/Manifests/HT.xml	TextureMap	Labeled aerial
http://local.live.com/Manifests/RT.xml	TextureMap	Roads only

For the best display, add the ElevationMap, Model and Aerial TextureMap layers as shown:

// setup the default aerial layer, buildings, and elevation data
AddFeederSource("http://local.live.com/Manifests/HD.xml", "Terrain", "Terrain", DataSourceUsage.ElevationMap);
AddFeederSource("http://local.live.com/Manifests/MO.xml", "Model", "Model", DataSourceUsage.Model);
AddFeederSource("http://local.live.com/Manifests/AT.xml", "Roads", "Roads", DataSourceUsage.TextureMap);

' setup the default aerial layer, buildings, and elevation data
AddFeederSource("http:'local.live.com/Manifests/HD.xml", "Terrain", "Terrain", DataSourceUsage.ElevationMap)
AddFeederSource("http:'local.live.com/Manifests/MO.xml", "Model", "Model", DataSourceUsage.Model)
AddFeederSource("http:'local.live.com/Manifests/AT.xml", "Roads", "Roads", DataSourceUsage.TextureMap)

If you were to run the application at this point, you would see a fully functioning Virtual Earth 3D control with proper data and navigation.

Control Scheme and Bindings

Controlling VE3D with the Wiimote will be accomplished using a control scheme that is very similar to most first person shooter (FPS) games on the Wii. The nunchuk, held in the left hand, will move the camera forward/back/left/right using the joystick. The C and Z buttons on the front of the nunchuk will be used to raise and lower the altitude of the camera. The Wiimote, held in the right hand, will be used to change the tilt and turn of the camera. The buttons on the Wiimote will also be hooked up to several VE3D functions. Home will center the map to an overhead view at the current camera position. The 1 button will toggle through the road layers. The 2 button will toggle the overlaid UI off and on.

VE3D bindings allow you to change or create new control schemes for VE3D. Open your %APPDATA%\Microsoft\Virtual Earth 3D directory. On Windows XP, %APPDATA% should resolve to \Documents and Settings\<user>\Application Data . On Windows Vista, it should resolve to \Users\<user>\AppData\Roaming . In this directory you will find a Bindings.xml file. This XML schema defines the default keyboard, mouse, Gamepad and other input device properties. Open the file to see the schema used to define events and parameters.

By default, VE3D will load any file named Bindings*.xml from this directory. For the Wiimote control scheme, create a new file named BindingsWiimote.xml in this directory. Set the contents of the file to the following:

<?xml version="1.0" encoding="utf-8" ?>
<Bindings>
    <BindingSet Name="wiimoteBindings" AutoUse="True" Cursor="Drag">
        <!-- Nunchuk joystick -->
        <Bind Event="Wiimote.NunchukX" Action="Strafe" Factor="1" />
        <Bind Event="Wiimote.NunchukY" Action="Move" Factor="1" />

        <!-- Nunchuk joystick with modifier since we can move and turn at the same time -->
        <Bind Event="Wiimote.B+Wiimote.NunchukX" Action="Strafe" Factor="1" />
        <Bind Event="Wiimote.B+Wiimote.NunchukY" Action="Move" Factor="1" />

        <!-- Nunchuk buttons -->
        <Bind Event="Wiimote.NunchukC" Action="Ascend" Factor="0.3" />
        <Bind Event="Wiimote.NunchukZ" Action="Ascend" Factor="-0.3" />

        <!-- Nunchuk joystick with modifier since we can move and change altitude -->
        <Bind Event="Wiimote.B+Wiimote.NunchukC" Action="Ascend" Factor="0.3" />
        <Bind Event="Wiimote.B+Wiimote.NunchukZ" Action="Ascend" Factor="-0.3" />

        <!-- Wiimote buttons -->
        <Bind Event="Wiimote.Home" Action="ResetOnCenter" />
        <Bind Event="Wiimote.One" Action="ToggleRoads,WiiEarth, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/>
        <Bind Event="Wiimote.Two" Action="ToggleUI,WiiEarth, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/>
        <Bind Event="Wiimote.A" Factor="1" Action="Locations,WiiEarth, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/>

        <!-- Wiimote IR for pitch/turn -->
        <Bind Event="Wiimote.B+Wiimote.IRX" Action="Turn" Factor="0.05" />
        <Bind Event="Wiimote.B+Wiimote.IRY" Action="Pitch" Factor="-0.05" />

        <!-- Wiimote accel for pitch/turn -->
        <Bind Event="Wiimote.B+Wiimote.AX" Action="Turn" Factor="-0.025" />
        <Bind Event="Wiimote.B+Wiimote.AY" Action="Pitch" Factor="-0.025" />

        <Bind Event="Wiimote.Left" Factor="-1" Action="Locations,WiiEarth, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/>
        <Bind Event="Wiimote.Up" Factor="-1" Action="LocationsMove,WiiEarth, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/>
        <Bind Event="Wiimote.Down" Factor="1" Action="LocationsMove,WiiEarth, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/>

        <!-- FPS-style keyboard controls in case we don't have a nunchuk -->
        <Bind Event="Key.W" Action="Move" Factor="22" />
        <Bind Event="Key.S" Action="Move" Factor="-22" />
        <Bind Event="Key.D" Action="Strafe" Factor="22" />
        <Bind Event="Key.A" Action="Strafe" Factor="-22" />
        <Bind Event="Key.Space" Action="Ascend" Factor="20" />
        <Bind Event="Key.C" Action="Ascend" Factor="-20" />
        <Bind Event="Key.Z" Action="ToggleRoads,WiiEarth, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/>
        <Bind Event="Key.X" Action="ToggleUI,WiiEarth, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/>
    </BindingSet>
</Bindings>

The <BindingSet> tags wrap groups of control bindings. It requires a Name and optionally a Cursor. If the binding set is to be used automatically, as it would be in most cases, set the AutoUse parameter to True. Inside of that are <Bind> tags. The tag requires the Event, Action parameters and optionally the Factor parameter. The Event parameter will be used to match the binding to its handler which will be written later. The syntax is <Handler Name>.<Event Name>. The Action parameter is used to map the specific binding to a particular method. The Factor parameter is optional and can be used to scale the data value up or down to increase or decrease sensitivity of the input method. Once the handler is written, these will make more sense.

Note that some bindings append two Events together with a + sign. This allows for button combinations. In this case, for the accelerometer and/or IR sensor, we only want to register the action if a button is pressed down. So, those events which require the button to be held down contain Wiimote.B+ and the event it is combined with. Also note that the combination events override other events that don't have a combination listed. So, for example, NunchukC is listed as working alone and with the Wiimote.B event.

<Bind Event="Wiimote.One" Action="ToggleRoads,
        WiiEarth, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/>

You can change any of these button bindings simply by changing this XML file and deploying to the directory above. So, for example, if you wanted the trigger button to be A, you would just change Wiimote.B to Wiimote.A in the above lines and re-deploy the bindings file.

The XML file also binds several keyboard keys in a first-person shooter style layout in the event the user does not have a nunchuk for the left hand functions.

Event Source

Next, add an enumeration named WiimoteEvent (the name isn't important) which contains all of the Name items from the bindings XML file above. It should look like this:

// all events handled by this event source from XML file
public enum WiimoteEvent
{
    IRX,        // IR X position
    IRY,        // IR Y position
    NunchukX,    // Nunchuk joystick X position
    NunchukY,    // Nunchuk joystick Y position
    NunchukC,    // Nunchuk C button
    NunchukZ,    // Nunchuk Z button
    AX,            // Wiimote accelerometer X
    AY,            // Wiimote accelerometer Y
    Up,            // Dpad up
    Down,        // Dpad down
    Left,        // Dpad left
    Right,        // Dpad right
    A,            // A button
    B,            // B button
    Minus,        // Minus button
    Home,        // Wiimote Home button
    Plus,        // Plus button
    One,        // Wiimote One button
    Two,        // Wiimote Two button
}

Public Enum WiimoteEvent
    IRX ' IR X position
    IRY ' IR Y position
    NunchukX ' Nunchuk joystick X position
    NunchukY ' Nunchuk joystick Y position
    NunchukC ' Nunchuk C button
    NunchukZ ' Nunchuk Z button
    AX ' Wiimote accelerometer X
    AY ' Wiimote accelerometer Y
    Up ' Dpad up
    Down ' Dpad down
    Left ' Dpad left
    Right ' Dpad right
    A ' A button
    B ' B button
    Minus ' Minus button
    Home ' Wiimote Home button
    Plus ' Plus button
    One ' Wiimote One button
    Two ' Wiimote Two button
End Enum

Method/Property	Description
GetEventData	Unsure at the moment...does not need to be implemented?
IsModifier	Returns a boolean stating whether the passed in event ID is a modifier (such as the Wiimote.B event above)
CanModify	Returns a boolean stating whether the current event is allowed as a modifier
TryGetEventID	Maps a string event name from the bindings file to the integer value in the enumeration above
TryGetEventName	Maps an integer event ID to the string name in the enumeration above
Name (property)	Returns the name of the handler which must match the name in the XML file above (Wiimote in this case)

The code for these methods is presented below:

// return out value of the passed enum
public override bool TryGetEventId(string eventName, out int eventId)
{
    eventId = (int)Enum.Parse(typeof(WiimoteEvent), eventName);
    return true;
}

// return out the string name of the passed in enum value
public override bool TryGetEventName(int eventId, out string eventName)
{
    eventName = ((WiimoteEvent)eventId).ToString();
    return true;
}

// unknown
public override EventData GetEventData(int eventId, EventActivateState state)
{
    throw new NotImplementedException();
}

// can the event be used as a modifier?
public override bool IsModifier(int eventId)
{
    // yes to all for now
    return true;
}

// can the supplied event be used as a modifier?
public override bool CanModify(int eventId, EventKey other)
{
    // only if it's from us
    return (other.Source == this);
}

// this must match the Source name in the bindings XML file
public override string Name
{
    get { return "Wiimote"; }
}

' return out value of the passed enum
Public Overrides Function TryGetEventId(ByVal eventName As String, <System.Runtime.InteropServices.Out()> ByRef eventId As Integer) As Boolean
    eventId = CInt(Fix(System.Enum.Parse(GetType(WiimoteEvent), eventName)))
    Return True
End Function

' return out the string name of the passed in enum value
Public Overrides Function TryGetEventName(ByVal eventId As Integer, <System.Runtime.InteropServices.Out()> ByRef eventName As String) As Boolean
    eventName = (CType(eventId, WiimoteEvent)).ToString()
    Return True
End Function

' unknown
Public Overrides Function GetEventData(ByVal eventId As Integer, ByVal state As EventActivateState) As EventData
    Throw New NotImplementedException()
End Function

' can the event be used as a modifier?
Public Overrides Function IsModifier(ByVal eventId As Integer) As Boolean
    ' yes to all for now
    Return True
End Function

' can the supplied event be used as a modifier?
Public Overrides Function CanModify(ByVal eventId As Integer, ByVal other As EventKey) As Boolean
    ' only if it's from us
    Return (other.Source Is Me)
End Function

' this must match the Source name in the bindings XML file
Public Overrides ReadOnly Property Name() As String
    Get
        Return "Wiimote"
    End Get
End Property

public WiimoteEventSource(ActionSystem actionSystem, MainForm form) : base(actionSystem)
{
    // store away an instance of the main form
    _form = form;

    // setup wiimote and event handlers
    _wm = new Wiimote();
    _wm.OnWiimoteChanged += new WiimoteChangedEventHandler(OnWiimoteChanged);
    _wm.OnWiimoteExtensionChanged += new WiimoteExtensionChanged(OnWiimoteExtensionChanged);
    _wm.Connect();

    // if we don't have an extension, set the report type to IR and accel's only
    if(!_wm.WiimoteState.Extension)
        _wm.SetReportType(Wiimote.InputReport.IRAccel, true);

    // turn off all LEDs
    _wm.SetLEDs(0x00);
}

Public Sub New(ByVal actionSystem As ActionSystem, ByVal form As MainForm)
    MyBase.New(actionSystem)
    ' store away an instance of the main form
    _form = form

    ' setup wiimote and event handlers
    _wm = New Wiimote()
    AddHandler _wm.OnWiimoteChanged, AddressOf OnWiimoteChanged
    AddHandler _wm.OnWiimoteExtensionChanged, AddressOf OnWiimoteExtensionChanged
    _wm.Connect()

    ' if we don't have an extension, set the report type to IR and accel's only
    If (Not _wm.WiimoteState.Extension) Then
        _wm.SetReportType(Wiimote.InputReport.IRAccel, True)
    End If

    ' turn off all LEDs
    _wm.SetLEDs(&H00)
End Sub

The OnWiimoteExtensionChanged method simply sets the report mode for the Wiimote based on whether or not a Nunchuk is inserted as shown:

private void OnWiimoteExtensionChanged(object sender, WiimoteExtensionChangedEventArgs args)
{
    // if nunchuk inserted, set the report type to return extension data
    if(args.ExtensionType == ExtensionType.Nunchuk && args.Inserted)
        _wm.SetReportType(Wiimote.InputReport.IRExtensionAccel, true);
    else // in all other cases, set it to the default IR and accel's
        _wm.SetReportType(Wiimote.InputReport.IRAccel, true);
}

Private Sub OnWiimoteExtensionChanged(ByVal sender As Object, ByVal args As WiimoteExtensionChangedEventArgs)
    ' if nunchuk inserted, set the report type to return extension data
    If args.ExtensionType = ExtensionType.Nunchuk AndAlso args.Inserted Then
        _wm.SetReportType(Wiimote.InputReport.IRExtensionAccel, True)
    Else ' in all other cases, set it to the default IR and accel's
        _wm.SetReportType(Wiimote.InputReport.IRAccel, True)
    End If
End Sub

The OnWiimoteChanged event handler is where the Wiimote data is handled and sent off to the VE3D control to reflect the changes. First, let's handle the IR and accelerometer data. The IR midpoint of the X and Y axes will be used from the WiimoteState object to activate the IRX and IRY events we defined above in the bindings XML file. The accelerometer X and Y values will be used to activate the AX and AY events.

The application linked above uses the following values for dead zones:

NunchukDeadX/Y -> 0.025
WiimoteDeadX/Y -> 0.15
IRDeadX/Y -> 0.1

// if we're using the IR
if(Properties.Settings.Default.UseIR)
{
    // and both LEDs are found
    if(ws.IRState.Found1 && ws.IRState.Found2)
    {
        // normalize the midpoints to -0.5 to 0.5 (from 0 to 1.0)
        float x = ws.IRState.MidX - 0.5f;
        float y = ws.IRState.MidY - 0.5f;

        // if we're beyond the thresholds, activate the events
        if(x > Properties.Settings.Default.IRDeadX || x < -Properties.Settings.Default.IRDeadX)
            this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.IRX), x));
        if(y > Properties.Settings.Default.IRDeadY || y < -Properties.Settings.Default.IRDeadY)
            this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.IRY), y));

        // save the last IR settings...these get used if we go beyond the range of the IRs.
        // in that case, the last used positions will be used until the Wiimote comes back in range
        this._lastIRX = x;
        this._lastIRY = y;
    }
    else // one or both LEDs aren't seen
    {
        // activate events based on the last known positions
        if(this._lastIRX > Properties.Settings.Default.IRDeadX || this._lastIRX < -Properties.Settings.Default.IRDeadX)
            this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.IRX), this._lastIRX));
        if(this._lastIRY > Properties.Settings.Default.IRDeadY || this._lastIRY < -Properties.Settings.Default.IRDeadY)
            this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.IRY), this._lastIRY));
    }
}
else // we're using motion controls
{
    // activate the events based on the accelerometer values
    if(ws.AccelState.X > Properties.Settings.Default.WiimoteDeadX || ws.AccelState.X < -Properties.Settings.Default.WiimoteDeadX)
        this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.AX), ws.AccelState.X));
    if(ws.AccelState.Y > Properties.Settings.Default.WiimoteDeadY || ws.AccelState.Y < -Properties.Settings.Default.WiimoteDeadY)
        this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.AY), ws.AccelState.Y));
}

' if we're using the IR
If My.Settings.Default.UseIR Then
    ' and both LEDs are found
    If ws.IRState.Found1 AndAlso ws.IRState.Found2 Then
        ' normalize the midpoints to -0.5 to 0.5 (from 0 to 1.0)
        Dim x As Single = ws.IRState.MidX - 0.5f
        Dim y As Single = ws.IRState.MidY - 0.5f

        ' if we're beyond the thresholds, activate the events
        If x > My.Settings.Default.IRDeadX OrElse x < -My.Settings.Default.IRDeadX Then
            Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.IRX))), x))
        End If
        If y > My.Settings.Default.IRDeadY OrElse y < -My.Settings.Default.IRDeadY Then
            Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.IRY))), y))
        End If

        ' save the last IR settings...these get used if we go beyond the range of the IRs.
        ' in that case, the last used positions will be used until the Wiimote comes back in range
        Me._lastIRX = x
        Me._lastIRY = y
    Else ' one or both LEDs aren't seen
        ' activate events based on the last known positions
        If Me._lastIRX > My.Settings.Default.IRDeadX OrElse Me._lastIRX < -My.Settings.Default.IRDeadX Then
            Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.IRX))), Me._lastIRX))
        End If
        If Me._lastIRY > My.Settings.Default.IRDeadY OrElse Me._lastIRY < -My.Settings.Default.IRDeadY Then
            Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.IRY))), Me._lastIRY))
        End If
    End If
Else ' we're using motion controls
    ' activate the events based on the accelerometer values
    If ws.AccelState.X > My.Settings.Default.WiimoteDeadX OrElse ws.AccelState.X < -My.Settings.Default.WiimoteDeadX Then
        Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.AX))), ws.AccelState.X))
    End If
    If ws.AccelState.Y > My.Settings.Default.WiimoteDeadY OrElse ws.AccelState.Y < -My.Settings.Default.WiimoteDeadY Then
        Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.AY))), ws.AccelState.Y))
    End If
End If

This code looks at the appropriate values, determines if they are beyond the specified thresholds for the dead zones, and, if they are, activates the event for that value using the Execute method. Execute is a method in the base EventSource class. This method will activate the event specified from the enumeration (which, remember, is contained in the bindings XML file) with the value associated with that event. An EventData object of some type must be created and passed to the Execute method. There are two EventData types to know about: AxisEventData and ButtonEventData. AxisEventData should be used when an event is activated that will modify the map position in some way. That is, if the map is being turned, elevation is changing, etc. ButtonEventData should be used if the event is a simple toggle like pressing a button down and releasing it.

Next, the nunchuk values need to be read and the associated events activated. This is done as follows:

// if the nunchuk is connected
if(ws.Extension && ws.ExtensionType == ExtensionType.Nunchuk)
{
    // activate the nunchuk-based events
    if(ws.NunchukState.X > Properties.Settings.Default.NunchukDeadX || ws.NunchukState.X < -Properties.Settings.Default.NunchukDeadX)
        this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.NunchukX), ws.NunchukState.X));
    if(ws.NunchukState.Y > Properties.Settings.Default.NunchukDeadY || ws.NunchukState.Y < -Properties.Settings.Default.NunchukDeadY)
        this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.NunchukY), ws.NunchukState.Y));
    if(ws.NunchukState.C)
        this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.NunchukC), 1.0f));
    if(ws.NunchukState.Z)
        this.Execute(new AxisEventData(new EventKey(this, (int)WiimoteEvent.NunchukZ), 1.0f));
}

' if the nunchuk is connected
If ws.Extension AndAlso ws.ExtensionType = ExtensionType.Nunchuk Then
    ' activate the nunchuk-based events
    If ws.NunchukState.X > My.Settings.Default.NunchukDeadX OrElse ws.NunchukState.X < -My.Settings.Default.NunchukDeadX Then
        Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.NunchukX))), ws.NunchukState.X))
    End If
    If ws.NunchukState.Y > My.Settings.Default.NunchukDeadY OrElse ws.NunchukState.Y < -My.Settings.Default.NunchukDeadY Then
        Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.NunchukY))), ws.NunchukState.Y))
    End If
    If ws.NunchukState.C Then
        Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.NunchukC))), 1.0f))
    End If
    If ws.NunchukState.Z Then
        Me.Execute(New AxisEventData(New EventKey(Me, CInt(Fix(WiimoteEvent.NunchukZ))), 1.0f))
    End If
End If

private void HandleButton(WiimoteEvent we, bool buttonState, bool lastButtonState)
{
    if(buttonState == lastButtonState)
        return;
    else
    {
        if(buttonState)
            this.Execute(new ButtonEventData(new EventKey(this, (int)we), EventActivateState.Activate));
        else
            this.Execute(new ButtonEventData(new EventKey(this, (int)we), EventActivateState.Deactivate));
    }
}

// handle all the Wiimote buttons
HandleButton(WiimoteEvent.Up, ws.ButtonState.Up, _lastBS.Up);
HandleButton(WiimoteEvent.Down, ws.ButtonState.Down, _lastBS.Down);
HandleButton(WiimoteEvent.Left, ws.ButtonState.Left, _lastBS.Left);
HandleButton(WiimoteEvent.Right, ws.ButtonState.Right, _lastBS.Right);
HandleButton(WiimoteEvent.A, ws.ButtonState.A, _lastBS.A);
HandleButton(WiimoteEvent.B, ws.ButtonState.B, _lastBS.B);
HandleButton(WiimoteEvent.Minus, ws.ButtonState.Minus, _lastBS.Minus);
HandleButton(WiimoteEvent.Home, ws.ButtonState.Home, _lastBS.Home);
HandleButton(WiimoteEvent.Plus, ws.ButtonState.Plus, _lastBS.Plus);
HandleButton(WiimoteEvent.One, ws.ButtonState.One, _lastBS.One);
HandleButton(WiimoteEvent.Two, ws.ButtonState.Two, _lastBS.Two);

...

// save off the current button state for next time
_lastBS = ws.ButtonState;
_lastNunchuk = ws.NunchukState;

Private Sub HandleButton(ByVal we As WiimoteEvent, ByVal buttonState As Boolean, ByVal lastButtonState As Boolean)
    If buttonState = lastButtonState Then
        Return
    Else
        If buttonState Then
            Me.Execute(New ButtonEventData(New EventKey(Me, CInt(Fix(we))), EventActivateState.Activate))
        Else
            Me.Execute(New ButtonEventData(New EventKey(Me, CInt(Fix(we))), EventActivateState.Deactivate))
        End If
    End If
End Sub

...

' handle all the Wiimote buttons
HandleButton(WiimoteEvent.Up, ws.ButtonState.Up, _lastBS.Up)
HandleButton(WiimoteEvent.Down, ws.ButtonState.Down, _lastBS.Down)
HandleButton(WiimoteEvent.Left, ws.ButtonState.Left, _lastBS.Left)
HandleButton(WiimoteEvent.Right, ws.ButtonState.Right, _lastBS.Right)
HandleButton(WiimoteEvent.A, ws.ButtonState.A, _lastBS.A)
HandleButton(WiimoteEvent.B, ws.ButtonState.B, _lastBS.B)
HandleButton(WiimoteEvent.Minus, ws.ButtonState.Minus, _lastBS.Minus)
HandleButton(WiimoteEvent.Home, ws.ButtonState.Home, _lastBS.Home)
HandleButton(WiimoteEvent.Plus, ws.ButtonState.Plus, _lastBS.Plus)
HandleButton(WiimoteEvent.One, ws.ButtonState.One, _lastBS.One)
HandleButton(WiimoteEvent.Two, ws.ButtonState.Two, _lastBS.Two)

' save off the current button state for next time
_lastBS = ws.ButtonState
_lastNunchuk = ws.NunchukState

And, the current button values are stored away to check on the next event so button events are only fired once.

Now that the event source object is written, it needs to be hooked up to the globeControl so it can be used. This can be done by creating an instance of the WiimoteEventSource object, passing in the VE3D's ActionSystem from the BindingsManager object. Then, the event source instance is passed to the ActionSystem's EventSourceManager and registered using the RegisterEventSource method. Event sources should re registered before the control is added to the form.

// wiimote events
private WiimoteEventSource _wiimoteEventSource;

...

// create a new instance of the Wiimote event handler
this._wiimoteEventSource = new WiimoteEventSource(this.globeControl.Host.BindingsManager.ActionSystem, this);

// register it in the event source list
this.globeControl.Host.BindingsManager.ActionSystem.EventSourceManager.RegisterEventSource(this._wiimoteEventSource);

' wiimote events
Private _wiimoteEventSource As WiimoteEventSource

...

' create a new instance of the Wiimote event handler
Me._wiimoteEventSource = New WiimoteEventSource(Me.globeControl.Host.BindingsManager.ActionSystem, Me)

' register it in the event source list
Me.globeControl.Host.BindingsManager.ActionSystem.EventSourceManager.RegisterEventSource(Me._wiimoteEventSource)

Our binding list contains three action types that are not defined by the default VE3D actions: ToggleRoads, Locations, LocationsMove, and ToggleUI. These actions and their handlers must be registered with the VE3D control. After the WiimoteEventSource is registered, the four actions can be registered as follows:

this.globeControl.Host.BindingsManager.RegisterAction(asmName, "ToggleRoads", new Action(this.ToggleRoadsHandler));
this.globeControl.Host.BindingsManager.RegisterAction(asmName, "Locations", new Action(this.LocationsHandler));
this.globeControl.Host.BindingsManager.RegisterAction(asmName, "LocationsMove", new Action(this.LocationsMoveHandler));
this.globeControl.Host.BindingsManager.RegisterAction(asmName, "ToggleUI", new Action(this.ToggleUIHandler));

Me.globeControl.Host.BindingsManager.RegisterAction(asmName, "ToggleRoads", New Action(AddressOf Me.ToggleRoadsHandler))
Me.globeControl.Host.BindingsManager.RegisterAction(asmName, "Locations", New Action(AddressOf Me.LocationsHandler))
Me.globeControl.Host.BindingsManager.RegisterAction(asmName, "LocationsMove", New Action(AddressOf Me.LocationsMoveHandler))
Me.globeControl.Host.BindingsManager.RegisterAction(asmName, "ToggleUI", New Action(AddressOf Me.ToggleUIHandler))

With the actions registered and handlers associated with them, the actual handlers need to be implemented. All event handler methods must be of the following signature:

public bool EventHandler(EventData cause)

Public Function EventHandler(ByVal cause As EventData) As Boolean

Let's look at the ToggleRoads event implementation:

private delegate void UIEventHandlerDelegate(EventData cause);

...

public bool ToggleRoadsHandler(EventData cause)
{
    if(cause.Activate)
    {
        // toggle through road types
        _roadState++;

        if(_roadState > RoadState.Roads)
            _roadState = RoadState.Aerial;

        // remove whatever one is currently there
        this.globeControl.Host.DataSources.Remove("Roads", "Roads");

        // add the new one
        switch(_roadState)
        {
            case RoadState.Aerial:
                AddFeederSource("http://local.live.com/Manifests/AT.xml", "Roads", "Roads", DataSourceUsage.TextureMap);
                break;
            case RoadState.Hybrid:
                AddFeederSource("http://local.live.com/Manifests/HT.xml", "Roads", "Roads", DataSourceUsage.TextureMap);
                break;
            case RoadState.Roads:
                AddFeederSource("http://local.live.com/Manifests/RT.xml", "Roads", "Roads", DataSourceUsage.TextureMap);
                break;
        }

        // update the status bar
        this.BeginInvoke(new UIEventHandlerDelegate(ToggleRoadsUI), cause);
    }
    return true;
}

private void ToggleRoadsUI(EventData cause)
{
    lblRoads.Text = _roadState.ToString();
    this.globeControl.Refresh();
}

Private Delegate Sub UIEventHandlerDelegate(ByVal cause As EventData)

...

Public Function ToggleRoadsHandler(ByVal cause As EventData) As Boolean
    If cause.Activate Then
        ' toggle through road types
        _roadState += 1

        If _roadState > RoadState.Roads Then
            _roadState = RoadState.Aerial
        End If

        ' remove whatever one is currently there
        Me.globeControl.Host.DataSources.Remove("Roads", "Roads")

        ' add the new one
        Select Case _roadState
            Case RoadState.Aerial
                AddFeederSource("http:'local.live.com/Manifests/AT.xml", "Roads", "Roads", DataSourceUsage.TextureMap)
            Case RoadState.Hybrid
                AddFeederSource("http:'local.live.com/Manifests/HT.xml", "Roads", "Roads", DataSourceUsage.TextureMap)
            Case RoadState.Roads
                AddFeederSource("http:'local.live.com/Manifests/RT.xml", "Roads", "Roads", DataSourceUsage.TextureMap)
        End Select

        ' update the status bar
        Me.BeginInvoke(New UIEventHandlerDelegate(AddressOf ToggleRoadsUI), cause)
    End If
    Return True
End Function

Private Sub ToggleRoadsUI(ByVal cause As EventData)
    lblRoads.Text = _roadState.ToString()
    Me.globeControl.Refresh()
End Sub

The implementations above simply check to see if the event is being activated and, if so, removes the current road layer and adds the next one in the series. The one thing to keep in mind is that the event handler methods are called inside the rendering thread of the VE3D control, which is not the thread the windows form UI is located on. Therefore, if the form UI needs to be updated in any way, one must use the BeginInvoke method and a delegate method to update any UI controls.

Next, let's look at the implementation of ToggleUI:

// toggle UI items off/on
public bool ToggleUIHandler(EventData cause)
{
    if(cause.Activate)
    {
        this.globeControl.Host.WorldEngine.ShowNavigationControl = !this.globeControl.Host.WorldEngine.ShowNavigationControl;
        this.globeControl.Host.WorldEngine.ShowCursorLocationInformation = !this.globeControl.Host.WorldEngine.ShowCursorLocationInformation;
        this.globeControl.Host.WorldEngine.ShowScale = !this.globeControl.Host.WorldEngine.ShowScale;
    }
    return true;
}

' toggle UI items off/on
Public Function ToggleUIHandler(ByVal cause As EventData) As Boolean
    If cause.Activate Then
        Me.globeControl.Host.WorldEngine.ShowNavigationControl = Not Me.globeControl.Host.WorldEngine.ShowNavigationControl
        Me.globeControl.Host.WorldEngine.ShowCursorLocationInformation = Not Me.globeControl.Host.WorldEngine.ShowCursorLocationInformation
        Me.globeControl.Host.WorldEngine.ShowScale = Not Me.globeControl.Host.WorldEngine.ShowScale
    End If
    Return True
End Function

This code simply changes the boolean values of several UI items. The WorldEngine object contains several other UI elements. The ShowUI property overrides all other properties and determines if anything is shown at all. This property also determines whether the globe in the lower-left corner is displayed.

Be sure to check the source code for the full demo linked above for the location handler methods. I omitted them here since it is just more of the same type of code above.

Running the Demo

To run the demo, do the following:

Copy BindingsWiimote.xml to the %APPDATA%\Microsoft\Virtual Earth 3D directory
Pair the Wiimote to the computer. See the WiimoteLib article for more information on how to do that
Run the executable

Conclusion

With the above code, we have written a Wiimote-driven interface for Virtual Earth 3D. The demo and source code linked above contain a few more features and bindings which enhance the application a bit more. Be sure to give the full demo a try and check out the full source code for a few more implementation details.

Additional Information

Managed Library for Nintendo's Wiimote - Updated version with this article!
Heptazane's Blog
Virtual Earth Blog
Virtual Earth Developer's Blog

Thanks

Thanks to Michelle Leavitt and Giovanni Montrone for testing the control scheme and helping to determine the best feel using the Wiimote.