• Eric Gunnerson's Compendium

    Miscellanea, and how to get a Wii


    I'm back at work today, after spending a record (for me) 23 days away from the office. The last time I spent that much time off was one of those involuntary vacations that happens sometimes ("the investors decided that they aren't going to fund us any more").

    Of that time, I spent a few days shopping, the week of Christmas in San Diego (beach/zoo/wild animal park/sea world/lego land/aquarium/balboa park), and then a few days skiing. Going from 65 and sunny to 20 and snowing was a bit hard, but we all have our crosses to bear.

    We also bought a wii, which has been a fair bit of fun. I got mine through the services of, which will send you both an email and a text message on your phone when stock shows up at one of the online retailers. You may be saying, "great, why didn't he tell me that before the holidays?", but I couldn't because a) my massive number of readers might have overloaded the site and b) my wife reads this blog, and she didn't know about the Wii (a risky move, but I got away with it). Probably more of "b" than "a". Anyway, a great site, and I'm sure I would not have gotten my Wii without it.

    Back at work, my group has moved to building 113, which is one of the old MS research buildings. It's pretty nice, though it isn't as nice as the brand new research building, which is both very pretty and has lots of innovative spaces (team rooms, working lounges, etc.) Plus, instead of glass doors between the lobby and the 4-story atrium in the middle of the building (I said it was pretty, right), it has some high-tech turnstiles that are operated with your badge.

  • Eric Gunnerson's Compendium

    Ring of Fire - Summary


    Ring of Fire is a tree ornament I built this year. A big, high-powered, custom-animated tree ornament.

    First, there was the idea and a parts list.

    Then I wrote a simulator to create animations against.

    When that was done, I built the ring and started on the firmware

    And then I did a whole bunch of wiring to hook up the microcontroller to the LEDs on the ring, finished the code, and put it up in the tree in my front yard.

    Finally, I took a low-quality video of it.


  • Eric Gunnerson's Compendium

    Santa Animation


    The oldest of my displays. The "landing lights" are blue lights in canning jars dyed blue. All the santas and the reindeer are made of plywood. Some of the lights on the mid and upper santa are burnt out, but I'm not sure I'm going to get around to fixing it before next year.

     Animated with an 8-channel custom Motorola HC11-based microcontroller system driving solid-state relays, all coded in assembler

    Santa Animation
    Santa Animation
  • Eric Gunnerson's Compendium

    Ring of Fire - Video


    Here's a video of the completed project. The video is from my Canon G3, so the quality isn't very good. For sake of comparison, the blue and red globes in the video are 50 watts each, while each LED is only 350mW. The LEDs a bit brighter than they appear because the video is about 20 degrees off-axis.

    Ring of Fire ornament
    Ring of Fire ornament
  • Eric Gunnerson's Compendium

    Tree of Lights Video


    My second animated display. This one has 16 channels and runs using a Motorola 68HC11 microcontroller. In the background you can see the house outline lights, which are a four-color dimming system also based on a 68HC11


    Tree of Lights
    Tree of Lights
  • Eric Gunnerson's Compendium

    Ring of Fire - Up in a tree


    The last few days have been fairly hectic. I wanted to get "Ring of Fire" into a demo-able state (to take into work on Friday), so that it could go up today.

    First up was building the project board. As I mentioned before, the LEDs pull 70mA, so I need a separate transistor to drive them, as the AVR can't handle the current requirements. The switch uses a 2n3904 NPN transistor, a 10K base resistor, and a 100K resistor between the emitter and base. And finally, an 42 ohm current-limiting resistor in line with the LED, in between the collector and 5V. That's 16 transitors and 48 resistors, and after about 5 minutes, it was clear that that wasn't going to fit on the board. I decided to put the current-limiting resistor next to the LEDs, and could barely fit the other components on the board. Here's a picture of the board:


    Yep, that's a whole bunch of wires.

    Next up was finishing the LED wiring. One of my bike wheels is built with 16 spokes, so I used it to mark the location the LEDs. Then I spent about 4 hours mounting the LEDs onto the ring. Here's a picture of a mounted LED:

    The black wire is the 5V bus, and continues all the way around the ring. The yellow part is a 42 ohm resistor attached to the ground side of the LED, which is then attached to a red wire that runs all the way back to the controller. The LED is then attached to the ring with hot glue. Here's what the completed ring looks like.

    That completed, I turned back to the software.

    Problems that I hit along the way:

    1. I hadn't set the avr to use the internal clock correctly. This meant that it used the clock on the STK500 board, which I changed now and then, so the animation timing made little sense. This also mean that the chip didn't run when plugged into the board.
    2. Reset needs to be pulled low for the avr to run until reset is disabled. This means that until you are ready to burn the final version, you need to pull it low in the final circuit or you can't test it. This also means that that LED is always on. When you are finally ready, you set a fuse, and reset is disabled, so you can't program it using the in-circuit programming approach, so it's a lot harder to do.

    Originally, I was going to put ping-pong balls on the LEDs to diffuse the light, but it turned out that the LEDs have a fairly wide viewing angle, so the diffuser wasn't needed.

    So, the project is done, up and the tree looks pretty nice.


  • Eric Gunnerson's Compendium

    Stay at Home Server


    The Windows team has launched Stay at Home Server, and they've produced a short book that introduces it.

    You should read it.

  • Eric Gunnerson's Compendium

    Ring of Fire - Ring and Firmware


    After firming up the animations that I wanted to do, I got two things accomplished in the last few days.

    First, I build the "ring" part. I had bought a 5' piece of pvc electrical conduit to use as a form, and my plan was to bend it into a circle by heating it with a heat gun.

    Initial results were disappointing. The circle that I got was about as circular as the ones I drew when I was in kindergarten, so I decided I needed a form.

    And there, sitting in a box because of my laziness at not putting them away, was a set of spare wheels for my bicycle. A little experimentation showed me three things:

    1. Using the wheel as a form was workable
    2. The piece of conduit I had wasn't long enough
    3. It was sort of bunged up for my first attempt

    A quick trip to Lowes and another 30 minutes with the heat gun, and I had a nice hoop (size 700c) completed.


    After digging out my development board (I use the Atmel STK500) and buying a copy of the CodeWarrior AVR C compiler, I was ready to get started. Well, not quite - I had to set up my work area since we're getting ready to gut and redo our office. My main tools are:

    • A nice 20MHz dual-trace Tektronix oscilloscope that I bought on the internets a few years ago.
    • A benchtop power supply (+5V, +12V, -12V, and 0-20V)
    • A Fluke multimeter
    • A Weller temperature-controlled soldering station
    • My 8 fingers and 2 thumbs

    I don't really need the power supply as I'm using a wall wart to power the development board, but it looks nice under the scope. The multimeter is useful to look at LED currents, and because I'm using PWM to control the brightness of the LEDs, the scope is invaluable for debugging. If you don't want a full-sized scope, there are some decent PC-based solutions, but keep in mind that the precision electronics required for 'scopes are never cheap. I like having the real buttons and knobs rather than having to use a UI on the laptop to do things, though I would really like a TDS1000B.


    I've often advocated that developers should learn multiple languages because knowing languages effects how you approach problems. If you are fluent in Perl, you're much more likely to know when to use a nice regex in C#.

    Similarly, it's good for your skills to spend some time working on a seriously constrained machine like a microcontroller. The 861 I'm using has a whopping 512 *bytes* of memory, which sounds pretty tiny until you note that the 261 only has 128, and some of the 8-pin versions only have 64 bytes.  Trying to structure your algorithms and code to use that amount of memory and the 8K of flash for the program memory will stretch your mind in ways that writing C# code won't.

    I'm undecided on the benefits of writing in assembler. I can be a learning experience, but I'm not sure if the pain of remembering how indexed address mode works transfers over to more rational languages.


    My first order of business was to get the chip in the board, make sure I could burn it, and get a skeleton program running. Getting the chip on the board was trivial, but I had real problems getting it to program (which is done with a PC program that sends the code over a serial cable to the board). After a lot of trial and error - mostly error - a kind soul on pointed out that the connections on the STK500 were wrong for my chip, and things started working correctly then.

    Codewarrior is a nice product. Microcontrollers require a lot of configuration up front - you need to set how you are using each of the pins, what interrupts you want, and 35 other things. You do this by writing code that stuffs particular values in specific registers on the chips. Something like:


    Where you found that by careful reading of the 227 page user's guide for the chip. It's pretty tedious, but CodeWarrior provides a "get started wizard" where you specify what chip you're using and then go through a dialog and choose what options you want and then it generates all the initialization code that is needed for your chip. It's a huge timesaver.

    After tracking down one problem that was keeping my interrupts from firing (traced to missing a bit in one of the interrupt configuration registers), my main loop woke up and started operating. I did a bit more debugging and then ported one of my animations over from C# to C.

    It looked okay on the scope (minus a few glitches), but it seemed slow. Plugging in the leds on the STK500 to 8 of the output bits I'm using showed that a) the animation was working, but very slowly and ii) the LEDs were flickering like nobody's business.

    As I suspectected, the 861 running at 8 MHz doesn't have enough horsepower to handle 256 levels of dimming. I'm not getting the 100 Hz that I need to keep the leds from flickering, and there's not enough free time after the interrupts to do the animation at any reasonable speed.

    I'll spend a little time seeing if I can't speed up the interrupt routine, but the real fix is going to be to reduce the number of dimming levels down to 16, which should give me enough time for everything to run smoothly.

    Then, it will be time to start attaching components to the ring 

  • Eric Gunnerson's Compendium

    Battery University


    A great resource for all things related to rechargeable batteries.

  • Eric Gunnerson's Compendium

    The life and times of a HealthVault piece of data...


    HealthVault has two temporarl views of data.

    The first temporal view is fairly obvious. If, for example, you're looking at the Height type, there is a When property that can tell the time at which the height was measured. That allows you to do fun things, like chart the growth of your children over time (as soon as somebody writes that application...)

    There's a second view that can provide a snapshot of the HealthVault state at a specific time. Though it's not obvious because you normally see the current view, much like a source code control system, HealthVault allows you to find out what an item's state was at a time in the past.

    Or, to put it another way, if you create a data item and then modify it over time, or even delete it, there's a way to figure out the whole lifecycle of that piece of data - when it was created, when it was updated, when it was deleted, and who performed each of those operations. Not to mention what the contents of the item was at each step.

    By default, you only see the current state of the data items, but this is controlled by the filter that you use when you query for items. Here are the filter properties that let you do this:


    This property is true by default. Set it to false to get all the versions back


    There are two states - Active and Deleted, along with Default (same as Active), and All (either Active or Deleted)

    If you want to get all the items, specify All for this property.

    Who Changed My Data?

    Those two properties are enough to get you all the items when you execute a query. The different snapshots of a single item have the same Key.Id value, but different version stamps.

    To find out who made the changes, you need to request the audit information. You do this through another filter property - View.Sections - by adding Audits to that field.

    Once you do that, the LastUpdated property on each item that you fetch will now contain a live HealthServiceAudit record, which tells you lots of fun information about the source of the record.

  • Eric Gunnerson's Compendium

    Ring of Fire - Simulator


    To make some progress before my parts arrive (they showed up yesterday), and to have a good way to test my animations, I decided to build a simulator.

    The simulator is build in windows forms, and is intended to prototype both the approach I'm going to take in the real software. I'm also hoping I can steal code, as the real software is going to be built in C this time, rather than directly in assembler.

    To build the simulator, I had to decide how I was going to encode the animations, and how the main loop would work.

    Because of time constraints, my plan was to build this as a sequencing system rather than something involving dimming. So, I coded up a very simple scheme of encoding the output in 3 bytes:

    Count byte            first 8 lights            second 8 lights

    Where the light bytes tell you what lights are on, and the count byte tells you how long to stay in that state.

    I then tried to write a "lights go around in a circle animation". The table looks roughly like this:

    0x08, 0x80, 0x00
    0x08, 0x40, 0x00
    0x08, 0x20, 0x00
    0x08, 0x00, 0x01

    So that's 16 lines of entries, taking up 48 bytes for a very simple animation, with me doing a lot of bitwise math in my head.

    I then decided to do a more complex animation - there's one light on at the bottom, the other lights circle around, and give a whack to the fixed one, which starts moving. So, in essence, a fixed spot moves backwards as the main animation goes forwards.

    A full cycle of that involves 16 full circle cycles, or 768 bytes. Each of which had to be hand-figured out, basically by overlaying a fixed mask on top of the moving circle animation. I got through two cycles through a lot of debugging and testing, and decided I'd had enough. It was like programming a system without any looping constructs.

    So, I decided to abandon the table-based approach to take a programmatic approach. Well, strictly-speaking, it's a hybrid approach, where the animation is controlled programatically, but the low-level stuff is handled on an interrupt.

    I also decided to incorporate dimming, since I really wanted to. So, here's the design.

    Interrupt Handler

    The interrupt handler executes N times for each cycle (arbitrarily set to around 1/100th of a second to avoid flickering), where N is the number of dim levels I want to support. Consider N to be 64 for sake of argument.

    PWM Cycle

    The lowest level operation is to implement the PWM loop. Each time the interrupt fires, a byte counter is incremented, and compared to the dim level array elements (one per led), and the output bit is cleared if the counter is greater than the dim level for that light.

    So, we have this array of lights all turned on, each with a number saying how bright it is, and as the PWM cycle continues, lights get turned off as their dim level is reached. Those with a low dim level are turned off early, those with a high level are turned off later.

    The current levels are stored in a

    byte current[16];

    When a PWM cycle is finished, we reset all the outputs to high and the counter to zero, and then move to the animation cycle to see what to do next...

    Animation Cycle

    The animation cycle implements changes to the current array values over time. This is done through a delta vector and a count. Basically, we have:

    byte delta[16];
    byte deltaCount;

    and every time we go through the animation cycle (ie each time a PWM cycle finishes), we add the elements of delta to current, and decrement the deltaCount.

    So, if current[0] was 63 (ie full on), and we want to dim it to zero and turn current[1] full one, we would set up the following:

    delta[0] = 255;
    delta[1] = 1;
    deltaCount = 63;

    The first time through the cycle, current[0] = 63 + 255 => 62, and current[0] = 0 + 1 => 1. We're just adding 1 to current[1] 63 times, and subtracting 1 (by adding 255 to it) from current[0] 63 times.

    If we didn't want dimming, we could also encode this as:

    delta[0] = 193;    // 256 - 63
    delta[1] = 63;
    deltaCount = 1;

    That would flip from one light to the next in a single cycle.

    Note that we can achieve a hold by clearing the delta array and setting deltaCount to whatever delay we want, which leaves current[] unchanged for that period.

    After the animation cycle has completed, we need to get the delta[] and count values for the next cycle. We do this by copying from deltaNext[] to delta[] and from deltaCountNext to deltaCount, and then setting deltaCount to zero. That gives us the delta set, and we continue as before.

    That's all for the interrupt routine, but it rather begs the question - where did the values in deltaNext[] and deltaCountNext get set?

    Main animation loop 

    In previous incarnations, the main animation was just handled in the same section of code as everything else - when the animation cycle ended, you'd figure out what you needed next (well, actually, you did it right before you needed it).

    This has two whopping disadvantages.

    The first is that you have to do it in the spare time between cycles. That's not bad if you are doing simple animation (ie not dimming) or you're doing dimming in hardware (which I'm not), but in this case there may not be enough time to do the main animation loop in the time left for an interrupt (at 64 dimming levels, about 156 microseconds for everything, including the interrupt routine).

    The second is that you have to write the animation as a state machine - for any given set of counters, you need to know what the next delta[] should be. That's not going to be much fun for more complex animations.

    I therefore decided to let the main animation loop run in the main execution thread (well, there is only one thread, which the interupt preempts as needed). It therefore has the full animation cycle to come up with the next set of values (though, if the count=1, that's the same as the PWM cycle), or 10 mS, and there should be enough time leftover to do the work I need to.

    The main loop will use a blocking function to tell it when it needs to proceed. It's:

    void SpinWait()
        while (deltaCountNext != 0)

    So, we'll just hang in that tight loop. An interrupt will come along, do what it needs to do, and if it's at the appropriate point, deltaCountNext will be set to zero and when the interrupt returns, we can go on to generate the next values.

    That allows the animation to be coded as something like:

    deltaNext[0] = 255;
    deltaNext[1] = 1;
    deltaCountNext = 63;

    deltaNext[1] = 255;
    deltaNext[2] = 1;
    deltaCountNext = 63;

    which is a whole lot easier to understand, and you can even use this thing called a "loop" so that it's easy to write. The simplification is roughtly analogous to how it is easier to write an enumerator using yield than the old way.

    Back to the Simulator

    The simulator implements this by having a timer fire off every 10 mS, and it calls into the interrupt code (which does everything except the PWM loop). The main loop runs in a separate thread, and everything is peachy.

    That's all for now. I think my next task is to write a few more simulations to make sure the code does anything, and then move the code to the microcontroller, hook up a scope, and see what I get.


  • Eric Gunnerson's Compendium

    Ring of Fire


    No, not that...

    Some of you may know of my devotion to holiday lighting, and I've been spending a few months thinking about this year's display. Last year's windstorm did a number on a few of the displays, but most were fixable (the spiral tree that took flight was not...).

    I wanted to build something new, something different. I settled on using an Atmel AVR as the microcontroller, and LEDs as my light source. I considered a lot of different designs, but procrastination and my unexpected encounter with the pavement have left me with less time and unable to do any heavy fabrication, so things had to be simplified.

    In the front of our house, we have a tall tree (30+ feet (2.34 hectares)) that I believe is of the Norway Spruce variety. Last year is was decorated with a number of the 50-light globes, and a big white 200 light globe for the top. I lost about half the 50 light globes and the 200 light globe only has about 20 working, so I've decided that the replacement top ornament will be a ring perhaps 18 inches (1.66 decalitres) across with 16 red leds distributed evenly around the ring. It's 16 because I think that's a nice number, and it lets me use a simpler microcontroller since I only need to control 16 outputs. And using LEDs will mean that I don't have to use solid state relays, which are a bit pricey...

    Hence "ring of fire"

    Here's the part slist:

    • Atmel AVR ATTiny861 (8K Flash, 512 bytes EEPROM, 512 bytes (!) SRAM, 16 I/O pins, 20 MHz)
    • 20 2n3904 transtors
    • 20 43 ohm resistors
    • 20 10K resistors
    • 20 100K resistors
    • 1 100uF capacitor
    • 1 waterproof box

    I chose the 861 mostly because it had 16 pins, and at $3, what's not to like?

    The LEDs are designed for sign and signal applications. They run at 70mA rather than the 20mA most LEDs use, and they have a wide viewing angle (ie the light is spread out rather than going straight ahead). The 43 ohm resistors will go in series with the leds to get that 70mA at 2V on the LEDs from the 5V power supply.

    The transistors and 10K/100K will be used to switch the LEDs off and on. The AVR can sink 20mA of current, which is a fair amount, but not enough for the LEDs, and even at 20mA it couldn't drive them all at once.

    I think I have a 5V power supply that will work. The LEDs pull 2.24 watts when they are all on, but unfortunately at 5V the 42 ohm resistors will pull 3.36 watts, which puts me up near 6 watts.  If I drop down to 3.3V, 20 ohm resistors would give me 65 mA, and that would mean 2.08 watts for the LEDs but only 1.35 watts for the resistors, so I could get by with a 4 watt supply.

    The AVR will run on anything from 1.8V to 5.5 V, though it won't be quite as fast at the lower voltages.

    Software is in two stages.

    The first stage will be a simple table-based animation system. I've built it a few times before, and last night I built a simple simulator in winforms. That will get it up and running.

    The second stage will be to add dimming to all the LEDs. That requires doing PWM on every channel, and I'm not sure that there is enough horsepower to do that in the AVR.

    To do flicker-free dimming will require an update frequency of around 100Hz - the dimmest setting would have a brief spike of "on" every 10 mS. That period would need to be divided by the number of light levels - to do 256 would mean that the code would have to update the output state about every 40 microseconds. That's roughly every 750 clock cycles at 20MHz, which seems possible but difficult.

    On the other hand, 64 levels would increase that to 3000 clocks, and 16 levels to 12000 clocks, so it looks like some sort of dimming is doable. It helps that the table-driven code doesn't use a lot of resources.

    All my previous projects have been written in assembler, but I've decided to use CodeVisionAVR as the development environment, which gets me a C compiler and IDE.

    I'll post other updates as things progress...

  • Eric Gunnerson's Compendium

    Storing CCR and CCD data in HealthVault


    There have been a couple of questions recently about storing CCR and CCD data in HealthVault, so I decided to do a quick sample. The sample will show up in the SDK sometime in the future, so consider this the accompanying discussion for the sample. If you want the actual bits before they show up in the SDK, let me know.

    Both the CCR and CCD are designed as ways of passing information between entities. For example, when you leave the hospital, information could be passed back to your primary care physician through one of these types. Or, at least, that's what I understand - my real aptitude relating to these types doesn't go much beyond being able to spell them.

    These types are examples of XML types that I talked a bit about in What Data Type Should I Use?, which simply means that they are types as far as the platform is concerned (ie they have an XML schema and thing type associated with them), but they don't have a client type (ie a type derived from HealthRecordItem) in the .NET API.

    To operate with such types, we'll need to use the HealthRecordItem type directly. To do so, I need to know the guid associated with each type. I get those from the Thing Types Reference page, which I encode using the following:

    readonly Guid CCR_THING_GUID = new Guid("1e1ccbfc-a55d-4d91-8940-fa2fbf73c195");
    readonly Guid CCD_THING_GUID = new Guid("9c48a2b8-952c-4f5a-935d-f3292326bf54");

    Then, it's really pretty simple. If I want to insert a CCR, I can use the following:

    void AddCCR()
        XmlDocument ccrDocument = new XmlDocument();
        HealthRecordItem ccr = new HealthRecordItem(CCR_THING_GUID, ccrDocument);


    You might want to use XPathDocument instead of XMLDocument, as I hear that it's cheaper than XmlDocument to use. Also note that the ExampleCCR.xml file that I open *does not* have anything before the "<ContinuityOfCareRecord>" element - if it does, the framework will reject it as poorly formatted.

    That's about all you need to do to add it in.

    To do the same thing with a CCD is pretty much equivalent to changing the appropriate "r" characters in the code to "d" characters.

    To get retrieve the types is also fairly simple. Assuming I want to look at all the CCDs that are stored in a record, I can do this:

    List<HealthRecordItem> ccdItems = GetValues<HealthRecordItem>(CCD_THING_GUID);

    foreach (HealthRecordItem ccd in ccdItems)
        // ccd xml data is in ccd.TypeSpecificData

    where GetValues() is from the HelloWorld sample.

    This is a simple form of support for these data types. Some partners have requested the ability to create a CCR/CCD from individual data items in HealthVault, or to break a CCR/CCD into individual items and store those items in HealthVault. We understand the utility of that scenario, but haven't announced any plans about supporting that.

  • Eric Gunnerson's Compendium

    The physics of rock guitar...


    I wish I had a PhD in guitar acoustics...

    The physics of rock guitar...

  • Eric Gunnerson's Compendium

    Psychic debugging...


    Ray wrote a post entitled "Psychic Debuggin: IP on heap", where he talks about somebody being amazed at his abilities to debug something immediately.

    Which brought something to mind (which I hope I haven't written about before (I did a quick search and didn't find anything ))

    Perhaps 4 or 5 times, I've come across the following question:

    I'm implementing an add-in architecture for my application. I'm able to load the add-in assembly and create an instance of a type, but I get an exception when I try to cast it to an interface.

    The answer - and it's *always* been this answer for me - is the following (highlight to view):

    You have the interface defined in your main program and in the add-in, and the assembly is part of the type's identity, so the two interfaces are not the same type.

    Which is really obvious when you've come across it a couple times, but there is a certain degree of satisfaction to be able to diagnose it correctly from afar.

    Do you have any examples of psychic debugging?


  • Eric Gunnerson's Compendium

    Request-Response Tracing (low-level debugging)


    If you want to do some low level debugging of the traffic between your application and the HealthVault servers, the following may be useful. Note that there's a lot of raw-xml there...


    1.       Make sure the account under which the ASP.NET worker process is running (w3wp.exe for W2K3 and Vista, aspnet_wp.exe for XP) has write access to the application directory.

    2.       Open the web.config file.

    3.  Add the following (if the system.diagnostics section already exists, just add the inner elements to the existing section):

              <source name="HealthVaultTraceSource"
                      switchType="System.Diagnostics.SourceSwitch" >
                      <add name="fileListener"
                          initializeData="sdk.log" />
                      <remove name ="Default" />
              <!-- You can set the level at which tracing is to occur -->
              <add name="HealthVaultTraceSourceSwitch" value="All" />
              <!-- You can turn tracing off -->
              <!--add name="HealthVaultTraceSourceSwitch" value="Off" -->

    4.       Run the scenario, and take look at the sdk.log file that gets generated in the application directory. You may need to shut down the application to get the complete set of data from the tracing infrastructure.

    5.       If the file doesn’t get created and you are sure a request was made to HealthVault, change “sdk.log” to be the complete path to a directory that the ASP.NET worker process has write access to and try again.


  • Eric Gunnerson's Compendium

    Extending HealthVault data types using a custom extension class


    This post will cover the second method of extending a HealthVault data type. If you haven't read about the first method, please do so now. We'll wait.

    In the first method, we had to deal with all the XML details ourselves, which made it a bit clumsy. This time, we'll encapsulate it in a class:

    public class UserSpineStateExtension: HealthRecordItemExtension
        static readonly string ExtensionSource = "Fabrikam.UserSpineState";

        string m_spineState;

        public UserSpineStateExtension()

        public UserSpineStateExtension(string spineState)
            m_spineState = spineState;
            Source = ExtensionSource;

        public string SpineState
            get { return m_spineState; }
            set { m_spineState = value; }


        protected override void ParseXml(System.Xml.XPath.IXPathNavigable extensionData)
            XPathNavigator navigator = extensionData.CreateNavigator();
            XPathNavigator spineStateNode = navigator.SelectSingleNode("extension/SpineState");

            if (spineStateNode != null)
                m_spineState = spineStateNode.Value;

        protected override void WriteXml(System.Xml.XmlWriter writer)

        public static void RegisterExtensionHandler()
            ItemTypeManager.RegisterExtensionHandler(ExtensionSource, typeof(UserSpineStateExtension), true);

    There are to constructors. The empty one is used by the system when it is creating an instance for us. The other one takes in the value that we want to store, and then sets the sources appropriately.

    In ParseXml(), we find the appropriate node, pull out the value, and store it in our own variable. In WriteXml(), we add in the xml for our spine state value.

    Note that the parse and write methods are not symmetrical. In the write one, we're already inside the <extension> tag, while in the parse one, we're up a level, pointing at the <extension> tag.

    Finally, the RegisterExtensionHandler() method registers the type with the system so it knows where to find it. That needs to be called someplace before you try to use the type.

    Using the custom extension

    To use the extension is very simple. To add an extension, we use this code:

    UserSpineStateExtension extension = new UserSpineStateExtension("Extended");

    And to get it back, here's what we write: 

    foreach (HealthRecordItemExtension extension in height.CommonData.Extensions)
        UserSpineStateExtension userSpineState = extension as UserSpineStateExtension;
        if (userSpineState != null)
    string spineState = userSpineState.SpineState;


  • Eric Gunnerson's Compendium

    Extending HealthVault data types using HealthRecordItemExtension


    So, let's say that you're building an application using the HealthVault Height type, and you find that you need to store another bit of information with that type. What should you do?

    Well, first, you should probably stop by HealthVault Forum and say, "I'm using Height, and I want to keep track of whether people were sleeping right beforehand, because spines compress during the day. How should I store that?"

    The reason you should come by and ask is that the types that are currently in HealthVault are "works in progress" - they support the applications that are currently written but may need to be extended to support further applications. We'd therefore like to know what you're thinking to see whether your addition belongs in the base type. It might also be the case that that bit of extra information shouldn't be stored in the Height type but stored as an instance of some other type.

    So, let's take it as read that you've already done that, and you want to extend an existing type. This is done through HealthRecordItem.CommonData.Extensions, which is a collection of HealthRecordItemExtension instances.

    By default, there's nothing in this list. If you have an instance and you add an extension instance to it, the framework will dutifully save that instance out to the server, and just as dutifully fetch it back when you read that instance back out.

    Assuming you do everything correctly.

    There are two ways to do this. The first is a quick-and-somewhat-dirty method where you just stuff some xml into the extension and pull it out on the other side, and the second involves a nice object that encapsulates everything for you. This post will talk about the first method, and the next post will talk about the second method.

    It's important to remember that this is a *list* of extensions, so you need to write your code so that it plays well with extensions that may have been put there by other users. That's fairly straightforward, but something to keep in mind...

    Saving data to the vault 

    Here's the code to add the extension data:

    string extensionSourceName = "Fabrikam.UserSpineState";

    HealthRecordItemExtension extension = new HealthRecordItemExtension(extensionSourceName);

    XPathNavigator navigator = extension.ExtensionData.CreateNavigator();
    navigator.InnerXml =
    @"<extension source=""" + extensionSourceName + @"""><SpineState>Compressed</SpineState></extension>";

    The first line defines the source string. This is the "unique identifier" that differentiates your data from all the other data out there, so it's a good place to put a company name or some other unique string at the beginning.

    We then create an instance of HealthRecordItemExtension with that name, and then finally add some xml to the extension. Note that the string in the extension source attribute has to be identical to the source you specified when you created the HealthRecordItemExtension instance.

    That's all there is to it - when the height instance is saved, that data is persisted.

    Pulling the data back out

    To get the data out, we need to find our extension amongst the other extensions out there. So, assuming I have a Height instance, I can write the following:

    foreach (HealthRecordItemExtension extension in height.CommonData.Extensions)
    (extension.Source == "Fabrikam.UserSpineState")
    XPathNavigator navigator = extension.ExtensionData.CreateNavigator();
    XPathNavigator spineStateNavigator = navigator.SelectSingleNode("extension/SpineState");

            if (spineStateNavigator != null)
                string spineState = spineStateNavigator.Value;


    And note that the source string I'm looking for is the same one I specified when I added the extension.

    There are a few other properties that you can set on the extension:


    If you'd like your extension data to be visible in applications that show data generically - such as the data view - you can provide the url of an SSL transform that can convert the xml into html. 


    Similarly, if you want a logo associated with the extension data, you can provide a url here.


    A version string that you can use for, surprise surprise, versioning.

  • Eric Gunnerson's Compendium

    Happy Birthday!


    30 years.

    30 years ago - give or take a few days - Digital Equipment Corporation introduced the VAX 11-780, running their brand new VMS operating system.

    That's "VAX' as in "Virtual Address eXtension" and VMS as "Virtual Memory System".

    "Virtual" as in "the system acts like it has more memory than it really has", something that we take for granted these days.

    Though not the first system to use virtual memory, it was one of the first popular ones.

    VMS was the system on which I first "cut my teeth" (so to speak) - I worked on it a couple of years in college, then professionally for a couple of years (and, arguably, I've been working on NT-class systems for quite a while).

    So, I'd like to take this opportunity to wish VMS (now, "OpenVMS") a happy 30th birthday.

  • Eric Gunnerson's Compendium

    Deep Breath...


    An unexpected vacation.

    Here's why...

  • Eric Gunnerson's Compendium

    Second use, third use


    A few days ago, I got a link to a post on Software Performance, which touches on a specific topic that is of interest to me.

    I use a lot of products that spend time trying to optimize the "first use" case, so it's a straightforward as possible. Which is good.

    What I don't see is those products trying to optimize the second use, the third use, etc. - very few products learn from what I asked them to do and provide reasonable defaults. Even fewer ask me questions up front about how I want to do the process to make my life easier.

    I'll pick on the picture import wizard in XP. I plug in my smart card adapter, the wizard starts up, and I have to walk through the same set of tasks every time, where what I'd really like to do is have one big button that says, "Do what I did last time".

    Why do products do so poorly at this? I think it's because they are generally optimized so that the demo scenario seems as simple as possible. In the extreme case this can result in "demo-ware" - software that demos really nicely but is actually kind of hard to use in the real world.

  • Eric Gunnerson's Compendium

    Prepare for liftoff...


    Last year, I came into work to find my main desktop machine off. Now, this machine had been misbehavin' for the last few weeks, but this was a new behavior. I sat down, pressed the power button, and sat back as I listened to the disks spool up, and the case fan spool up. and up. and up. and up, until it was spinning at approximately 12 million RPM and sounding very much like a 1800watt Clairol hair dryer (without a diffuser attachment).

    The problem was easy to repro, and after showing it off to a few of my co-workers, I called up our helpdesk. After the initial preliminaries (verifying who I was and what my phone number is), the tech asked me to describe my problem, and I said, "my computer is having a fan problem."

    "What sort of fan problem?"

    "Well, I start up my computer and.. well, I'll let you listen to it"

    I pressed the button, and held the headset near the system. As the fan passed "hair dryer" and was rapidly approach "leaf blower", over the din I heard the tech yelling

    "Turn it off! TURN IT OFF!"

    Which I did, though I didn't understand his anxiety. It's not like it was leaking magic smoke or anything.

    And the computer got fixed through a simple motherboard swap.

    And I hadn't thought about it for many months, until...

    Last weekend I was at the end of a bridge, standing in the shadows while I moved slowly back and forth, looking for the right time to toss a grenade or go for a headshot. And then suddenly, my world froze, and I was back to keeping my couch from floating away.

    A quick test showed that my controller was lifeless, and from the component cabinet in the corner, I heard a loud sound, which became a very loud sound when I opened the cabinet up.

    My 360 had decided to destructively test my Halo 3 DVD. I hit the eject button (which spit out a still-spinning dvd), and now have a console that is no longer literate (think about it).

    The good news is that the warrany expires a month from now, but my presence seems to be having a bad effect on speed controllers...

    Alternate post titles I considered. Did I make the right choice?

    1. Prepare for liftoff...
    2. I feel the need - the need for WHAT IS THAT NOISE?
    3. Captain, if you keep this up, I cannot be responsible for the safety of the ship...
    4. You know, plastic is stronger than it looks...
  • Eric Gunnerson's Compendium

    What data type should I use?


    In my previous posts, I kindof glossed over the different kinds of data types you might have in your application and when you might choose a specific type. I now seek to rectify that oversight, but first I need to talk about types in a more general sense:

    Eric's Taxonomy of HealthVault data types...

    With the exception of custom types, all types are defined by the HealthVault platform. Take a minute to look at the platform ThingType Reference (which is always the definitive reference to what data types are deployed on the HealthVault servers).

    You probably already figured out that each of the classes derived from HealthRecordItem correspond to a "thing type" here - the AerobicSession class is a AerobicExerciseSession thing type on the server, the Appointment class is the Appointment thingtype. And you may also have figured out that if you click on the guid for the thingtype, you'll get a page that shows more information about that type.

    If you spend a lot of time looking at that list, you may notice that there are certain thing types that don't have corresponding classes (Continuity of Care Record, for example). For those thing types, there is an XML schema underneath the type.

    Sometimes that's because we haven't written a .NET wrapper around the thing yet (just adding a new thingtype is easier and quicker than creating/testing the wrapper), and sometimes it's because we've decided not to write a wrapper for the type at this time.

    And then, of course, there are the custom types, so the overall taxonomy is:

    • API Class types
    • XML-only types
    • Custom types

    Choosing the right option for your information

    The right option for your information depends on the kind of information it is and how you expect to use it.

    Initially, it will probably be simplest to develop new types as custom types - that way you understand what should be in them, how they fit into your application, etc.

    Then, if the information would be widely useful to other applications (ie if we didn't have a Weight type and you wanted to store weight), that's a good indication that it belongs in the platform. At that point you should request a new type, which may involve some discussion between you and MS over the details, and, after a (undefined but hopefully fairly short) period of time, the new type will show up in the platform.

    Conversely, if the information is very specific to what your app does, you may want to leave it as a custom type.

    That's for types that are very different. If you have a type that is close but not quite the same as a type we already have, you may want to extend the type yourself (using CommonData.Extensions), or you may want to request a modification of an existing type (which may or may not be feasible, depending on the type and what you want to do to it).

    If you have questions about what to do with a specific bit of data, please ask on the forums.

  • Eric Gunnerson's Compendium

    HealthVault Data Types - a custom data type architecture


    In my last post, I showed how to create custom data types, but there's an obvious problem with that approach.

    There's only one of them.

    That means that when you do a query for your custom type, you get all the custom types that meet your filter.

    If your application creates multiple custom types, then it needs a layer that can do the appropriate thing and give you the types that you want.

    As is often the case in computer science, we're going to approach it by adding in a layer of indirection. We will write a wrapper object that is the only custom object around as far as the platform is concerned, and that wrapper object will know how to deal with multiple custom objects.

    This approach is used in the HealthAndFitness sample - look in the website\app_code\ThingTypes directory to find the full implementations of the code I list here. Go use those instead of using this code.

    We start with a base class for our custom objects. It looks a lot like the HealthRecordItem class, but the ParseXml() method is public. We will also use it to hold a bit of magic later...

    public class HealthRecordItemCustomBase
        protected CustomHealthTypeWrapper m_wrapper;

        public CustomHealthTypeWrapper Wrapper
    get { return m_wrapper; }
    set { m_wrapper = value; }

        public virtual void ParseXml(IXPathNavigable typeSpecificXml) {}
        public virtual void WriteXml(XmlWriter writer) {}

    The Wrapper property lets us get back to the wrapper for this instance. The usefullness of that will become apparent later (oooo, foreshadowing...).

    The wrapper class looks like this:

    public class CustomHealthTypeWrapper: HealthRecordItem
        const string NullObjectTypeName = "NULLOBJECT";

        private HealthServiceDateTime m_when;
        private HealthRecordItemCustomBase m_wrappedObject = null;

        public CustomHealthTypeWrapper() : this(null) {}

        public CustomHealthTypeWrapper(HealthRecordItemCustomBase wrappedInstance) :
    base(new Guid(ApplicationCustomTypeID))
            m_wrappedObject = wrappedInstance;

                // Test for null here because the deserialization code needs to create this object first...

            if (wrappedInstance != null)
    wrappedInstance.Wrapper = this;

    public HealthRecordItemCustomBase WrappedObject
            get { return m_wrappedObject; }
    set { m_wrappedObject = value; }

    So, the wrapped object has a reference to the wrapper, and the wrapper has a reference to the wrapped object. We also have a nice constructor to create a wrapper around an object, and when we use that one, it sets both pointers.

    It's now time to add some code to the wrapper class to save objects. Here's WriteXml():

    public override void WriteXml(XmlWriter writer)
        if (writer == null)
    throw new ArgumentNullException("null writer");


            string wrappedTypeName = NullObjectTypeName;

            if (m_wrappedObject != null)
                Type type = m_wrappedObject.GetType();
                wrappedTypeName = type.FullName;


            m_when.WriteXml("when", writer);


            if (m_wrappedObject != null)

    That's essentially unchanged except for two changes. First, if there's a wrapped object, we write out the actual type name, and then when it comes to writing out the custom type data, we delegate that to the wrapped object.

    On to ParseXml()

    protected override void ParseXml(IXPathNavigable typeSpecificXml)
        XPathNavigator navigator = typeSpecificXml.CreateNavigator();
        navigator = navigator.SelectSingleNode(

        if (navigator == null)
    throw new ArgumentNullException("null navigator");

        XPathNavigator when = navigator.SelectSingleNode("when");

        m_when = new HealthServiceDateTime();

        XPathNavigator formatAppid = navigator.SelectSingleNode("format-appid");
        string appid = formatAppid.Value;

        XPathNavigator formatTag = navigator.SelectSingleNode("format-tag");
        string wrappedTypeName = formatTag.Value;

        if (wrappedTypeName != NullObjectTypeName)
            m_wrappedObject = (
    HealthRecordItemCustomBase) CreateObjectByName(wrappedTypeName);

            if (m_wrappedObject != null)
                m_wrappedObject.Wrapper =
                XPathNavigator customType = navigator.SelectSingleNode("CustomType");
                if (customType != null)

    This is also not very different - the big change is that if there is a type name stored in format-tag, we create an instance of that type, find the custom data for it, and then call ParseXml() on that object. Note that it's possible that format-tag isn't a type that we know about, so we have to deal with those cases.

    The object instances are created by CreateObjectByName():

    public object CreateObjectByName(string typeName)
    Type type = Type.GetType(typeName);
    object o = null;

    if (type != null)
    if (type.BaseType != typeof(HealthRecordItemCustomBase))
    throw new ApplicationException("Custom type not derived from HealthRecordItemCustomBase");

            o =

        return o;

    We look up the type to see if it's one we recognize, make sure that it's derived from our base type (avoid lots of weird scenarios that might show up), and then creating an instance of it.

    That's the basic scheme. We also provide a helper method to register the custom type:

    public const string ApplicationCustomTypeID = "a5033c9d-08cf-4204-9bd3-cb412ce39fc0";

    public static void RegisterCustomDataType()
        ItemTypeManager.RegisterTypeHandler(new Guid(ApplicationCustomTypeID), typeof(CustomHealthTypeWrapper), true);

    You'll need to make sure this is called before saving or loading any custom types, or weird things happen.

    Custom type XML format

    Because the custom type XML is no longer handled by the platform, it can be whatever XML format you want. On the theory that many custom types may migrate to become platform types, I'd suggest using the XML format that the platform likes rather than inventing your own.

    Using the wrapper type

    The wrapper type is fairly easy to use - instead of saving an instance of the custom item, you create a wrapper around it, and then pass the wrapper to NewItem(), or whatever method you're calling.

    When you're querying, there is an extra step. You will get back a HealthRecordItemCollection full of CustomHealthTypeWrapper instances. Some of them will wrap the type you want, some of them some other type, and some of them won't wrap anything. So, you need to write something like the following:

    List<MyCustomObject> customItems = new List<MyCustomObject>();

    foreach (HealthRecordItem item in items)
        CustomHealthWrapper wrapper = (CustomHealthTypeWrapper) item;
        MyCustomObject custom = wrapper.WrappedObject as BPZone;
        if (custom != null)

    Then you end up with an array that only has the types you care about.

    I think that it's possible to modify the query filter so that it will only fetch the custom items of a specific type, but I haven't gotten around to trying it yet.

    A final bit of goodness

    The scheme as described is a bit unfortunate, in that the type derived from HealthRecordItemCustomBase doesn't have the base class items that are present in the HealthRecordItem. So, if I want to get the Key value, I'm forced to write:


    and similarly for the CommonData stuff:


    The final bit of goodness is to add a few properties in the HealthRecordItemCustomBase class that forward on requests to the wrapper class, so you can write:


    and have it refer to the information in the wrapper class.

  • Eric Gunnerson's Compendium

    HealthVault Data Types - Custom Data Types


    So, you're using the SDK, and it turns out that you need to store some data, and there isn't a built-in type that stores what you need. What do you do?

    The SDK provides a way for you to create your own custom type. There's an important limitation to this technique that I'll touch on later, so make sure to read all the way through. I would tell you now, but that would ruin the suspense.

    Creating the class

    First of all, you'll need to create the class. I'm going to be using a simplified version of a custom class that I created, without all the niceties like properties, validation logic, etc.  

    using Microsoft.Health;
    using Microsoft.Health.ItemTypes;

    public class PeakFlowZone: HealthRecordItem
        private double m_redOrangeBoundary;
        private double m_orangeYellowBoundary;
    private double m_yellowGreenBoundary;

        private HealthServiceDateTime m_when;

        public PeakFlowZone() : base(new Guid("a5033c9d-08cf-4204-9bd3-cb412ce39fc0"))

    The two important features of the class are that it derives from HealthRecordItem - as all good custom types should - and that it uses specifies a specific Guid in the constructor. This is the magic "custom type guid", and you need to use that exact one.

    Okay, so that seemed pretty easy. Next, we'll need to write the code to serialize and deserialize the data. This works a little like .NET serialization.

    The custom data type XML format

    The platform expects that you conform to a specific XML schema when you store your data. For our class, here's the way it needs to look:



      <summary />

      <PeakZone RedOrangeBoundary="3" OrangeYellowBoundary="4" YellowGreenBoundary="5" />


    The format-appid and format-tag elements are for your use - stuff whatever you want in there. When is the timestamp for the element, and summary is the summary text.

    Those elements are required. The format of the xml that you actually use to store your data is up to you.

    The XML Methods

    Now that we know what the format is, we need to write two methods. Here they are:

    protected override void ParseXml(IXPathNavigable typeSpecificXml)
        XPathNavigator navigator = typeSpecificXml.CreateNavigator();
        navigator = navigator.SelectSingleNode(

        XPathNavigator when = navigator.SelectSingleNode("when");
        m_when =
    new HealthServiceDateTime();

        XPathNavigator formatAppid = navigator.SelectSingleNode("format-appid");
        string appid = formatAppid.Value;

        XPathNavigator peakZone = navigator.SelectSingleNode("PeakZone");

        for (; ; )
            switch (peakZone.LocalName)
                case "RedOrangeBoundary":
                    m_redOrangeBoundary = peakZone.ValueAsDouble;

                case "OrangeYellowBoundary":
                    m_orangeYellowBoundary = peakZone.ValueAsDouble;

                case "YellowGreenBoundary":
                    m_yellowGreenBoundary = peakZone.ValueAsDouble;

            if (!peakZone.MoveToNextAttribute())

    public override void WriteXml(XmlWriter writer)


            m_when.WriteXml("when", writer);


    "RedOrangeBoundary", m_redOrangeBoundary.ToString());
    "OrangeYellowBoundary", m_orangeYellowBoundary.ToString());
    "YellowGreenBoundary", m_yellowGreenBoundary.ToString());

    Using The Class

    Now, we need to use the class. That's just like dealing with any other data type, with one exception. For the built-in types, the api layer already knows the correspondence between a specific guid and the type that it represents. If a type comes across and the type id is:


    then the system knows that it should create a Weight item. I found that value by looking at Weight.TypeId.

    In our case, the system doesn't know what to do with our custom type yet, so we need to register it. We do that through the following bit of code:

    ItemTypeManager.RegisterTypeHandler(new Guid("a5033c9d-08cf-4204-9bd3-cb412ce39fc0"), typeof(PeakFlowZone), true);

    Note that we're using the magic "custom data type" guid again, and then we're telling it the type of the data. The last parameter tells it not to worry if there's already a type registered for this guid, just do whatever we said... 

    Caveats and Limitations

    The biggest caveat is that there is currently only support for one registered custom type per application. So, once you've defined that PeakFlowZone type, you're through.

    There are a couple of ways to get around this limitation.

    First, you could introduce a "invalid data" concept for all the custom types that you wrote. The ParseXml() method would look at the format-tag attribute, and if it wasn't a PeakZone tag, it would just set the InvalidData flag. Then, before you do the query, you would register whatever type you cared about, and then filter out the instances with invalid data (which are really valid, just not the type you wanted).

    Another option is to build a "custom data type" wrapper that does this sort of thing for you. The wrapper type is the one that would be registered, and it would know how to create the appropriate type based on the format-tag setting. A query would then return a list of the wrapper types, and you'd need to drill into them to get the values you wanted out.

    I think the second approach is preferable, as it avoids having to get the registration correct, but it does involve a bit more plumbing.

Page 6 of 46 (1,136 items) «45678»