Posts
  • Eric Gunnerson's Compendium

    Simulators or not?

    • 0 Comments

    I’ve been spending some time playing with Cockburn’s hexagonal architecture  (aka “ports and adapters”), and the extension I learned from Arlo, simulators. I’ve found it to be quite useful.

    I was writing some code, and I ended up at a place I didn’t expect. Here’s the situation. I have the following external class (ie “port”).

    class EntityLoader
    {
        public EntityLoader(string connectionInformation) {}

        public IEnumerable<Entity> Fetch(EntithyType itemType) { … }
    }

    I need to use this class to some different kinds of entities, do some light massaging of data, and then query against the data. I’ll start figuring out what the adapter should be, and I’ll define it by the question that I want to ask it:

    interface IPeopleStore
    {
        IEnumerable<Employee> GetAllEmployeesForManager(Employee employee);
    }

    Now that I have the interface, I can use TDD to write a simulator that implements the interface:

    class PeopleStoreSimulator: IPeopleStore
    {
        public IEnumerable<Employee> GetAllEmployeesForManager(Employee employee) { ...}
    }

    The implementation for this will be pretty simple; I just add a way to get the list of employees for a manager into the simulator. Now I have unblocked my team members; they can code against the interface and use the simulator for their testing while I figure out how to write the version that talks to the EntityLoader.

    And this is where it got interesting…

    One of the cool things about port/simulator/adapter is that you can write one set of tests and run them against all of the adapters, including the simulator. This verifies that the simulator and the real adapter have the same behavior.

    That’s going to be problematic because the interface for Entity doesn’t give me any way to put data into it, so I can’t use the simulator tests on it. It will also do two things; fetch the data from the entity and implement the GetAllEmployeesForManager() method, and because I can’t put data into it, I don’t have a way to write a test for the method().

    It also violates one of my guidelines, which is to separate the fetching of data and the processing of data whenever possible. The problem is that we have the lookup method logic in a class that we can’t test – ie so we can’t adapt the data into what we need. That’s a good sign that adapter may not be a good choice here. How about a simpler approach, such as wrapper?

    Let’s start with the lookup logic. We’ll make PeopleStore a simple container class, and that will make the lookup logic trivial to test.

    class PeopleStore
    {
        IList<Employee> m_employees;
        IList<Employee> m_managers;

        public PeopleStore(IList<Employee> employees, IList<Employee> managers)
        {
            m_employees = employees;
            m_managers = managers;
        }
       
        public IEnumerable<Employee> GetAllEmployeesForManager(Employee employee)
        {
            …
        }
    }

    Now, I’ll work on the wrapper level. After going with an interface, I end up switching to an abstract class, because there is a lot of shared code.

    abstract class EntityStoreBase
    {
        protected IEnumerable<Employee> m_employees;
        protected IEnumerable<Employee> m_managers;

        IEnumerable<Employee> FetchEmployees() { return m_employees; }
        IEnumerable<Employee> FetchManagers() { return m_managers; }
    }

    class EntityStoreSimulator: EntityStoreBase
    {
        public EntityStoreSimulator(IEnumerable<Employee> employees, IEnumerable<Employee> managers)
        {
            m_employees = employees;
            m_managers = managers;
        }
    }

    class EntityStore : EntityStoreBase
    {
        public EntityStore(string connectionInformation)
        {
            EntityLoader loader = new EntityLoader(connectionInformation);

            m_employees = loader.Fetch(EntityType.Employee)
                                .Select(entity => new Employee(entity));
            m_managers = loader.Fetch(EntityType.Manager)
                                .Select(entity => new Employee(entity));
        }
    }

    That seems to work fine. Now I need a way to create the PeopleStore appropriately. How about a factory class?

    public static class EntityStoreFactory
    {
        public static EntityStoreBase Create(IEnumerable<Employee> employees, IEnumerable<Employee> managers)
        {
            return new EntityStoreSimulator(employees, managers);
        }

        public static EntityStoreBase Create(string connectionInformation)
        {
            return new EntityStore(connectionInformation);
        }
    }

    This feels close; it’s easy to create the right accessor and the EntityLoader class is fully encapsulated from the rest of the system. But looking through the code, I’m using 4 classes just for the entity-side part, and the code there is either not testable (the code to fetch the employees from the EntityLoader), or trivial. Is there a simpler solution? I think so…

    public static class PeopleStoreFactory
    {
        public static PeopleStore Create(IEnumerable<Employee> employees, IEnumerable<Employee> managers)
        {
            return new PeopleStore(employees, managers);
        }

        public static PeopleStore Create(string connectionInformation)
        {
            EntityLoader loader = new EntityLoader(connectionInformation);

            var employees = loader.Fetch(EntityType.Employee)
                                .Select(entity => new Employee(entity));
            var managers = loader.Fetch(EntityType.Manager)
                                .Select(entity => new Employee(entity));

            return Create(employees, managers);
        }
    }

    This is where I ended up, and I think it’s a good place to be. I have a class that is well-adapted to what the program needs (the PeopleStore), and very simple ways to create it (PeopleStoreFactory).

    Thinking at the meta level, I think the issue with the initial design was the read-only nature of the EntityStore; that’s what made the additional code untestable. So, as fond as I am of port/adapter/simulator, there are situations where a simple factory method is a better choice.

  • Eric Gunnerson's Compendium

    Identifying your vertical story skeleton

    • 0 Comments

    I’ve been leading an agile team for a while now, and I thought I would share some of the things we’ve learned. This one is about vertical slices, and learning how to do this has made the team more efficient and happier.

    To make this work you need a team that is cross-functional and has the skills to work on your whole stack (database/server/ui/whatever).

    As an example, assume that the team is working on the following story as part of a library application:

    As a authenticated library user, I can view the list of the books that I have checked out.

    The team discusses what this means, and here’s what they come up with:

    1. Authenticate the current user
    2. Fetch the list of book ids that are checked out by that user
    3. Fetch the book title and author for each book id
    4. Display the nicely-formatted information in a web page

    My old-school reaction is to take these four items, assign each of them to a pair, and when all of them are done, integrate them together, and the story will be complete. And that will work; lots of teams have used that approach over the years.

    But I don’t really like it. Actually, that’s not quite strong enough – I really don’t like it, for a bunch of reasons:

    • It requires a lot of coordination on details to keep everybody in sync. For example, changes need to be coordinated across the teams.
    • We won’t have anything to show until the whole story is done, so we can’t benefit from customer feedback.
    • Teams will likely be waiting for other teams to do things before they can make progress.
    • The different areas will take different amounts of time to finish, so some people are going to be idle.
    • Our architecture is going to be a bit clunky in how the pieces fit together.
    • It encourages specialization.
    • Nobody owns the end-to-end experience
    • Integrations are expensive; when we integrate the parts together we will likely find issues that we will have to address.

    The root problem is that the units of work are too coupled together. What I want is a work organization where the units of work are an end-to-end slice, and a pair (or whatever grouping makes sense) can go from start to finish on it.

    That seems to be problematic; the story describes a simple business goal, and it’s unclear how we can make it simpler. We *need* all the things we thought of to achieve success.

    This situation blocks most teams. And they are right in their analysis; there is no way to be simpler and to achieve success. Therefore, the only thing that might work is to redefine success.

    That’s right, we’re going to cheat.

    This cheating involves taking the real-world story and simplifying it by making it less real-world. Here’s a quick list of ways that we could make this story simpler:

    • We don’t really need the book title and author, so we redefine “list of books” to “list of book ids”.
    • The display doesn’t have to be nicely formatted, it could just be a list of book ids on a web page.
    • The display could even just be the results of a call to a web API that we make in a browser.
    • We could build the initial version as a console app, not as a web app.
    • The story doesn’t have to work for every user, it could just work for one user.
    • The list of returned books doesn’t have to be an actual list of checked-out books, it could be a dummy list.

    This is just a quick list I came up with, so there may be others. Once we have this list, we can come up with our first story:

    As a developer, I can call an API and get a predefined list of book ids back

    I’ve taken to calling this a “skeleton story”, because it’s a bare-bones implementation that we will flesh out later.

    We will go off and implement this story, deploy it as a working system, and – most importantly – verify that it behaves as it should.

    Getting to this story is the hard part, and at this point the remaining vertical slices are merely adding back the parts of the story that we took out. Here’s a possible list of enhancements:

    1. Fetch the list of book ids for a predefined user
    2. Fetch the list of book ids for a user passed into the api.
    3. Display the book ids in web page
    4. Display a book description instead of just a book id.
    5. Make the web page pretty.

    These will all be converted to stories, and we will verify that each one makes the system more real in a user-visible way. They aren’t perfect; some of the slices depend on the result of earlier slices, so we can’t parallelize across all 5 of them, and we will still need to have some coordination around the changes we make. These issues are more tractable, however, because they are in relation to a working system; discussions happen in the context of actual working code that both parties understand, and it’s easy to tell if there are issues because the system is working.

  • Eric Gunnerson's Compendium

    Rational behavior and the Gilded Rose kata…

    • 0 Comments

    The following is based on a reply to an internal post that I almost wrote this morning, before I decided that it might be of more general interest. It will take a little time to get to my point so perhaps this would be a good time to grab whatever beverage is at the top of your beverage preference backlog.

    Are you back? Okay, I’ll get started…

    A while back my team spent an afternoon working on the Gilded Rose kata. We used it to focus on our development skills using a pairing & TDD approach. This kata involves taking a very tangled routine (deliberately tangled AFAICT) and extending it to support a new requirement. It is traditionally viewed as an exercise in refactoring, and in the introduction to the kata, I suggested that my team work on pinning tests before refactoring or adding new functionality. My team found the kata to be quite enjoyable, as it involved a puzzle (unraveling the code) and the opportunity to work on code that is much worse than our existing codebase.

    This week, another team did the String Calculator kata (a nice introduction to katas and one that works well with pairing/TDD), and somebody suggested that Gilded Rose would be a good next step. I started to write a quick reply, which became a longer reply, which then evolved into this post.

    My initial reply was around how to encourage people to take the proper approach (if it’s not obvious, the “proper” approach is the one I took when I last did the kata…) – which was “write pinning tests, refactor to something good, and then add new functionality”. When writing it, however, it seemed like I was being overly prescriptive, and it would make more sense to remind people of a general principle that would lead them to a good approach.

    At this point, I realized that I had a problem. Based on the way the kata is written, I was not longer sure that my approach *was* the proper choice. The reason is that the kata provides a detailed and accurate description of the current behavior of the system and the desired changed behavior. After a bit of thought, I came up with four alternatives to move forward:

    1. Attempt to extract out the part of the code that is related to the area that I need to change, write pinning tests for it, and then TDD in the new functionality.
    2. Write pinning tests on the current implementation, refactor the code, then add new functionality.
    3. Write pinning tests on the current implementation, write new code that passes all the pinning tests, then add new functionality.
    4. Reimplement the current functionality from scratch using TDD and based on the detailed requirements, then add new functionality.

    Which one to choose?

    Well, the first one is something I commonly do in production code to make incremental changes. You can make things a little better with a little investment, and many times that’s the right choice WRT business value. If you want more info on this approach, Feather’s “Working Effectively With Legacy Code” is all about this sort of approach. It’s one of the two books that sits on my desk (anybody want to guess the other one?).

    Presuming that a kata that I finish quickly isn’t really my going, that leaves me with three options. My experience says that, if I have the Gilded Rose code and a complete specification of the desired behavior, I’m going for the last option. The problem isn’t very complex and, doing TDD is going to be straightforward, so I’m confident that I can get to an equivalent or better endpoint with less effort than dealing with refactoring the existing code.

    That conclusion was a bit surprising to me, given that this is commonly thought to be a refactoring kata, but it’s the logical outcome of having a perfect specification for the current functionality available. Last year I had the chance to fix a buggy area of code that handled UI button enabling/disabling, and I took this exact approach; I knew what the functionality would be, wrote a nice clean class & tests, and tossed the old code out. Worked great.

    In reality, however, this case is somewhat rare; most times you just have the code and your requirement is to add something new without changing the existing behavior, whatever it might be. Or, you have some requirements and some code, but the requirements are out of date and don’t match the code. So… I think the Gilded Rose kata would be a lot better if you didn’t have the description of the current behavior of the system, because:

    1. That’s when pinning becomes important, to document the existing behavior.
    2. Automated refactoring becomes more interesting because it allows me to make *safe* changes even if my tests aren’t perfect.
    3. I get led towards these approaches automatically.

    If I don’t have the description, I’m going to choose between the last two approaches. The tradeoff likely depends on how good my pinning tests are; if they are perfect I should just choose the option that is quicker. If they are less than perfect – which is generally going to be the case - then doing it through refactoring is likely a better choice.

    Or, to put it another way, the effort to write a reasonable set of pinning tests and refactor the code is generally going to be less than the effort to write a perfect set of pinning tests and write the code from scratch.

  • Eric Gunnerson's Compendium

    7 Hills 2014

    • 3 Comments

    The forecast did not look good. In fact, it looked pretty bad.

    It was Sunday of Memorial day weekend, and I was forecast-shopping. That’s what I do when I want to ride and the weather is marginal; I look at the different weather forecasts (Accuweather, wunderground, weather.com, national weather service) to see if I can find one that I like. They said – if I recall correctly – Rain, showers, showers, rain.

    I was registered to ride 7 hills for the nth time (where 5 < N < 10) on Memorial day. To be specific, I was registered to ride the 11 hills “metric century”. Quotes because a kilometer is about 8% shorter on this ride, needing only 58 miles to reach the metric century mark.

    I had tentatively agreed to ride with a few friends, which is not my usual modus operandi; after a few rides where a group ride turned into a single ride, I started doing most rides by myself.

    I rolled out of bed at 6AM on Memorial day, and took a look outside. It was wet but not raining. A look at the radar (the NWS National Mosaic is my favorite) showed that not only was there no rain showing, it looked like it was going to be that way for the next 6 hours or so.

    Normally, my ride prep would be done the night before; I’d have everything that I wanted out on the counter, appropriate clothes chosen, and a couple of premixed bottles in the fridge. Since I expected not to be riding, I had to do all of this in a bit of a hurry. I got packed, grabbed my wallet, keys, phone, and GPS, and headed out.

    I passed the first group parking on Lake Washington Blvd (people always park too far to the south), find a spot and unload. I roll into the park, get my registration band, route sheet, and find my companions. I’ll be riding with riding friends Joe and Molly, and their friends Bill and Alex. We roll out at 8:20 or so.

    Market street (Hill 1) is quickly dispatched, and we head up Juanita (Hill 2). The first two hills are fairly easy; something like 5-7% gradient max. We regroup at the top of Juanita (well, actually not the top of the hill, but the part where we head back down). My legs have felt pretty good so far, but we are coming to Seminary hill (#3), which is steeper and harder than the other two. I think it’s the second-hardest climb of the ride. It also is a bit misleading; there’s a steep kicker right at the beginning, a flat part, and then it steepens up again for the remainder of the climb.

    I start the climb. I’m have a secret weapon – my power meter. I know from the intervals that I’ve been doing that I can hold 300 watts for 2 minutes. I also know that I can hold 240 watts for 10 minutes, so I set that as my “do not exceed” level. I pass a few people, pass a few more, and before I know it, I’m at the top. I do have legs today.

    The others filter up soon after. Well, that’s not factually true; Joe and Alex finished quite a bit faster than me, and Molly and Bill filter up soon after. Joe is my benchmark for comparative insanity, so I know that him finishing in front of me just means that things are right with the world.

    We head north to descend; Joe/Molly/Bill have an almost-incident with a right-turning truck. We get on the trail and spin to Norway hill. As we approach the base, Joe is talking with a few friends, and we turn right and the climb starts. The road turns left, and I see a bunch of people on the hill. I start passing people, and strangely, nobody is passing me. I hit the stop sign, keep climbing, and eventually top out. I passed 40 people on the way up, get passed by none. Though in the spirit of full disclosure, I did pass the last 5 as they were getting ready to pull off near the top, and most of these riders are out here for the “7 hills” version of the ride.

    We head south, and turn left on 132nd. The previous course would take us all the way to my favorite intersection  - 132nd st and 132nd ave – but this year they instead route us south, and then to a food stop near Evergreen Hospital. Somewhere on the last section, the sun has popped out, and we feel pretty good. I get some sort of energy bar and pretty tasteless bagelette. After a bit too long waiting, we head out again, and take 116th north. We descend down brickyard, and turn right, heading towards back on the south towards Winery hill.

    And into the headwind. I go into ride leader mode, and settle in with the rest of the group somewhere behind me. After a few minutes, Bill – who is tall and wide like me – passes and pulls for a little bit. Soon enough, we reach the base of Winery. The route that we are taking – through the neighborhood – is a series of climbs and flats. We hit the first one, which is something like 15%, and Joe and Alex ride off. I try to stay around 300 watts on the climbs and recover a bit on the flats. Soon enough, I hit the top, and find the the 7 hills bagpiper is too busy having his picture taken with riders to play. He starts playing as Molly pulls up and we ride off down to the next food stop. The new route has changed this experience; previously you would have to climb north while being demoralized by the riders approaching because they had already finished winery, and then have the opposite feeling when you come down the same road after Winery. The new route is fine but is missing a bit of the emotional experience of the old one.

    I grab a dark chocolate chip cookie, refill my Nuun bottle and deploy some cheez-its, my wonder ride food.

    We now have a decision to make. We have done 6 hills, and we can either descend down into the Sammamish River Valley, ride south, and climb up hill #7, Old Redmond Road, or we can head east to grab an extra 4 hills before returning for the last climb. We decide to do the full metric and head east. This takes us on 116th to a short but really steep (say, 17%) climb. There’s a route via 124th that is much more gradual, so I’m not sure whether this route is because the organizers don’t know about the other route or it’s a deliberate choice.

    This is one of the downsides of being a ride leader; I know the vast majority of the roads out here and if I’m on an organized ride I’m constantly plotting where we are going versus what the other options are.

    The next climb is Novelty Hill. There really isn’t a lot of novelty involved; it’s a 500’ or so climb with a lot of fast traffic. On the way up, I find myself stuck on “If you’re happy and you know it, clap your hands”, planted by Joe a few minutes before. A few minutes later, it morphs to the surprisingly appropriate “I’ve been through the desert on a horse with no name, it felt good to get out of the rain” (America, 1971).

    We finish, regroup, and head south to Union hill road. There’s a bonus half hill here that isn’t part of the 11 hills, we finish that section, and head north to descend Novelty again, and head up NE Redmond Road (not to be confused with Old Redmond Road, which we will climb later). This is a fairly easy climb but everybody’s legs are a bit tired. Even Joe’s, though his are tired because of the miles that he has put in the past few days. Another hill top, another descent, and we head up education hill on 116th for the second time (re-education hill). That takes us to the last food stop, where I have a fairly pedestrian ham and cheese wrap and make up another bottle of Nuun. Unfortunately, it seems that I chose “moldy fruit” flavor, so I’m not too excited about it, but I choke a bit down.

    We descend, head across the valley with a vicious sidewind which turns into a headwind as we head south. I pull for Molly for the couple of miles, then Molly and Bill and I hit the base of Old Redmond Road at the same time. This is the last hill, and I open it up a bit, passing X people (5 < X < 300,000) on the way up. We crest, regroup, and head down  the last descents and the final run on Lake Washington Blvd back into Redmond. I get ahead, wait for the group, Joe goes by, and I find that I have one last sprint in my legs, so I open it up, and catch him.

    Then it’s through to the finish, chocolate milk, and strawberry shortcake.

    Normally at this point, I would talk about stats, but I only have 30 miles of the ride. I *can* say that I got PRs on Seminary, Norway, and Winery hills, so it’s pretty clear that I did have legs.

     

  • Eric Gunnerson's Compendium

    A Programmer's Guide to C# 5.0

    • 3 Comments

    My author's copies of the Fourth Edition of my book showed up today:

    It is significantly updated from the previous version. I especially enjoyed writing the sections on Linq and asynchronous features.

     

     

     

  • Eric Gunnerson's Compendium

    Small in, Big out

    • 10 Comments

    I’ve come across some code that is using – and overusing – IEnumerable<T>, and thought it might be of general interest.

    Consider the following method signatures:

    IEnumerable<string> Process(List<string> input);

    IList<string> Process(IEnumerable<string> input);

    Of the two, which is the better choice? Write down your answer.

    There are really two questions – what is the best choice for input parameters, and what is the best choice for return values. I’ll cover them separately:

    Small In

    The input parameter should be the smallest (ie least functional) type that allows the method to efficiently do what it needs to do. I’ve seen methods like this:

    void Process(List<string> input)
    {
        foreach (string in input)
        {
            …
        }
    }

    (or maybe the Linq equivalent).

    In this case, you are just making things harder for the caller; all you really need is an IEnumerable<string>, so that’s what you should specify. On the other hand, I’ve also seen code like this:

    void Process(IEnumerable<string> items, IEnumerable<ReferenceItem> referenceItems)
    {
        var lookup = referenceItems.ToDictionary(referenceItem => referenceItem.Name, referenceItem => referenceItem.Contents)'
        …
    }

    This code takes a simple enumerable, and constructs a dictionary out of it. This is worse than the first case; if you need a dictionary, ask for a dictionary.

    Big Out

    Output parameters should be the biggest (ie most functional) type that is acceptable for the scenario. I’ve seen methods like this:

    IEnumerable<string> Process(List<string> input)
    {
        List<string> items = new List<string>();

        // fill list here

        return items;
    }

    The developer has created a beautiful, functional list object, but they’re only going to let the caller enumerate over it. This often leads to my favorite outcome, where the caller writes something like:

    List<string> items = new List<string>(myClass,Process(input));

    Don’t do that. Just return the List<string> or perhaps the IList<string>.

    Return values aren’t as cut-and-dried as input parameters, however. If we modify the previous example as follows:

    IList<string> Process(List<string> input)
    {
        m_items = new List<string>();

        // fill list here

        return m_items;

    In this example, the class is retaining ownership of the list, and in that case, it probably doesn’t want to give it out to somebody who could clear it, or replace all the strings with the string “Haddock”. If the class is going to retain ownership, it should use a return type that prevents bad things like that from happening.

  • Eric Gunnerson's Compendium

    A Programmer’s Guide to C# 2012

    • 0 Comments

    More months than I’d like to think about, I decided to undertake an update of my sorely out-of-date C# book, and Saturday I submitted the last of the chapters for the rewrite. I still have the author review (after the tech review) and the final review to do on each chapter, but that’s a huge amount of work out of the way.

    Back when the last version of the book came out, C# was an upstart language that had just added generics to it. Now, several * 3 years later, it has matured into a language with a lot going for it. Over that same time period, I’ve written a lot of C# code and have much more informed opinions about what works and what doesn’t.

    When I was working on the update, I was once again reminded that there is a huge gulf between “I understand how that works enough to use it” and “I understand how that works enough to write about it”, and I’ve enjoyed writing the chapters on Linq, the new asynchronous support, and a bit about Roslyn.

  • Eric Gunnerson's Compendium

    A new year and a new group

    • 4 Comments

    I’m starting off the new year (well, in a few weeks…) in a new group. I am leaving the HealthVault team, and moving to an internal Engineering Excellence team where I will be working on… making our engineering 28.5% more excellent.

    Actually, I’m not sure what I’m able to talk about yet, so I’ll leave it at that.

  • Eric Gunnerson's Compendium

    Improve HealthVault query efficiency with final transforms…

    • 0 Comments

    A while back I wrote a post about using the built-in HealthVault transforms to get a different view on your data. That post discusses the mtt, stt, and form transforms.

    There’s another transform option in HealthVault that gives you the option to transform the response that comes back. It is best explained through a scenario. Most of this discussions is going to be at the raw xml request/response level. Some

    Getting the instance ids of all Weight items in a record

    If you want to do a full synchronization of weight values in a HealthVault record (and please read the guidance here before you decide to do so), you will need to get the instance ids of all the items. This simplest way to do this is just to ask for all the weight items, which will result in the following response:

    <response>
      <status>
        <code>0</code>
      </status>
      <wc:info xmlns:wc="urn:com.microsoft.wc.methods.response.GetThings3">
        <group>
            <thing>
            <thing-id version-stamp="8853d460-3c79-4284-aa12-08cb9730d151">645cae6f-942d-40bb-bc36-5529b6f9f46a</thing-id>
            <type-id name="Weight Measurement">3d34d87e-7fc1-4153-800f-f56592cb0d17</type-id>
            <thing-state>Active</thing-state>
            <flags>0</flags>
            <eff-date>2011-10-26T10:14:01.17</eff-date>
            <data-xml>
              <weight>
                <when>
                  <date>
                    <y>2011</y>
                    <m>10</m>
                    <d>26</d>
                  </date>
                  <time>
                    <h>10</h>
                    <m>14</m>
                    <s>1</s>
                    <f>170</f>
                  </time>
                </when>
                <value>
                  <kg>79.4010889292196</kg>
                  <display units="lbs">175</display>
                </value>
              </weight>
            </data-xml>
          </thing>
        </group>
      </wc:info>
    </response>

    Where the <thing/> element is repeated for each weight item.

    That’s a lot of data (almost 2K) just to get the single instance guid. The first thing we can do is change our request so that the response only contains the key information. This is done by telling the platform not to return you any full items, but just return them as partial items. You do that by setting the max-full attribute to “0”. If you are working with the SDK, you can set the MaxFullItemsReturnedPerRequest property and use GetMatchingItemsRaw().

    If you do this, you will get a response that looks like this:

    <response>
      <status>
        <code>0</code>
      </status>
      <wc:info xmlns:wc="urn:com.microsoft.wc.methods.response.GetThings3">
        <group>
          <unprocessed-thing-key-info>
            <thing-id version-stamp="8853d460-3c79-4284-aa12-08cb9730d151">645cae6f-942d-40bb-bc36-5529b6f9f46a</thing-id>
            <type-id name="Weight Measurement">3d34d87e-7fc1-4153-800f-f56592cb0d17</type-id>
            <eff-date>2011-10-26T10:14:01.17</eff-date>
          </unprocessed-thing-key-info>
        </group>
      </wc:info>
    </response>

    That takes us down to about 500 bytes, which is much smaller. But there’s still a lot of extraneous information there. What we really need is a way to get back only the information that we care about. That something is a HealthVault final xsl.

    A final-xsl is an application-supplied transform that is run on the response right before it is sent back to the client. In the .NET sdk, the GetTransformedItems method is used. In the pure xml world, it is done by adding a <final-xsl> element to the request after the <country> element and before the <msg-time> element. The xsl is represented as a string in the request, so it must be appropriately escaped in the request xml.

    Here’s a transform I wrote to process the unprocessed items response:

    <xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:fo="http://www.w3.org/1999/XSL/Format" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:fn="http://www.w3.org/2005/xpath-functions" xmlns:wc="urn:com.microsoft.wc.methods.response.GetThings3" exclude-result-prefixes="fn fo wc xs">
      <xsl:output omit-xml-declaration="yes"/>
      <xsl:template match="/response">
        <response>
          <xsl:copy-of select="status"/>
          <xsl:copy-of select="error"/>
          <wc:info>
            <xsl:for-each select="wc:info/group">
              <group>
                <xsl:for-each select="unprocessed-thing-key-info">
                  <id>
                    <xsl:value-of select="thing-id"/>
                  </id>
                </xsl:for-each>
              </group>
            </xsl:for-each>
          </wc:info>
        </response>
      </xsl:template>
    </xsl:stylesheet>

    Which results in the following output:

    <response>
      <status>
        <code>0</code>
      </status>
      <wc:info xmlns:wc="urn:com.microsoft.wc.methods.response.GetThings3">
        <group>
          <id>645cae6f-942d-40bb-bc36-5529b6f9f46a</id>
        </group>
      </wc:info>
    </response>

    That takes us down to about 250 bytes, but still preserves the basic form of the response; the mobile libraries will still be able to parse it successfully. If the application is willing to do more parsing, we can simply further to:

    <xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:fo="http://www.w3.org/1999/XSL/Format" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:fn="http://www.w3.org/2005/xpath-functions" xmlns:wc="urn:com.microsoft.wc.methods.response.GetThings3" exclude-result-prefixes="fn fo wc xs">
      <xsl:output omit-xml-declaration="yes"/>
      <xsl:template match="/response">
            <xsl:for-each select="wc:info/group">
                <xsl:for-each select="unprocessed-thing-key-info">
                  <id>
                    <xsl:value-of select="thing-id"/>
                  </id>
                </xsl:for-each>
            </xsl:for-each>
      </xsl:template>
    </xsl:stylesheet>

    which yields:

    <group>
      <id>645cae6f-942d-40bb-bc36-5529b6f9f46a</id>
    </group>

    which takes us to 66 easy-to-parse bytes.

    Final XSL parameters

    The following set of parameters are passed to all final-xsl transforms:

    • currentDateTimeUtc - the date and time just before the transform started executing
    • requestingApplicationName - the name of the application that made the request to HealthVault.
    • countryCode - the ISO 3166 country code from the request.
    • languageCode - the ISO 639-1 language code from the request.
    • personName - the name of the person making the request.
    • recordName - if the request identified a HealthVault record to be used, this parameter contains the name of that record.
  • Eric Gunnerson's Compendium

    Mobile HealthVault Apps and Sync

    • 0 Comments

    I have been working with some developers working on mobile applications that connect to HealthVault, and the issue of synchronizing data between the mobile device and HealthVault has come up a couple of times.  We’ve come up with some guidance around this area and I’d like to share it more widely.

    I’m assuming that the mobile application needs to keep the data around to work effectively – for example, it may want to produce a chart of weight values for the last few years. If your application only works with small amounts of data, you may not need to store any data locally.

    Our goal

    First, a bit of expectation setting about what our goal is.

    We can start with a goal of “perfect sync” between the mobile application and HealthVault. It is possible to be current within a few seconds – but to do so will result in:

    1. A slower mobile application
    2. More bandwidth usage by the application
    3. Lots of extra requests to the HealthVault servers

     

    The first two aren’t good for the user of the application, and the HealthVault team would like to avoid the third one.

    Instead of trying for the “perfect sync”, we’re going to try for a “good enough sync” – a scheme that is right the majority of time for the majority of users.

    “Good enough sync” guidelines

    These are our current guidelines for keeping data synced on mobile devices.

    1. Fetch only the items that have changed

    If your application fetches all of the weight items in a user’s record and the user has recorded daily weights for the last 3 years, it’s going to take a while to process 1000+ weights.

    Instead, when the application fetches data items, request the audit section as part of the filter:

    <info>
      <group>
        <filter>
          <type-id>30cafccc-047d-4288-94ef-643571f7919d</type-id>
          <thing-state>Active</thing-state>
        </filter>
        <format>
          <section>audits</section>
          <section>core</section>
          <xml/>
        </format>
      </group>
    </info>

    You will get back XML that looks like this:

    <thing>
            <thing-id version-stamp="cdb934a8-08c9-44a2-85e2-429cde027d01">1a868d6f-26c2-4504-bbc5-7b28b11de351</thing-id>
            <type-id name="Medication">30cafccc-047d-4288-94ef-643571f7919d</type-id>
            <thing-state>Active</thing-state>
            <flags>0</flags>
            <eff-date>2008-09-12T09:46:27.987</eff-date>
            <created>
              <timestamp>2008-09-12T16:46:28.22Z</timestamp>
              <app-id name="HelloWorld-SDK">05a059c9-c309-46af-9b86-b06d42510550</app-id>
              <person-id name="Eric Gunnerson">41b87f12-9b6f-40b6-a6ee-be4fbfd12170</person-id>
              <access-avenue>Online</access-avenue>
              <audit-action>Created</audit-action>
            </created>
            <updated>
              <timestamp>2008-09-12T16:46:28.22Z</timestamp>
              <app-id name="HelloWorld-SDK">05a059c9-c309-46af-9b86-b06d42510550</app-id>
              <person-id name="Eric Gunnerson">41b87f12-9b6f-40b6-a6ee-be4fbfd12170</person-id>
              <access-avenue>Online</access-avenue>
              <audit-action>Created</audit-action>
            </updated>
            <data-xml>
              <medication>
                <name>
                  <text>Marmoset</text>
                </name>
              </medication>
              <common/>
            </data-xml>
          </thing>

     

    When you process the items, extract the updated timestamp out, and figure out the maximum timestamp value for all of the items that you have fetched. Then, when you call next time, you can only get items that are newer than that timestamp:

    <filter>
      <type-id>30cafccc-047d-4288-94ef-643571f7919d</type-id>
      <thing-state>Active</thing-state>
      <updated-date-min>2011-10-10T16:50:45.334Z</updated-date-min>
    </filter>

    That will reduce the amount of data that you fetch considerably.

    2. Fetch partial things initially

    When a filter would return a large number of items, HealthVault will return the more recently updated items in their entirety, and then it will return the remainder of the items as partial items. Instead of a full “<thing></thing>” structure, you will see something like this:

    <unprocessed-thing-key-info>
      <thing-id version-stamp="fdd3f6cd-6eb6-46d7-bd7d-3d2c2fccc08b">34b0f978-f454-476a-b4a2-004c4a504e66</thing-id>
      <type-id name="Medication">5c5f1223-f63c-4464-870c-3e36ba471def</type-id>
      <eff-date>2011-07-22T19:22:30.94</eff-date>
    </unprocessed-thing-key-info>

     

    The number of full items that are returned can be specified by the caller, in the following manner:

    <group max-full="1">
      <filter>
        <type-id>30cafccc-047d-4288-94ef-643571f7919d</type-id>
        <thing-state>Active</thing-state>
      </filter>
      <format>
        <section>audits</section>
        <section>core</section>
        <xml/>
        <type-version-format>30cafccc-047d-4288-94ef-643571f7919d</type-version-format>
      </format>
    </group>

    Where the “max-full” attribute sets the number of full items returned in this request. There is a hard limit of 240 full items returned per request.

    Instead of fetching a large number of instances all at once, applications should fetch a few full items initially and save the unprocessed keys around. It can then fetch any later items in the background. This can be done by listing all of the instance ids (obtained from the <thing-id> element of the unprocessed thing key info section) in the filter:

    <group>
      <id>34b0f978-f454-476a-b4a2-004c4a504e66</id>
      <id>e8655aca-eb68-4670-b253-4bc3585fa513</id>
      <format>
        <section>audits</section>
        <section>core</section>
        <xml/>
      </format>
    </group>

     

    3. Consider fetching more than one data type in a single request

    If your application needs to fetch data for more than one data type, you should consider whether you can fetch all the data in a single get request. This will reduce the number of trips you make to the server and the amount of data you send over the network.

    This is done by including multiple filters in a single request:

    <group>
      <filter>
        <type-id>30cafccc-047d-4288-94ef-643571f7919d</type-id>
        <thing-state>Active</thing-state>
      </filter>
      …
    </group>
    <group>
      <filter>
        <type-id>3d34d87e-7fc1-4153-800f-f56592cb0d17</type-id>
        <thing-state>Active</thing-state>
      </filter>
       …
    </group>

    The response will have one group for each group in the request.

    4. Fetch only the data that you need now

    HealthVault records can contain a considerable amount of data in them. A weight tracking application may work fine with your set of test data, but if it gets pointed to a record that contains daily weights for the past 3 years, it may roll over and die.

    Applications should instead limit their queries to fetch bounded amounts of data. If the application is only going to show the user their weight for the last month, it should only fetch that data.

    5. Fetch only on startup and when data has been changed locally

    Mobile apps tend to be run for short periods of time, so checking for new data at startup and when you know there is new data (ie you put it there) is the recommended approach.

    Applications may add a “refresh” option if desired.

    Deleted data

    HealthVault does not currently provide a simple way for applications to determine when data has been deleted from a user’s HealthVault record. Since deletions are rare in HealthVault, we recommend that mobile applications do not attempt to track deletions automatically.

    An app may choose to provide a “synchronize” option that the user may choose to invoke. In that case the app should fetch using max-full=”0”, which will return the minimal amount of data.

    If an application does want to detect deleted data, it should fetch the things using max-full=”0”, and then remove any items in the local cache that were not returned from the query.

  • Eric Gunnerson's Compendium

    A movie scene

    • 1 Comments

    Anybody who has watched movies about software development knows that reality is much different. None of us work with huge screens on the wall, and debugging is a lot of unexciting detective work.

    Today, we were doing some remote debugging over live meeting in a conference room, trying to figure out why an interaction between a website and a web service wasn’t working correctly. We worked through some backend verification, which is as exciting as writing and executing ad-hoc SQL queries always is (not very). Then we moved on to running Fiddler, where we started looking at requests. The URL was pretty complicated and we were verifying all the items in it when I noticed that there was a problem with the base part of the URL. I couldn’t figure out any easy way to tell people what it was because the screen was busy, so I jumped up and pointed at the spot on the giant projected desktop and exclaimed “that’s the problem!”.

    And it was, and I got to do a happy dance. But the soundtrack was a bit lacking.

  • Eric Gunnerson's Compendium

    Dealing with blob data in HealthVault XML

    • 0 Comments

    I recently had a request for more information on dealing with blob data using our newly-released Mobile support, which finally got me to write something about low-level blob support.

    This information applies to any application that isn’t using the .NET SDK to talk to HealthVault. If you are using the .NET SDK, I recommend using the support that is provided in the SDK.

    HealthVault Blob Options

    HealthVault provides the capability of storing additional data – such as documents, images, etc. – as a blob of data attached to any of the HealthVault data types.

    HealthVault provides two ways for applications to manage blob data; an inline method where the blob data is passed with the rest of the data for an item instance, and a streaming approach where blob data is accessed through http get and put requests.

    Inline blob data

    This method passes the blob data in a base64-encoded format with the rest of the item data. It’s the simpler of the methods to use, but it has the following drawbacks:

    • The total size of a HealthVault request has a maximum, which limits the space available for blob data to approximately 5MB (this limit is subject to change).
    • The total size of a HealthVault response has a maximum size, which limits how much blob data can be retrieved, especially if multiple objects with blobs are returned.
    • When the blob data is fetched for an object, data for all blobs is fetched; it isn’t possible to fetch the data for a single blob.
    • Network use will be a little higher and data will be slower since the base64 encoding is a little (about 1/3) bigger than the raw bytes.

    Here’s how you deal with blob data using the inline method:

    Fetching blob data as inline data

    Here’s the “<info>” section of a GetThings request:

    <info>
        <group>
            <filter>
                <type-id>3d34d87e-7fc1-4153-800f-f56592cb0d17</type-id>
                <thing-state>Active</thing-state>
            </filter>
            <format>
                <section>core</section>
                <section>blobpayload</section>
                <xml/>
                <type-version-format>3d34d87e-7fc1-4153-800f-f56592cb0d17</type-version-format>
                <blob-payload-request>
                    <blob-format>
                        <blob-format-spec>inline</blob-format-spec>
                    </blob-format>
                </blob-payload-request>
            </format>
        </group>
    </info>

    The “<section>blobpayload</section>” line specifies that in addition to the core part of the data item, any blob data should be passed back along with the rest of the XML. If you forget to do that, you won’t get any blob data…

    This query returns a <blob-payload> section that comes after the <data-xml> section:

    <blob-payload>
      <blob>
        <blob-info>
          <name />
          <content-type>text/plain</content-type>
          <hash-info>
            <algorithm>SHA256Block</algorithm>
            <params>
              <block-size>2097152</block-size>
            </params>
            <hash>K9OYLkNyHLtTKRinTVTPh4WqbsZQ5l3jcUBIoMieNIc=</hash>
          </hash-info>
        </blob-info>
        <content-length>11</content-length>
        <base64data>U2FtcGxlIFRleHQ=</base64data>
      </blob>
    </blob-payload>

    The following elements are important:

    • name: The name of the blob.
    • content-type: The content type that was specified when the blob was saved
    • content-length: The length of the blob
    • base64data: The blob data

    Uploading blob data as inline data

    Here’s an example of saving a blob as inline data. The blob is passed at the same time the rest of the data for the instance is passed.

    <info>
        <thing>
            <type-id>3d34d87e-7fc1-4153-800f-f56592cb0d17</type-id>
            <thing-state>Active</thing-state>
            <flags>0</flags>
            <data-xml>...</data-xml>
            <blob-payload>
                <blob>
                    <blob-info>
                        <name/>
                        <content-type>text/plain</content-type>
                    </blob-info>
                    <content-length>4</content-length>
                    <base64data>U2FtcGxlIFRleHQ=</base64data>
                </blob>
            </blob-payload>
        </thing>
    </info>

    The <data-xml> section is removed for the sake of brevity. The important parts are:

    • name: The name of the blob
    • content-type: The encoding of the data
    • content-length: The length of the data in bytes
    • base64data: The data expressed in base64 format

    Streamed blob data

    In the streamed method, the blog data is fetched or stored using separate interfaces or methods.

    Fetching blob data as streamed data

    Fetching the blob data is simple. If you change the filter to be:

                        <blob-format-spec>streamed</blob-format-spec>

    Instead of the base64data element, you will get a blob-ref-url element that looks something like this:

      <blob-ref-url>https://platform.healthvault-int.com/streaming/wildcatblob.ashx?blob-ref-token=ASAAAA%2br2bo24GxMiX1j8dyBpoQhGSTp1z9laFjW0PVfgRzyT3PzSWgLdGubgPEK4bUbZG0ni4nY5mlXTW9ZYZQi2iFVXOstWasUtNmtNxJj2NBLolSd9J4MI4aSPID0w1l%2bHwSq%2fEM1BADh%2b01iMtnEIBEMQt%2fuDu0yQrazxjaXR0QokGLtRFOduPvo0LyMDWuo8meFRTPoCNfQoE3IkqIgkvY3gS1KssuQtDXmjVkKCKLSi8lHZV0WjYnCdnilHUAKWoWSOp4d6QX3%2fAUbRbXzzdIr00WGtrDOuoZwXWs5MA%2fKSfi%2fbp4Gfh96ZXJP%2bRy9lDDj8RkWmorT5SQW9QC4QEUONDxW1vSelijjvno451gA%2fi3Ufa57k2%2fs0x6swKrFRDTF0OZknjc5Id92zAnjvOFa5rs4v430bpZiD5MScUsWmEwTpqucCsKorXodAN823Wx4J1VPkeoOZiejnGJePYdyN91g</blob-ref-url>

    You can then download the blob data directly using the URL.

    Uploading blob data as streamed data

    Uploading blob data through the streamed interface is considerably more complex; it requires calling the BeginPutBlob method to get the destination url and then uploading in chunks of data that are a specific size (it also supports signing data, which makes things even more complex).

    If you are interested in using this to load large blobs, let me know, and I’ll try to update this to show that approach as well.

  • Eric Gunnerson's Compendium

    CHC 2011 HealthVault talk, and what is new for HealthVault this past year…

    • 0 Comments

    When we were creating our talk, we had a problem.

    Well, to be fair, we had a few problems, but the problem that I want to talk about is the “for more information” slide, the one that you put at the end to direct people to a location where they can find more information.

    There were actually two problems.

    The first was was that we covered 13 different things during the talk, and there was no way to fit enough information onto a single slide (or even a few slides) to help people out.

    The second was that a fair number (1 “fair” = 6) of items aren’t yet released and this was the first time we were talking about them, so there are no public resources to reference.

    We decided to solve that by doing a “for more information” blog post on the HealthVault blog, which would help attendees find more information and also be of use to those who didn’t attend the talk.

    And then I didn’t remember to write anything until Tuesday night, and had to do a bit of scrambling to get it done. But now, for your reading pleasure, you can read HealthVault at CHC 2011, find out what we demoed, what we announced in the mobile space, and other juicy tidbits of information.

    The conference itself was pretty good, but at only a day and a half tends to feel pretty rushed. We spoke at 8AM on the second day of the conference, second only to the immediately after lunch as a slot that I would like to avoid, but the crowd seemed pretty good, at least, to the extent that I could see them; I tried to look around in the crowd but was a bit hampered by the lights; two searchlights previously used to illuminate Saturn V rockets were aimed directly at my face, and I literally could not tell whether people were standing or sitting; they could all have been wearing clown makeup for all I knew. And if you looked up too much, you’d end up with a bunch of spots in your eyes that made it hard to read your slides.

    Our demo had a lot of moving parts, and it mostly worked correctly. What Vaibhav did was clean as far as I could tell; in my part, the conference center WiFi timed out and I had to re-initiate it during the talk, and I ended up provisioning the mobile application to talk to our practice account, not our live account, but luckily Vaibhav had written the eventing application so that either account worked. Oh, and I had unexpected lack of success with our CAPTCHA, even with the help of others.

    Our goal was to time the talk to 45 minutes, and after answering a few questions during the talk, we finished at 8:52 which is pretty much perfect in my book.

    Please let us know if you have questions or comments.

  • Eric Gunnerson's Compendium

    Speaking at CHC 2011

    • 0 Comments

    I'm in Chicago this week to speak at the Microsoft Connected Health Conference, 2011 edition. It's been nice to get away from the cool and rainy northwest and spend some time in Chicago,where the weather has been... well, it's been cool, and rainy, though the rain has been a bit torrential at times. We are rain wimps in Seattle.

    I ended up with some spare time earlier this week and spent a morning at the Museum of History and Industry, where I toured the U505 (an authentic WWII U boat captured in 1944 IIRC), and the bodies in motion exhibit, which was Oh my god did you see that guy? that's so gross I think I'm going to be sick!, or at least that was what the group of teenagers next to me thought. I found it fascinating and disturbing at the same time.

    I'm co-presenting at an overview presentation on HealthVault, where we have an hour to cover everything we've done in HealthVault in the last year or so. We have an involved demo and technical explanation, then a lot of short items. I'm thoroughly practiced-out the night before the talk, but I feel pretty good about it.

    The conference so far has been pretty good. We had a HealthVault-specific event last night and I was able to put some faces with a few names of people, and talk with partners about what they are doing.

    We will be posting more information about what we talk about on the HealthVault blog; I'll link here when the post is up.

  • Eric Gunnerson's Compendium

    Data sorting dances

    • 1 Comments

    Intercultural Computer Science Education

    Data sorting dances

    I enjoyed the dances, though I'm not sure that insertion sort is correctly danced.

     

  • Eric Gunnerson's Compendium

    Default parameters in C#

    • 4 Comments

    From an internal discussion we're having on the advisability of using default parameters in C#:

    Currently, the pain and limitation of doing overloads forces you to rethink how a method should work. Consider the following:

     Process(int a);

    Process(int a, float b);

    Process(int a, float b, string c);

     

    If I now need to change how that works in some situations, I could add a boolean to control that behavior, but it’s not obvious how to add it to the current overload scheme. I can do something like:

     

    Process(int a);

    Process(int a, bool doBackground);

    Process(int a, float b);

    Process(int a, float b, bool doBackground);

    Process(int a, float b, string c);

    Process(int a, float b, string c, bool doBackground);

     

    But not only is that a bit hard to write, it’s a bit confusing, and if I need to add another parameter in the future, I’m pretty much SOL. So, that forces me to consider the alternatives; should I go with a “Settings” class (like XmlWriterSettings), should I live with it, or should I think about refactoring the process to simplify things? That forced stop gives me the chance to think about better ways to approach things.

     

    With default parameters, it’s going to be really tempting to just add a default, and it’s more likely you’ll end up with methods like this:

     

    Process(int a, float b, string c, bool doBackground = false, bool writeToLog = true, string database=null, string method=”jumble”);

     

    That is bad not only for the caller of the api, but it suggests that the process method is pretty complex internally.

     

    On the other hand, I can’t count how many times I’ve written a well-constrained series of overloads that purely added in default values and had to write nearly duplicate xml docs for each of them, and I’d be really happy to save that time and not have those methods clutter up the code. 

     

    Or, to put it another way, default parameters are great if you use them to simplify scenarios that you would have written with overloads. If you start doing things that would be hard to express in overloads, I’d look harder at the overall design.

     

  • Eric Gunnerson's Compendium

    Writing Tests for HealthVault Applications

    • 0 Comments

    We have added some useful new functionality designed to make it easier to test a HealthVault application.

    The existing HealthVault SDKs didn’t make it easy if you wanted to write isolated (or method-level) unit tests for features that talked to the HealthVault Platform. You could usually do it by designing your own layer that could return simulated results (sometimes called “mocking” in agile methodologies), but that was sometimes difficult because of the way the SDK worked.

    In this release, we’ve made some changes that should make this a whole lot easier.

    For the sake of discussion, consider a bit of application code that fetches medications from HealthVault and filters them:

             HealthRecordItemCollection GetNewMedications(HealthRecordAccessor record)
             {
                 HealthRecordSearcher searcher = record.CreateSearcher(Medication.TypeId);
                 HealthRecordItemCollection items = searcher.GetMatchingItems()[0];
                 // filter items here... 
     
                 return  items;
             }
     

    We want to write a test that verifies that the “filter items here” part of the method works correctly, and we’d like the test to run without talking to the HealthVault platform. That’s not easy to do in the current SDK, because there’s no way to control what comes back from GetMatchingItems().

    In the new SDK, there is a way to do that. If we debug down we will find that the call to GetMatchingItems ends up in the following method in a new class named HealthVaultPlatform.

     

             public  static  ReadOnlyCollection <HealthRecordItemCollection > GetMatchingItems(
                 ApplicationConnection  connection,
                 HealthRecordAccessor  accessor,
                 HealthRecordSearcher  searcher)
             {
                 return  HealthVaultPlatformItem .Current.GetMatchingItems(connection, accessor, searcher);
             }
     

    The HealthVaultPlatform class centralizes all operations (except for one exception I’ll cover later) in a single class – if the SDK needs to talk to HealthVault it goes through that class. You can call into that class directly if you wish, or just troll through to see what operations can be performed.

    To create our test, we are going to be hooking in underneath that level. The method above just forwards into a method in the HealthVaultPlatformItem class, and that class provides a way for us to override the behavior.

    To get started, we need to create a class that derives from HealthVaultPlatformItem and overrides the GetMatchingItems() method. The first version looks like this:

         public  class  HealthVaultPlatformItemMock  : HealthVaultPlatformItem
         {
             HealthRecordItemCollection _itemsToReturn;
     
             public  HealthVaultPlatformItemMock(params  HealthRecordItem[] items)
             {
                 _itemsToReturn = new  HealthRecordItemCollection(items);
             }
     
             public  override  ReadOnlyCollection <HealthRecordItemCollection> GetMatchingItems(
                 ApplicationConnection connection, 
                 HealthRecordAccessor accessor, 
                 HealthRecordSearcher searcher)
             {
                 List <HealthRecordItemCollection> collections = 
                     new List <HealthRecordItemCollection>();
                 collections.Add(_itemsToReturn);
     
                 return  new  ReadOnlyCollection <HealthRecordItemCollection>(collections);
             }
         }
     

    We can use it like this (this is an NUnit test):

             [Test]
             public  void  GetMatchingItems()
             {
                 Medication medication = new  Medication(new  CodableValue("Ibuprofen" ));
                 Medication medication2 = new  Medication(new  CodableValue("Vitamin C" ));
     
                 HealthRecordItemCollection newItems = null ;
                 HealthVaultPlatformItemMock mock = new  HealthVaultPlatformItemMock(medication, medication2);
                 HealthVaultPlatformItem.EnableMock(mock);
                 ApplicationConnection connection = new  ApplicationConnection(Guid .NewGuid());
                 HealthRecordAccessor accessor = new  HealthRecordAccessor(connection, Guid .NewGuid());
                 newItems = GetNewMedications(accessor);
                 HealthVaultPlatformItem.DisableMock(mock);
     
                 Assert.AreEqual(2, newItems.Count);
                 Assert.AreEqual("Ibuprofen" , ((Medication)newItems[0]).Name.Text);
                 Assert.AreEqual("Vitamin C" , ((Medication)newItems[1]).Name.Text);
             }
     

    When the call to GetMatchingItems() gets down to HealthVaultPlatformItems, it will end up calling our mocked method rather than the built-in one.

    The code requires us to do a few things:

    1. Create an instance of the mock class.
    2. Enable the mock.
    3. Disable the mock.

    We can make it nicer by having the mock class itself handle enabling and disabling the mock, using the following:

         public  class  HealthVaultPlatformItemMock  : HealthVaultPlatformItem, IDisposable 
         {
             HealthRecordItemCollection _itemsToReturn;
     
             public  HealthVaultPlatformItemMock(params  HealthRecordItem[] items)
             {
                 _itemsToReturn = new  HealthRecordItemCollection(items);
                 HealthVaultPlatformItem.EnableMock(this );
             }
     
             public  override  ReadOnlyCollection <HealthRecordItemCollection> GetMatchingItems(
                 ApplicationConnection connection, 
                 HealthRecordAccessor accessor, 
                 HealthRecordSearcher searcher)
             {
                 List <HealthRecordItemCollection> collections = new  List <HealthRecordItemCollection>();
                 collections.Add(_itemsToReturn);
     
                 return  new  ReadOnlyCollection <HealthRecordItemCollection>(collections);
             }
     
             #region  IDisposable
             ~HealthVaultPlatformItemMock()
             {
                 Dispose(false );
             }
     
             /// <summary> 
             /// Disposes the request. 
             /// </summary> 
             ///  
             public  void  Dispose()
             {
                 Dispose(true );
                 GC.SuppressFinalize(this );
             }
     
             /// <summary> 
             /// Disables the mocking. 
             /// </summary> 
             ///  
             /// <param name="disposing"></param> 
             ///  
             protected  void  Dispose(bool  disposing)
             {
                 HealthVaultPlatformItem.DisableMock();
             }
     
             #endregion  IDisposable
      
         }
     
    That allows us to simplify our test code to this:
     
             [Test]
             public  void  GetMatchingItems()
             {
                 Medication medication = new  Medication(new  CodableValue("Ibuprofen" ));
                 Medication medication2 = new  Medication(new  CodableValue("Vitamin C" ));
     
                 HealthRecordItemCollection newItems = null ;
                 using  (HealthVaultPlatformItemMock mock = new HealthVaultPlatformItemMock(medication, medication2))
                 {
                     ApplicationConnection connection = new  ApplicationConnection(Guid .NewGuid());
                     HealthRecordAccessor accessor = new  HealthRecordAccessor(connection, Guid .NewGuid());
                     newItems = GetNewMedications(accessor);
                 }
     
                 Assert.AreEqual(2, newItems.Count);
                 Assert.AreEqual("Ibuprofen" , ((Medication) newItems[0]).Name.Text);
                 Assert.AreEqual("Vitamin C" , ((Medication) newItems[1]).Name.Text);
             }
     

    Special Classes

    There are a few classes where it’s not straightforward to create the class. For classes such as ServiceInfo, we don’t provide a way to create and modify them directly. To create an instance of those classes, you need to derive a new class and use that:

         public  class  ServiceInfoTest  : ServiceInfo
         {
             public  ServiceInfoTest(string  version)
             {
                 Version  = version;
             }
         }
     

    and the associated test uses this class instead of ServiceInfo:

             [Test]
             public  void  GetServiceInfoTest()
             {
                 ApplicationConnection connection = new  ApplicationConnection(Guid .NewGuid());
     
                 ServiceInfoTest serviceInfo = new  ServiceInfoTest("V2.x" );
     
                 ServiceInfo serviceInfoBack = null ;
                 using  (HealthVaultPlatformInformationMock mock = new  HealthVaultPlatformInformationMock(serviceInfo))
                 {
                     serviceInfoBack = connection.GetServiceDefinition();
                 }
     
                 Assert.AreEqual("V2.x" , serviceInfoBack.Version);
             }
     

     

    Limitations

    We currently don’t have mockable interfaces for blob operations. We hope to do that in a future release.

  • Eric Gunnerson's Compendium

    HealthVault Event Notifications

    • 0 Comments

    The HealthVault platform now provides the ability to notify applications when specific conditions are met.

    A scenario

    A blood-pressure-tracking application wants to be notified whenever a new blood pressure measurement is added to any of the user records that the application has access to, so it can perform some operation with the data.

    With previous releases, the only way to do this was for the application to periodically call GetUpdatedRecordsForApplication(), and then look at each record that was updated to see if the update was a new blood pressure instance.

    The solution

    Each application can now create a series of subscriptions, where each subscription specifies the event to detect and how to notify the application when the event occurs.

    The BloodPressureTracker application creates a new subscription, specifies that it wants to be notified when a blood pressure measurement is added, updated or removed, and that the notification should be sent to www.example.com/notificationBloodPressurePage.ashx. The subscription is persistent until the application deletes it.

    The notification page must be in a location that is accessible to HealthVault, which means it is accessible to other internet programs. To allow the application to verify that a notification came from the HealthVault platform, the application registers a key with the subscription, and when the notification arrives the application can verify that the HMAC in the message is identical to one computed by the application.

    The life and times of a notification

    Notification Dispatch

    The dispatching of a notification happens on the HealthVault Platform.

    • An operation such as PutThings is performed on the HealthVault Platform.
    • The HealthVault platform finds subscriptions that match the event.
    • The HealthVault platform notifies the application using the following steps:
    • The key registered with the matching subscription is used to create an HMAC of the notification payload.
    • That hash, a version id that was specified with the key, and the subscription id are included in the Authentication header of a request.
    • The request is sent to the URL defined in the subscription as a POST with the XML notification text as the POST payload.
    • The server waits for a response.
    • If it gets a “200 OK” response, it considers the notification to be delivered.
    • If it gets any other response or does not receive a response, it will hold onto the notification and try again later.
    • If the notification cannot be delivered after a period of days (currently set to 10 days but subject to change), the notification is abandoned.

      Notification Processing

      The notification processing happens in the HealthVault application.

    • The notification handler reads the XML notification text into a string.
    • The key version id, subscription id, and HMAC of the notification payload are extracted from the authentication header.
    • The notification handler determines which subscription was notified based on the key version id that was passed.
    • The expected key is determined based on the key version id that was passed. This allows keys to be updated to new versions while not breaking the handling using the old keys.
    • An HMAC of the xml notification text is calculated, and compared to the one passed in the header. If the hmac does not match the notification should be ignored and discarded as it did not originate from the HealthVault service.
    • The notification handler returns a status of “200 OK” so that the HealthVault platform knows that the delivery was successful.
    • The XML notification text is processed.

      The processing of the notification should be performed on a separate thread to prevent the possibility of taking so much time that the timeout is reached.

      Event types and notification methods

      For this release, the platform supports one event type – a change (add/update/delete) of an instance of a specific set of data types in a user’s record – and one delivery method – over an https: connection. We are planning to extend support in future releases – if you would like to influence which events and delivery methods we consider, please send us feedback.

      Health Record Item Changed Event details

      The health record item changed event passes the following information in the notification:

      • The person id and record id that specify the record in which the change was made.
      • A list of the health record item ids (aka “thing ids”) that were changed.

      After the notification is received the application will need to fetch the item to determine what change was made. If the object was deleted, it will not be returned from the fetch operation.

      Limitations

      Notification URL

      The notification URL must be on the same domain as the action url that is registered with the application.

      Authorization

      The user must have granted the application offline read access to the data type that the subscription refers to.

      Number of subscriptions

      An application can only register 25 subscriptions at a time. This number is subject to change.

      Delivery timeliness and guarantee

      The HealthVault platform makes a “best effort” to deliver each notification in a timely manner, but does not guarantee delivery. It is not designed for real-time monitoring scenarios.

      Notification of changes only

      Notifications are delivered only for changes that are detected in records – the platform does not notify for items that are already existing in a record when the user first authorizes the application, nor does it notify for deletion if a user de-authorizes the application.

      Eventing sample and test application

      We have created a sample application which serves three purposes:

      1. It demonstrates how to use the subscription manager api calls to create, modify, and delete subscriptions.
      2. It provides a sample implementation of a subscription notification handler that processes incoming notifications.

      Getting started

      The first time that you run the application, it will generate an application id and a key for you to use. The sample will tell you how to properly define these in the web.config file.

      Managing subscriptions

      The management part of the application is pretty simple – you merely add a new subscription and then list the data type ids that you want to be monitored.

      Testing notification handlers

      In many cases, developer machines are not directly reachable from the internet and therefore there is no address that can be used in a subscription. To make it easier to develop notification handlers, the sample application can send simulated notifications to a notification handler for debugging purposes. It provides the following options:

      Notification Destination

      Choose between the URL defined in the subscription, the test notification handler defined in the project, or a URL that you enter.

      Authentication

      Choose Normal to have correct authentication headers, send bad HMAC to send an HMAC that is incorrect, or send bad key version to send a key version that is different than the one in the subscription.

      Instances

      The sample application can generate fake instance ids (if you just want to check that the notification handler is set up correctly), send an empty instance list, or select actual instance IDs from the current record.

  • Eric Gunnerson's Compendium

    Naked came the null delegate

    • 0 Comments

    I few weeks ago James Curran came up with the idea of a number of .NET bloggers (or, in my case, bloggers who remember vaguely what .NET is about) write a serial story. I, who am easily flattered by the smallest of attentions to my previous brush with semi-fame, signed on.

    And then when it came around to me, I procrastinated for a few days, wrote something I didn’t like and threw it away, wrote something I liked that I couldn’t figure out how to fit into the existing story, then finally wrote something that I’m somewhat fond of that fits into the story, more or less. Which is kindof the point.

    My contribution is here. I recommend reading the first two chapters so that you won’t be lost. You can also read James' explanation for the title.

    If you have questions about the obscure parts (ie – what is he writing about) feel free to ask in comments, and I’ll try to answer when I get a few seconds away from my adoring fans.

  • Eric Gunnerson's Compendium

    HealthVault SDK and Visual Studio 2005

    • 4 Comments

    The HealthVault SDK is currently built on top of the .NET 2.0/Visual Studio 2005 toolset. We are thinking about moving forward a few years and switching to the .Net 3.5/Visual Studio 2008 toolset, but would like some feedback from customers on what they are using first.

    If you are building HealthVault applications, can you reply with the version of VS that you are using? Thanks.

  • Eric Gunnerson's Compendium

    Photographers at Microsoft Fundraiser

    • 0 Comments

    There’s a fairly active photography alias at Microsoft, and last year during October – the annual Microsoft Giving Campaign – about 200 photographers got together and produced a Blurb book titled “Photographers @ Microsoft”. They put it on sale for a price that would raise $25 per copy.

    The ended up raising $50,000.

    This year there has been more participation, and the book is printed on an offset press. I’ve seen advance copies and they’re as nice as any coffee table book you’ve seen (well, perhaps not as nice as Kramer’s…). They are slightly cheaper than the previous year’s books (offset printing is cheaper if you print enough), and still raise $25 per copy.

    You can preview and order the book here (if you’re at MS, enter your alias and employee number and matching will happen to make it $50 a copy).

    If you click on the image above, you can see small versions of the photos – they’re stunning. If you look carefully, you might find my contribution.

    Bubo bubo

  • Eric Gunnerson's Compendium

    Bohemiam Rhapsody… on the slide whistle…

    • 0 Comments

    Just wonderful.

    I love how he went to the trouble to overdub the way the original video was and matched the cheesy video.

  • Eric Gunnerson's Compendium

    Powerpoint, audio, and packaging…

    • 2 Comments

    This is a post about how to take conference audio and add it to a powerpoint presentation to give you a self-contained package you can give out. It took me a while to figure out how to do this effectively, so I’m hoping this will help others.

    But, since I’m all about the story, there’s a bit of background first. If you just want to understand the mechanics of how to do it, scroll down until you see “Mechanics”.

    Earlier this year I did a presentation (with Lowell Meyer) for the Microsoft Connected Health Conference, entitled “Practical HealthVault: Challenges and Opportunities”.

    The goal of the deck was to cover the things that are different and/or confusing about HealthVault – the things that we’ve answered over and over in the past couple of years. We wanted to come out with something that we could give to people who were new to HealthVault (anybody from the technical side, be they developer or manager) and therefore make our lives easier. And – which will come as no surprise to those who have seen me speak – I was interested in the adulation of my fans.

    In both writing and presenting, I’m a big fan of progressive revelation, where you start simple and build things up. For this talk, that meant a lot of custom animations in powerpoint. A *lot* of custom animations in powerpoint. Five of my slides have more custom animation steps than will fit in the custom animation box on the side.

    We had a company doing video production for the conference, and a few weeks after our presentation (which was a lot of fun – I always forget how much I enjoy presenting), the video showed up.

    It consists of a lot of mostly-dark shots of us presenting, cutting back and forth with the slides. If they happen to be showing the slides when the animation happens, things generally work okay, but at times they showed the design view. And the resolution is what you get with video, which isn’t great.

    What I want is a narrated presentation, so I set out to do that.

    Mechanics

    1. Pulling the audio off of the video

    For this I used AoA Audio Extractor, which gave me 155 MB MP3 file.

    2. Editing the audio

    For editing the audio, I used Audacity, which is pretty darn nice. My one caveat is that with the version I have, you can’t perform any operations on the audio if the audio is paused. So, you have to hit “stop” on the transport controls and then do a trim/cut/export/whatever.

    If you just took that whole audio track and put it on a powerpoint presentation, it would work fine at the beginning but would get out of sync on some machines. To prevent this from happening, we need to use per-slide audio.

    It’s also true that working with a 90-minute track is not a lot of fun, so breaking it up will help in that realm as well.

    I started by going though the audio track and putting labels on all of the slide breaks. This support is there to break albums up into songs, but it works very well for this as well. I had the presentation open on another monitor so I could reference it during the audio. Name the labels “Slide<x>”, where <x> is the number of the slide. Make sure to put one at the beginning for the start of the presentation.

    Once you have the labels, you choose “File->Export multiple”, and it creates individual .wav files for each slide.

    At that point I walked through all the slides and did some judicious editing. This is a bottomless pit of time consumption if you let it, so I tried to just pull out the things that were distracting where it was simple to do so. I also highly recommend adding 5 seconds of silence at the end of each slide’s audio – I didn’t do this initially and had to go back and re-edit all of them.

    3. Conversion to lossy format

    Audicity gave me files in .wav format, which are a bit wasteful in size. I download Lame and converted them into .mp3s. You can use the encoder of your choice (the list of formats that PP supports is here); I used lame because it’s what my home system runs on and is very simple to use from the command-line.

    4. Adding the audio to your presentation

    The following steps are all manual – if you have powerpoint 2010 (which has a macro recorder) and/or want to write some macros, you may be able to automate it. I just suffered doing it by hand in PP 2007.

    1. Select the slide
    2. Pick the insert tab
    3. Click on the sound icon
    4. Pick the appropriate audio file for the slide.
    5. Choose “automatic” when it asks you if it should start playing automatically.
    6. Drag the sound icon someplace that isn’t too annoying.
    7. In the custom-animation tab, drag the audio to the top of the list
    8. Right click on the audio, choose “Timing”, and then set “repeat” to “until end of slide”. If you don’t do this, the audio will only play until you hit the spacebar to start your animation.
    9. Repeat this 4000 times for the rest of your slides.

    At this point, you should be able to start the slideshow and have your audio start playing.

    5. Recording the animation timings

    For some reason that is not apparent to me, powerpoint calls this “rehearsing” the timings. And by default, it only supports it for the whole presentation, and there’s no (obvious) way to do it for a single slide.

    However, my mad search-fu led to a workaround. When you rehearse the timings, it only does it for the slides that are visible, so you can hide all the slides but one and rehearse the timings only for that slide.

    Here’s the progression:

    1. Hide all the slides
    2. Unhide the one you want to record the timings for.
    3. On the slideshow page, make sure “use rehearsed timings” is not checked.
    4. Choose “rehearse timings”
    5. Listen to your audio and hit the spacebar at the appropriate point to run the animation.
    6. When your audio is done, wait a couple of seconds and hit pause on the rehearse controls, then close the window.
    7. PP should ask you if you want to save the timings. You do.
    8. Repeat for each slide.

    You may find that you need to slightly modify your animations – some of mine seemed to change the “1” elements to “0” elements so they showed up too early. I think this happened when I added the audio to the slides.

    The 5 seconds of silence is critical here. If not, you get to step 6, and your audio will start to repeat.

    We had a few slides with 2 minutes of presentation and 10 minutes of questions. If you have this, stop the recording when the animation is done, and accept the timings. Then go to the Animations tab – on the right side you will see “advance slide automatically after” and a time. Look at the length of the audio for that slide (any player will tell you that), and set the value to 2 seconds shorter than that. That will put the advance right in the middle of the silent section you added.

    6. Polish

    At this point, you may need to polish the animation on certain slides. You may have to go back and redo some audio.

    7. Publish

    At this point you will want to publish the presentation. My plan is to just use PP’s publish to folder functionality, though there are also ways to publish to video if you are willing to deal with the loss in resolution. Publish to HD video might be worthwhile, but it would be pretty big, and right now the publish to folder is about 80MB in size.

  • Eric Gunnerson's Compendium

    Metricism

    • 4 Comments

    Your neologism for the day...

    Metricism

    A devotion to creating and implementing metrics for a system or process while loosing sight of the real goal.

    Examples:

    • Evaluating software developers on how many bugs they fix.
    • Evaluating newsgroup interaction quality based on the percentage of posts answered in the first day.
  • Eric Gunnerson's Compendium

    Floating point numbers and string representations

    • 0 Comments

    I’ve recently been doing some work on the HealthVault SDK to improve the consistency of how it deals with floating-point numbers, and thought that the information would be of general interest.

    First off, I’ll note that if you’re new to the often-surprising world of floating-point arithmetic, a few minutes reading What Every Computer Scientist Should Know About Floating-Point Arithmetic would be a good introduction.

    While my code examples are C#, this is a general issue, not just a .NET one.

    Anyway, off to the issue. Consider the following:

    static void Main(string[] args)
    {
        double original = 3.1;
        string stringRepresentation = original.ToString();
        double parsed = Double.Parse(stringRepresentation);

        Console.WriteLine("Equal: " + (original == parsed).ToString());
    }

    What is the output of this program?

    If you said “Equal: true”, you are correct. You might try a few more numbers before deciding that this is a general solution. Or maybe you don’t even think about it..

    But what if you choose a different number:

    double original = 667345.67232599994;

    What is the output this time?

    It is “false”.

    I hope that there was sufficient foreshadowing earlier in the post so that you are not uncomfortably disturbed by this turn of events.

    We could modify our code to check whether the two numbers are equal within an epsilon value. Or, we could dig a bit deeper…

    If we look at the wikipedia, we’ll find that under the IEEE 745 standard for floating-point arithmetic, there are 53 significant bits in the fractional part of the number. That is not quite 16 digits, so to be correct, we treat the numbers as if they only have 15 digits of precision when we convert them to strings. The last few bits are ignored.

    Another way of saying that is to say that it is possible to find two numbers that have the same ToString() representation but are different in those last few bits. Which is what is going on her.

    If we are printing out numbers for somebody to look at, the ToString() behavior is what we want, because the numbers really only have 15 digits of precision.

    In this scenario, however, what we want is a string format that will ensure that we get the exact same number back.  We can do that by using the “R” numeric format:

    string stringRepresentation = original.ToString("R");

    That gets us the behavior that we want. We could also have called XmlConvert.ToString(), which has the same behavior.

Page 1 of 46 (1,130 items) 12345»