Patterns and Practices in Computational Science

Introducing the Microsoft Visualization Language

martinca — Thu, 03 Dec 2009 13:42:22 GMT

Microsoft Visualization Language
The Vedea Project
Create. See. Understand.

When I introduced the Microsoft Computational Science Studio a couple weeks ago, I also hinted at the ‘Vedea’ project – a new visualization language that we have been working on in the Computational Science Laboratory here in Microsoft Research in Cambridge, UK. I’ll now use this post to introduce you to the Microsoft Visualization Language, Vedea in more detail.

Vedea is a prototype of a new experimental language for creating interactive infographics, data visualizations and computational art. It is designed to be accessible to people who are either new to programming or whose primary domain of expertise is something other than programming. We wanted to give those users a tool that they can use to realize their own vision and visualizations without having to engage skilled programmers, but have it be an environment that skilled programmers would not find limiting.

The motivation for building Vedea comes from the Microsoft Computational Science Studio. It’s no use facilitating modelling and computation if you don’t give the user a way to visualize the results. Simple charts are ok in general for simple data sets, but not for facilitating deep interactive exploration of data with many dimensions or for facilitating the type of exploration that leads to speculative visual exploration or visually-inspired ‘aha’ moments.

The inspiration for Vedea is the same set of goals underlying the language “Processing” (http://processing.org/). Both Processing and Vedea have similar goals and similar audiences. We’re all (the scientists and technologists in MSRC’s Computational Science Laboratory) frequent users and admirers of Processing, but we felt there was an opportunity here for us to take those concepts further by capitalizing on some new Microsoft technologies.

The Language

The Microsoft Visualization Language is built on .net 4.0’s new Dynamic Language Runtime. This gives us some important advantages over more traditional language implementations. Syntactically, the Vedea language looks a lot like C#. In its simplest form though, there are no class decorations – just a collection of functions. You can introduce classes if you want to do object-oriented programming, but they are not required and your topmost functions aren’t wrapped in any of the syntactic trappings of a class.

The second thing to notice about the language is that it is dynamically typed. Variables take on the type of whatever is assigned to them and, if you like, you don’t have to declare variables at all – they are defined the first time they appear on the left of an assignment.

The DLR also allows us to play some neat late-binding tricks with data sources, COM objects and the like by defining properties and methods based on what we find out about those objects at runtime. More about that below.

Another unique aspect of the language is its implementation of bindings. You might be familiar with bindings from WPF and Silverlight, and the concept here is similar, though expanded. The simplest example of a binding is as follows:

textbox.Text := slider.Value;

This is a ‘binding’ that you call once in your program, but which ‘forever’ binds the value of the slider to the text in the textbox. If the slider is moved, the text will change to match the current slider value. Since bindings can be cancelled, ‘forever’ really means ‘until you do something to cancel the binding.

You can also do the following:

textbox.Text :=: slider.Value;

In this case, if you move the slider, the textbox’s text will change, but if you type a value in the textbox, the slider will change to match that value. Text and slider values are of different types, so some value conversion is happening under the covers here.

Things get much more interesting when you combine bindings with collections defined with LINQ syntax:

myData = DataSet(“mydata.csv”);
currentYear := slider.Value + 1900;
bubbles := from row in myData
where row.Year :== currentYear
select new Circle()
    {
      X = row.Latitude,
      Y = row.Longitude,
      Radius = row.Population * scalingFactor,
      Fill = BlackBodyPalette(1., 1., row.DeltaCarbon)
    };
Scene[“USMap”].Add(bubbles);

The first line connects the variable ‘myData’ to the dataset in mydata.csv. The CSV file contains columns for Year, Latitude, Longitude, Population and DeltaCarbon (possibly in addition to other columns) which is why we can access those columns as if they were properties of ‘row’.

The second line binds the variable currentYear to the slider’s value plus 1900. As the slider is moved by the user, currentYear will get updated.

The third line creates a collection of circles (‘bubbles’), one for each row in the csv file whose Year value matches currentYear. The Boolean comparison :== is the same as == except that the Boolean value is bound to both variables in the conditional expression. A change to either variable in the comparison will invalidate the comparison, which will cause the collection to be re-evaluated. As a result, as the user moves the slider, the visual collection of bubbles will get updated to reflect the current year’s data. (nb: an identifier column can be specified in order to obtain least-flow visualizations or transitional animations). Using ‘==’ instead of ‘:==’ would have resulted in a one-time comparison and the collection would not be updated as the slider is moved (which might be desirable in some cases). In the ‘select’ portion of the LINQ expression, we create the circles and set their properties based on values found in that row of the data. So as you move the slider, not only will bubbles move and change size and color, but the number of bubbles that are visible will change based on the outcome of the ‘where’ clause.

Finally, we add the entire collection of bubbles to some member of the scenegraph called ‘USMap’. Presumably that is one of our map objects and adding the collection as a child of that map will use the map’s lat/lon coordinate system and position the bubbles according to the lat/lon values found in X and Y.

The end result is that we have produced an interactive timeline visualization using less than a dozen lines of code (or four statements). It is true that the code is ‘denser’ – that is, more functionality in less typed space – and consequently a user might spend more time working on fewer lines of code, but we are betting that the number of failure modalities is smaller (fewer ways to mess up); and there’s less code ‘real estate’ to observe and comprehend. Of course, we still support immediate-mode graphics for anyone who wants to ignore this declarative syntax and create old-school visuals and animations in the render loop.

Data

With a few exceptions, when it comes to visualizations, data is king and time is precious so you want to spend a minimum amount of time thinking about the ‘plumbing’ required to get your data into your visualization (things like file formats, storage location, memory allocation, virtualization, pagination, etc.) and spend the majority of your time working with the data itself.

In the Microsoft Visualization Language, we leverage another project of the Computational Science Laboratory called ‘Scientific Data Set’ to make working with data as simple and efficient as possible. Scientific Data Set is a hugely important piece of technology for both Vedea and Computational Science Studio, so it will get a lot of ‘airtime’ here.

Scientific Data Set uses a collection of ‘providers’ to support access to a wide variety of back-end storage, including comma-separated variable (csv) and variations on csv; netCDF and HDF; SQL; Excel; and a host of others. The data from all of those back-end storage formats is presented through a consistent API that supports named variables, meta-data on variables (units of measure, dimensionality and other annotations) and the use of variables to form coordinate systems onto other variables. For instance, you could have separate vectors of latitude, longitude and altitude. You could use latitude, longitude and altitude to store a three-dimensional structure of variables containing temperature, pressure and particulate count and then re-use the latitude and longitude vectors to index into a two-dimensional structure containing variables with calibration data for the recording station at that lat/lon.

In Scientific Data Set (ScDS), you can then access your variables not only by their integer indices, but also by the lat/lon variables. In the future, ScDS will also support interpolation so that if you turn on interpolation and ask for data at a lat/lon/alt that does not exist in the data, you will get an interpolated value which you could use for re-sampling the data.

In Vedea, ScDS data is a top-level syntactic construct. When you open a .CSV file, the variable you use for that CSV file will have member variables for each column in the CSV. For instance, given this CSV file (mydata.csv):

If you use the following Vedea code:

myData = DataSet(“mydata.csv”);

Then myData will have the following indexed member variables available:

          myData.Sample
     myData.Population
     myData.Reading

and by ‘indexed’ member variables, we mean that you can get at the values like this:

print(myData.Reading[1]);

which would output 0.0201. The purpose behind this is to make less typing yield more results. That means that beginning programmers have less syntactic overhead to deal with and experienced programmers can get where they want to go more quickly.

Graphics

The graphical capabilities, especially when combined with simple data handling, are the most exciting aspect of Vedea. The current generation of infographics don’t look anything like the traditional pie and bar charts. Take a look at information aesthetics or Bestiario and you’ll get an immediate sense of the types of graphics that we are targeting with Vedea. They combine color, hierarchy, shape and line into clear, concise and informative (or entertaining) visuals. Vedea seeks to bring these visual capabilities to people who are more in tune with their data; their vision for representing that data; and their audience than they are with programming.

The graphics features of Vedea are designed to build upon the native capabilities of XNA and GDI to make common tasks much simpler and to require less code to achieve those tasks. Those features include the following (which will all be discussed in more depth in upcoming posts):

Hierarchical scenegraph
A full set of 2d primitives (polygons, lines, ellipses, circles, curves, arcs, images, text, etc)
A full set of 3d primitives (cubes, spheres, capsules, cylinders, pipes, images, text, meshes, textures, materials, height-maps, cameras and lighting)
Solid and alpha-blended (semi-transparent) colors
Map object (using data from Bing Maps/Virtual Earth) which can be combined with 2d and 3d primitives
A rich library of rendering utilities (e.g., Perlin noise, warping functions, color management)
Animation features (linear and exponential interpolation for smooth variation of any Vedea variable or visual property)
Direct binding of data to visuals (e.g., create one circle for every row in a database; bind properties of the circles to columns in the database)
Planned support for Network viewers, volumetric renderers, physics effects and other high-level graphics constructs

Retained-mode Graphics

In Processing, a programmer works without a scene-graph. We call this ‘immediate mode’. In order to change a visual object you have to erase some or all of the scene and then redraw it. In general, this is a very easy system for beginning programmers to grasp. But for complex scenes with large numbers of objects, some portion of which change from frame to frame, the overhead quickly adds up either in terms of the cost of redrawing or in terms of the amount of code you need to dedicate to optimizing your drawing. Very quickly, either performance suffers or the programmer is required to do more work than they might be prepared for. Experienced programmers can also find the situation frustrating due to the amount of code overhead they face. Similarly, the Z-order (back-to-front) order of objects is determined by the order in which you draw objects. Implementing move-back or bring-forward functionality means that you have to implement an internal list of graphical objects which amounts to implementing a scene-graph in your program’s code.

Vedea supports the immediate drawing mode, but also allows the user to add objects to a ‘scene graph’. Use of a scene graph is called ‘retained mode’ graphics. In Vedea, you can type:

Ellipse(100,100,20,20);

to create a 20x20 circle at 100, 100. But you can also do this :

circle = new Ellipse(100,100,20,20);
Scene.Add(circle);

This creates a circle and displays it just like in the immediate-mode example. However in the immediate mode example, to change the size of the circle, you must erase your drawing and re-draw the circle. In the retained-mode example, you can do the following:

circle.Width = 25;
circle.Height = 25;

Now your circle has changed sizes. Likewise, you can change the z-order like this:

circle.ZOrder = 500;

This means that your circle will appear behind any objects with a ZOrder less than 500 and in front of any with a Z-order greater than 500. In the immediate mode example, if you change your circle, you would have to redraw everything that appeared in front of that circle (which means you had to keep track of that or redraw the whole scene). In Vedea, the Vedea runtime will keep track of that bookkeeping for you.

Hierarchical Scene-Graph

Above, we saw that we could add a circle to the scene graph. In Vedea, it is also possible to add objects to each other. For instance:

r1 = new Rectangle(100,100,10,10);
r1.Fill = “Red”;
Scene.Add(r1);
r2 = new Rectangle(30,30,5,5);
r2.Fill = “Green”;
r1.Add(r2);

We created a rectangle r1, set its fill color to red and then added it to the scene. Then we created a green rectangle called ‘r2’ and added it to r1. r2 will actually be drawn at 130,130 because it is a child of r1 and we said that r2 should be at 30,30 relative to its parent, which in this case is r1. (Note that r1’s parent is the scene itself). It is also useful here to note that although I used “Red” and “Green” as colors, there are six different valid ways to specify colors.

Now if I rotate r1 with r1.Rotate(45) then not only will rectangle r1 rotate, but rectangle r2 will orbit 45 degrees around the center of r1. When you rotate a parent, the frame of reference for all the children rotates too.

This becomes especially interesting with maps (or any other container object that implements its own coordinate system). If you add a map to a scene, then you can add shapes, images, etc as children of the map image and the coordinates, width and height of these child objects will be interpreted in the coordinate system of the map instead of as pixel locations. If the map is zoomed, rotated or panned, the child objects will behave correctly without requiring any further code to adjust their position.

Runtime and Authoring Environment

Vedea programs can be edited in Notepad or an html text input and are compiled by the Vedea runtime. We are working on a standalone compiler that will allow you to create intermediate code that can be used at runtime instead of source code. The Vedea runtime (the program that compiles and runs Vedea code and hosts the graphical display window) can run either on the desktop or as a client-side web control. One of the key areas we are concerned with in the runtime is to increase the performance and cross-platform compatibility of the web-based runtime, though Vedea is dependent on .net 4.0 and the DLR so even with OpenGL compatibility, we may have to wait for ongoing Mono development before we can reach beyond the Windows family.

Getting Vedea

The Microsoft Visualization Language and its runtime will first be available on via http://research.microsoft.com/Vedea. You won’t find it there now (you just looked, didn’t you?) but we will be posting it very early in the coming year. We’re eager to get it out for people to play with, but we have a bit of work to finish and a fair bit of packaging to do before we can post it on the web site. Forums will be available for questions and discussion and I’ll post samples and discuss features here. Vedea is an ongoing experiment here at Microsoft Research Computational Science Laboratory and like all experiments (and all pre-release software) it’s final shape and outcome is uncertain, but there has been a lot of interest in it and we’re eager to get some community experience and feedback. I hope you found this interesting and hope you will give the Microsoft Visualization Language a spin when we post the first downloadable version.

Microsoft Computational Science Studio

martinca — Mon, 02 Nov 2009 18:11:14 GMT

It’s been awhile (to say the least) since I last posted – a year or so. It’s been an intense year of work on two very exciting projects. I plan to spend the coming weeks blogging extensively about these two projects – what you can do with them and how they work and ultimately, how you can get them too.

The two projects are Vedea and Microsoft Computational Science Studio. Vedea is a new language for creating interactive data-driven visualizations, and I’ll blog separately about that in a few days. Vedea will be demonstrated publicly for the first time at PDC 09 November 16-19 in Los Angeles and should be broadly available from research.microsoft.com shortly thereafter.

Today, I want to write about Microsoft Computational Science Studio which is getting its first public demonstrations today as part of Craig Mundie’s 2009 Campus Tour. The Seattle Times has a nice article on Craig’s tour and our part of his demo.

The Microsoft Research Computational Science Laboratory within Microsoft Research in Cambridge, UK consists of three groups : Computational Biology, Computational Ecology and Environmental Science and Tools and Technology. Craig’s demo is the culmination of a unique collaboration between the scientists in the laboratory and the software designers and developers in the Tools and Technology group to arrive at what we intend to be a tool for enabling non-programmer scientists and researchers to harness vast amounts of storage and compute power for running the multi-scale models that are needed to truly understand and predict complex natural systems.

Craig is quoted in the Seattle Times article as saying that he tells people “this is sort of doing for scientists and policymakers what Excel did for the average business guy 20 years ago”. Its really gratifying to hear him make that observation since simplifying the plumbing in multi-scale modelling is a core goal. He’s been a huge supporter of our efforts to do something really new and unique here and we’re grateful for that support.

Microsoft Computational Science Studio (MSCSS) is unique in a few ways and I will touch on each of those in more depth in upcoming posts. MSCSS attempts to address issues in collaboration, publication, presentation, data management, and most importantly to facilitate the construction of large data- and compute-intensive experiments and visualizations by non-programmers.

One of the first ways in which MSCSS is unique is in its collaboration story. You don’t have to copy files around in order to work on multiple computers or to collaborate with or share results with other people. You can share experiments among your own computers or with collaborators using some nifty cloud-based replication that’s built into MSCSS. You can still use MSCSS standalone and disconnected from the internet, but when coupled with some of Microsoft’s Windows Live cloud services you get local-file performance with the convenience of being able to access your projects from any computer. The collaboration possibilities will get a few posts all their own here.

MSCSS allows the user to create ‘data flows’ which connect the various data sources, computational elements, and presentation elements of an experiment together. The data flow is the ‘code’ that drives an MSCSS experiment.

I’ll spend quite a bit of space here talking about the data-flow layer and how you can construct flows for various trade-offs of realtime visualization performance, data size, etc and how to construct interactive and multi-scale simulations.

MSCSS also allows the user to create a separate presentation layer of views and viewports that contain the UI that lets someone drive your experiment or simulation and see the results without having to get tangled up in the internal details of the data flow.

You can think of these views and viewports as the ‘dashboard’ view into your experiment’s inner workings. This one, at left, exposes many of the UI elements seen in the data flow above. (You can expand these images by clicking on them).

Ultimately, you might use these views to tell a story, create publication-ready graphics or create a dashboard or application for others to explore your results. We’ve discussed various ways of exporting just these views to create standalone desktop or web applications, but that’s still a ways down the road before we’ll see it implemented in any usable form.

MSCSS itself is a shell into which you plug in extensions – for visualization, data management, computation, modelling, etc. One extension might give you access to remote data on Azure; another might allow you to draw heat-maps over Virtual Earth; and another might support Perfect Plasticity Approximation models or computations on the Hadley climate model data (both of which are components in Craig’s demo and some of the screenshots above). You then compose the elements from those extensions into a data flow and create UI that interacts with your data flow. That’s the exact process we used to create the demo Craig is giving. There’s no smoke and mirrors there – these are real data flows and UI being driven by MSCSS.

Of course, that extensibility is there to allow users to create their own extensions that add new data sources; new calculations, analyses and data transformations; and new visualizations. Down the road, we’re planning on making an SDK available to let you build your own extensions. Extensions also have a fairly unique model for publishing, sharing, discovering, installing and updating – a model designed to support direct user-to-user sharing of extensions and the SDK and entire extension pipeline will get a lot of space in upcoming posts as well.

MSCSS has been a substantial two-year effort by the entire team of scientists and engineers in the Computational Science Laboratory. It is still a research project in every sense but we’re eager to share our work with the community at large (with the usual non-commercial-use-only stipulation and with the understanding that this is a prototype and we’re not promising to ever actually ship or support it or any specific features in it). We do plan to make early versions of it available for download, perhaps in January of next year, with updates as we continue to evolve the project. I’m sure we’ll get an MSCSS team blog and such set up at some point, but in the meantime, I will use this blog to disseminate public announcements and discuss features related to MSCSS and Vedea

Instrumenting Manx Shearwater Burrows

martinca — Wed, 25 Jun 2008 17:05:35 GMT

I certainly can't complain that I have a boring job. In addition to being responsible for architecting and building some very interesting horizontal software components for computational science investigations, I get to apply my electronics and mechanics skills to creating some interesting special-purpose components. These can range from collaborating on creation of a sub-15-gram solar GPS logger backpack to creating special purpose sensor interfaces to engineering a mechanism to get birds to weigh themselves every time they enter or leave their burrow. Its that last one that I will talk about today.

Among the goals is to try to determine how much food a Manx Shearwater brings home and how much their weight varies over a nesting season by weighing them with gram accuracy each time they enter or leave their burrows. In addition to weight, we need to read the bird's identification (and RFID tag) and sense motion (so that we only power up the very hungry scale and RFID circuits when they are needed). Many of the birds will also be carrying a preliminary version of the GPS backpacks, so we will also know where they have been foraging. In this article, I'll show you how we went about getting the birds to weigh themselves.

This Manx Shearwater research project on Skomer Island (off the western coast of Wales) is led by Prof. Tim Guilford of Oxford and Robin Freeman, a Post-Doc Researcher here at Microsoft Research in Cambridge, UK. Tomek Naumowicz is an endlessly energetic doctoral candidate from Freie Universitat Berlin and Tomek has built the firmware that is being used on the ScatterWeb sensors that monitor each burrow. My contribution has been to design and build some additional circuit cards along with the outdoor housings, the weighing tubes, and a solar-powered WindowsCE + GPRS base station for relaying the data back to the mainland.

As you will see below, the goals of cheap and fast build-out of less than a dozen low-environmental-impact weighing tubes means that the engineering is closer to the Scrapheap Challenge end of the spectrum than the NASA/JPL end.

The central component of the weighing tubes is an aluminum strain gauge harvested from a Tesco kitchen scale. The Tesco scale has 2g resolution from 0-5kg, but through the use of a different A/D converter, different choices for the amplification of the strain gauge signal and a narrower total weight range, we manage to get several times that resolution, down to the sub-gram range.

Out of each scale, we get remove the strain gauge assembly (seen lying on the right half of the dissassembled scale above), and bend the strain gauge base slightly to match the curve of the tube where we will mount it. The left photo below shows gauges before and after bending. The working part of the strain gauges are the aluminum bars, each of which have a patches of material applied which varies in resistance in proportion to the stress on the bar. In the original scales, an amplifier and A/D converter takes the signals from these gauges and amplifies and converts the signals into the weight readouts displayed on the readout. We replaced this with our own amplifier board (green board in the right-hand picture below). Our A/D converter is part of the ScatterWeb board. The amplifier takes the micro-volt changes given off by the strain gauges and boosts it to a 0-2V range which will be interpreted by the ScatterWeb A/D converter a meter or two away.

The strain gauges are mounted in a 5" piece of PVC drainage pipe. They are counter-sunk so that we have room to nest a 4" tube inside this assembly. An earlier version of the weighing setup used a 6" outer tube (which actually has a nearly 6.5" O.D.) but we found that this required too much excavation of the bird's burrows and would have permanently and irreversibly altered the habitat. This led us to this smaller and less-invasive design.

In the right-hand picture above, you can see that the Tesco scales came with two different sized strain gauges. The one on the left came from scales purchased through the Internet. The smaller assembly on the right came from scales purchased in Tesco stores. The inside of the scales had mounting holes for both sizes of strain gauge assembly - we had to accommodate both as well. The aluminum bar in each gauge has one 'free' end and the other end fastened to the white metal base. The important measurement was to make sure that the inner tube would be centered above the free end of the aluminum strain-gauge bar so that the weight would bear directly down on this free end of the bar.

Next, the 4" inner tube was fastened to the strain gauge forming this axial assembly. In the field, this tube was inserted into the burrow entrance and the bird would have to pass through this inner tube on each trip in or out. The axial gap was covered in the field. Field tests indicated that the birds were perfectly comfortable moving through this tube.

The amplifier boards were attached to the inner tube at about the 3:00 position and a small access hole allowed for connection of the cable that would connect the amplifier to the ScatterWeb processor board.

The completed tubes were combined with the ScatterWeb nodes and supporting circuitry and 12V gell batteries and prepared for the field.

The deployment to Skomer Island is just underway now. Robin is making frequent trips to Skomer to deploy and configure hardware. There are lots of unanswered questions about the equipment and its installation. It is unclear what quality of readings we will get from this assembly. Robin did a significant amount of experimentation with weights so that we could calibrate the circuitry. Since the birds are almost continuously moving while being weighed, an actual weight will be a matter of some statistical inference rather than being a straightforward readout. This will probably substantially reduce the effective accuracy of the scales, but hopefully not down to where the data is not meaningful. There are also significant challenges remaining for Tomek as he tweaks the MSP430 code on the ScatterWeb for maximum power savings while juggling the limited interface resources and nearly unlimited need to talk to every interface and storage device all at the same time.

And for me, there is still the matter of the base station. The philosophy we want to support is one of 'Deploy and Depart' research, where you deploy the sensors and then go back to your office to monitor the data stream over a long period of time, with little or no field maintenance. To realize that goal, we need our base station in place. Last year's base station was a partial success and we want to take those lessons and improve on them this year, but the birds wait for no one, and with time running out before hatching, we had to ensure that the data acquisition equipment that I have talked about here, was working first. I will follow up in this blog with more on last year's PC-based base station and this year's solar-powered Windows CE base station.

You will be able to read more about the actual results from this research by tracking the publications of the principal investigators listed in this article.

RoboParty

martinca — Wed, 02 Apr 2008 00:08:18 GMT

I had the immense pleasure of being an invited speaker at Roboparty 2008 in Guimarães, Portugal at the University of Minho. RoboParty is part rock concert, part cultural event, and part old-school homebrew club meeting (and don't tell anyone, but it also manages to teach and to bring families together). It brings close to 400 young enthusiastic students of primary, secondary and early undergraduate ages together for three days of building, customizing, and programming robot kits on-site and then competing with those robots in maze exploring, choreographed dance and on their artistic/creative merit.

RoboParty is is held over a weekend (March 14-16 this year) in a University of Minho gymnasium that has been subdivided into sleeping and construction/competition areas. Not much sleeping appeared to go on as many of the competitors built their robots or programmed through the night.

Fernando Rebeiro, Associate Professor at the University is the organizer and force of nature behind RoboParty. Fernando appeared to run on only oxygen and enthusiasm for the full three days. I never saw him stop, and in between everything, he gave us a great tour of the city.

The image at right shows the mass of teams engaged in building their kits. The teams generally consisted of four people - students or students and a parent.

One of the amazing things to witness was the way so many families worked as a unit to build, decorate and program their robotic creations. Technology is usually such a solitary pursuit, but here it has been used as a remarkably effective venue for families and teams of young people to create something together.

Each team started with a standardized "bot'n roll One" kit from SAR (Soluções de Automação e Robótica) - check out their web mascot. The robot is uses the PICaxe processor and sports DC motors, rechargable batteries, and proximity sensors. Teams could also add line-follower kits and LCD readouts. SAR staff members were on hand to help sort out any construction mishaps. The folks at SAR were kind enough to give us one of these kits, and my wife Petra is going to put it together as soon as we finish relocating to the UK. It'll be her first soldering and programming experience too.

Teams generally arrived with little or no robotics experience - just a desire to learn and have fun. Many soldered and worked with electronics for the first time. For most this was their first experience programming or at least, programming robots. Over three days, they went from no experience and a pile of parts to wildly decorated robots that danced routines choreographed by the teams to music they selected. Teams could also compete in a maze-running exercise. The robot decorations rivaled anything you might find on an art-car or the Burning Man playa.

My role at RoboParty was to give a one-hour introduction to Microsoft Robotics Studio on the morning of the second day. Even under normal circumstances this is a challenging talk to give at TechEd or TechDays. People just want to see robots move. The percentage of the audience who are there to actually learn about robotics or robotics-like problem spaces is very small. But here, I had a full house of young people, parents and teachers who, in spite of being one day into a robot-building sleep-deprivation experiment, not only stayed awake, but asked some of the most insightful questions I've gotten from an audience. They covered everything from technical questions to ethics and philosophy. These young people were engaged and intelligent and I hope they enjoyed the talk because they really energized and inspired me.

It truly was a rock-concert experience. After the talks, I stood for perhaps a dozen photos and signed autographs. Professor Changjiu Zhou, Director, Advanced Robotics and Intelligent Control Centre at Singapore Polytechnic gave an excellent talk with videos and demonstrations of their work in Singapore, much of which is now MSRS-based. He enjoyed a similarly warm reception. I can safely say we were both very impressed with the intelligence and enthusiasm of the whole crowd - not to mention their stamina for staying awake.

RoboParty featured a continuous background thread of alternative activities too. There were sports exhibitions; a folk-dancing demonstration; a Tae Kwon Do demonstration; two movie nights and technical talks, including a very popular, creative and highly approachable programming course taught by Fernando.

In between all of this robo-enthusiasm we were treated to a spectacular tour of Guimarães, led by Fernando. Guimarães is the historic birthplace of Portugal. This tiny town is home to the castle, church and the remains of a medieval wall that together circumscribe much of the founding history of Portugal. We walked some of the wall and as you see at right, climbed to the top of the castle wall - a substantial (and slightly nerve-wracking) climb but a stunning view. I've made myself a promise to make it back to Guimarães. It's a city that deserves to be taken in patiently and appreciated for its beauty and its important place in history.

I am grateful to Fernando, the University of Minho and SAR for inviting me; and to SAR, Microsoft and the other sponsors for having the vision to support RoboParty; and especially to Fernando for being such a gracious and generous host. And, I am grateful to all of the students, teachers and families at RoboParty for being such an enthusiastic and intelligent bunch - I came away from RoboParty energized and inspired about technology and the students who will be bringing forth the next wave.

Low-Flying Bird

martinca — Wed, 19 Mar 2008 12:05:21 GMT

I am just back from Madrid where I visited the UK's visa division to sort out my visa. I now get to enjoy a week or two without traveling. Yesterday was my first ride on the Spanish 'Ave' (ah'-vay, Spanish for 'bird') high-speed train and I loved it. From Barcelona to Madrid it was faster than a plane and a far more pleasant travel experience.

The train takes between 2.5 and 2.75 hours to make the trip at between 200 and 300 km/hr (125-190 mi/hr) with a nice smooth ride. The Ave was faster than a plane because the train station was easier to get to (two Renfe stops from my apartment to the station at Barcelona Sants) and I didn't waste all that time at the airport getting through checkin and security. On top of that, the British Consulate was just a few blocks from the Madrid station. How many 'conveniently located' airports have you been to?

I arrived 20 minutes before departure and walked right onto the train. There's a lot to be said for not having restrictions on what electronics you can use nor when you can use them. The whole experience is far more comfortable, sensible and relaxed than any airport experience I've ever had. Since I'm a big fellow (198cm/6'6") I really appreciated the fact that I fit very comfortably into even the Tourist class seats. Having a real cafeteria, office space, and big bathrooms counts for a lot too. The trains run every hour to hour and a half from BCN to Madrid and back. A same-day purchase cost me €177 out and €70 for the return.

Overall, this is definitely the way to (not) fly for short- and medium-haul trips. Sensible urban and inter-city transportation continues to be one of the great joys of living in Europe.

On the Road

martinca — Mon, 17 Mar 2008 09:22:31 GMT

I've been on the road since the 25th of February and I have a fair bit to write about from those travels. To do it all justice, I'll have to spread it out over several postings. I started back in February with one week in Cambridge, UK preparing for Microsoft Research's TechFest and then ten days in Redmond at TechFest presenting with most of the rest of the MSRC ERO team; meeting with people on the Redmond campus; and re-connecting with colleagues. Then it was off to Lisbon Portugal for three days to present at Microsoft TechDays Portugal followed by another presentation and three amazing days at RoboParty in Guimarães Portugal.

I'm about to head off to Madrid today to sort out my UK visa, but upon my return I'll post a series of articles trying to capture these three road trip stops that have re-invigorated and inspired me both technically and culturally.

Autonomous Monitoring of Vulnerable Habitats

martinca — Fri, 22 Feb 2008 20:42:04 GMT

Robin Freeman, a post-doc in Computational Ecology and Environmental Science here in MSR Cambridge, and a number of the rest of us here in MSRC, European Science Initiative, together with partners at Freie Universitat Berlin and Oxford University have just concluded the first phase of an interesting project in the monitoring of vulnerable habitats using a wireless autonomous sensor web. You can learn more here.

This is an ongoing research effort, with the next field season on Skomer Island starting in just a few weeks. We're hopeful to see additional deployments in a variety of habitats around the world. This work will help enable more convenient, less-invasive and longer-term monitoring of vulnerable species and habitats.

A Return to Microsoft and MSR

martinca — Wed, 20 Feb 2008 10:36:18 GMT

After almost exactly two years outside Microsoft, I have returned to a full-time position in the company. I left my full-time position in February of 2006 when I moved to Barcelona, Spain. I haven't been very far 'outside' of Microsoft - I've spent most of the last two years working as a consultant or vendor on a couple different Microsoft projects. For most of the past year, I have been working under contract with Microsoft Research in Cambridge in the External Research/European Science Initiative's Technology Group, and that is the group that I have now joined full-time.

Most of what you will read here will be in some way related to our work in MSR Cambridge's ERO/ESI group (see: http://research.microsoft.com/ero), but I am not speaking for the group, Microsoft, or MSR - this blog is just me speaking for myself. In particular, our focus in ESI is on addressing the challenges of Science in the 21st Century through computational science approaches. In the main, we will be assisting researchers doing modeling and research in biology and ecology. Computational Science encompasses advanced data collection, annotation and analysis; data-driven and theory-based modeling; new collaboration mechanisms; advanced visualization; and augmenting electronic and traditional publication.

MSRC's ESI group is a research group and not a product group. As such, my development efforts are completely driven by the needs of our in-house researchers and their external collaborators. I am a developer and not a researcher, so what you will read here are practical applications of Microsoft and other technologies to the challenges of Computational Science. I'll share links, observations, commentary and code samples relating to data, computational tools and methods, cloud computing, visualization and publication.

One thing that I won't be doing is representing the science behind these projects. There will be Technical Reports (TRs) and published papers that will do that much better than I can here, and I will link to the ones that are related to our work in ESI. This blog is simply a source of sample material, observations, musings and discussions on the tools, methods and various practical matters of Computational Science.

I am not an official spokesperson in any manner for MSR or any of the MSR labs or Microsoft or MSR groups, including ESI. The opinions here are my own, and any sample code that I post or link to will be correct to the best of my knowledge, but is not warranted for any use or purpose and no license or rights are conferred by my posting or linking to any code or reference material.