It’s been awhile (to say the least) since I last posted – a year or so. It’s been an intense year of work on two very exciting projects. I plan to spend the coming weeks blogging extensively about these two projects – what you can do with them and how they work and ultimately, how you can get them too.
The two projects are Vedea and Microsoft Computational Science Studio. Vedea is a new language for creating interactive data-driven visualizations, and I’ll blog separately about that in a few days. Vedea will be demonstrated publicly for the first time at PDC 09 November 16-19 in Los Angeles and should be broadly available from research.microsoft.com shortly thereafter.
Today, I want to write about Microsoft Computational Science Studio which is getting its first public demonstrations today as part of Craig Mundie’s 2009 Campus Tour. The Seattle Times has a nice article on Craig’s tour and our part of his demo.
The Microsoft Research Computational Science Laboratory within Microsoft Research in Cambridge, UK consists of three groups : Computational Biology, Computational Ecology and Environmental Science and Tools and Technology. Craig’s demo is the culmination of a unique collaboration between the scientists in the laboratory and the software designers and developers in the Tools and Technology group to arrive at what we intend to be a tool for enabling non-programmer scientists and researchers to harness vast amounts of storage and compute power for running the multi-scale models that are needed to truly understand and predict complex natural systems.
Craig is quoted in the Seattle Times article as saying that he tells people “this is sort of doing for scientists and policymakers what Excel did for the average business guy 20 years ago”. Its really gratifying to hear him make that observation since simplifying the plumbing in multi-scale modelling is a core goal. He’s been a huge supporter of our efforts to do something really new and unique here and we’re grateful for that support.
Microsoft Computational Science Studio (MSCSS) is unique in a few ways and I will touch on each of those in more depth in upcoming posts. MSCSS attempts to address issues in collaboration, publication, presentation, data management, and most importantly to facilitate the construction of large data- and compute-intensive experiments and visualizations by non-programmers.
One of the first ways in which MSCSS is unique is in its collaboration story. You don’t have to copy files around in order to work on multiple computers or to collaborate with or share results with other people. You can share experiments among your own computers or with collaborators using some nifty cloud-based replication that’s built into MSCSS. You can still use MSCSS standalone and disconnected from the internet, but when coupled with some of Microsoft’s Windows Live cloud services you get local-file performance with the convenience of being able to access your projects from any computer. The collaboration possibilities will get a few posts all their own here.
MSCSS allows the user to create ‘data flows’ which connect the various data sources, computational elements, and presentation elements of an experiment together. The data flow is the ‘code’ that drives an MSCSS experiment.
I’ll spend quite a bit of space here talking about the data-flow layer and how you can construct flows for various trade-offs of realtime visualization performance, data size, etc and how to construct interactive and multi-scale simulations.
MSCSS also allows the user to create a separate presentation layer of views and viewports that contain the UI that lets someone drive your experiment or simulation and see the results without having to get tangled up in the internal details of the data flow.
You can think of these views and viewports as the ‘dashboard’ view into your experiment’s inner workings. This one, at left, exposes many of the UI elements seen in the data flow above. (You can expand these images by clicking on them).
Ultimately, you might use these views to tell a story, create publication-ready graphics or create a dashboard or application for others to explore your results. We’ve discussed various ways of exporting just these views to create standalone desktop or web applications, but that’s still a ways down the road before we’ll see it implemented in any usable form.
MSCSS itself is a shell into which you plug in extensions – for visualization, data management, computation, modelling, etc. One extension might give you access to remote data on Azure; another might allow you to draw heat-maps over Virtual Earth; and another might support Perfect Plasticity Approximation models or computations on the Hadley climate model data (both of which are components in Craig’s demo and some of the screenshots above). You then compose the elements from those extensions into a data flow and create UI that interacts with your data flow. That’s the exact process we used to create the demo Craig is giving. There’s no smoke and mirrors there – these are real data flows and UI being driven by MSCSS.
Of course, that extensibility is there to allow users to create their own extensions that add new data sources; new calculations, analyses and data transformations; and new visualizations. Down the road, we’re planning on making an SDK available to let you build your own extensions. Extensions also have a fairly unique model for publishing, sharing, discovering, installing and updating – a model designed to support direct user-to-user sharing of extensions and the SDK and entire extension pipeline will get a lot of space in upcoming posts as well.
MSCSS has been a substantial two-year effort by the entire team of scientists and engineers in the Computational Science Laboratory. It is still a research project in every sense but we’re eager to share our work with the community at large (with the usual non-commercial-use-only stipulation and with the understanding that this is a prototype and we’re not promising to ever actually ship or support it or any specific features in it). We do plan to make early versions of it available for download, perhaps in January of next year, with updates as we continue to evolve the project. I’m sure we’ll get an MSCSS team blog and such set up at some point, but in the meantime, I will use this blog to disseminate public announcements and discuss features related to MSCSS and Vedea
I certainly can't complain that I have a boring job. In addition to being responsible for architecting and building some very interesting horizontal software components for computational science investigations, I get to apply my electronics and mechanics skills to creating some interesting special-purpose components. These can range from collaborating on creation of a sub-15-gram solar GPS logger backpack to creating special purpose sensor interfaces to engineering a mechanism to get birds to weigh themselves every time they enter or leave their burrow. Its that last one that I will talk about today.
Among the goals is to try to determine how much food a Manx Shearwater brings home and how much their weight varies over a nesting season by weighing them with gram accuracy each time they enter or leave their burrows. In addition to weight, we need to read the bird's identification (and RFID tag) and sense motion (so that we only power up the very hungry scale and RFID circuits when they are needed). Many of the birds will also be carrying a preliminary version of the GPS backpacks, so we will also know where they have been foraging. In this article, I'll show you how we went about getting the birds to weigh themselves.
This Manx Shearwater research project on Skomer Island (off the western coast of Wales) is led by Prof. Tim Guilford of Oxford and Robin Freeman, a Post-Doc Researcher here at Microsoft Research in Cambridge, UK. Tomek Naumowicz is an endlessly energetic doctoral candidate from Freie Universitat Berlin and Tomek has built the firmware that is being used on the ScatterWeb sensors that monitor each burrow. My contribution has been to design and build some additional circuit cards along with the outdoor housings, the weighing tubes, and a solar-powered WindowsCE + GPRS base station for relaying the data back to the mainland.
As you will see below, the goals of cheap and fast build-out of less than a dozen low-environmental-impact weighing tubes means that the engineering is closer to the Scrapheap Challenge end of the spectrum than the NASA/JPL end.
The central component of the weighing tubes is an aluminum strain gauge harvested from a Tesco kitchen scale. The Tesco scale has 2g resolution from 0-5kg, but through the use of a different A/D converter, different choices for the amplification of the strain gauge signal and a narrower total weight range, we manage to get several times that resolution, down to the sub-gram range.
Out of each scale, we get remove the strain gauge assembly (seen lying on the right half of the dissassembled scale above), and bend the strain gauge base slightly to match the curve of the tube where we will mount it. The left photo below shows gauges before and after bending. The working part of the strain gauges are the aluminum bars, each of which have a patches of material applied which varies in resistance in proportion to the stress on the bar. In the original scales, an amplifier and A/D converter takes the signals from these gauges and amplifies and converts the signals into the weight readouts displayed on the readout. We replaced this with our own amplifier board (green board in the right-hand picture below). Our A/D converter is part of the ScatterWeb board. The amplifier takes the micro-volt changes given off by the strain gauges and boosts it to a 0-2V range which will be interpreted by the ScatterWeb A/D converter a meter or two away.
The strain gauges are mounted in a 5" piece of PVC drainage pipe. They are counter-sunk so that we have room to nest a 4" tube inside this assembly. An earlier version of the weighing setup used a 6" outer tube (which actually has a nearly 6.5" O.D.) but we found that this required too much excavation of the bird's burrows and would have permanently and irreversibly altered the habitat. This led us to this smaller and less-invasive design.
In the right-hand picture above, you can see that the Tesco scales came with two different sized strain gauges. The one on the left came from scales purchased through the Internet. The smaller assembly on the right came from scales purchased in Tesco stores. The inside of the scales had mounting holes for both sizes of strain gauge assembly - we had to accommodate both as well. The aluminum bar in each gauge has one 'free' end and the other end fastened to the white metal base. The important measurement was to make sure that the inner tube would be centered above the free end of the aluminum strain-gauge bar so that the weight would bear directly down on this free end of the bar.
Next, the 4" inner tube was fastened to the strain gauge forming this axial assembly. In the field, this tube was inserted into the burrow entrance and the bird would have to pass through this inner tube on each trip in or out. The axial gap was covered in the field. Field tests indicated that the birds were perfectly comfortable moving through this tube.
The amplifier boards were attached to the inner tube at about the 3:00 position and a small access hole allowed for connection of the cable that would connect the amplifier to the ScatterWeb processor board.
The completed tubes were combined with the ScatterWeb nodes and supporting circuitry and 12V gell batteries and prepared for the field.
The deployment to Skomer Island is just underway now. Robin is making frequent trips to Skomer to deploy and configure hardware. There are lots of unanswered questions about the equipment and its installation. It is unclear what quality of readings we will get from this assembly. Robin did a significant amount of experimentation with weights so that we could calibrate the circuitry. Since the birds are almost continuously moving while being weighed, an actual weight will be a matter of some statistical inference rather than being a straightforward readout. This will probably substantially reduce the effective accuracy of the scales, but hopefully not down to where the data is not meaningful. There are also significant challenges remaining for Tomek as he tweaks the MSP430 code on the ScatterWeb for maximum power savings while juggling the limited interface resources and nearly unlimited need to talk to every interface and storage device all at the same time.
And for me, there is still the matter of the base station. The philosophy we want to support is one of 'Deploy and Depart' research, where you deploy the sensors and then go back to your office to monitor the data stream over a long period of time, with little or no field maintenance. To realize that goal, we need our base station in place. Last year's base station was a partial success and we want to take those lessons and improve on them this year, but the birds wait for no one, and with time running out before hatching, we had to ensure that the data acquisition equipment that I have talked about here, was working first. I will follow up in this blog with more on last year's PC-based base station and this year's solar-powered Windows CE base station.
You will be able to read more about the actual results from this research by tracking the publications of the principal investigators listed in this article.
I had the immense pleasure of being an invited speaker at Roboparty 2008 in Guimarães, Portugal at the University of Minho. RoboParty is part rock concert, part cultural event, and part old-school homebrew club meeting (and don't tell anyone, but it also manages to teach and to bring families together). It brings close to 400 young enthusiastic students of primary, secondary and early undergraduate ages together for three days of building, customizing, and programming robot kits on-site and then competing with those robots in maze exploring, choreographed dance and on their artistic/creative merit.
RoboParty is is held over a weekend (March 14-16 this year) in a University of Minho gymnasium that has been subdivided into sleeping and construction/competition areas. Not much sleeping appeared to go on as many of the competitors built their robots or programmed through the night.
Fernando Rebeiro, Associate Professor at the University is the organizer and force of nature behind RoboParty. Fernando appeared to run on only oxygen and enthusiasm for the full three days. I never saw him stop, and in between everything, he gave us a great tour of the city.
The image at right shows the mass of teams engaged in building their kits. The teams generally consisted of four people - students or students and a parent.
One of the amazing things to witness was the way so many families worked as a unit to build, decorate and program their robotic creations. Technology is usually such a solitary pursuit, but here it has been used as a remarkably effective venue for families and teams of young people to create something together.
Each team started with a standardized "bot'n roll One" kit from SAR (Soluções de Automação e Robótica) - check out their web mascot. The robot is uses the PICaxe processor and sports DC motors, rechargable batteries, and proximity sensors. Teams could also add line-follower kits and LCD readouts. SAR staff members were on hand to help sort out any construction mishaps. The folks at SAR were kind enough to give us one of these kits, and my wife Petra is going to put it together as soon as we finish relocating to the UK. It'll be her first soldering and programming experience too.
Teams generally arrived with little or no robotics experience - just a desire to learn and have fun. Many soldered and worked with electronics for the first time. For most this was their first experience programming or at least, programming robots. Over three days, they went from no experience and a pile of parts to wildly decorated robots that danced routines choreographed by the teams to music they selected. Teams could also compete in a maze-running exercise. The robot decorations rivaled anything you might find on an art-car or the Burning Man playa.
My role at RoboParty was to give a one-hour introduction to Microsoft Robotics Studio on the morning of the second day. Even under normal circumstances this is a challenging talk to give at TechEd or TechDays. People just want to see robots move. The percentage of the audience who are there to actually learn about robotics or robotics-like problem spaces is very small. But here, I had a full house of young people, parents and teachers who, in spite of being one day into a robot-building sleep-deprivation experiment, not only stayed awake, but asked some of the most insightful questions I've gotten from an audience. They covered everything from technical questions to ethics and philosophy. These young people were engaged and intelligent and I hope they enjoyed the talk because they really energized and inspired me.
It truly was a rock-concert experience. After the talks, I stood for perhaps a dozen photos and signed autographs. Professor Changjiu Zhou, Director, Advanced Robotics and Intelligent Control Centre at Singapore Polytechnic gave an excellent talk with videos and demonstrations of their work in Singapore, much of which is now MSRS-based. He enjoyed a similarly warm reception. I can safely say we were both very impressed with the intelligence and enthusiasm of the whole crowd - not to mention their stamina for staying awake.
RoboParty featured a continuous background thread of alternative activities too. There were sports exhibitions; a folk-dancing demonstration; a Tae Kwon Do demonstration; two movie nights and technical talks, including a very popular, creative and highly approachable programming course taught by Fernando.
In between all of this robo-enthusiasm we were treated to a spectacular tour of Guimarães, led by Fernando. Guimarães is the historic birthplace of Portugal. This tiny town is home to the castle, church and the remains of a medieval wall that together circumscribe much of the founding history of Portugal. We walked some of the wall and as you see at right, climbed to the top of the castle wall - a substantial (and slightly nerve-wracking) climb but a stunning view. I've made myself a promise to make it back to Guimarães. It's a city that deserves to be taken in patiently and appreciated for its beauty and its important place in history.
I am grateful to Fernando, the University of Minho and SAR for inviting me; and to SAR, Microsoft and the other sponsors for having the vision to support RoboParty; and especially to Fernando for being such a gracious and generous host. And, I am grateful to all of the students, teachers and families at RoboParty for being such an enthusiastic and intelligent bunch - I came away from RoboParty energized and inspired about technology and the students who will be bringing forth the next wave.
Photos all (c) 2008 Fernando Rebeiro - Many thanks to Fernando for sharing them.
I am just back from Madrid where I visited the UK's visa division to sort out my visa. I now get to enjoy a week or two without traveling. Yesterday was my first ride on the Spanish 'Ave' (ah'-vay, Spanish for 'bird') high-speed train and I loved it. From Barcelona to Madrid it was faster than a plane and a far more pleasant travel experience.
The train takes between 2.5 and 2.75 hours to make the trip at between 200 and 300 km/hr (125-190 mi/hr) with a nice smooth ride. The Ave was faster than a plane because the train station was easier to get to (two Renfe stops from my apartment to the station at Barcelona Sants) and I didn't waste all that time at the airport getting through checkin and security. On top of that, the British Consulate was just a few blocks from the Madrid station. How many 'conveniently located' airports have you been to?
I arrived 20 minutes before departure and walked right onto the train. There's a lot to be said for not having restrictions on what electronics you can use nor when you can use them. The whole experience is far more comfortable, sensible and relaxed than any airport experience I've ever had. Since I'm a big fellow (198cm/6'6") I really appreciated the fact that I fit very comfortably into even the Tourist class seats. Having a real cafeteria, office space, and big bathrooms counts for a lot too. The trains run every hour to hour and a half from BCN to Madrid and back. A same-day purchase cost me €177 out and €70 for the return.
Overall, this is definitely the way to (not) fly for short- and medium-haul trips. Sensible urban and inter-city transportation continues to be one of the great joys of living in Europe.
I've been on the road since the 25th of February and I have a fair bit to write about from those travels. To do it all justice, I'll have to spread it out over several postings. I started back in February with one week in Cambridge, UK preparing for Microsoft Research's TechFest and then ten days in Redmond at TechFest presenting with most of the rest of the MSRC ERO team; meeting with people on the Redmond campus; and re-connecting with colleagues. Then it was off to Lisbon Portugal for three days to present at Microsoft TechDays Portugal followed by another presentation and three amazing days at RoboParty in Guimarães Portugal.
I'm about to head off to Madrid today to sort out my UK visa, but upon my return I'll post a series of articles trying to capture these three road trip stops that have re-invigorated and inspired me both technically and culturally.
Robin Freeman, a post-doc in Computational Ecology and Environmental Science here in MSR Cambridge, and a number of the rest of us here in MSRC, European Science Initiative, together with partners at Freie Universitat Berlin and Oxford University have just concluded the first phase of an interesting project in the monitoring of vulnerable habitats using a wireless autonomous sensor web. You can learn more here.
This is an ongoing research effort, with the next field season on Skomer Island starting in just a few weeks. We're hopeful to see additional deployments in a variety of habitats around the world. This work will help enable more convenient, less-invasive and longer-term monitoring of vulnerable species and habitats.
After almost exactly two years outside Microsoft, I have returned to a full-time position in the company. I left my full-time position in February of 2006 when I moved to Barcelona, Spain. I haven't been very far 'outside' of Microsoft - I've spent most of the last two years working as a consultant or vendor on a couple different Microsoft projects. For most of the past year, I have been working under contract with Microsoft Research in Cambridge in the External Research/European Science Initiative's Technology Group, and that is the group that I have now joined full-time.
Most of what you will read here will be in some way related to our work in MSR Cambridge's ERO/ESI group (see: http://research.microsoft.com/ero), but I am not speaking for the group, Microsoft, or MSR - this blog is just me speaking for myself. In particular, our focus in ESI is on addressing the challenges of Science in the 21st Century through computational science approaches. In the main, we will be assisting researchers doing modeling and research in biology and ecology. Computational Science encompasses advanced data collection, annotation and analysis; data-driven and theory-based modeling; new collaboration mechanisms; advanced visualization; and augmenting electronic and traditional publication.
MSRC's ESI group is a research group and not a product group. As such, my development efforts are completely driven by the needs of our in-house researchers and their external collaborators. I am a developer and not a researcher, so what you will read here are practical applications of Microsoft and other technologies to the challenges of Computational Science. I'll share links, observations, commentary and code samples relating to data, computational tools and methods, cloud computing, visualization and publication.
One thing that I won't be doing is representing the science behind these projects. There will be Technical Reports (TRs) and published papers that will do that much better than I can here, and I will link to the ones that are related to our work in ESI. This blog is simply a source of sample material, observations, musings and discussions on the tools, methods and various practical matters of Computational Science.
I am not an official spokesperson in any manner for MSR or any of the MSR labs or Microsoft or MSR groups, including ESI. The opinions here are my own, and any sample code that I post or link to will be correct to the best of my knowledge, but is not warranted for any use or purpose and no license or rights are conferred by my posting or linking to any code or reference material.
