It’s been awhile (to say the least) since I last posted – a year or so. It’s been an intense year of work on two very exciting projects. I plan to spend the coming weeks blogging extensively about these two projects – what you can do with them and how they work and ultimately, how you can get them too.
The two projects are Vedea and Microsoft Computational Science Studio. Vedea is a new language for creating interactive data-driven visualizations, and I’ll blog separately about that in a few days. Vedea will be demonstrated publicly for the first time at PDC 09 November 16-19 in Los Angeles and should be broadly available from research.microsoft.com shortly thereafter.
Today, I want to write about Microsoft Computational Science Studio which is getting its first public demonstrations today as part of Craig Mundie’s 2009 Campus Tour. The Seattle Times has a nice article on Craig’s tour and our part of his demo.
The Microsoft Research Computational Science Laboratory within Microsoft Research in Cambridge, UK consists of three groups : Computational Biology, Computational Ecology and Environmental Science and Tools and Technology. Craig’s demo is the culmination of a unique collaboration between the scientists in the laboratory and the software designers and developers in the Tools and Technology group to arrive at what we intend to be a tool for enabling non-programmer scientists and researchers to harness vast amounts of storage and compute power for running the multi-scale models that are needed to truly understand and predict complex natural systems.
Craig is quoted in the Seattle Times article as saying that he tells people “this is sort of doing for scientists and policymakers what Excel did for the average business guy 20 years ago”. Its really gratifying to hear him make that observation since simplifying the plumbing in multi-scale modelling is a core goal. He’s been a huge supporter of our efforts to do something really new and unique here and we’re grateful for that support.
Microsoft Computational Science Studio (MSCSS) is unique in a few ways and I will touch on each of those in more depth in upcoming posts. MSCSS attempts to address issues in collaboration, publication, presentation, data management, and most importantly to facilitate the construction of large data- and compute-intensive experiments and visualizations by non-programmers.
One of the first ways in which MSCSS is unique is in its collaboration story. You don’t have to copy files around in order to work on multiple computers or to collaborate with or share results with other people. You can share experiments among your own computers or with collaborators using some nifty cloud-based replication that’s built into MSCSS. You can still use MSCSS standalone and disconnected from the internet, but when coupled with some of Microsoft’s Windows Live cloud services you get local-file performance with the convenience of being able to access your projects from any computer. The collaboration possibilities will get a few posts all their own here.
MSCSS allows the user to create ‘data flows’ which connect the various data sources, computational elements, and presentation elements of an experiment together. The data flow is the ‘code’ that drives an MSCSS experiment.
I’ll spend quite a bit of space here talking about the data-flow layer and how you can construct flows for various trade-offs of realtime visualization performance, data size, etc and how to construct interactive and multi-scale simulations.
MSCSS also allows the user to create a separate presentation layer of views and viewports that contain the UI that lets someone drive your experiment or simulation and see the results without having to get tangled up in the internal details of the data flow.
You can think of these views and viewports as the ‘dashboard’ view into your experiment’s inner workings. This one, at left, exposes many of the UI elements seen in the data flow above. (You can expand these images by clicking on them).
Ultimately, you might use these views to tell a story, create publication-ready graphics or create a dashboard or application for others to explore your results. We’ve discussed various ways of exporting just these views to create standalone desktop or web applications, but that’s still a ways down the road before we’ll see it implemented in any usable form.
MSCSS itself is a shell into which you plug in extensions – for visualization, data management, computation, modelling, etc. One extension might give you access to remote data on Azure; another might allow you to draw heat-maps over Virtual Earth; and another might support Perfect Plasticity Approximation models or computations on the Hadley climate model data (both of which are components in Craig’s demo and some of the screenshots above). You then compose the elements from those extensions into a data flow and create UI that interacts with your data flow. That’s the exact process we used to create the demo Craig is giving. There’s no smoke and mirrors there – these are real data flows and UI being driven by MSCSS.
Of course, that extensibility is there to allow users to create their own extensions that add new data sources; new calculations, analyses and data transformations; and new visualizations. Down the road, we’re planning on making an SDK available to let you build your own extensions. Extensions also have a fairly unique model for publishing, sharing, discovering, installing and updating – a model designed to support direct user-to-user sharing of extensions and the SDK and entire extension pipeline will get a lot of space in upcoming posts as well.
MSCSS has been a substantial two-year effort by the entire team of scientists and engineers in the Computational Science Laboratory. It is still a research project in every sense but we’re eager to share our work with the community at large (with the usual non-commercial-use-only stipulation and with the understanding that this is a prototype and we’re not promising to ever actually ship or support it or any specific features in it). We do plan to make early versions of it available for download, perhaps in January of next year, with updates as we continue to evolve the project. I’m sure we’ll get an MSCSS team blog and such set up at some point, but in the meantime, I will use this blog to disseminate public announcements and discuss features related to MSCSS and Vedea
How does this relate to Project Trident that was made available recently from MSR?
You mentioned "Vedea will be demonstrated publicly for the first time at PDC 09 November 16-19 in Los Angeles ..." - Where and when will this demo be held at PDC? It's not on any of the current schedules, and I definitely am interested in seeing this...
Apologies for the slow response, but I have been out ill for the past week.
@davidacoder: The two projects were separately developed and have slightly different purposes, but we have, in fact, started, stopped and managed Trident workflows from within MSCSS and we will likely continue to increase the level of integration so that you can reap the benefits of using Trident distributed workflows together with the orchestration, integration, collaboration and visualization benefits of MSCSS.
@reed: I probably should have been clearer there. We are a late addition to the program and will only be on the product demo floor - not a scheduled session. Because of my illness, I won't be there myself, but Vedea will be well represented on the demo floor and I encourage you to check it out. I plan to catch up on my posting this coming week too and talk quite a bit about Vedea and its features.
I was looking out for announcements about Verdea but could not find anything on the PDC site at all. I am very interested to know more. When will you guys post information ?
Sorry folks, I was ill and off work for most of Nov (including missing PDC, but Vedea was represented there on the demo floor) and I am only slowly returning to the digital world. Updates are coming soon - both here and on the main research.microsoft.com site.
Any news on this? You wrote that you plan to release an early preview download in Jan. That has sort of passed ;) Any update on this (as well as on Vedea) would be much appreciated.