Stuart Kent - Software Modeling and Visualization
We’ve been enjoying injecting MEF extension points in our tools to make it easy for customers to write extensions using very simple code. Jean-Marc has just posted a sample demonstrating how MEF-enable your own DSL so you too can add MEF extension points for customers. See http://blogs.msdn.com/jmprieur/archive/2010/05/17/enabling-your-dsl-for-mef-extensions.aspx for details.
Videos of the VS2010 launch and deep dive sessions at the recent Tech Days UK event are now available. Also See my earlier post about this event.
Goto http://www.microsoft.com/uk/techdays/ click on the Resources tab and then choose the video you’re interested in watching.
I’ve embedded the video of my session below:
Writing this as I come back from speaking at Tech Days in London, in a series of deep dive sessions following the launch of VS2010. The venue was a cinema in Fulham Broadway. The place was full (between 500 and 600 people), indeed over-subscribed. I understand that some folks who turned up were turned away (Health and Safety dictates that you can’t go over capacity in this kind of venue); desperately sorry for those unlucky few, but perhaps they enjoyed a day in London instead. It’s great to see so much enthusiasm for the new tools in Visual Studio. Yesterday’s VS2010 launch was packed out as well.
The decision to use a modern cinema was inspired. The seating is very comfortable, much better than you usually get in conference centres, and I think this must be the largest screen I’ve ever given a presentation on; there were no worries about people not being able to see the text. Though slightly disconcerting to see a few participants tucking into their popcorn! The audience was attentive, and I could see some heads nodding in the front rows, with, of course, the usual politeness and reserve to be expected from a British audience. I’ll be interested to see the feedback scores. A couple of questions: one asking about the version of Visual Studio you need to use these tools (Ultimate for all the functionality, though some limited functionality is available in lower versions), another about the relationship to M and Oslo (M is now part of SQL Modeling Services, and we look forward to being able to hook some of our visualizations to models in databases at some point in the future).
The speaker after me was Colin Bird, formerly of Conchango and now founder of a new company RippleRock. He gave a very interesting talk on Agile Development, making the point that Microsoft is really providing a tooling platform, with some tools, templates and so on in the box to get you started, but to get the most out of it you need to invest some resource every sprint into building up and maintaining your tools and automation, which should include customizing and extending Visual Studio to fit your own engineering practices and particular circumstances. I couldn’t agree more, and I think that it is our job in Microsoft to make it as easy as possible to implement and deploy those customizations. As an illustration of how far we’ve come, towards the end of my talk I was able to create a new extension to the UML designers and deploy it to VS in less than a couple minutes, as well as create a new DSL from scratch. A few years ago, I’d have been much more careful about demonstrating DSL Tools because of the wait times it took to do the code generation and build. Deployment was also a case of adding another project to create a .msi. And I wouldn’t have dared tried creating other kinds of VS extension during a demo.
This trip also made me realize how old I must be getting (that and the recently acquired reading glasses). I used to live in London when a PhD student, and would have expected to have needed to go outside at least once to get to my destination. I travelled on the high speed rail service from my home town of Canterbury which takes you straight into St Pancras station. It’s the first time I’ve used this service (I usually take the slower train into Charing Cross), and I have to say that St. Pancras really is beautiful, with a broad arching roof over the platforms. So I arrived under cover, and from there you can walk to Kings Cross tube without stepping outside, then round on the Circle line to Edgware Road, and a step across the platform onto the District line service to Fulham Broadway. The entrance to Fulham Broadway is integrated into the shopping complex and multi-screen cinema, so no need to go outside there, either. The experience was slightly surreal.
It’s great to see that the RTM version of the Visualization and Modeling SDK is also available at the launch and release of Visual Studio 2010. For details check out http://code.msdn.microsoft.com/vsvmsdk. See also Jean-Marc’s announcement of this.
As well as the DSL Tools which has been much improved in this release (see What’s new), it also includes some project templates to get you started with extending the UML Tools in Visual Studio. Enjoy.
Visual Studio 2010 and .Net 4.0 launches today across the world, and I have the pleasure of giving a talk tomorrow at the UK Tech Days event in London as part of a set of deep dive sessions into the new release. For details see http://www.microsoft.com/uk/techdays/daydev.aspx.
I’m going to be demonstrating the new visualization and modelling tools in Visual Studio, which I’ve helped to build and have been enjoying using on our own development projects.
Back from holiday (in Cyprus, 30 degrees C, very relaxing thanks), and as I suggested might happen in my previous entry, I see that Jean-Marc has blogged about DSL Tools in VS2010 Beta2: We released the DSL SDK for Visual Studio Beta2 this morning.
I see that Jean-Marc has just posted another sample on the DSL Tools Code Gallery landing page, this time about the new IMS Locks capability. This allows you to make some parts of your models read-only, which is good for situations where some users (say an architect) are allowed to make changes to some aspects of the model (say the core architecture components), and other users (say developers) are only allowed to make changes to other aspects (say the properties specifically focused on driving the code generators). As author of the DSL you have complete control in the locking policy used. More details in Jean-Marc’s post.
Jean-Marc has just posted a ModelBus Adapters project template on the Visual Studio Gallery. This template makes it easy to a modelbus adapter and adapter manager to your own DSL, which exposes your DSL for reference and access from other DSLs. This compliments the sample recently posted, as described in my last post.
One of the most common asks from customers of the DSL Tools has been support for integrating models and designers, that is the ability to have models of the same or different DSL cross-reference one another, and the ability to write simple code that exploits these cross-references to implement interesting interactions between designers.
In VS2010 we’re providing this support which can now be previewed in VS2010 Beta1 release. Jean-Marc has just posted a sample (StateMachineOverModelBus.zip, documentation) that illustrates how the modelbus can be used to achieve the behaviors described above, and I’ve just posted a short video demonstrating the behaviors and giving a quick insight into what you need to do to realize them in your own designers.
Details on how you can provide feedback can be found on our the DSL Tools Code Gallery landing page. Or by all means provide feedback on this blog.
In Visual Studio 2010 you’ll find a new extension manager that makes it much easier to package and deploy extensions to Visual Studio. Pedro has more information: Early Buzz on the VS SDK and Extension Manager, Introducing VS Extension Manager.
We’ve exploited this to deploy DSL Tools. Gone has the DSL Setup project which used WiX to create and MSI. Instead, a DSL authoring solution builds a VSIX package and uses that to install the DSL in the experimental instance of Visual Studio. But even better, to install the DSL in the main instance of Visual Studio, all you have to do is open the VSIX file in windows explorer and respond to the prompts. Restart Visual Studio and your DSL is installed! How easy is that?
I’ve captured all this on video.
I’m speaking at the Code Generation 2009 conference in Cambridge in June on the topic of Code-Centric or Model-Centric – Approaches to developing software.
Jean-Marc is also speaking on What’s new in the DSL Tools and T4 in Visual Studio 2010.
I’m really looking forward to going to this conference which is at the cutting edge of model-driven and code generation techniques. It will be great to meet up again with folks I haven’t seen for some time, especially colleagues I used to work with before I joined Microsoft.
And beware: “LATE BOOKING FEES APPLY FROM JUNE 1ST”
Gareth has been posting a lot about T4 over the past few months. In case you missed it, here’s a roundup of all his posts, in chronological order. Lots there to get your teeth into.
You can now try out the VS2010 features mentioned above with the release of VS2010 Beta1, VS2010 SDK Beta1, and VS2010 DSL SDK Beta1: see Dsl Tools for Visual Studio 2010.
I’ll also add one more link:
As you can see, T4 is getting popular!
And for VS2010 there’s more to come on top of the features that Gareth describes. Not available in Beta1, but in Beta2 we’ll be providing support for accessing models in T4 via the modelbus. This will enable text templates to access models created with the new UML designers in Visual Studio and more enterprise-scale orchestration of code generation.
When we released DSL Tools for VS2008, the migration story from VS2005 was not great: basically we provided a sheet of manual instructions which led to a fragile and long-winded experience. This time we’ve done a lot better. In the VS2010 DSL SDK Beta1 you’ll find a migration tool that converts your projects and solutions for you automatically. We’ve been using this internally to convert samples, test projects and so on, and it does a lovely job. There are links to a video showing the tool in use, and some documentation on the Downloads tab of the DSL Tools Code Gallery. Life is good.
It seems that my blog entries and those of some of my colleagues (Steve, Jean-Marc, Gareth) are like London buses: you wait for ages and then three come at once. And I’m definitely the worst offender – that bus which breaks down all the time.
Anyway, the reason why we’ve been so lax is that we’ve all been desperately busy getting Visual Studio 2010 out the door.
So it’s really exciting to announce the availability of DSL Tools for Visual Studio 2010 Beta1, hot on the heels of Visual Studio 2010 Beta1 itself and the corresponding Visual Studio SDK. Jean-Marc has all the details. We have a page summarizing the new features.
Various members of the team will be following up with blogs and samples, and we look forward to your feedback: the good, bad and ugly (actually, I’m sure you’re all extremely eloquent, so I don’t expect any ugliness). We’ll keep you up to date through the DSL Tools Code Gallery page.
As Cameron blogged in Visual Studio Team System 2010 Architecture: Prologue, the DSL Tools team have recently moved back to Team Architect from the Visual Studio Platform team. We’ve been working increasingly closely with Team Architect since they renewed their focus on building VS modeling tools integrated using the DSL Tools. By moving back to that team we’ll be better placed to focus on three goals:
We expect to make progress on all three goals in VS2010.
That said, our sojourn with the VS Platform Team was time well spent. We understand a lot better the technologies and the direction of the core VS platform and, just as importantly, know the people in that team a lot better. DSL Tools is layered on top of that platform and, already in VS2010, we’ll be taking advantage of the integration of the Managed Extensibility Framework (MEF) into the heart of VS (see e.g. this post) and the technologies underpinning the Extension Manager.
Jean-Marc has just posted a lab on code gallery, which provides a fully worked, step-by-step example of creating a DSL from scratch. If you're new to DSL Tools, or want to teach others how to use them, then you may find this useful.
Nearly forgot the link. It's at:
http://code.msdn.microsoft.com/DSLToolsLab
The Oslo modeling platform was announced at Microsoft's PDC and we've been asked by a number of customers what the relationship is between DSL Tools and Oslo. So I thought it would be worth clearing the air on this. Keith Short from the Oslo team has just posted on this very same question. I haven’t much to add really, except to clarify a couple of things about DSL Tools and VSTS Team Architect.
As Keith pointed out, some commentators have suggested that DSL Tools is dead. This couldn’t be further from the truth. Keith himself points out that "both products have a lifecycle in front of them". In DSL Tools in Visual Studio 2010 I summarize the new features that we're shipping for DSL Tools in VS 2010, and we'll be providing more details in future posts. In short, the platform has expanded to support forms-based designers and interaction between models and designers. There's also the new suite of designers from Team Architect including a set of UML designers and technology specific DSLs coming in VS 2010. These have been built using DSL Tools. Cameron has blogged about this, and there are now some great videos describing the features, including some new technology for visualizing existing code and artifacts. See this entry from Steve for details.
The new features in DSL Tools support integration with the designers from team architect, for example with DSLs of your own, using the new modelbus, and we're exploring other ways in which you can enhance and customize those designers without having to taking the step of creating your own DSL. Our T4 text templating technology will also work with these designers for code generation and will allow access to models across the modelbus. You may also be interested in my post Long Time No Blog, UML and DSLs which talks more about the relationship between DSLs and UML.
The first public CTP of Visual Studio 10
Anyone attending or watching the goings on at Microsoft's Professional Developers Conference (PDC) will have heard about the next version of Visual Studio.
If you were at PDC, you will have received a VPC containing a Community Technology Preview of Visual Studio 10. If you were not, you might know that you can download it at: Visual Studio 2010 and .NET Framework 4.0 CTP.
What about the DSL Tools?
If you are interested in the DSL Tools, and try this CTP, you might be disappointed not to find anything new in DSL Tools. This doesn't mean we haven't been busy. In fact we've been developing a number of highly-requested new features, but unfortunately did not get these integrated into this CTP release. We'll share them with customers in our next preview release.
Also, in the CTP you'll find a suite of new designers from Team Architect, including a set of UML designers. These have all been built using DSL Tools, and some of the new features we're delivering are directly to support this effort.
The new features
So what are the new features then? Below is a summary. They've been driven by a combination of supporting internal teams such as Team Architect, and responding to customer feedback. We'll blog more about these features over the coming weeks, including, we hope, publishing some videos demo'ing them as a taster whilst you wait for the next preview.
Dsl Libraries. Share fragments of DSL definitions between different designers.
Dsl extensibility. Extend the domain model and behavior for a DSL after it's been deployed, including DSLs shipped by a third party.
Readonly. Selectively switch off the ability to edit models and model elements in a designer.
Forms-based UI. Easily bind models to winforms and WPF-based forms UI. IMS now implements the necessary databinding interfaces.
Modelbus. A new piece of platform to support cross-referencing between models and interaction between designers. This has been one of customers' main requests.
T4 precompile. Precompile text templates so that they can be deployed for use on machines that do not have VS installed. (This applies to scenarios where text templates take inputs from data sources other than DSL models. We've had requests from internal teams to use text templates in scenarios which don't use models created from DSLs.)
UI enhancements:
Our colleagues in China have been busy translating our DSL Book into Chinese. It's taken 6 months and 9 translators. See http://blogs.msdn.com/stbcblog/archive/2008/07/24/Domain-Specific-Development-with-Visual-Studio-DSL-Tools.aspx for the story. The translation is available for sale online.
We're holding our first Visual Studio Extensibility Developer conference in September. See http://msdn.microsoft.com/en-us/vsx/cc676517.aspx for details, including information on how to sign up.
We've finally converted all the samples for our book on Domain Specific Development for VS2008. They can be downloaded from www.domainspecificdevelopment.com.
Embarrassing, but true. Last time I blogged was back in February.
I'm on one of my regular trips to Redmond, and jet lag still means I'm getting up at 4am. Usually on these trips I use the early morning to do a bit of work, then go for a run. However, last trip I stumbled on a tree root when out running and fractured my ankle. It's the first time I've broken a bone, so I didn't know what to expect. In fact, at first I didn't really believe the doctor; x-ray looked fine to me. Little did I expect that I'd be back in Redmond 7 weeks later, still limping and definitely unable to run. So I find myself with more time in the morning, and an opportunity to contribute once again to the blogsphere, though any readership I may have had has probably already given up on me.
So what's there to talk about. As the title suggests, I thought I would touch on the subject of UML and DSLs given a recent post on UML + DSL from Cameron Skinner, who now leads Visual Studio Team Architect. In the article, Cameron talks about a new set of designers that Team Architect are working on, which includes some UML designers, as well as some graphical DSLs. The goal is to 'make modeling more central to a broader range of people' as well as to 'seamlessly connect the two approaches [UML and DSL]'. We've found that DSL Tools is very appealing to customers who've already bought into modeling. It allows them to create their own customized model driven solutions fairly quickly. However, for those not already into modeling it's a harder sell (although, we have had customers who have seen what others have produced and decide to build something similar which focuses on their domain). To really get modeling out to a broader audience it's necessary to have tools that people can use straight out of the box. I'm still convinced that if you want to dramatically increase productivity then you should be using DSLs driving code generators, or further model transformations, or visualizing abstractions discovered from existing artifacts, or some combination of all three. On the other hand, whatever your views are about the effectiveness or not of UML, it has significantly raised the awareness of modeling with a broad audience and it is something that many people are familiar with. So I'm really pleased that Team Architect has now stepped up to drive a strategy that delivers on both sides of the modeling coin, and connects them up.
What is more, I'm really enjoying helping them deliver on this strategy by providing a great platform to build on. Because, as Cameron points out, many of the designers being built by Team Architect (including the UML ones) are being built on the DSL Tools platform. Not only is this further proof for the robustness and flexibility of this platform, it is also helping to drive new features into the platform which will benefit anyone wanting to build their own DSLs. We're not ready to talk about those features in detail yet, but my list posted a few months ago is still fairly accurate, although there are a couple of additions and some changes in priority. Worth calling out are the features to allow a designer to be extended with new metadata once it has been deployed, and support for designers and models to interact. This will allow the designers delivered by Team Architect to be extended, and for new, third party DSLs to interact with them. Both these are key aspects of connecting UML with DSLs in the UML + DSL story that Cameron speaks about.
Finally, you may also have noticed that Steve is changing his role, by going back to Team Architect in order to help them deliver on the UML + DSL vision. This means that Steve will be more of a consumer of the platform that he helped create, and I'm sure will be driving hard to ensure that it meets his needs. Steve and I still share an office in the UK and will still be working closely together. Should be a fun ride, this.
Over on Deep's blog I came across the following comment to his post on Tools for Tools.
I had looked at creating a DSL for one of our projects a while ago, and in the end decided against it because the deployment was too complicated. In particular, here is how I want deployment to work: In my normal solution of my project, I add a new DSL project, along with my other C# projects that constitute my end piece of software. In the DSL project I define a new DSL that I can then use right away in the C# projects that are part of the solution as well. That is, I don't want any setup at all, but rather want the DSL to be loaded as for example WinForm custom controls and the designers that go along are loaded from a dll project that is simply part of the solution of the consuming project. Why is this so important? First, this would automatically solve the version problem. In my case we would rapidly (or shall I say "agile" ;) change the DSL as the project goes along. With a model where the DSL is installed wia a setup program on a machine wide basis, we run into huge trouble when we want to work with a previous version of our source code (say a branch from the source code repository that corresponds to a previous version). We would need to make sure that our DSL project can always load old versions, i.e. we would need to be super backword compatible. But I don't want that, I want the DSL versioned along with the source code so that I get a predictable match of tools and source code artefacts. Also, of course, deployment via setup is just a pain, I really just want my solution file to have all the info about tools (be it DSL, compilers what have you) needed to build it and take care of that. Any chance that we might end up with something like that?
I had looked at creating a DSL for one of our projects a while ago, and in the end decided against it because the deployment was too complicated. In particular, here is how I want deployment to work:
In my normal solution of my project, I add a new DSL project, along with my other C# projects that constitute my end piece of software. In the DSL project I define a new DSL that I can then use right away in the C# projects that are part of the solution as well. That is, I don't want any setup at all, but rather want the DSL to be loaded as for example WinForm custom controls and the designers that go along are loaded from a dll project that is simply part of the solution of the consuming project.
Why is this so important? First, this would automatically solve the version problem. In my case we would rapidly (or shall I say "agile" ;) change the DSL as the project goes along. With a model where the DSL is installed wia a setup program on a machine wide basis, we run into huge trouble when we want to work with a previous version of our source code (say a branch from the source code repository that corresponds to a previous version). We would need to make sure that our DSL project can always load old versions, i.e. we would need to be super backword compatible. But I don't want that, I want the DSL versioned along with the source code so that I get a predictable match of tools and source code artefacts.
Also, of course, deployment via setup is just a pain, I really just want my solution file to have all the info about tools (be it DSL, compilers what have you) needed to build it and take care of that.
Any chance that we might end up with something like that?
Generalizing this scenario a bit, I think what the commenter is asking for is the following two capabilities:
1) For VS extensions (in this case DSLs) to be checked in the rest of the code on a project, so that you can manage the tools used to build your code just in the same way that you manage the code itself.
2) Not to have to go through an expensive process (e.g. install an MSI) every time you want to reconfigure VS with the tools (extensions) needed for a particular project, or when you want to update those tools.
There are things you can do now to get over these two problems.
For (1) you can have a separate tools authoring solution that is checked in and used to rebuild the tools. You can do this now, and I don't see any issues around this. Indeed I would argue that it's not a good idea to mix your tools authoring solution and application development solutions, just because I think it's a good idea to keep these two concerns separate. However, you still need a way of getting revisions of tooling quickly out to developers, rolling back to past versions and so on. And making sure that before you build your application code, the right version of the tools is built and installed.
For (2), if you don't want to go to the trouble of installing an MSI, then, today, you could do the application building in the experimental version of VS (though you'll need to have the VS SDK installed for that). Building the authoring solution will install the tools in the experimental hive, and then you just need to remember to launch VS Exp rather than VS to do your work. So once you've updated your machine from source control, you'd go an build the authoring solution, then you'd go an open the experimental version of VS and continue with your development. You can reset the experimental version of VS anytime to mirror main VS, when you want to switch projects.
An alternative is to work in virtual PCs (our team does much of its development this way). Then to roll out a new version of the tools to your team, you build a new VPC with the tools installed and distribute this. The advantage of this approach is that you can set up multiple VPCs for different projects and then it's very easy to switch between them. Of course there is more upfront cost taking this approach. This is a good solution if you are distributing tools to a team of developers, you want to retain some stability in the toolset they are using, and you expect developers to be needing to switch context between projects.
Ideally, what I'd like to see happen is that when a developer signs up for a Team System project, they get their machine configured automatically with the necessary tools, and, further, that designated members of the team (which could be anyone) can go and make changes and rebuild those tools, which then get pushed out to other members of the team via Team System. When you switch to a different project, VS reconfigures itself with the tools required for that project. This won't be achieved overnight - it requires some platform changes to VS - but it's certainly on the radar.
A new sample has been published of a hosting a DSL in the VS2008 isolated shell. See http://www.codeplex.com/storyboarddesigner.
The isolated shell is a stripped out VS shell who's UI and splash screen you can customize. VS packages, including DSLs, can be hosted in both full VS and in your own customized shell without change. This sample shows how to do this specifically in the case where you want to host a DSL in a custom shell, for example to distribute to a designer to non-developer stakeholders. See the VSX development center for more details.
This sample shows a storyboarding DSL, which a product manager or business analyst might do to envision the requirements of an application. In addition to demonstrating hosting in the shell, the sample also shows a couple of cool customizations on a DSL, including a very simple approach to linking diagrams, generation of web pages for the storyboard, and export of images off the design surface.
Here's a screenshot:
Thanks to Clarius for putting this up.
Pablo has written a series of posts that should be of help to anyone thinking of hosting their applications in the VS2008 shell:
Customizing a VS Shell application
Integrating a VS Package with a VS Shell application
VS Shell application deployment
Creating a bootstrapper for a VS Shell application
VS Shell and the registry
VS Shell application switches