VS2008 and VS2008 SDK have just shipped. The announcement about the SDK over on the VSX blog didn't get into much detail about changes in DSL Tools for VS2008, so I thought it would be worth summarizing them here:

• The runtime is now part of the VS platform. There is no need for a DSL author to redistribute a separate DSL runtime.
• The authoring experience now works with Vista RANU (Run as Normal User). You don’t have to run VS as an admin in order to develop a DSL.
• We’ve added a path editor to the DSL Designer. Paths are used largely to express mappings between domain classes / relationships and shapes / connectors. The new editor provides UI that allows you to select valid navigation paths through domain classes and relationships in the domain model.
• LINQ can be used in code written against the code generated from a domain model, for example when writing validation constraints or text templates for code generation. For those of you familiar with UML / MOF, LINQ gives you OCL-like querying capability (actually its far more powerful than that) against domain models.
• We’ve fixed a large number of bugs, in both runtime and authoring.

I promised a while ago to publish a roadmap for what we're doing with DSL Tools, post VS2008. Now that VS2008 and the VS2008 SDK have just shipped (thanks Gareth for providing a post which points at both announecements) now seems a good time honour that promise.

There are two main themes in how we expect DSL Tools to develop in the future:

• Evolve the graphical designer and code generation platform to provide more capabilities, to integrate better with other parts of the VS platform, to componentize it so it is easier to choose which parts you want to use, to better support extensibility and customization, and to improve performance.
• Generalize the authoring approach pioneered in DSL Tools, to the creation and maintainence other kinds of Visual Studio extension, and integrate the DSL authoring experience with that. These are the tools to build tools that I've used as a tagline for my blog, although, in the context of Visual Studio, VSX Tools might be a better term.

I mentioned the second theme soon after the DSL Tools team joined the Visual Studio platform team. That is still in our sights and will be part of our roadmap for the VS SDK as a whole, which we'll publicize when it's ready. For the remainder of this post, I’ll focus on the first theme.

Roadmap for VS codename Rosario

Below is a list of features we have in the backlog for Rosario. We’re not promising to complete all of them, but expect to complete a significant subset. Some of the features may appear earlier in a service pack for VS2008.

• DSL extensibility. This enables you to extend a designer that has already been deployed – i.e. without cracking open the source. An extension defines a set of domain class extensions (adding new domain properties, new relationships, new domain classes), which appear on elements of that domain class when the extension is enabled. An extension also defines a root domain class and a scope. You activate an extension on an element of the root domain class, and then all elements in scope also get that extension enabled. Extension activation is dynamic through the UI of the designer (we’ll provide some standard UI, and the base designer author will be at liberty to write their own custom UI). Multiple extensions may be activated on the same element. The root domain class for an extension may be the root of the domain model of the DSL being extended, which then allows an extension to apply to the whole model. Extensions may also define additional validation constraints, in the usual way. I’ll write a separate blog entry on this with more detail a bit later. The authoring experience for extensions will require support for referencing one dsl definition from another, which will also benefit customers who want to build a set of designers that share some parts of their definition (e.g. a common core set of domain classes).
• Support for databinding of domain model to Winforms. Although it is possible to create a custom editor with a placeholder for you to write your own forms-based editing surface, and it is also possible to add new tool windows to supplement the graphical design surface, there is currently no support for easily binding the form to underlying model elements. This feature involves generating the appropriate APIs from a domain model so that the standard Winforms databinding experience can be used.
• Nested shapes. Although there is some support for nested shapes, it is harder work than it should be to get them to work. This feature involves improving the overall experience and fixing some bugs in the runtime which currently need to be worked around.
• Various fit & finish items, for example:
• Print preview. Enable the print preview menu for designers built with DSL Tools, so that you do not have to print to a file in order to preview the page layout. Page setup already provides some control on how to layout the pages.
• Search and replace. Enable search and replace to work with designers built with DSL Tools. You should be able to search for a text string and replace that string throughout the model.
• Sticky tools on the toolbox. Once a tool is sticky, you don't need to keep going back to the tool to use it repeatedly on the design surface.
• Standard layout options. The underlying framework supports a number of layout options. This feature would surface it through some standard UI in the generated designer.
• Pan/zoom control. A pan/zoom control provided as part of the standard UI in the generated designer, rather than having to custom code this as you do now.
• Provide better support for orchestrated code generation. At present we have a button in the solution explorer for executing all templates in a solution. It is possible, with a little custom code, to hook templates up to a menu command (there's an example in our book), and then write code which does things like apply the same template multiple times to a model, generating a new file for each application. One option for improving this (though we're not fully decided yet) is to hook code generation into the build system, using MsBuild to coordinate it, and make it easy to write custom build actions that do the kind of things you can do when you launch code generation off a menu command. 

Post-Rosario

The two main investments we’re considering post-Rosario are:

• WPF-based design surface. We'd like to replace the current graphical design surface with one founded on WPF (WIndows Presentation Foundation). Not only will this open up many new possibilities in the quality of visualization that one can achieve, we also believe it will make it much easier to customize the graphical notation. At the same time, we'd take the opportunity to componentize the architecture, so that it would be possible to bind a graphical design surface built on top of this component to other data sources, such as XML or a database.
• First class support for cross-referencing between different models and multiple views. This has been requested many times by customers, although it's also interesting to note that many customers have been able to achieve their scenarios by writing custom code.

Well, that's all for now. Please provide your feedback on this. What do you like? What don't you like? What's on your wish list?

Yesterday I posted about VSX talks at TechEd Europe, including one I'm giving on DSL Tools.

I forgot to mention a couple of other talks involving the use of DSL Tools in software factories. Jezz Santos gives the details. Unfortunately Jezz can't be there himself, but Don Smith, from patterns and practices, will be giving the talks. I had the opportunity to work with Don a few months back on an assignment, and the work his team has been doing on a modeling version of the service factory is impressive. If you want to see a very real, practical application of graphical DSLs in taking some of the grunt out of the development process, then Don's talk on the service factory is a 'must see'.

I'm speaking at TechEd Europe in Barcelona on DSL Tools. Go here and look me up under the list of speakers to see details of my session on DSL Tools. There's also a talk from the VSX (VS Ecosystem) team on the new VS Shell given by James (including a demo of hosting a DSL in that shell), and James and Steve are participating in a Q&A session on customising Visual Studio. The ecosystem team will also be running a booth on Visual Studio Extensibility.

I'm arriving Sunday night, and leaving Friday evening. Come to the booth if you want to talk. See you there.

I've recently received a number of questions about DSL Tools, making me realize that our messaging about the future of this technology has not been very clear. One such question has just appeared on the DSL Tools forum, so I've spent a little time replying to it - take a look here. I've restated the important bit of my answer below:

"DSL Tools has recently moved from Team Architect to the Visual Studio platform, together with some of the team that developed it. This has been announced in various places including on my blog.

We are still adjusting to the move and producing a roadmap that makes sense in the new context. Please be patient for a few more weeks. I'll begin by publicising what we're doing to support Visual Studio Team System codename 'Rosario'. For VS2008 (Beta 2 available), the main change is to move the runtime into every Visual Studio box, including the new VS2008 shell (redist of run time no longer required). We've also fixed bugs and improved the path editing experience in the Dsl Designer."

See Gareth's post for details.

I was sent email by Alastair McKeeman from Connective Logic Systems, a UK based company.

They've done some cool work on tools to build concurrent apps targeted at multi-core processors. They have a graphical designer hosted in Visual Studio, implemented using DSL Tools. Take a look at http://www.connectivelogic.co.uk/toolset.html which contains a screenshot. They've also built graphical analysers and debuggers.

Fascinating stuff.

That's the title of an article in e-week reporting on an interview with Soma. Thanks to Gareth and Pedro for pointing this out.

When I joined Microsoft four and half years ago, I left a career as a reasearcher in model driven development to join a company that was not recognized as a player in this approach to software development. Now, as Soma points out, there are a range of initiatives across the company looking at model driven development. It wil lbe an exciting ride ahead...

One of those initiatives is Software Factories (mentioned in Soma's article), and DSL Tools has been instrumental in supporting that vision - see for example the web services software factory, modeling edition, as described in this article by Don.

As mentioned in my earlier post, DSL Tools, and our team, have moved into the VS Platform, and it's great that we are now in a position to support the adoption of this technology by more teams inside Microsoft, as well as continued adoption by external partners and customers. We're working on our roadmap, but the general direction is to continue to evolve and enhance the DSL Tools to support richer modeling scenarios, and at the same time generalize our model-driven approach to creating designers (the tools in DSL Tools are built using DSL Tools) for creating other kinds of tools and extensions to Visual Studio, not just modeling tools.

James and Gareth have posted about the move of the DSL Tools team to VS Ecosystem team.

Gareth provides some of the reasoning behind this move. Here's a bit more.

One of the most innovative aspects of DSL Tools is the model-driven approach we take to building a graphical designer which brings the cost of creating such tools to the point where it is cost-effective to invest in building the tools within the first project that they are used. This move opens up the opportunity to broaden this approach to creating other kinds of Visual Studio Extensions. I'm looking forward to the challenge, and to mark this broadening of focus, I've provided a new one-liner to describe my blog: tools to build tools. If you have ideas of software development tools you'd like to be able to build on top of Visual Studio then I'd love to hear from you. Just send email or add a comment.

The move also makes it easier for us to exploit the new VS2008 shell, in providing a way to deliver DSL-based tooling to folks involved in software development who are not in the engineering role, and so don't need or want all the capabilities that Visual Studio has to offer. This has been requested over and over by customers, and it's great that we'll now be able to address that need.

And, of course, we'll be using DSL Tools ourselves to build the VSX authoring tools: there's nothing like dogfooding your own stuff to drive improvements! 

I'm also pleased that I still get to work with friends in Team Architect albeit in a different capacity: they are building some exciting products using DSL Tools, and we'll be making some enhancements to help them. Watch this space.

As mentioned in this earlier blog post: http://blogs.msdn.com/stuart_kent/archive/2007/06/01/talking-at-dev-days-in-amsterdam.aspx

The presentation is at http://download.microsoft.com/download/1/9/5/195c1f00-bc52-44d8-9950-a00b4a7bc751/Track06__%2014.track6_sessie5_16h30_Stuard_Kent.pdf

I'm giving a talk on Extending Visual Studio with tools for Model Driven Development at Dev Days in Amsterdam on Thursday June 14th.

I should be there for the whole day on Thursday (my talk is at the end of the day), so if you want to meet just contact me through this blog.

I'm looking forward to meeting the vibrant Software Factory community in the Netherlands, just outed by Jezz.

Olaf has an interesting post on how you can use fonts as shapes in a DSL Tool using a little bit of custom code. Here's the screenshot:

Jezz has just announced the release of the next milestone in this project. To quote:

<quote>We released the next milestone of the DSL Editor PowerToy today on CodePlex.

"A means to add multiple editors to your DSL, either hosted in a tool-window, or replace the graphical designer of your DSL"

</quote>

Thanks Jezz for implementing my suggestion of allowing the editors to be hosted in a a window that replaces the graphical design surface. When this powertoy is complete, it will be really easy to produce a forms-based DSL for languages which either don't suit a graphical representation, or where the effort of designing the graphical syntax is deemed not worth the investment. That's in addition to using the tool window in conjunction with a graphical surface. I'm really looking forward to the next milestone where we start to see some generated UI for the content.

Also, related to my last post, it's great to see Jezz exploiting our approach of generating code from a dsl definition then allowing that code to be further extended through custom code. This has allowed Jezz to look for patterns in custom code you would need to write to implement his editors, and then add new code generators (and, his case, a new DSL as well) to generate to those patterns. This allows the capabilities of the tools to be extended by third parties, to cover all those bits we haven't thought of yet or haven't had time to implement.

I see that Steven is having another go at Microsoft's DSL Tools.

Not for the first time, my impression is that he makes it look worse than it actually is. To quote:

"As long as Microsoft thing of building modeling languages as a programming project, it's going to be impossible for them to make the changes they need to, without breaking existing modeling languages built with their tools. If they have the possibility to make major improvements to DSL Tools, my suggestion would be to go right ahead: the important market is those who have not yet been reached. Those brave enough to use CTPs, betas and version 1.0 knew what they were letting themselves in for.

This is of course a problem with tools that require programming to build modeling languages, or a separate compilation step between the definition of the metamodel and using it to model with. In MetaEdit+, the metamodel is specified directly in the tool, with no programming, and models are made based directly on it, with no intermediate compilation. In other words, the metamodel is expressed as pure data, which makes both metamodel updates and tool updates easy and painless for the users."

In DSL Tools, a large proportion of the code for a designer is generated from a dsl definition. If we make changes to the code generators, then to migrate to a new version is a matter of regenerating the code. If we make changes to the designer definition, then, in the case where those changes are not pure extensions to the designer definition language, we can provide a tool to migrate existing definitions across to the new format. If the designer author has written custom code to finish off their designer, then we could break that custom code if we make sigificant changes to the APIs it is written against, although there are various techniques that can be used to mitigate some of the pain (for example, deprecating APIs over versions). Now, as we add more features to DSL Tools, we will do our best to reduce the amount of custom code that designer authors will feel the need to write, but I think we'll always want to provide that outlet, as authors will always want to push the boundaries of the designers they wish to create beyond what we have thought about in advance when designing the dsl definition format.

Now, I wonder what happens in the MetaEdit+ case. If the format in which the metamodel (dsl definition) is expressed changes (which it might if they want to add new features), then there has to be code somewhere which migrates metamodels expressed using previous versions to the new version. And I don't see how this is any different to us providing a tool (which may be integrated with the DSL designer itself) that migrates a dsl definition expressed in previous versions of the dsl definition language into newer versions. The difference in the two situations is that with DSL Tools, the extra custom code needs to be converted, but I wonder how much effort that really is, given appropriate guidance and the fact that significant changes to the APIs are not exactly frequent occurrences.

Our book on DSL Tools is now available. Available at amazon and other online stores. You can also read it online at http://safari.informit.com/9780321398208.

BTW, if you're interested in meeting with the authors, then you can find Gareth at Tech Ed in the US, Steve at Tools Europe, and me at ECMDA and Microsoft Developer Days in the Netherlands.

A version of the VS SDK (download from http://www.vsipmembers.com/), which includes DSL Tools, is now available for Visual Studio Orcas Beta 1. Gareth has the details.

There's a nice post from Steven Kelly which contrasts different ways of interacting with a DSL in a tool.

I say 'interacting with a DSL', to stress the difference between that and working with a static syntax on a piece of paper. Working through a tool provides you with many more options not only in how you view and browse expressions in your language, but how you create and update those expressions.

However, I'd like to pick Steven up on a couple of points:

• He says that graphical languages are much slower for data entry than text. Whilst this has been true in the past, I'm wondering if it will remain so. I think that some of the mindmapping tools out there are showing that you can achieve rapid data entry through a graphical surface by minimizing mouse movement and using keyboard shortcuts much more.
• We've also found that there is often rich data in a DSL that is not amenable to graphical representation (or at best adds considerable clutter to the diagram, thereby reducing it's value) and also isn't that well served by an explorer (tree view) and/or properties window. In these case, you really want to design domain specific forms for dealing with that data. So it's nice to see Jezz kick off a project along those lines: check out http://www.codeplex.com/dsltreegrideditor.

So I've not been blogging for a while. Sorry about that.

What's my excuse? Well, I've been writing a book with my colleagues Steve Cook, Gareth Jones and Alan Wills.

And, yes, they've kept up their blog, so it's not a very good excuse really (though book and five children doesn't leave much time for anything else).

I've also been tagged now by both Steve and Gareth. Gareth tagged me a while back, which obviously failed to wake me from my slumber, and Steve has just tagged me. Now, as Steve says about himself "I never do chain letters", but I quite like the idea of telling you a few things about me you might not know. I won't tag anyone else, mostly because I see that those people I would have felt comfortable tagging have already been tagged.

1. I'm married with five children and live in a converted barn on the top of the North Downs near Canterbury in the UK.
2. I sing (more like used to sing) counter tenor in church choirs. Well, I guess it was a little more than church choirs. I was a choral scholar at Exeter cathedral whilst at University there, and then a Gentleman of Her Majesty's Chapels Royal in St James's Palace, London, whilst I studied for my PhD at Imperial College. There are two chapels in St James's palace: the Chapel Royal and the Queen's Chapel. I was married in the Chapel Royal. Whilst there, I also attended one of the Queen's garden parties at Buckingham Palace, and got to sing at a number of Royal events. 
3. My first computer was a Sinclair ZX80 that my Dad built. My next was a Sincalir ZX81 that I bought myself (I remember spending a week in the sanatorium at school reading the manual just after I got the computer - I wasn't allowed to take the computer in). I learnt Z80 assembly code programming on the ZX81 in an attempt to build a arcade game. My third computer was a Grundy New Brain - a very rare thing indeed as the company wnet bust soon after I bought it. I still have the ZX81 and New Brain boxed away somewhere.
4. My first programming job was in a robotics firm based in the village in Hampshire where I was brought up. I wrote Z80 assembly code for the controller of a steel rod cutting machine. Now that was fun.
5. I went to a public boarding school (a 'private school' is called a 'public school' in the UK, and a 'public school' is called a 'state school' - don't ask me why) from the age of 8 to 16, when I left to go to university. There I learnt to play lots of sports - football, rugby, squash, cricket - though was not particularly good at any of them, although I did enjoy trying. I did get into the school cross country running team, though, and still enjoy pounding through the mud when I can find the time. And I was part of the choir that won the first ever National School Choir Competition.

Gareth has the low-down: http://blogs.msdn.com/garethj/archive/2007/03/02/visual-studio-sdk-v4-and-dsl-tools-update-is-released.aspx

This is a bug-fix release for DSL Tools, and the final SDK release for VS2005.

Yes it's true - we've shipped!

The DSL Tools web pages will be updated shortly, with more details about this V1 release. For those who can't wait, the VS2005 SDK V3 containing DSL Tools V1, can be downloaded from http://affiliate.vsipmembers.com/affiliate/downloadfiles.aspx.

..at the link http://msdn2.microsoft.com/en-us/library/aa396774.aspx

This is actually the documentation for our version 1 release which will be available soon as part of the Visual Studio 2005 SDK version 3 release - see http://blogs.msdn.com/stuart_kent/archive/2006/08/31/733372.aspx.

Note that the documentation isn't finished - we have a dedicated team still working on it - so expect further documentation releases until the end of the year.

Having just returned from vacation, I thought I'd update folks about the V1 release of DSL Tools. This will be shipped as part of the Visual Studio 2005 SDK Version 3 in the first part of September.

We have signed, sealed and delivered our code to the VS SDK team who are now just wrapping up. There will be a significant new documentation drop at the same time, although we will continue to update the documentation until the end of the year.

Apart from bug fixes, the main feature in this release is a completed Dsl Designer for editing .dsl definitions.

I'll post again as soon as the toolkit has been released.

A while back I posted about how useful I find storyboards in the specification process.

Now there's a tool integrated with VSTS for doing storyboarding. Check it out at http://www.stpsoft.co.uk/story/.

Here is a great post from Jos Warmer over on the DSL Tools forum.

He makes some good arguments. Please add your own opinions to the discussion. We'll definitely take note as we think about the next set of features...