Keith Short's Blog

Codename “Oslo” Repository Becomes SQL Server Modeling Services

At last week’s PDC, the new name for our repository and modeling project was announced – we are the SQL Server Modeling Services!

James Baker and Shoshanna Budzianowski did an awesome job of describing the Modeling Services database and how to get value from the domain models which are delivered in the latest CTP. Check out the video of their session here. This shows the value of a queriable database of UML and Assembly metadata in realistic developer scenarios. The use of regular SQL Server Reporting Services reports and Excel to analyze and drill in to information is a direct result of making the metadata schemas ordinary relational structures using clear language from the domains. I especially like the query that returns all uses of an interface across the System.Runtime and Microsoft.Uml2 domains including a UML diagram loaded into our database from IBM’s Rational Software Architect using an exported XMI file.

Also worth looking at is Kim Cameron and Gert Drapers’ presentation of the System.Identity model which is part of the Modeling Services CTP, and will play an increasingly important role across other domains in the future. You can see their presentation here.

Get the latest Modeling Services CTP from here.

What happened to the Service Model?

I just looked at Mikael Håkansson’s  blog article on building a Web Service Model using M and the Oslo Repository (http://blogical.se/blogs/mikael/default.aspx ) It’s a very nice article. As Mikael observes, the Service Model we shipped in January was removed from the May CTP. His article correctly points out that such a model allows the repository to store all configuration information for live services.

Here’s some background as to why it was removed. First, the Service Model in M was derived from another model we shipped at PDC/January – the Application Model. Those who downloaded that CTP might recall that this model allowed a “logical” description of composable components that communicated with one another via contract-bearing “ports”. The web service model bindings and configuration information was, in effect, a platform-specific extension to the platform-independent component collaboration model.

As I pointed out in an earlier posting , we elected to move to the increasingly accepted UML2 model which offered an alternative platform-independent model of component collaboration to our original System.Application model shipped at PDC/January. Despite various well-known issues with the UML2 component/port metamodel (often fixed with specific UML2 profiles), we’ve generally had a warm reception from customers for our decision to switch to the effective industry standard.

In the May CTP, you can see the beginnings of the UML2 model we will support – in the next CTP this will cover the entire UML2 model. I fully expect you’ll see the old System.ServiceModel re-emerge as an extension to the UML2 model of components/ports either as a custom UML profile, or just a related M-based DSL, though I can’t say right now exactly when/how this will happen.

In a private communication, Mikael asked me which approach I’d favor:

A. Create a new model

B. Use the Microsoft.Uml2 model

C. Create a new ServiceModel deriving from the Uml2 model 

My first thoughts are I’d personally prefer to go with A since this would give a cleaner model with a natural representation of the concepts – I’m a big fan of custom DSLs. But given the widespread use of existing UML2 tools, I’d choose C if an organization was using existing UML2 tools that supported appropriate extension capabilities.

Oslo and UML

It’s been a while since my last posting. Why? The truth is quite simple – about five months ago I became involved in a small team building some specific content for our Oslo repository. This team was working using Scrum techniques with two week milestones. Initially I was supposed to be “part-time” on the project – a kind of advisor – as my duties at Microsoft as an Architect usually involve me in cross-divisional work, “strategic” projects, and early-stage prototypes. But I quickly realized that it’s hard to be a useful member of a fast moving multi-disciplinary team working on tight schedules if you are not completely engaged. Consequently I have thrown myself right into this project, and have had a blast ignoring all other duties just to get something done, and have fallen right back into check-ins, code reviews, triaging, and bug debts. Not to mention working long hours and weekends – the not-so-fun partJ.

Now I can reveal that the “specific content” we’ve been working on is an M implementation of the UML2 (version 2. 2) metamodel – which will be released as part of the Oslo CTP to be made public next week. We’ve taken (a subset of) the OMG’s specifications for UML2 (available from the OMG site here), used the M language to represent the classes described in the specification, created M extents using a mapping of the classes to extents that I’ll describe in more detail later, and created storage structures in the Oslo repository into which we can load UML2 models using a loader that is also part of the upcoming CTP. I’ll give a few more details of the technical specifics in a moment, but some readers may be already asking the question Why would the Oslo team do this?

Well, there are a couple of very good reasons why we should place a bet on UML in this way:

1.       Our customers have told us that increasingly UML2-based tools are used as part of their development processes. The usage varies from non-rigorous drawings used for communications, to rigorous specifications used in sophisticated ways as part of various model-driven development methods. Often tools from several different vendors are used, and specifically in the case of UML models used in model-driven development, it is challenging to impose standards on their use, to ensure UML elements are reused consistently across models, and to conduct analyses (such as conformance to the UML specification) upon them.  By placing these models into a database, in our case the Oslo repository, we can address all of these challenges. Once loaded into the repository, customers can search, analyze, query and add linkages between elements in various models. Our colleagues in Visual Studio have of course also responded to this customer demand, and are in the process of adding to Visual Studio Team Architect (Dev10) a set of UML2 modeling tools. Once these tools are released, we plan to have loaders to allow VSTA customers to load their UML models into the Oslo repository, and we’ll commit to supporting the main variations of the industry-standard XMI format for loading models created with third party tools into the Oslo repository.

 

2.       UML2 contains a collection of specific sub-models which address many of the needs of analysts, architects, developers, testers and IT staff. In early releases of the Oslo CTP, we had provided M models for some of these domains (Composite Application Structure, Workflow, etc.), which were a mixture of Microsoft platform-specific concepts and general platform-independent concepts. We provided these models specifically to encourage model-driven development upon our platform. But again, feedback from our customers led us to understand that where possible, they would like to use UML2 for concepts that are defined in the OMG specification or easily extended.   Consequently we have decided to double-down on our provision for UML2 as a set of modeling concepts. Naturally these concepts may then be inter-related as required to other non-UML2 domains represented as DSLs or overlapping domains already covered by DSLs.  In any case, once we have the UML2 and non-UML2 information in the Oslo repository, other tools such as model-to-model transformation tools and code generators now have a common place to look up and insert UML model elements. Runtimes directly targeting the repository may utilize metadata irrespective of which modeling standard was used to create it.

Some folks may be surprised by a perceived about-turn towards UML2. We, as have many other industry thought-leaders, have criticized UML for its complexity and extension mechanisms. I’ve seen its specification reviewed as anything from an over-complex, compromise-laden monstrosity to a pinnacle of human achievement. Suffice to say that having spent the last few months deeply immersed in it, my judgment tends to the latter. Sure the specification has bugs and will improve in places in subsequent revisions, but the bulk of the concepts are proving themselves useful and workable as evidenced by a growing number of tools, and as variants of XMI reduce in number, interoperability between tools will increase.

So, back to the Oslo CTP. The UML2 model we have provided is a subset of the full specification. There are whole domains missing, and domains present are not complete. However the loader we have provided can load a restricted family of XMI artifacts into the repository. Specifically, only a few of the specification’s OCL-based constraints are represented in the M source. When we finally ship Oslo, we intend to have the whole model built to OMG Level 3 compliance. Our main goal is to seek feedback and comments on our approach and garner scenarios from early reviewers that can help us shape the product’s future.

Anyone who already knows the UML2 specification in detail will understand the complexity involved in translating such a complex, intertwined class model that relies heavily on long trails of multiple inheritance into essentially a relational style model using a language such as M. We have developed a set of M patterns which we apply to patterns of UML model elements (related to industry standard approaches for object to relational transformation) which we’ll publish in further articles and blog entries. Further details of the UML2 Loader will also appear subsequently.

We await your comments. Many thanks.

 

Oslo Makes the Top Ten

We’re just getting back into the swing of things after a couple of weeks break over the Christmas holidays, not to mention the disruption of unusually heavy snow followed by floods here in the Seattle area. Two articles caught my attention this week.

One, an article in eWeek by Darryl Taft, positions Oslo at number 9 in the Top Ten Development Products for 2008 (even ahead of the iPhone SDK). Also, in the text version of the story, Oslo makes it to number 2! Granted, in this latter story, Darryl makes more of the emerging relationship between Microsoft and the OMG (though I’d push back against the “indifferent if not hostile” comment).

It’s the text version of the story that links to my second article, a blog posting by Charles Young called Oslo and the OMG. This is an amazing article, a great summary of OMG technologies and some erudite interpretation of our Oslo strategy gained from the PDC bits and such information as we’ve made public. Charles pieced together some useful comparisons between the OMGs layered metamodel stack (M0, M1, M2, etc)and MSchema, as well as relationships between software factories and MDA, and highlights some shortcomings of the previous Microsoft Repository (he missed a few critical ones J ) versus the Oslo Repository approach.

As we firm up our Oslo ship plans, we’ll have a lot more light to shed publicly about Oslo and its relationship to UML and other OMG technologies, but for a summary of the key issues, Charles’ article is well worth reading.

Oslo and Software Factories

I’ve had a few people ask me about the connection between the Microsoft Software Factories initiative and Oslo. First, the SF initiative is alive and well and a good summary of the current state appears here. I just had a demo from Jack Greenfield’s DPE team of the new Blueprint technology which is very exciting and I think will be the basis for renewed attention to the ideas around software factories we first described in the book.

Since Oslo is a technology platform addressing a way to improve development using frameworks, DSL’s and models you’d expect us also to be considering the other limb of the software factory idea – guidance and development process.  You’d be right. I can’t say at this stage when we’ll be able to deliver Oslo bits that support something akin to the Blueprint concepts, but the attraction of being able to store the guidance metadata in the repository is very strong. Since a software factory schema is a schema of all the information, tools, DSLs and other resources and processes required to build a family of like applications (a family having similar core architectural features and patterns), using the repository as the store and advising a Quadrant developer using an embedded guidance framework would be very cool.

Comments on Communication Between Doug Purdy and Lars Corneliussen

Doug has published a response to an Open Letter addressed to him from Lars Corneliussen.  I think Doug did a good job in addressing Lars’ concerns, and where not, to invite discussion.  I wanted to add a couple of comments of my own.

First: terminology connected with modeling, code generation and runtimes. Given my background in various modeling efforts at Microsoft over the last ten years, I know of the difficulty in seeming to redefine established terms. We faced a lot of the same concerns when, in 2004, we tried to help people understand the subtle distinctions between modeling with DSLs and using UML.  As history showed, in many cases we were not very successful then, though these days (see several entries on Steve Cook’s blog especially this one and this paper from Andrew Watson, Technical Director at the OMG), most people are willing to see how both approaches may combine to bring benefits to developers across the lifecycle.

I for one have been using the term “model-driven software development” (or just “model-driven development” for short) for a number of years (actually going back to my pre-Microsoft years at Texas Instruments during the era of CASE tools). When Jack and I wrote the Software Factories book, we used the term model-driven development to encompass both “model-assisted” and “model-driven” as Doug uses those terms. We  saw then, and still do today, that MDSD could involve models that transform to other models, which transform to code and which just “complete frameworks” by transforming into whatever is necessary (including no transformation) to drive a framework at runtime. For example, if I use a transform to build a logical data model, several workflows, and some service descriptions from a set of business process models, I think I’m doing more than just drawing, and I may be doing some code generation and model generation. Which of the three terms best describes this activity?

 As I explained in my previous blog entry concerning the DSL Toolkit and Oslo, we are trying to help folks understand the specifics of Oslo in its first incarnation, with respect to pre-existing technologies from Visual Studio (and other parts of Microsoft). This leads us to seek terminology that helps us conduct that discussion, and in cases such as this, involves some subtle distinctions that are tough to differentiate. As Doug says: “when you a birth a new product, naming/terminology is often the most difficult aspect of the process”.

Second: Open Microsoft and Eclipse Modeling Project. I second Doug’s remarks wholeheartedly. I’d especially like to see discussion on relationships between EMF and oAW technologies.

Oslo and the DSL Toolkit

I’ve received a number of questions concerning the relationship of Oslo to the Visual Studio DSL Toolkit.  Comments have varied from the sublime (see David Ing’s comment on his blog “My best guess is that the DSL Toolkit is research road kill in front of the big Oslo truck and that VSTS Architecture Edition was just about a necessary cycle too early”) to the ridiculous. Since I’ve been a founder member of both projects, I thought I’d try to start the discussion with a few comments of my own.

Firstly, it’s true that the two efforts are built on different technology stacks – the DSL Toolkit works on file-based artifacts (schemas, model instances, etc.) and produces graphical and forms-based tools that run as add-ins to Visual Studio – dramatically simplifying the task of creating tools hosted via VS extensibility.

On the other hand, Oslo is based on an underlying SQL database. Quadrant depends on the underlying database for both the data it is processing and its own configuration data. In other words, Quadrant’s equivalent of the DSL Tools’ domain modeling language and the shape and shape mapping languages is MSchema. Concrete textual languages are of course defined in MGrammar.

Despite these differences, two things need to be made really clear:

1.       Both the Oslo and the DSL Toolkit have grown from a common belief in the role DSLs can play in the development lifecycle. Not just during development, but DSLs that help record Business Objectives, Business Processes and Entities, System Architectures, Software components and connections, Deployment Information, Data Center Configuration, and System Management to name just some of the lifecycle stages. This is a shared vision, well documented elsewhere, though each project has focused on a different aspect initially. The DSL Toolkit builds great graphical (box and line) tools that run in Visual Studio and may be translated into code-based artifacts. Oslo is focused on textual and graphical developer experiences around models that initially represent code and configuration that “completes” the underlying frameworks that are part of the application platform – in other words – models that are mostly executable by the underlying servers and frameworks (e.g. WF, WCF, and Identity Services).

 

2.       Both products have a lifecycle in front of them. The two teams, already aligned around vision, are working together to bridge differences over releases.  Would it have been nice to have gone dark for a period while we resolved these technology stack issues and re-emerged with a fully aligned set of technologies? You bet – but such a strategy rarely ends with the right thing being built for customers. Ideas we are tossing around include (a) storing DSL Toolkit artifacts (including those created with the emerging UML tools from VSTA) in the Oslo Repository, (b) using MSchema as the domain language for the DSL Toolkit, and (c) converging on a single way to specify concrete DSL syntax whether it is graphical or textual. Sadly, I can’t give dates at this point.

Stuart Kent is the architect for DSL Tools. If you take a look at Stuart’s blog, you’ll find the latest blog entry where he responds to the same questions from his point of view.

 

Oslo at the PDC

The PDC is now over and by all measures has been a great success. The Oslo talks have been received with great enthusiasm. Any one not following the activity on our MSDN forum who missed the sessions might want to look at this entry on the forum that summarizes the links to our various sessions.

Also, the main Oslo site is now active at http://msdn.microsoft.com/en-us/oslo/default.aspx and of course don’t miss the fun video at the site http://modelsremixed.com .

Yet Another Promise to Start Blogging !!

Yet again, it’s been a long time since I wrote to my blog. This posting is my restart.

 

I know, you just have to look at the last posting of over two years ago to realize I’ve said that before!

 

But here’s what happened. Shortly after writing that last posting, I moved from Visual Studio to join the group that was working on the earliest ideas that have now become known by the codeword “Oslo”. Obviously we couldn’t talk about this work, but now, at last, we are about to make the project public in a *big* way at next week’s PDC. You just have to look at blogs such as Don Box’s or Doug Purdy’s to realize “Oslo” is Microsoft’s modeling platform consisting of a new language for describing models and DSLs, a database in which to store models of various kinds, and a new tool to view and edit them.

 

I’ve mostly been working on the design of the model store (a SQL Server database) and using the new language to create schemas for the new store, but I’ve been able to bring a long background in model driven development, software factories and DSL tools to help with the design and testing of the language and tool.

 

For me this is very exciting since the “Oslo” investment represents a serious commitment from Microsoft to change the game and make modeling mainstream. That’s almost been the story of my life!

 

Once the excitement over the various announcements that will be made at the PDC subsides, I’ll try to give more background from my perspective, and begin the task of helping folks understand exactly what we are doing in this space.

 

Recent Postings by Enterprise Tools Architects

I’ve been lax with my blog for the last two months. I plan to do better this year! During the last two months, we have of course shipped Visual Studio 2005 so our modeling tools for developers and architects finally are in the hands of customers. This has been very exciting!

 

There are a couple of recent postings by architects in the team that I wanted to advertise.

 

Firstly, Bill Gibson, a long time colleague of mine both at Microsoft and before, has written a set of Technical Notes which I think offer invaluable advice to users of our tools in Visual Studio Team System 2005 edition for Architects. Among other helpful insights, Bill describes ways to use the tools that many users will find helpful. You can find these indexed on Bill’s blog here.

 

Also, in response to a question posed by Tad Anderson about our DSL strategy, VSTS and Software Architecture (posting is here), Jack Greenfield wrote a great summary of our Software Factories approach that is embellished with several up-to-date links to various podcasts, papers and other references. I can definitely recommend reading Jack’s posting here.

Software Factories at OOPSLA 2005

OOPSLA 2005 in San Diego was a fun event. It was great to chat with lots of old friends and sit in on some interesting sessions and panels. Our team was represented at a number of events. Jack Greenfield and Steve Cook presented a one day tutorial on Software Factories using some updated material from last year, and showed a fewof interesting demos. One, given by Mauro Regio, showed Visual Studio 2005 configured with various assets to help developers build collaborating services based on the HL7 (a set of standards for B2B communication in the Health Services Industry) software factory I talked about in a previous post which you can read about here. Steve also demoed the latest builds of the DSL Tools.

Also, we attended the Software Factories Workshop, a one-day event which Jack chaired. This was voted a great success by those who attended and gave us all a chance to drill into a bunch of interesting topics around model-driven development, DSLs and software factories. The papers which were reviewed and accepted by the review committee may be read here, but will be published in some form, yet to be decided, as soon as possible along with the proceedings of the workshop. I'll point to those proceedings when they're ready.

Windows Workflow Foundation

After the announcement last week of this great piece of technology, we can finally all start to talk about it. Although you can code directly against the WWF, it contains a nice example of a couple of graphical DSLs that allow the developer to declaratively describe workflows consisting of human and system activities. The lead architect for WWF, Dave Green, has started a blog here, but you'll also want to check out a great article written by David Chappell published on MSDN here.

Steve Cook Interview

Back off three weeks vacation always causes severe "re-entry" problems! But digging myself out of the email stack I noticed Steve Cook has posted a very good interview with Roy Osherove on Software Factories and DSLs. You can download the interview as an MP3 from here.

Steve explains, among other things, the relationship between the DSL Toolkit and the Guidance Automation Toolkit, both of which can be downloaded from the MSDNCommunity Technology Preview site for Visaul Studio Team System.

Software Factories & DSLs

A couple of interesting items have appeared recently that will be of interest to folks following our stories on Software Factories and DSL’s.  I had posted earlier some examples of Software Factories that generated quite a bit of comment from readers. We have had a team of people working on a specification for implementing business-to-business interactions under HL7 (Health Level 7) – a set of standards for B2B communications in the Health industry. Since building these kinds of systems uses common design patterns, common tooling, and the implementations share a common architecture across a diverse range of applications and scenarios, they are ideally suited to a Software Factory approach.  You can read about this on MSDN here.

 

Renowned author and industry leader Martin Fowler has published an excellent set of papers describing the use of DSLs in the development process. The starting point is a paper entitled “Language Workbenches: The Killer Application for Domain Specific Languages”.

 

I thoroughly recommend the article.

Microsoft's Modeling Strategy

My team has written a document that explains Microsoft’s Modeling Strategy and addresses a number of FAQ’s we get from customers and partners. The outline is:

Why Modeling?
How Are DSLs Used in Model-Driven Development?
What About UML?
What About MDA?
What Are Software Factories?
Other Frequently Asked Questions

Astute readers of my blog may even recognize parts of this document that have appeared in earlier postings. That’s a great value to us of blogs – getting feedback on how we communicate products and strategies so we can improve our ideas.  We’d love to get feedback on this document too.

 

Steve Cook posted these links too, but I do not suffer the same anguish that he does over British/American spelling. I guess I’ve lived in the USA for too long now to continue worrying about that, and happily type ‘z’s instead of ‘s’s with hardly a thought.