Keith Short's Blog

Codename “Oslo” Repository Becomes SQL Server Modeling Services

Keith Short — Mon, 23 Nov 2009 19:33:00 GMT

At last week’s PDC, the new name for our repository and modeling project was announced – we are the SQL Server Modeling Services! James Baker and Shoshanna Budzianowski did an awesome job of describing the Modeling Services database and how to get...(read more)

What happened to the Service Model?

Keith Short — Fri, 10 Jul 2009 01:17:00 GMT

I just looked at Mikael Håkansson’s blog article on building a Web Service Model using M and the Oslo Repository ( http://blogical.se/blogs/mikael/default.aspx ) It’s a very nice article. As Mikael observes, the Service Model we shipped in January was...(read more)

Oslo and UML

Keith Short — Wed, 27 May 2009 00:10:00 GMT

It’s been a while since my last posting. Why? The truth is quite simple – about five months ago I became involved in a small team building some specific content for our Oslo repository. This team was working using Scrum techniques with two week milestones...(read more)

Oslo Makes the Top Ten

Keith Short — Sat, 10 Jan 2009 00:29:00 GMT

We’re just getting back into the swing of things after a couple of weeks break over the Christmas holidays, not to mention the disruption of unusually heavy snow followed by floods here in the Seattle area. Two articles caught my attention this week....(read more)

Oslo and Software Factories

Keith Short — Mon, 08 Dec 2008 22:19:00 GMT

I’ve had a few people ask me about the connection between the Microsoft Software Factories initiative and Oslo. First, the SF initiative is alive and well and a good summary of the current state appears here . I just had a demo from Jack Greenfield’s...(read more)

Comments on Communication Between Doug Purdy and Lars Corneliussen

Keith Short — Tue, 18 Nov 2008 22:37:00 GMT

Doug has published a response to an Open Letter addressed to him from Lars Corneliussen. I think Doug did a good job in addressing Lars’ concerns, and where not, to invite discussion. I wanted to add a couple of comments of my own.

First: terminology connected with modeling, code generation and runtimes. Given my background in various modeling efforts at Microsoft over the last ten years, I know of the difficulty in seeming to redefine established terms. We faced a lot of the same concerns when, in 2004, we tried to help people understand the subtle distinctions between modeling with DSLs and using UML. As history showed, in many cases we were not very successful then, though these days (see several entries on Steve Cook’s blog especially this one and this paper from Andrew Watson, Technical Director at the OMG), most people are willing to see how both approaches may combine to bring benefits to developers across the lifecycle.

I for one have been using the term “model-driven software development” (or just “model-driven development” for short) for a number of years (actually going back to my pre-Microsoft years at Texas Instruments during the era of CASE tools). When Jack and I wrote the Software Factories book, we used the term model-driven development to encompass both “model-assisted” and “model-driven” as Doug uses those terms. We saw then, and still do today, that MDSD could involve models that transform to other models, which transform to code and which just “complete frameworks” by transforming into whatever is necessary (including no transformation) to drive a framework at runtime. For example, if I use a transform to build a logical data model, several workflows, and some service descriptions from a set of business process models, I think I’m doing more than just drawing, and I may be doing some code generation and model generation. Which of the three terms best describes this activity?

As I explained in my previous blog entry concerning the DSL Toolkit and Oslo, we are trying to help folks understand the specifics of Oslo in its first incarnation, with respect to pre-existing technologies from Visual Studio (and other parts of Microsoft). This leads us to seek terminology that helps us conduct that discussion, and in cases such as this, involves some subtle distinctions that are tough to differentiate. As Doug says: “when you a birth a new product, naming/terminology is often the most difficult aspect of the process”.

Second: Open Microsoft and Eclipse Modeling Project. I second Doug’s remarks wholeheartedly. I’d especially like to see discussion on relationships between EMF and oAW technologies.

Oslo and the DSL Toolkit

Keith Short — Fri, 07 Nov 2008 01:30:00 GMT

I’ve received a number of questions concerning the relationship of Oslo to the Visual Studio DSL Toolkit. Comments have varied from the sublime (see David Ing’s comment on his blog “My best guess is that the DSL Toolkit is research road kill in front of the big Oslo truck and that VSTS Architecture Edition was just about a necessary cycle too early”) to the ridiculous. Since I’ve been a founder member of both projects, I thought I’d try to start the discussion with a few comments of my own.

Firstly, it’s true that the two efforts are built on different technology stacks – the DSL Toolkit works on file-based artifacts (schemas, model instances, etc.) and produces graphical and forms-based tools that run as add-ins to Visual Studio – dramatically simplifying the task of creating tools hosted via VS extensibility.

On the other hand, Oslo is based on an underlying SQL database. Quadrant depends on the underlying database for both the data it is processing and its own configuration data. In other words, Quadrant’s equivalent of the DSL Tools’ domain modeling language and the shape and shape mapping languages is MSchema. Concrete textual languages are of course defined in MGrammar.

Despite these differences, two things need to be made really clear:

1. Both the Oslo and the DSL Toolkit have grown from a common belief in the role DSLs can play in the development lifecycle. Not just during development, but DSLs that help record Business Objectives, Business Processes and Entities, System Architectures, Software components and connections, Deployment Information, Data Center Configuration, and System Management to name just some of the lifecycle stages. This is a shared vision, well documented elsewhere, though each project has focused on a different aspect initially. The DSL Toolkit builds great graphical (box and line) tools that run in Visual Studio and may be translated into code-based artifacts. Oslo is focused on textual and graphical developer experiences around models that initially represent code and configuration that “completes” the underlying frameworks that are part of the application platform – in other words – models that are mostly executable by the underlying servers and frameworks (e.g. WF, WCF, and Identity Services).

2. Both products have a lifecycle in front of them. The two teams, already aligned around vision, are working together to bridge differences over releases. Would it have been nice to have gone dark for a period while we resolved these technology stack issues and re-emerged with a fully aligned set of technologies? You bet – but such a strategy rarely ends with the right thing being built for customers. Ideas we are tossing around include (a) storing DSL Toolkit artifacts (including those created with the emerging UML tools from VSTA) in the Oslo Repository, (b) using MSchema as the domain language for the DSL Toolkit, and (c) converging on a single way to specify concrete DSL syntax whether it is graphical or textual. Sadly, I can’t give dates at this point.

Stuart Kent is the architect for DSL Tools. If you take a look at Stuart’s blog, you’ll find the latest blog entry where he responds to the same questions from his point of view.

Oslo at the PDC

Keith Short — Fri, 31 Oct 2008 21:13:00 GMT

The PDC is now over and by all measures has been a great success. The Oslo talks have been received with great enthusiasm. Any one not following the activity on our MSDN forum who missed the sessions might want to look at this entry on the forum that summarizes the links to our various sessions.

Also, the main Oslo site is now active at http://msdn.microsoft.com/en-us/oslo/default.aspx and of course don’t miss the fun video at the site http://modelsremixed.com .

Yet Another Promise to Start Blogging !!

Keith Short — Sat, 25 Oct 2008 02:10:00 GMT

Yet again, it’s been a long time since I wrote to my blog. This posting is my restart.

I know, you just have to look at the last posting of over two years ago to realize I’ve said that before!

But here’s what happened. Shortly after writing that last posting, I moved from Visual Studio to join the group that was working on the earliest ideas that have now become known by the codeword “Oslo”. Obviously we couldn’t talk about this work, but now, at last, we are about to make the project public in a *big* way at next week’s PDC. You just have to look at blogs such as Don Box’s or Doug Purdy’s to realize “Oslo” is Microsoft’s modeling platform consisting of a new language for describing models and DSLs, a database in which to store models of various kinds, and a new tool to view and edit them.

I’ve mostly been working on the design of the model store (a SQL Server database) and using the new language to create schemas for the new store, but I’ve been able to bring a long background in model driven development, software factories and DSL tools to help with the design and testing of the language and tool.

For me this is very exciting since the “Oslo” investment represents a serious commitment from Microsoft to change the game and make modeling mainstream. That’s almost been the story of my life!

Once the excitement over the various announcements that will be made at the PDC subsides, I’ll try to give more background from my perspective, and begin the task of helping folks understand exactly what we are doing in this space.

Recent Postings by Enterprise Tools Architects

Keith Short — Sat, 07 Jan 2006 01:17:00 GMT

I’ve been lax with my blog for the last two months. I plan to do better this year! During the last two months, we have of course shipped Visual Studio 2005 so our modeling tools for developers and architects finally are in the hands of customers. This has been very exciting!

There are a couple of recent postings by architects in the team that I wanted to advertise.

Firstly, Bill Gibson, a long time colleague of mine both at Microsoft and before, has written a set of Technical Notes which I think offer invaluable advice to users of our tools in Visual Studio Team System 2005 edition for Architects. Among other helpful insights, Bill describes ways to use the tools that many users will find helpful. You can find these indexed on Bill’s blog here.

Also, in response to a question posed by Tad Anderson about our DSL strategy, VSTS and Software Architecture (posting is here), Jack Greenfield wrote a great summary of our Software Factories approach that is embellished with several up-to-date links to various podcasts, papers and other references. I can definitely recommend reading Jack’s posting here.

Software Factories at OOPSLA 2005

Keith Short — Tue, 01 Nov 2005 01:24:00 GMT

OOPSLA 2005 in San Diego was a fun event. It was great to chat with lots of old friends and sit in on some interesting sessions and panels. Our team was represented at a number of events. Jack Greenfield and Steve Cook presented a one day tutorial on Software Factories using some updated material from last year, and showed a fewof interesting demos. One, given by Mauro Regio, showed Visual Studio 2005 configured with various assets to help developers build collaborating services based on the HL7 (a set of standards for B2B communication in the Health Services Industry) software factory I talked about in a previous post which you can read about here. Steve also demoed the latest builds of the DSL Tools.

Also, we attended the Software Factories Workshop, a one-day event which Jack chaired. This was voted a great success by those who attended and gave us all a chance to drill into a bunch of interesting topics around model-driven development, DSLs and software factories. The papers which were reviewed and accepted by the review committee may be read here, but will be published in some form, yet to be decided, as soon as possible along with the proceedings of the workshop. I'll point to those proceedings when they're ready.

Windows Workflow Foundation

Keith Short — Fri, 23 Sep 2005 19:51:00 GMT

After the announcement last week of this great piece of technology, we can finally all start to talk about it. Although you can code directly against the WWF, it contains a nice example of a couple of graphical DSLs that allow the developer to declaratively describe workflows consisting of human and system activities. The lead architect for WWF, Dave Green, has started a blog here, but you'll also want to check out a great article written by David Chappell published on MSDN here.

Steve Cook Interview

Keith Short — Thu, 25 Aug 2005 01:35:00 GMT

Back off three weeks vacation always causes severe "re-entry" problems! But digging myself out of the email stack I noticed Steve Cook has posted a very good interview with Roy Osherove on Software Factories and DSLs. You can download the interview as an MP3 from here.

Steve explains, among other things, the relationship between the DSL Toolkit and the Guidance Automation Toolkit, both of which can be downloaded from the MSDNCommunity Technology Preview site for Visaul Studio Team System.

Software Factories & DSLs

Keith Short — Sat, 02 Jul 2005 00:37:00 GMT

A couple of interesting items have appeared recently that will be of interest to folks following our stories on Software Factories and DSL’s. I had posted earlier some examples of Software Factories that generated quite a bit of comment from readers. We have had a team of people working on a specification for implementing business-to-business interactions under HL7 (Health Level 7) – a set of standards for B2B communications in the Health industry. Since building these kinds of systems uses common design patterns, common tooling, and the implementations share a common architecture across a diverse range of applications and scenarios, they are ideally suited to a Software Factory approach. You can read about this on MSDN here.

Renowned author and industry leader Martin Fowler has published an excellent set of papers describing the use of DSLs in the development process. The starting point is a paper entitled “Language Workbenches: The Killer Application for Domain Specific Languages”.

I thoroughly recommend the article.

Microsoft's Modeling Strategy

Keith Short — Fri, 27 May 2005 22:33:00 GMT

My team has written a document that explains Microsoft’s Modeling Strategy and addresses a number of FAQ’s we get from customers and partners. The outline is:

Why Modeling?
How Are DSLs Used in Model-Driven Development?
What About UML?
What About MDA?
What Are Software Factories?
Other Frequently Asked Questions

Astute readers of my blog may even recognize parts of this document that have appeared in earlier postings. That’s a great value to us of blogs – getting feedback on how we communicate products and strategies so we can improve our ideas. We’d love to get feedback on this document too.

Steve Cook posted these links too, but I do not suffer the same anguish that he does over British/American spelling. I guess I’ve lived in the USA for too long now to continue worrying about that, and happily type ‘z’s instead of ‘s’s with hardly a thought.

New Website for Software Factories

Keith Short — Sat, 21 May 2005 01:55:00 GMT

Although I have been lax at blogging again this last month, the team has been extremely busy with the various efforts around Software Factories. In the course of the last few weeks we have opened a new public portal that will become the focus of all activities under the umbrella term of the Microsoft “Software Factories Initiative”. You can visit this site at http://lab.msdn.microsoft.com/teamsystem/workshop/sf/default.aspx .

You notice that on this site we’ve placed links to the two software download sites for the DSL Tools and for the Guidance Automation Toolkit (GAT). Some of you may be familiar with the DSL Tools (visit the blogs of Steve, Stuart, Alan and Gareth for details) but less familiar with the GAT. Please check-out the article that describes what the GAT is used for. In a forthcoming posting, I’ll explain how the GAT forms the second of three chunks of technology that constitute our initial implementation of basic Software Factory functionality in the Software Factories Initiative.

Examples of Software Factories

Keith Short — Tue, 19 Apr 2005 04:05:00 GMT

I have been speaking to customers and partners recently (last week in Japan) about Software Factories and our modeling strategy in general. During those talks, the audience found it useful to see a couple of different Software Factory examples that fall near the opposite ends of a size scale.

A baseline architecture for part of a Presentation Layer

The Pattern & Practices team User Interface Process (UIP) block available from http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnpag/html/uipab.asp. This is a small framework, based on the MVC pattern, which helps a developer build that part of a layered application architecture that concerns transitions between related Web pages or Windows forms in the Presentation layer
A solution-building agent (we call these recipes) that adds a UIP project to an existing solution from a project template
Process guidance for building User Interface Processes

This mini-Software Factory could be further enhanced by a DSL-based Design tool for diagrammatically showing links between pages or forms, which generates completion code for the UIP block.

A set of assets spanning the entire life cycle for B2C ecommerce systems for small to medium sized businesses that includes:

Feature models of system requirements that help the developer configure the Software Factory assets when building a particular instance of the family
A baseline architecture description for this family of applications
A set of assemblies containing pre-built frameworks and components for all major subsystems
DSL based tools for both development and deployment tasks
A set of patterns to be used by developers when coding or using one of the abovementioned DSLs
Configurable BizTalk schedules for orchestrating drop ship dialogs with suppliers
A tailored development process – exactly fit to the needs of the various roles engaged in building this kind of application
Recipes that create the initial solution, and then lazily build out project clusters for optional subsystems
A deployment topology
A set of test cases and data sets
A set of build scripts

Software Factories can be composed. For example, the second SF may include (and possibly specialize to the domain of ecommerce applications) the first one.

SD Magazine Jolt Productivity Award 2005

Keith Short — Thu, 24 Mar 2005 22:54:00 GMT

Wow! We heard that our Software Factories book was awarded a “Productivity Award” in the General Books Category at the Software Development Magazine Jolt Awards last week.

http://www.sdmagazine.com/pressroom/jolt_winners_2005.pdf

This is a great honor, and we are deeply grateful to the judges and organizers for this result. Unfortunately, neither Jack Greenfield nor I was able to attend the ceremony at Santa Clara.

On Definitions

Keith Short — Thu, 17 Mar 2005 16:45:00 GMT

In Steve Cook’s interview, Steve gives a great one sentence definition of a Software Factory. I realize I should have provided a one sentence definition as part of my “elevator story” posted earlier. I was inspired by Steve’s statement to dig out an earlier attempt we had to produce a single brief sentence that defines what a Software Factory is:

A set of integrated tools, process and content assets used to accelerate life cycle tasks for a specific type of software component, application or system.

Steve Cook on Software Factories and DSLs

Keith Short — Wed, 16 Mar 2005 18:29:00 GMT

Steve Cook, my colleague based in Cambridge, UK, has just posted a very interesting interview with journalist Matt Nicholson:

http://dnjonline.com/article.aspx?ID=mar05_stevecook

It comes complete with a rather fetching picture of Steve at his desk in Cambridge J

Software Factory Elevator Story

Keith Short — Tue, 08 Mar 2005 20:42:00 GMT

To start the ball rolling, I thought I’d write down the Software Factory elevator story. People are always asking us “Can you please just quickly summarize what one is?” That’s right, boil down a 600 page book, and thousands of words in white papers and articles to just three points. So here’s what I think are the key ideas, resisting the urge to expand on them here (that’ll come later).

1. Models are first–class artifacts in software development

Model-driven development uses higher levels of abstraction than code oriented development to reduce the complexity of the development process, to enable faster response to changing technologies and requirements, to make developers more productive, and to make development projects more predictable. It is a continuation of the constant migration of platforms and tools to higher levels of abstraction that has characterized software development since its inception as a discipline. Models designed to be used as source artifacts can support analysis and validation, provide holistic views of otherwise scattered details, and streamline communication between different groups involved in designing, building and deploying complex modern applications. Such models do not get out of sync with the software because unlike models acting only as documentation, they define the software, and must be changed to change the software.

2. There is no single, fixed collection of DSLs that can be pre-specified for all kinds of applications

To be effective, models are instances of well-focused, inter-related DSLs arranged into a collection that provides layers of useful abstractions with which to specify, design, build and manage applications. Each collection should be customized so that the collection overall exactly meets the needs of specific families of applications – like e-commerce, financial arbitrage or home banking – that have a well-defined architecture, and well-defined dependencies on platform frameworks and components. Common features of applications in the same family, and how each may family member may differ, should be clearly identified. How each member of a family should be built depends on how variability highlighted in the family architecture is mapped to variability in the requirements the family is designed to meet. Software product line thinking emphasizes the importance of well-defined architecture, and sets the context for effective reuse.

3. It’s about more than just models

Also important are customized development processes – matched to the specific needs of the architecture and assets of the application family. Likewise for frameworks, components and patterns – these should be arranged for use by the developers and architects in a way that is custom-fit for the family of applications. Developers should not be forced to scan through catalogs and repositories in the hope that they can find something to reuse. Note that models are used not only for analysis and design. With Software Factories, models are used to support many varied types of computation across the entire software life cycle – even at run time – a fundamental principle of Microsoft’s DSI Strategy. Model-driven deployment, operations and management tools are equally important. Ensuring design metadata and runtime metadata is available wherever it can be utilized is a key principle of Software Factories.

Restarting My Blog

Keith Short — Mon, 07 Mar 2005 18:31:00 GMT

It’s been a long time since I wrote to my blog. This posting is my restart.

One of the reasons I’ve been so quiet is that I’ve been involved in some start up projects here at Microsoft that we are not yet ready to blog about. These have taken quite a bit of my time, and although are related to the general universe of modeling, are concerned with longer lead projects. Fortunately, much good discourse on the topics of Software Factories, Visual Studio Team System, DSLs and model-driven development has been continued in the blogs of my colleagues in my team - Steve Cook, Stuart Kent, Jack Greenfield, Gareth Jones and Alan Wills, as well as the usual great commentary from Harry Pierson and many others at Microsoft and outside.

But during this period, I have done a fair bit of traveling to visit customers, and speaking at conferences – usually to explain our strategy for model-driven development and the roadmap for Microsoft’s support of it. At most of these occasions, I’m asked to explain more carefully our positioning of DSLs and the broader Software Factories story of which they are part, relative to the OMG branded technologies and approaches – specifically UML and MDA. I intend to comment on how we address these questions in the next few blog entries.

Restarting My Blog

Keith Short — Mon, 07 Mar 2005 18:25:00 GMT

It’s been a long time since I wrote to my blog. This posting is my restart.

Report from OOPSLA

Keith Short — Wed, 27 Oct 2004 17:12:00 GMT

We’re into our third day here at OOPSLA. On Monday, Jack Greenfield, Steve Cook and I delivered our full-day tutorial as I described in my last posting. I think it was received well by those who attended – based on chats we had with those who shared the day with us. Of course, we won’t really be able to tell until we see the evaluation forms.

We had to explain to those that attended that we were performing something of an experiment on them. Several weeks ago we had submitted a day’s worth of slides to the organizers, which were duly duplicated and ready for the attendees when they arrived. But we took a risk to completely ignore that deck, which was a synopsis of our book on Software Factories, and instead spend the whole day working through an end-to-end scenario showing how four variations of an application could be specified and built using assets provided by a Software Factory. We took this risky step based on some feedback we received at the Strategic Architects Forum in Redmond a couple of weeks ago. Most people we spoke to there were interested in the ideas, some very much so, but many people told us that they found the concepts a little abstract, and needed many more examples of DSLs, customized guidance and custom content.

So that’s what we did. It involved us in a long week of rebuilding a deck and building applications (ably assisted by Michael Lehman). We took the famous Pet Shop application (the .Net version which can be downloaded from msdn), and defined two organizations building e-commerce pet shop applications: PetShoppe, a small business with modest requirements, and PetCluCo, a larger organization with more sophisticated requirements. We first asked the audience to suspend belief and imagine a Software Factory from which these applications could be built and then modified when requirements changed, or in the case of PetClubCo, the business needs changed.

We then showed a number of tools, based on DSLs, that could be used to configure business capabilities and system requirements, and produce implementations from these, first in the case of PetShoppe in a largely manual coding style, and second, in the case of PetCluCo, using a powerful set of DSL based tools to allow the developer to be more efficient, and do less rote programming tasks. Only at the end of the day, did we show how the Software Factory assets were designed and created. I’ll be writing a more detailed summary of the tutorial in a few days time, and I’ll post here when it’s ready.

Some of these tools exist and will ship with Visual Studio 2005 Team System. Others could be built using the framework and tools we use in Microsoft to define DSLs and build tools that support them. We also showed a demo of these tools (building a simple Web page transition design tool – Harry Pierson has posted a screen shot of the simple tool we built in the demo). This is the framework and tool kit we announced yesterday at OOPSLA as described in various press coverage (the ServerSide, CRN, and Computerworld). We’ll be posting a lot more information about this framework and tools in the next few days. Stuart Kent has set the ball rolling in his blog. We hope that many of our customers and partners will build the tools we showed in the tutorial as mock-ups.

Unfortunately, yesterday’s announcement of what we’re doing with DSL building tools and Software Factories was somewhat marred by some unintentional clumsiness on our part at OOPSLA. The description of the availability of the technology and the things we’re working on came across as a product announcement – an unwelcome thing at OOPSLA conferences. Several people at the conference expressed their disgust in various obvious ways (including Alan Kay who chose to lampoon Microsoft about this in remarks sprinkled through his Turing Award keynote). All I can say is that we’re sorry this happened in this way. We’ll learn from this and make sure we respect OOPSLA practices in the future.

Right now I’m sitting at the back of the GPCE keynote in which Jack is giving the Software Factory talk. I’ll report on feedback later.

Getting Ready for OOPSLA 2004 in Vancouver, BC.

Keith Short — Sat, 16 Oct 2004 20:18:00 GMT

I’m very much looking forward to going to OOPSLA next weekend. Microsoft will be making several major announcements around Software Factories and Domain Specific Languages at both OOPSLA and GPCE ’04 on October 24-28 in Vancouver, British Columbia. I hope to run into old friends and make new ones. In addition to two keynotes, BoF sessions and panel discussions, members of my team will be presenting two tutorials on topics including Generative Software Development and Domain Specific Languages. Please see conference site for more information.

The last couple of weeks have been pretty much devoted to preparing for these various activities and announcements. We’ve got the slides for the keynotes wrapped, and various members of the team have been focused on software demos for the tutorials and keynotes. In the day-long tutorial that runs on Monday 25^th (T40 : Using Domain Specific Languages, Patterns, Frameworks and Tools to Assemble Applications) we plan to use a large proportion of the time to walk through a detailed example of using Software Factories ideas to build four different variations of the same application. This involves us using some existing technology, some technology that will ship in Visual Studio 2005 Team System, and some technology that isn’t yet planned, but which could readily be built by Microsoft or one our partners. As we plan to show the whole process – from soup (business case, business capabilities and business processes) to nuts (running code), this involves a lot of work with help from several teams here at Microsoft. It is coming together well though, thankfully!