Recent Postings by Enterprise Tools Architects

I’ve been lax with my blog for the last two months. I plan to do better this year! During the last two months, we have of course shipped Visual Studio 2005 so our modeling tools for developers and architects finally are in the hands of customers. This has been very exciting!

 

There are a couple of recent postings by architects in the team that I wanted to advertise.

 

Firstly, Bill Gibson, a long time colleague of mine both at Microsoft and before, has written a set of Technical Notes which I think offer invaluable advice to users of our tools in Visual Studio Team System 2005 edition for Architects. Among other helpful insights, Bill describes ways to use the tools that many users will find helpful. You can find these indexed on Bill’s blog here.

 

Also, in response to a question posed by Tad Anderson about our DSL strategy, VSTS and Software Architecture (posting is here), Jack Greenfield wrote a great summary of our Software Factories approach that is embellished with several up-to-date links to various podcasts, papers and other references. I can definitely recommend reading Jack’s posting here.

Software Factories at OOPSLA 2005

OOPSLA 2005 in San Diego was a fun event. It was great to chat with lots of old friends and sit in on some interesting sessions and panels. Our team was represented at a number of events. Jack Greenfield and Steve Cook presented a one day tutorial on Software Factories using some updated material from last year, and showed a fewof interesting demos. One, given by Mauro Regio, showed Visual Studio 2005 configured with various assets to help developers build collaborating services based on the HL7 (a set of standards for B2B communication in the Health Services Industry) software factory I talked about in a previous post which you can read about here. Steve also demoed the latest builds of the DSL Tools.

Also, we attended the Software Factories Workshop, a one-day event which Jack chaired. This was voted a great success by those who attended and gave us all a chance to drill into a bunch of interesting topics around model-driven development, DSLs and software factories. The papers which were reviewed and accepted by the review committee may be read here, but will be published in some form, yet to be decided, as soon as possible along with the proceedings of the workshop. I'll point to those proceedings when they're ready.

Windows Workflow Foundation

After the announcement last week of this great piece of technology, we can finally all start to talk about it. Although you can code directly against the WWF, it contains a nice example of a couple of graphical DSLs that allow the developer to declaratively describe workflows consisting of human and system activities. The lead architect for WWF, Dave Green, has started a blog here, but you'll also want to check out a great article written by David Chappell published on MSDN here.

Steve Cook Interview

Back off three weeks vacation always causes severe "re-entry" problems! But digging myself out of the email stack I noticed Steve Cook has posted a very good interview with Roy Osherove on Software Factories and DSLs. You can download the interview as an MP3 from here.

Steve explains, among other things, the relationship between the DSL Toolkit and the Guidance Automation Toolkit, both of which can be downloaded from the MSDNCommunity Technology Preview site for Visaul Studio Team System.

Software Factories & DSLs

A couple of interesting items have appeared recently that will be of interest to folks following our stories on Software Factories and DSL’s.  I had posted earlier some examples of Software Factories that generated quite a bit of comment from readers. We have had a team of people working on a specification for implementing business-to-business interactions under HL7 (Health Level 7) – a set of standards for B2B communications in the Health industry. Since building these kinds of systems uses common design patterns, common tooling, and the implementations share a common architecture across a diverse range of applications and scenarios, they are ideally suited to a Software Factory approach.  You can read about this on MSDN here.

 

Renowned author and industry leader Martin Fowler has published an excellent set of papers describing the use of DSLs in the development process. The starting point is a paper entitled “Language Workbenches: The Killer Application for Domain Specific Languages”.

 

I thoroughly recommend the article.

Microsoft's Modeling Strategy

My team has written a document that explains Microsoft’s Modeling Strategy and addresses a number of FAQ’s we get from customers and partners. The outline is:

Why Modeling?
How Are DSLs Used in Model-Driven Development?
What About UML?
What About MDA?
What Are Software Factories?
Other Frequently Asked Questions

Astute readers of my blog may even recognize parts of this document that have appeared in earlier postings. That’s a great value to us of blogs – getting feedback on how we communicate products and strategies so we can improve our ideas.  We’d love to get feedback on this document too.

 

Steve Cook posted these links too, but I do not suffer the same anguish that he does over British/American spelling. I guess I’ve lived in the USA for too long now to continue worrying about that, and happily type ‘z’s instead of ‘s’s with hardly a thought.

 

 

New Website for Software Factories

Although I have been lax at blogging again this last month, the team has been extremely busy with the various efforts around Software Factories. In the course of the last few weeks we have opened a new public portal that will become the focus of all activities under the umbrella term of the Microsoft “Software Factories Initiative”. You can visit this site at http://lab.msdn.microsoft.com/teamsystem/workshop/sf/default.aspx .

 

You notice that on this site we’ve placed links to the two software download sites for the DSL Tools and for the Guidance Automation Toolkit (GAT). Some of you may be familiar with the DSL Tools (visit the blogs of Steve, Stuart, Alan and Gareth for details) but less familiar with the GAT. Please check-out the article that describes what the GAT is used for. In a forthcoming posting, I’ll explain how the GAT forms the second of three chunks of technology that constitute our initial implementation of basic Software Factory functionality in the Software Factories Initiative.

 

Examples of Software Factories

I have been speaking to customers and partners recently (last week in Japan) about Software Factories and our modeling strategy in general. During those talks, the audience found it useful to see a couple of different Software Factory examples that fall near the opposite ends of a size scale.

 

1. A baseline architecture for part of a Presentation Layer

• The Pattern & Practices team User Interface Process (UIP) block available from http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnpag/html/uipab.asp. This is a small framework, based on the MVC pattern, which helps a developer build that part of a layered application architecture that concerns transitions between related Web pages or Windows forms in the Presentation layer
• A solution-building agent (we call these recipes) that adds a UIP project to an existing solution from a project template
• Process guidance for building User Interface Processes

This mini-Software Factory could be further enhanced by a DSL-based Design tool for diagrammatically showing links between pages or forms, which generates completion code for the UIP block.

 

1. A set of assets spanning the entire life cycle for B2C ecommerce systems for small to medium sized businesses that includes:

• Feature models of system requirements that help the developer configure the Software Factory assets when building a particular instance of the family
• A baseline architecture description for this family of applications
• A set of assemblies containing pre-built frameworks and components for all major subsystems
• DSL based tools for both development and deployment tasks
• A set of patterns to be used by developers when coding or using one of the abovementioned DSLs
• Configurable BizTalk schedules for orchestrating drop ship dialogs with suppliers
• A tailored development process – exactly fit to the needs of the various roles engaged in building this kind of application
• Recipes that create the initial solution, and then lazily build out project clusters for optional subsystems
• A deployment topology
• A set of test cases and data sets
• A set of build scripts

 

Software Factories can be composed. For example, the second SF may include (and possibly specialize to the domain of ecommerce applications) the first one.

SD Magazine Jolt Productivity Award 2005

Wow!  We heard that our Software Factories book was awarded a “Productivity Award” in the General Books Category at the Software Development Magazine Jolt Awards last week.

 

http://www.sdmagazine.com/pressroom/jolt_winners_2005.pdf

 

This is a great honor, and we are deeply grateful to the judges and organizers for this result.  Unfortunately, neither Jack Greenfield nor I was able to attend the ceremony at Santa Clara.

 

On Definitions

In Steve Cook’s interview, Steve gives a great one sentence definition of a Software Factory. I realize I should have provided a one sentence definition as part of my “elevator story” posted earlier. I was inspired by Steve’s statement to dig out an earlier attempt we had to produce a single brief sentence that defines what a Software Factory is:

 

A set of integrated tools, process and content assets used to accelerate life cycle tasks for a specific type of software component, application or system.

 

Steve Cook on Software Factories and DSLs

Steve Cook, my colleague based in Cambridge, UK, has just posted a very interesting interview with journalist Matt Nicholson:

 

http://dnjonline.com/article.aspx?ID=mar05_stevecook

 

It comes complete with a rather fetching picture of Steve at his desk in Cambridge J

 

Software Factory Elevator Story

To start the ball rolling, I thought I’d write down the Software Factory elevator story. People are always asking us “Can you please just quickly summarize what one is?” That’s right, boil down a 600 page book, and thousands of words in white papers and articles to just three points. So here’s what I think are the key ideas, resisting the urge to expand on them here (that’ll come later).

1.      Models are first–class artifacts in software development

Model-driven development uses higher levels of abstraction than code oriented development to reduce the complexity of the development process, to enable faster response to changing technologies and requirements, to make developers more productive, and to make development projects more predictable. It is a continuation of the constant migration of platforms and tools to higher levels of abstraction that has characterized software development since its inception as a discipline. Models designed to be used as source artifacts can support analysis and validation, provide holistic views of otherwise scattered details, and streamline communication between different groups involved in designing, building and deploying complex modern applications. Such models do not get out of sync with the software because unlike models acting only as documentation, they define the software, and must be changed to change the software.

2.      There is no single, fixed collection of DSLs that can be pre-specified for all kinds of applications

To be effective, models are instances of well-focused, inter-related DSLs arranged into a collection that provides layers of useful abstractions with which to specify, design, build and manage applications. Each collection should be customized so that the collection overall exactly meets the needs of specific families of applications – like e-commerce, financial arbitrage or home banking – that have a well-defined architecture, and well-defined dependencies on platform frameworks and components. Common features of applications in the same family, and how each may family member may differ, should be clearly identified. How each member of a family should be built depends on how variability highlighted in the family architecture is mapped to variability in the requirements the family is designed to meet. Software product line thinking emphasizes the importance of well-defined architecture, and sets the context for effective reuse.

3.      It’s about more than just models

Also important are customized development processes – matched to the specific needs of the architecture and assets of the application family. Likewise for frameworks, components and patterns – these should be arranged for use by the developers and architects in a way that is custom-fit for the family of applications. Developers should not be forced to scan through catalogs and repositories in the hope that they can find something to reuse. Note that models are used not only for analysis and design. With Software Factories, models are used to support many varied types of computation across the entire software life cycle – even at run time – a fundamental principle of Microsoft’s DSI Strategy. Model-driven deployment, operations and management tools are equally important. Ensuring design metadata and runtime metadata is available wherever it can be utilized is a key principle of Software Factories. 

Restarting My Blog

It’s been a long time since I wrote to my blog. This posting is my restart.

 

One of the reasons I’ve been so quiet is that I’ve been involved in some start up projects here at Microsoft that we are not yet ready to blog about. These have taken quite a bit of my time, and although are related to the general universe of modeling, are concerned with longer lead projects. Fortunately, much good discourse on the topics of Software Factories, Visual Studio Team System, DSLs and model-driven development has been continued in the blogs of my colleagues in my team - Steve Cook, Stuart Kent, Jack Greenfield, Gareth Jones and Alan Wills, as well as the usual great commentary from Harry Pierson and many others at Microsoft and outside.

 

But during this period, I have done a fair bit of traveling to visit customers, and speaking at conferences – usually to explain our strategy for model-driven development and the roadmap for Microsoft’s support of it. At most of these occasions, I’m asked to explain more carefully our positioning of DSLs and the broader Software Factories story of which they are part, relative to the OMG branded technologies and approaches – specifically UML and MDA. I intend to comment on how we address these questions in the next few blog entries.

Report from OOPSLA

We’re into our third day here at OOPSLA. On Monday, Jack Greenfield, Steve Cook and I delivered our full-day tutorial as I described in my last posting. I think it was received well by those who attended – based on chats we had with those who shared the day with us. Of course, we won’t really be able to tell until we see the evaluation forms.

 

We had to explain to those that attended that we were performing something of an experiment on them. Several weeks ago we had submitted a day’s worth of slides to the organizers, which were duly duplicated and ready for the attendees when they arrived. But we took a risk to completely ignore that deck, which was a synopsis of our book on Software Factories, and instead spend the whole day working through an end-to-end scenario showing how four variations of an application could be specified and built using assets provided by a Software Factory. We took this risky step based on some feedback we received at the Strategic Architects Forum in Redmond a couple of weeks ago. Most people we spoke to there were interested in the ideas, some very much so, but many people told us that they found the concepts a little abstract, and needed many more examples of DSLs, customized guidance and custom content.

 

So that’s what we did. It involved us in a long week of rebuilding a deck and building applications (ably assisted by Michael Lehman). We took the famous Pet Shop application (the .Net version which can be downloaded from msdn), and defined two organizations building e-commerce pet shop applications: PetShoppe, a small business with modest requirements, and PetCluCo, a larger organization with more sophisticated requirements. We first asked the audience to suspend belief and imagine a Software Factory from which these applications could be built and then modified when requirements changed, or in the case of PetClubCo, the business needs changed.

 

We then showed a number of tools, based on DSLs, that could be used to configure business capabilities and system requirements, and produce implementations from these, first in the case of PetShoppe in a largely manual coding style, and second, in the case of PetCluCo, using a powerful set of DSL based tools to allow the developer to be more efficient, and do less rote programming tasks. Only at the end of the day, did we show how the Software Factory assets were designed and created. I’ll be writing a more detailed summary of the tutorial in a few days time, and I’ll post here when it’s ready.

 

Some of these tools exist and will ship with Visual Studio 2005 Team System. Others could be built using the framework and tools we use in Microsoft to define DSLs and build tools that support them. We also showed a demo of these tools (building a simple Web page transition design tool – Harry Pierson has posted a screen shot of the simple tool we built in the demo). This is the framework and tool kit we announced yesterday at OOPSLA as described in various press coverage (the ServerSide, CRN, and Computerworld). We’ll be posting a lot more information about this framework and tools in the next few days. Stuart Kent has set the ball rolling in his blog. We hope that many of our customers and partners will build the tools we showed in the tutorial as mock-ups.

 

Unfortunately, yesterday’s announcement of what we’re doing with DSL building tools and Software Factories was somewhat marred by some unintentional clumsiness on our part at OOPSLA. The description of the availability of the technology and the things we’re working on came across as a product announcement – an unwelcome thing at OOPSLA conferences. Several people at the conference expressed their disgust in various obvious ways (including Alan Kay who chose to lampoon Microsoft about this in remarks sprinkled through his Turing Award keynote). All I can say is that we’re sorry this happened in this way. We’ll learn from this and make sure we respect OOPSLA practices in the future.

 

Right now I’m sitting at the back of the GPCE keynote in which Jack is giving the Software Factory talk.  I’ll report on feedback later.

Getting Ready for OOPSLA 2004 in Vancouver, BC.

I’m very much looking forward to going to OOPSLA next weekend. Microsoft will be making several major announcements around Software Factories and Domain Specific Languages at both OOPSLA and GPCE ’04 on October 24-28 in Vancouver, British Columbia. I hope to run into old friends and make new ones. In addition to two keynotes, BoF sessions and panel discussions, members of my team will be presenting two tutorials on topics including Generative Software Development and Domain Specific Languages.  Please see conference site for more information.

 

The last couple of weeks have been pretty much devoted to preparing for these various activities and announcements. We’ve got the slides for the keynotes wrapped, and various members of the team have been focused on software demos for the tutorials and keynotes. In the day-long tutorial that runs on Monday 25^th (T40 : Using Domain Specific Languages, Patterns, Frameworks and Tools to Assemble Applications) we plan to use a large proportion of the time to walk through a detailed example of using Software Factories ideas to build four different variations of the same application. This involves us using some existing technology, some technology that will ship in Visual Studio 2005 Team System, and some technology that isn’t yet planned, but which could readily be built by Microsoft or one our partners. As we plan to show the whole process – from soup (business case, business capabilities and business processes) to nuts (running code), this involves a lot of work with help from several teams here at Microsoft. It is coming together well though, thankfully!