Jack Greenfield's Blog

Geographically Distributed High Availability

JackGre — Mon, 26 Dec 2011 23:54:00 GMT

This is the last post in the series on business continuity. It briefly describes how the service developer can provide geographically distributed high availability on the Azure platform. The Azure platform currently does not directly support highly...(read more)

Disaster Recovery

JackGre — Sun, 23 Oct 2011 19:04:00 GMT

While the Azure platform provides high availability within a single data center, as discussed in the previous post , it currently does not explicitly support or enable disaster recovery or geographically distributed high availability. This post and the...(read more)

High Availability On The Azure Platform

JackGre — Sun, 23 Oct 2011 05:54:00 GMT

Currently, both Windows Azure and SQL Azure offer high availability within a single data center. As long as a data center remains operational and accessible from the Internet, services hosted there can achieve high availability. Windows Azure Windows...(read more)

Business Continuity Basics

JackGre — Wed, 19 Oct 2011 19:46:00 GMT

This post describes the basics of business continuity. Business continuity poses both business and technical challenges. In this series, we focus on the technical challenges, which include failure detection, response, diagnosis, and defect correction...(read more)

Business Continuity On Azure

JackGre — Tue, 18 Oct 2011 18:49:00 GMT

Azure customers want their services to be continuously available to their users, despite component failures, platform degradations and data center outages. In other words, they want business continuity. To achieve it, the services must be highly available...(read more)

Remembering Steve by Remembering NeXT

JackGre — Tue, 11 Oct 2011 17:08:15 GMT

Last week, JJ Dubray asked me to write something about NeXT in Steve's honor for InfoQ. It's part of a post about Steve with thoughts from several people, including Eric Schmidt and Tim Berners-Lee. My thoughts are near the bottom of the post....(read more)

FabrikamJets Example Updated (Really)

JackGre — Fri, 09 Jul 2010 00:19:00 GMT

I just discovered that the 2.0 release of the Fabrikam Jets example on code gallery (described in this blog post ) contained the original version 1.0 code that worked with OAuth WRAP v0.8, instead of the new version 2.0 code that works with OAuth WRAP...(read more)

Silverlight Samples for OData Over SQL Azure

JackGre — Tue, 20 Apr 2010 20:00:00 GMT

How to build a Silverlight client for the OData Service for SQL Azure, and how to build a service that validates Secure Web Tokens (SWTs) issued by AppFabric Access Control...(read more)

Silverlight Clients and AppFabric Access Control

JackGre — Mon, 19 Apr 2010 22:08:00 GMT

The Silverlight behavior is designed to prevent cross site attacks in which request headers on a POST could be redirected to unknown domains. Clearly, this is preferable to the .NET framework behavior, which puts the client at risk. However, the HTTP specification says that in the case of a 301 or 303 response code, the user agent should not redirect the request without explicit permission from the user, unless the request is either a GET or HEAD, for precisely this reason. So, according to the specification, neither behavior is correct....(read more)

How to Use OData for SQL Azure with AppFabric Access Control

JackGre — Fri, 16 Apr 2010 23:16:00 GMT

While developers can build and deploy custom OData based services to expose data from SQL Azure, other data sources, the OData Portal and Service provide OData access to SQL Azure through simple configuration.

...(read more)

FabrikamJets Example Updated to AppFabric Access Control V1

JackGre — Sat, 09 Jan 2010 02:41:00 GMT

I just finished migrating the FabrikamJets example from the CTP to V1 of AppFabric Access Control. There were two changes that made an update necessary: 1) V1 uses version 0.9 of the WRAP protocol, instead of version 0.8, and 2) the issuer name generated...(read more)

It's a wrap... WRAP v0.9 to be precise...

JackGre — Thu, 17 Dec 2009 00:21:00 GMT

I just finished migrating the integration between Microsoft Codename "Dallas" and Microsoft AppFabric Access Control from WRAP v0.8 to WRAP v0.9. What? You haven't heard of the Web Resource Authorization Protocol (WRAP) ? Well, you're really missing...(read more)

Fabrikam Jets - Integrating Codename "Dallas" with AppFabric Access Control

JackGre — Tue, 08 Dec 2009 18:49:00 GMT

I recently posted a screencast of an integration between Microsoft AppFabric Access Control and Microsoft Codename "Dallas" here on Channel 9. This was my first project in CST, and took about three weeks to implement. Producing the screencast and packaging the code as an example, soon to be posted on Microsoft Code Gallery, took another week. It was an interesting and educational exercise, as it touched on several technologies that I didn't know much about before the project. Both Access Control and Dallas are intuitive and easy to use, with REST based APIs. I'm looking forward to more work in the areas of access control, identity federation, trust delegation and web data management.

A New Team, A New Mission

JackGre — Thu, 03 Dec 2009 20:42:00 GMT

Just over a month ago, I moved from p&p to the Cloud Services Team (CST), which is part of the Business Platform Division (BPD) of the Server and Tools Business (STB), reporting to John Shewchuk. CST is responsible for SQL Azure, Windows Azure platform AppFabric, Microsoft Codename "Dallas", and many other cloud platform technologies. It's a talented and innovative team in one of the most significant and fast moving areas of the industry. As a Principal Architect here, I'll be helping to advance Microsoft's vision for an industry-defining cloud platform by creating and driving product, technical, and marketing strategy; developing new product offerings; and collaborating with other CST leaders on execution and product delivery.

Software Factories Focus Groups at Tech Ed 2008 in Orlando

JackGre — Fri, 06 Jun 2008 03:25:00 GMT

Michael Lehman, creator of two factory based initiatives, S+S Blueprints and Project Glidepath, will be hosting a pair of focus groups at Tech Ed 2008, discussing his work with S+S Blueprints, and the relationship between S+S Blueprints and Software Factories. To attend, register via the post on Michael's blog.

Articles on Software Factories

JackGre — Wed, 10 Oct 2007 23:26:00 GMT

I recently published an article on using software factories in supply chains for Methods and Tools. Also, many people have asked about the article I wrote about factories vs. MDA for the Perspectives of the IASA.

Views and Patterns in VSTS

JackGre — Fri, 24 Nov 2006 23:50:00 GMT

Recently, some of our partners asked how best VSTS allows the architect to define architectural views, such as the 4+1 views, and how VSTS supports ready made architectural styles and design patterns, such as MVC, and the patterns from Microsoft Patterns and Practices.

Both questions are best answered in terms of our Software Factory strategy.

Architects using VSTS can use software factories to define sets of views, either standard sets of views, such as the 4+1 views, or custom sets of views, usually the set of view needed to build a specific type of deliverable, such as a smart client or a web service. At the heart of a software factory is a model that defines a set of views. It is called a factory schema. Future releases of VSTS will provide first class support for developing and applying factory schemas, allowing the factory author to define a set of views, and the factory user to interact with the set of views defined by the factory author. Currently, views can be mapped to various VSTS mechanisms, as described in this white paper on MSDN.

Architectural styles and design patterns can be captured and delivered to application developers using software factories. In a software factory, architectural styles and design patterns are packaged as reusable assets. A software factory can also package other kinds of reusable assets, such as tools, templates, libraries, frameworks, guidelines, checklists, tests, models, requirements, and so on. These assets are contextualized by activities that describe simple process steps, and the conditions under which those steps make sense to perform. The activities, in turn, are contextualized by the views defined in the factory schema, so that it is easy for the factory user to determine where in the architecture the factory author intends a given activity to be performed, and consequently where in the architecture and under what conditions the factory author intends an asset to be used.

Performance Analysis and Prediction

JackGre — Sun, 29 Oct 2006 23:49:00 GMT

Irfan Idrees wrote with the observation that while performance analysis and prediction are of critical importance for software developers, the rigorous approaches prevalent in the marketplace have not been widely accepted in the community. Irfan then asked how software factories deal with this issue.

Software factories provide leverage by reducing the amount of variability in application development. With a software factory, one effectively builds a variant of a known application type. All features common to the product family are designed and implemented in known ways. The only variability in a given member of the family is due to the features that make it unique.

Unique features include both fully custom features and variations on known variable features. A fully custom feature is one that has not been predicted or taken into account in any way by the software factory. Variations on known variable features may be either fully constrained (e.g., picking one of five predefined transaction policies from a drop down), or partially constrained, (e.g., writing custom code to complete a template generated event handler designed to work within a predefined event architecture).

To the extent that the performance of that application type is known and understood, then the task of analyzing and predicting the performance of a given variant becomes a matter of analyzing and predicting the performance of its unique features, and of analyzing and predicting the impact of their performance on the known performance characteristics of the application type, such as overall performance, the performance of specific operations or scenarios, and the way those measurements may vary with variations on known variable features.

Ad Hoc and Systematic Reuse

JackGre — Wed, 18 Oct 2006 01:19:00 GMT

In the software factories book, we explain that systematic reuse is effective, but ad hoc reuse is not.

Ad hoc reuse is the “Field of Dreams” approach… “If we build it, they will come”. Great line for a movie, but it doesn’t work well in the real world of software development.

Systematic reuse relies on the recognition that every component embodies many assumptions about functional requirements, technology choices, operational requirements, architecture, deployment topology, testing strategy, development process, and many other aspects of the software development life cycle.

Failing to recognize and plan around these assumptions leads to the well known problem called “architectural mismatch”. This problem is described in detail in a series of papers from David Garlan at Carnegie Mellon. The patterns and practices team arrived at this same conclusion “bottom up”, by listening to customer feedback regarding the guidance offerings they were developing. Customers were asking how to use those offerings to build a specific type of application.

Recognizing and planning around these assumptions leads to defining families of similar solutions, and separating their common features, which make them similar, from their variable features, which make one member of the family different from the next. The common features are used to develop a custom process for building the members of the family, and of a set of reusable assets supporting the process. The variable features are used to drive parameterization, configuration, assembly and other techniques for accommodating the differences between the family members.

This is the thinking underlying the development of the components that are truly reusable, such as RDBMSs and GUI frameworks, although it may not have been recognized at the time by all of the people involved in building those components.

Work at the Software Engineering Institute (SEI) formalized this thinking in the concept of Software Product Lines. Software Product Line practices have been well established through many books, case studies and experience reports, and are widely accepted in the software engineering community.

Software Factories is a methodology for using domain specific languages and other technologies to automate Software Product Lines. They were developed by leveraging the work of David Garlan, as described in the book, in consultation with the authors of Software Product Lines at the SEI, and with other experts in related disciplines, such as pattern languages, generative programming and feature based development.

Some Inaccurate Statements About Software Factories

JackGre — Mon, 24 Jul 2006 20:58:00 GMT

The CEO of 6th Sense Analytics recently published an article containing some inaccurate statements about Software Factories.

The article asserts that SFs are about making developers into battery chickens who squeeze out code. On the contrary, SFs are designed to automate the rote and menial tasks that make developers feel like battery chickens, freeing up time for the creative work of software development.
The article asserts that SFs attempt to solve software development problems by imposing process. On the contrary, SFs seek to enable the project team to be more agile by maintaining invariants to keep the product well formed and the project manageable, so that the overall process can follow any path that the team chooses to take.
The article asserts that Software Factories are imposed by people outside the team that uses them. On the contrary, they are generally developed by their users and are most effective when they serve as a repository of shared experience.

These misconceptions suggest that the author of the article does not know much about Software Factories, and perhaps has not even read what we have written about them. By themselves, they would not be serious enough to merit the time required to post a reply, as most people who evaluate a technology look at it deeply enough to get past shallow misconceptions.

However, the article goes on to say that SFs “crush the flexibility and creativity needed to build software and meet constantly changing requirements”. This statement is profoundly misleading, and requires a response for the sake of the those who have read or may read the article.

We have published a book, as well as many articles, papers and pod casts describing Software Factories. Reading just a few of them would make it easy to see why this statement is off the mark. Therefore, rather than explain SFs yet again here, I will refer to them. You can find pointers to many of them in my previous blog postings.

I would also like to ask the author of the article to answer the following 3 questions:

1. Which of the following innovations crush flexibility and creativity?

Compilers
Class Libraries
Frameworks
Patterns
Best Practices
Databases
Form Builders
Transforms

2. Which of the following practices crush flexibility and creativity?

Writing User Stories
Defining Service Contracts
Refactoring
Testing
Configuration Management
Defect Tracking

3. What evidence do you have that SFs crush flexibility and creativity?

SFs are a lot like the innovations and practices listed above in the way they organize and support software development, and in the way they help developers harvest experience and make it easy for others to reuse.

The author goes on to suggest that the key to success is visibility into project status, supported by metrics. I agree and recommend the article that I co-wrote with Marcel DeVries on using Software Factories with Visual Studio Team System, soon to be published on MSDN.

It explains how, by providing a simple but effective architectural framework for organizing a project, called a schema, Software Factories help their developers define better metrics, and how, by providing a framework for harvesting and reusing experience, Software Factories help their developers use metrics more effectively to improve future projects.

Architecture Interview and Podcasts

JackGre — Mon, 23 Jan 2006 04:44:16 GMT

Here’s an update about Software Factories on ARCast, the podcast series on the Architecture Resource Center. Last month, five panelists (counting myself) recorded three shows on model driven development. It's been quite successful, with the largest number of downloads to date in the series. You can find the first show here, the second one here and the third one here.

Following up the panel discussion, Martin Danner and I did another podcast on MDA vs. Software Factories.

I also joined Mauro Regio recently for an ArcTalk interview with Ron Jacobs about the HL7 Software Factory described in my recent post, and Steve Cook also did an interesting one on one with Ron Jacobs about DSLs and Software Factories.

RE: VSTS 2005, DSL, and Software Architecture

JackGre — Fri, 06 Jan 2006 21:45:00 GMT

This is a response to a blog posting by Tad Anderson, and the pointer to it on the MSDN Architecture General forum.

First, I agree with your observation, Tad, about the lack of support for the Software Architect in VSTS 2005. We did indeed focus on the System Architect not the Software Architect. We had limited resources to allocate to architecture tools, and felt it was more important to support Microsoft’s drive toward connected systems for the 2005 release. As you suggest, however, we do intend to address the needs of the Software Architect in subsequent releases, and most of that support will come from our Software Factory strategy.

However, I disagree with your assertion that our Software Factory strategy, with its reliance on DSLs, will “make a huge mess”. From your comments, I gather that you haven’t read the Software Factories book, or been exposed to the many white papers, blog postings, conference proceedings, articles and podcasts that we’ve produced, since they address the issues you raise at length. I will summarize them here, but I'll also point you to the those other resources, where you learn a lot more about what we’re doing and why.

While it’s certainly true that unqualified practitioners might abuse our technology, there is nothing about our technology that makes it particularly susceptible to abuse. As we know, there is nothing to stop people from writing bad code in Java or from creating ineffective UML models.

However, we have admittedly not done a good enough job of making clear that we intend people to build DSLs primarily in the context of Software Factories. Yes, we do expect them to build standalone DSLs, but the primary goal of productizing the DSL technology was to support factory development. Why? Two reasons:

1) Factories provide a methodology for determining what to model for a given product family. The key to using DSLs effectively is to understand how solutions are developed for a given problem domain. Factories are organized around a model of the development process for a given problem domain. This model, called a factory schema, is a graph of viewpoints. A viewpoint defines the perspective of a specific stakeholder or a specific role in the project team. As recommended by IEE 1471, and the book Documenting Software Architectures by Clements, et. al., we use the graph of viewpoints to define the product family architecture. We also use it to define the development process, or what Product Line Engineering calls the production plan. Each viewpoint describes the set of workflows used by its stakeholders, the work products acted upon by those workflows, and a set of assets, such as tools, templates, class libraries and guidelines, that support the enactment of those workflows. How does this relate to modeling? The schema shows us what to model. Each viewpoint defines a domain, such as logical database design, user interface process or business rule definition. These domains are aspects or subsets of the larger problem domain defined by the product family. We may only know enough from experience about such a domain to supply documentation assets, such as patterns or practices, to its stakeholders. However, if we know enough about it to model its key concepts, then we can supply more powerful assets, such as a DSL and tools that perform generation, validation, trace, analysis and other forms of automation using models based on that DSL. Of course, implicit in this approach to modeling is the assumption that DSLs are more effective than general purpose modeling languages like UML for model driven development because they are highly focused on specific problem domains. That point is discussed at length in many of our publications, including our Modeling Strategy paper. In summary, a factory contains a model that provides a basis for determining which aspects of a problem can and/or should be supported by DSLs, and how DSL based models should relate to each other and to the work products of non-model driven activities.

2) Software development is about more than just models. While it’s nice to think that some day, we might develop every part of a system using models, just as we currently develop every part of an application using 3 GLs rather than assembly language, it’s just not feasible today. You can hear more on this topic in the recent ARCast on MSDN. Because we take this pragmatic view, we require factories to provide whatever assets are most appropriate to support the enactment of the workflows for a given viewpoint, and not just on modeling. A modeling tool is just one kind of asset that factory builders can provide to support factory users, but there are many others. This is one of the fundamental differences between Software Factories and MDA. You can read more about the differences between them in the article that I wrote for the newsletter of the International Association of Software Architects.

Regarding the effect of our strategy on industry standards, I think it was inevitable that people would demand more focused modeling technologies. Within the OMG, there is very strong momentum toward domain specific modeling. It just happens to be expressed using UML profiles, which are awkward at best. A large block of OMG members would prefer to see domain specific languages defined using the Meta Object Facility (MOF). In effect, that’s what we’re doing, too. However, we don’t use the standard MOF. Neither does IBM, Sun, or anyone else. Why? Because the MOF maps to the internal architecture of the modeling tool. Experiences like IBM AD/Cycle show that tool internals cannot be standardized effectively, since vendors must be free to design and implement tools in response to market changes, not standards committee proceedings. Based on these experiences, we prefer the standardization of interachange formats – not interchange formats like XMI, which depend on MOF alignment to achieve interoperability, but interchange formats like the ones defined by UN/CEFACT, RosettaNet and HL7 to name a few.

As for learning a new language with every gig, you have to do that anyway, whether you’re using UML profiles, or no modeling language, at all. While the UML defines a set of widely understood notational styles, the meaning of any given profile must be learned. This should come as no surprise. What you really have to learn with every gig is the problem domain, or rather the many subsets or aspects of the overall problem domain, and how they all relate to each other. The goal of software factories is to help solve that problem. The software factory schema provides a way to model all of the aspects and their interrelationships. While the DSLs that people build using our tools are unique, they are usually based on the notational styles supplied by the templates that we provide. They happen to look a lot like UML notational styles, of course, and that’s intentional. We have no interest in reinventing the wheel when it comes to notational styles that have proven effective. By the same token, we have not followed any of UML notational styles that have not proven effective, such as the deployment diagram.

It’s true that that we have not yet aligned Software Factories and MSF. That is currently work in progress. When the alignment is published, I think you will find that the result quite different from RUP and XP. For one thing, the development process in a factory is organized around viewpoints, as I’ve just described. For another, the workflow in a factory is driven not by sequential execution, by pre- and post- conditions associated with each activity. Activities come into scope when their pre- conditions are satisfied and go out of scope when their post- conditions are satisfied. Work can be done in any order subject to these constraints.

It’s also quite true that we need to provide a more complete set of tools and templates to support the Software Architect, as you suggest. That’s what we’re busy doing.

Finally, I should point out that a critical mass of industry experts supports our work.

For example, if you read the Acknowledgements in the Software Factories book, you’ll find that we leaned heavily on the work of Paul Clements and Linda Northrop, and met with them multiple times in person. You’ll also learn that Paul was a reviewer, and you’ll find his endorsement of our work inside the front cover.

If you look at our web site, you’ll discover that Linda Northrop invited me to give the keynote at the SEI sponsored Software Product Lines Conference 2004, giving an overview of Software Factories.

If you look at our blogs, you’ll find that Martin Fowler had some complimentary things to say about our strategy in his paper on Language Workbenches.

It may be worth mentioning, while we’re talking about industry expert response to our work, that we have had and continue to have critical interaction with some of the other people on your list, including Mr. Jacobson, Mr. Ambler and Mr. Evans.

Inside the front cover of the book, you will also find endorsements from Doug Schmidt, lead author of Pattern Oriented Software Architecture, Volume 2 (POSA2), David Frankel, author of Model Driven Architecture, Krzysztof Czarnecki, author of Generative Programming and leading voice in the industry on feature modeling, and John Crupi, lead author of Core J2EE Patterns and the writer of our foreword.

There are many other industry experts who continue to review our work, share their thinking with us, and generally advise us in our efforts to craft a pragmatic approach to software development that is firmly rooted in proven practices from software product lines, model driven development and service oriented architecture. You can read what many of them have written in the papers they submitted to the International Conference On Software Factories at OOPSLA 2005.

Moving to Software Factories

JackGre — Sat, 10 Dec 2005 07:40:00 GMT

Here’s an excerpt from an article that Keith and I wrote some time ago.

Total global demand for software will grow by an order of magnitude over the next decade, driven by new forces in the global economy like the growing role of software in social infrastructure, by new application types like business integration and medical informatics, and by new platform technologies like web services, mobile devices and smart appliances. Without comparable increases in productivity, total software development capacity seems destined to fall far short of total demand by the end of the decade. What will change to provide the massive increase in capacity required to meet demand? It is not likely to come from adding developers. Instead, software development methods and practices will have to change dramatically to make developers much more productive.

I’d forgotten about it, but rediscovered it this evening while browsing around. You can read the rest of it here.

Software Factories Applied

JackGre — Fri, 09 Dec 2005 20:24:00 GMT

Keith Short, Mauro Regio and I have agreed to write a new book called Software Factories Applied. Our goal is to write a short book for practitioners that complements the theoretical foundation established in the original Software Factories book by showing how to build a software factory using currently available technologies, namely the DSL tools, the Guidance Automation Toolkit and Visual Studio 2005 Team System. A key topic covered in the book will be how to build a software factory schema, which is the model at the heart of a software factory. Our plan is to walk through a complete worked example from end to end, showing what we did at each step. It will probably be based on a Business Collaboration Factory that generalizes the HL7 factory that Mauro and I developed. You can read about the lessons learned in the development of that factory in this paper that we contributed to the International Workshop on Software Factories at OOPSLA 2005.

OOPSLA 2005 Recap

JackGre — Fri, 09 Dec 2005 05:10:00 GMT

Keith Short has already written a post that provides an overview our activities at OOPSLA 2005. Here is a compendium of my activities at the conference.

On Sunday, 10/16, I hosted a full day tutorial on Software Factories with Steve Cook.
On Monday, 10/17, Steve and I hosted a full day workshop on Software Factories. The workshop announcement and proceedings are posted here.
On Tuesday, 10/18, I spoke on a panel called Aspects – Passing Fad or New Foundation with Dave Thomas, Gregor Kiczales, Ivar Jacobson and Matthew Webster hosted by Steve Berczuk.
On Wednesday, 10/19, I wrote an article for the OOPSLA Conference Newsletter called A Not So Random Walk.
On Wednesday, 10/19, I attended a half day tutorial on Agile Software Development in the Large.
On Wednesday, 10/19, I met with Brian Henderson-Sellers and Don Firesmith to discuss the Open Process Framework.