Visual Studio Team System : Team System Column

The Microsoft Solutions Framework: An Integrated Approach to Agile or Formal Software Development Process

AskBurton — Thu, 23 Dec 2004 03:56:00 GMT

Granville G. Miller, Program Manager, Visual Studio 2005 Team System

It would certainly be nice if there could be one way of building software. However, the truth is, there are many different types (e.g. embedded, packaged application, web service, three-tier client server, or web) of applications and so there must be just as many ways to build them. Compound this with the different software development cultures and the different types of competitive environments (hyper-competitive to regulated) and you can reach only a single conclusion. There can be no single software development process for all software development projects. This is the central philosophy behind the Microsoft Solutions Framework.

What every software veteran learns is that the way to adapt to all of the different project climates is to develop a set of good techniques for dealing with all of the different challenges that they might face. In an agile environment, these tools can be used at any time to meet these challenges. In a formal environment, the goal is to pick the correct set for the project and refine them over time. Each of these approaches has merits.

The trouble is that these techniques are divorced from the tooling that they use. The techniques are often shoe horned into a set of disparate tools that were never meant to support them. These tools do not share a common meta-model. Hence, information has to be manually moved from one tool to another. Worse, during the move, the semantic meaning of this information is lost.

The Problem with Process

Most people see the problem with software development processes being a constant struggle between productivity and repeatability. When project schedules get tight, and we rarely have room to spare in the schedule, the process goes “out the window”. But does it really? Even when we go “out-of-band” on our processes, we continue to use our configuration management system and our bug tracking systems so some elements of process remain. These elements are those that are ingrained in our culture and supported by necessary tooling.

When process is divorced from tooling, it creates an impedance mismatch. Ask yourself, when I make a change to the process, do I make a change to the tooling? No? Why not? This illustrates the impedance mismatch precisely. We expect our developers to translate our process on to our tooling. This translation requires unnecessary effort which is often seen as unnecessary when the project schedule gets tight.

The fact is, most processes are disconnected from the tooling that is used to enact them. Tooling is built to support multiple processes (because there can be no single software development process) but it is built to the least common denominator. In other words, it handles the things that all software development processes require but fails to handle the elements that provide the value in differentiating a process. The result is that we fall back to this least common denominator instead of holding on to the parts of the process that provide competitive advantage.

Click here to read more...

Moving to Software Factories

AskBurton — Tue, 21 Sep 2004 00:00:00 GMT

Jack Greenfield and Keith Short, Architects, Visual Studio Enterprise Tools, Microsoft Corporation

Industrializing Software Development

Total global demand for software will grow by an order of magnitude over the next decade, driven by new forces in the global economy like the growing role of software in social infrastructure, by new application types like business integration and medical informatics, and by new platform technologies like web services, mobile devices and smart appliances. Without comparable increases in productivity, total software development capacity seems destined to fall far short of total demand by the end of the decade. What will change to provide the massive increase in capacity required to meet demand? It is not likely to come from adding developers. Instead, software development methods and practices will have to change dramatically to make developers much more productive.

Other industries multiplied their capacity by moving from craftsmanship, where whole products are created from scratch by individuals or small teams, to manufacturing, where a wide range of product variants is rapidly assembled from reusable components created by multiple suppliers, and where machines automate rote or menial tasks. They standardized processes, designs and packaging, using product lines to facilitate systematic reuse, and supply chains to distribute cost and risk.

What will industrialization look like in the software industry? We cannot know with certainty until it happens, of course, but we can make educated guesses based on the way the software industry has evolved, and on what industrialization looks like in other industries. The key is to leverage experienced developers by encapsulating their knowledge as reusable assets that others can apply. Patterns demonstrate limited but effective knowledge reuse. The next step is to move from documentation to automation, using languages, frameworks and tools to automate more of the software life cycle.

Automating Software Development

Can we automate software development? Of course, we can, and we have already. Widget frameworks and WYSIWYG editors, for example, make it easier to build and maintain graphical user interfaces, providing benefits like device independence and visual assembly. Database design offers similar forms of automation. Looking closely at how this was done, we can see a recurring pattern.

After developing a number of systems in a given problem domain, we identify a set of reusable abstractions for that domain, and then we document a set of patterns for using those abstractions.
We then develop a run time, such as a framework or server, to codify the abstractions and patterns. This lets us build systems in the domain by instantiating, adapting, configuring and assembling runtime components.
We then define a language for the domain and build tools that support the language, such as editors, compilers and debuggers, to automate the assembly process. This helps us respond faster to changing requirements, since part of the implementation is generated, and can be easily changed.

The Role of Models

Raising the level of abstraction for developers using higher level languages, such as modeling or visual assembly languages, is one of the key elements of this pattern. We are using UML for documentation, but we are using models based on small, focused, domain specific languages (DSLs) for automation. DSL based tools can help developers define and assemble components, such as web services, generate their implementations using framework completion, and capture metadata used to automate validation, packaging, deployment, configuration management, test generation, defect tracking and many other aspects of the software life cycle. We are using high fidelity DSL based models as first class software development artifacts.

Is that Model Driven Architecture (MDA)? No, not quite. Like MDA, we are interested in models. However, we are less concerned with portability and platform independence than MDA, and more concerned with productivity. While stereotypes and tags can be used to decorate UML models, experience shows that more precise language features are required to support compilation, debugging, testing and other development tasks. Unlike MDA, we do not propose to use UML where programmatic manipulation of models is a key requirement. We use UML for discussion, sketching diagrams on whiteboards and napkins, using the UML static structure and sequence chart notations that are now almost universally recognized by developers.

Not Just Models

While models play an important role, they are not the whole solution. Scaling up to higher levels of productivity will require the ability to rapidly configure, adapt and assemble independently developed, self-describing, location independent components to produce families of similar but distinct systems. It will require a transition from craftsmanship to manufacturing like the ones we have seen in other industries, and it will eventually produce more advanced earmarks of industrialization, such as supply chains, value chain integration, and mass customization. This vision cannot be achieved with models alone. Domain specific patterns, frameworks, and tools must also be developed and applied systematically, and must be aligned with both the product architecture and the life cycle process.

Also, we must address the processes by which we analyze requirements, develop software and deploy it to our datacenters. While prescriptive methods optimize for complexity not change, modern agile methods optimize for change not complexity. To scale agile methods requires the ready availability of best practices, reusable content and patterns. To ensure prescriptive methods are not overly rigid requires flexibility and variability in their description. We are using a methodology, based on these ideas, called software factories.

Click here to read more...

As Simple As Possible, But No Simpler

AskBurton — Tue, 31 Aug 2004 22:10:00 GMT

Sam Guckenheimer, Group Product Planner, Visual Studio Team System, Microsoft Corporation

A Centennial Shift

2005 marks the centennial of Einstein’s publication of the Theory of Special Relativity. Einstein’s work not only settled forty years of debate on the nature of time and synchronicity, but largely set the agenda for much of science, technology, and world affairs for the 20th century. According to most popular legends, Einstein was a solitary theoretician, who discovered relativity in nearly monastic isolation, despite the distractions of his day job reviewing patent applications.

The popular view of Einstein is quite incomplete. Contrary to the popular image, Einstein was very concerned with the inventions he reviewed at the patent office, the majority of which concerned the synchronization of time for multiple practical purposes: industrial (railroads and maritime navigation), technical (coordination of clocks over distance), and political (territorial boundaries).

The culmination of this multidisciplinary interplay was the theory of special relativity. Once in a great while a scientific-technological shift occurs that cannot be understood in the cleanly separated domains of technology, science, or philosophy.¹

A shift similar to the contrasting views of physics 100 years ago is occurring today in software development. 2005 also marks forty years of debate about software development methodology and process, much of which is chronicled in the September 2004 issue of Software Development. During that time, software development process has been described largely as though it were a “pure” pursuit, independent of the tools and technology that implement it. According to most viewpoints, process has been treated as an exercise for managers, specialist Program Management Offices and skilled practitioners, who could define metrics and activities quite divorced from the tools used to implement them.

Two general methodology factions have emerged. Software process, according to the traditional plan-driven view, is about prescribed artifacts and activities that measure intermediate progress toward a goal. This style has been easily caricatured by the Agile Community as unwanted overhead, creating as much harm as good. The Agile Manifesto² drew a sharp contrast between the two viewpoints. Yet neither version is complete.

A Different Approach

Microsoft Visual Studio 2005 Team System takes a different approach, with a big simplifying assumption. The major impedance to process adoption is simply the lack of integration of process and development tools, requiring process enforcement to be a set of purely manual tasks. Once the tooling automates the process guidance and the collection of metrics, then most of the overhead associated with process, and most of the resistance to compliance, will disappear.

Visual Studio Team System extends the concept of an Integrated Development Environment to an Integrated Services Environment for the extended software team – project managers, architects, and testers, as well as developers. Central to Team System is the integration of the tools with the workflow of Microsoft Solutions Framework, a knowledge base of process guidance, based on proven practices. This integration allows Team System to tailor the experience to the individual person and current activity, surfacing the right guidance to the right person at the right time.

Click here to read more...

¹Peter Galison, Einstein’s Clocks, Poincare’s Maps (Norton, 2003), p.40

²http://www.agilemanifesto.org/

Announcing Visual Studio Team System

AskBurton — Mon, 24 May 2004 18:57:00 GMT

With the Microsoft announcement of Visual Studio Team System at Tech·Ed, a dream of mine is coming true. You see, I’m sort of an odd duck here at Microsoft. Although I’ve been in the IT industry for twenty-some years, I don’t have a computer science degree. I never mastered C++. I’ve spent most of my career as a tester, project manager, analyst, and only sometimes as a developer. The only real Windows programs I’ve written were in Visual Basic. I’m not sure how I got hired here; if they’d asked me the tough programming questions, I’m sure I would have flunked.

As a tester, I’ve always understood the theoretical value of advanced developer practices, like unit testing, code coverage, static analysis, memory and performance profiling. At the same time, I never understood how anyone had the patience to learn the obscure tools that you needed to follow the right practices.

As a project manager, I was always miffed that the only decent data we could get was about bugs. Driving a project from bug data alone is like driving a car with your eyes closed, but turning the wheel every time you hit something. You really want to see the right indicators that you are on course, not just feel the bumps when you stray off it. Here too, I always understood the value of metrics like code coverage and project velocity, but I never understood how anyone could realistically collect all that stuff.

As an analyst, I fell in love with modeling. I think visually, and I found graphical models compelling ways to document and communicate. But the models always got out of date as soon as it came time to implement anything. And the models just didn’t handle the key concerns of developers, testers and operations.

And in all these cases, I was miffed by how hard it was to connect the dots for the whole team. I loved the idea in Scrum (one of the agile processes) of a “single product backlog”—one place where you could see all the work—but the tools people could actually use would fragment the work every which way. What do these requirements have to do with those tasks, and the model elements here, and the tests over there? And where’s the source code in that mix?

From a historical perspective, I think IT turned the corner when it stopped trying to automate manual processes and instead asked the question, “With automation, how can we re-engineer our core business processes?” That’s when IT started to deliver real business value.

They say the cobbler’s children go shoeless. That’s true for IT, too. While we’ve been busy automating other business processes, we’ve largely neglected our own. Virtually all tools targeted to IT professionals and teams seem to still be automating the old manual processes. Those processes were high overhead before automation and they’re high overhead still. How many times have you gone to a one-hour project meeting where the first ninety minutes were an argument about whose numbers were right?

Now, with Visual Studio Team System, we are seriously asking, “With automation, how can we re-engineer our core IT processes? How can we remove the overhead from following good process? How can we make all these different roles individually more productive while integrating them as a high-performance team?”

We don’t have all the answers yet, but we’re taking a huge leap forward. When I joined Microsoft from Rational Software a year ago, the Application Life-Cycle Tools market was experiencing another spasm of consolidation, as it had seven years earlier. Customers were seeing more re-labeling of the same old bottles. On the other hand, Microsoft did not join the acquisition fray. Instead, we talked to our customers about their issues and brainstormed about innovative solutions. I’ve had the privilege to meet with hundreds of customers and partners in the last year and ask how we can do it better. We’re starting to show what we’ve learned, but we certainly haven’t stopped asking.

Please join the conversation by signing up for the newsgroups and participating in blog discussions that you will find on our developer center in the “community” section. If you’re a casual reader, we’ll also distill some of the highlights in this column for you every couple weeks-. And if you didn’t make it to Tech·Ed, you’ll be able to see some of the highlights on this site.

I look forward to “meeting” you soon.

Sam Guckenheimer
Group Product Planner
Visual Studio Team System