As many of you may know, Microsoft has a vocal and thriving Agile Software Development community.  Recently, on our community forum, a question appeared about the ability of Agile development to “scale” to a large team.  In other words, if we can make agile development practices work in a dev group with hundreds of people, can we make it work in a dev group with thousands of people?

There was a lot of discussion on the alias.  Much focused on process improvements.  E.g. how to create scrum of scrums, and how to automate test and build processes so that large systems can be integrated continuously.  That is part of the answer.

However one quote, from a seasoned and experienced engineering leader, Nathan McCoy, who joined Microsoft as part of our acquisition of aQuantive , provides a real clue to the rest of the answer.

The answer is yes, agile can scale to larger systems...  Here’s the quote:

When we were in waterfall mode, we tended to batch up our releases.  They were complicated to plan and manage.  We burned people out on death march projects that culminated in release weekends where we would work 72 hours with little sleep and little contact with our families. 

We turned to agile engineering practices – in my case, not simply because I believed it would be a panacea, but rather it gave me a whole arsenal of techniques to make improvements, techniques that built on engineering practices that made a lot of sense to me.

We evolved away from the big batch release by decoupling on component boundaries, putting in services, adding contracts and other techniques mentioned often on this [forum], to the place where we have not done such a big batch release weekend in years.

Let’s look at that for a minute. 

The Nathan McCoy is talking about a painful deployment process that could not scale.  Early on, deployment to live servers would take hours, but as the code complexity and number of customers grew, hours turned to days.  Deployment suffered.  People suffered.  Quality suffered. 

This team turned to agile techniques, and solved their scalability problem.  They did it with decoupling, interfaces, and services.  They did it with architecture.

The real lesson is this: using architecture allowed an agile team to decouple various parts of a system, which enabled agility to go further.  In other words, the success of the agile project depended on the addition of architecture, at the right time, in the right manner.  The problem could not be solved by agile processes alone. 

They solved their problem , in an agile environment, using agile architecture.  What makes it agile architecture?

• The architecture was introduced through refactoring. 
• The architecture supports a specific business problem, and the minimum amount was applied to solve the problem.
• The architecture was not described in a 200 page document beforehand.  It was designed in small increments and expressed directly in code.
• The practice emphasized all of the principles of the agile manifesto: working code, delivered at a sustainable pace, in small quantities, with direct customer involvement, using the best practices available.

My conclusion it threefold:

1. Solution architecture can be applied in an agile manner. 
   
2. As the solutions get larger or teams grow in size and scope, Agile practices alone are not sufficient to solve every problem.  For some problems, architecture is required. 
   
3. Therefore: solution architecture is a necessary and critical skill for agile project teams to master.