Tom Hollander's blog : Agile

The role of an architect in an agile team

Tom Hollander — Wed, 08 Sep 2010 06:18:00 GMT

I’ve just come back from Microsoft’s Tech.Ed conferences in Australia and New Zealand, where I presented a session called The role of an architect in an agile team. Thanks to everybody who attended the session, and for the great questions and eval results. If you weren’t able to come along, the good people behind Tech.Ed have recorded the session and published it to the web. The embedded version is below, or go to the Tech.Ed Online Site for the downloadable version, PPT and links to other sessions.

Looking forward to seeing you all again next year!

Your guided tour of the Microsoft Solutions Development Centre

Tom Hollander — Sun, 03 May 2009 03:21:00 GMT

When I decided to leave the patterns & practices team to move back to Australia, one of my big concerns was whether I would be able to work on teams with the quality and dedication I experienced on projects such as Enterprise Library. It turned out that my fears were unfounded, as I’ve found myself working in the most high-performing team I’ve ever experienced, Microsoft Australia’s Solutions Development Centre (SDC). The SDC is basically a software engineering team available for hire, building complex solutions for external customers. We pride ourselves on excellence in process, technology and outcomes, using the latest Microsoft technologies and agile development practices to drive the best possible outcomes for customers.

About a month ago, the SDC opened its doors to a wider audience in an event we called the SDC Open Day. This consisted of a series of presentations describing all aspects of how we manage our projects, followed by a tour of our facilities with drinks, nibblies and networking. I had originally planned on blogging about this before the event so more of you could attend, but it turned out that we reached our maximum capacity through direct invitations and word of mouth.

The event went off without a hitch, and the feedback from the 150-odd attendees was overwhelmingly positive. However right from the start we wanted to find a way that this event would continue to provide value long after the projectors went dark. So we decided to video the entire event (over two hours of it!) for you to enjoy no matter where you are. The videos have just been posted live on microsoft.com.

If you’re in Australia (or are happy to move here :-), we’d obviously like you to consider how the SDC could help your team become more effective and your organisation to deliver critical solutions. However no matter where you are, I’m hoping you’ll find the videos interesting. Whether you’re a developer, tester, project manager or architect, the videos will show you some techniques that our team has found very successful, which you may want to apply to your own projects.

The event was divided up into 20 minute sessions presented by people playing different roles in the SDC team (including both Microsoft staff and the partners that work with us), which you can view individually in any order. There are also some introductory interviews and “vox pops” to add a bit of spice. The main sessions are:

An Introduction to the SDC, presented by Rob Mawston (SDC Lead at Microsoft). Rob provides some background into why the SDC exists and how we approach software development.
A Day In the Life of the SDC, presented by me (Solution Architect at Microsoft). I take the audience through a typical day, describing the key activities performed by the team and each individual role.
How we do: Project Management, presented by Prasadi de Silva (Senior Project Manager at Microsoft). Prasadi discusses how we use agile requirements management and metrics to ensure we successfully deliver what the customer actually needs.
How we do: Development, presented by Corneliu Tusnea (Senior Consultant at Readify). Corneliu describes some of the techniques our developers use to ensure ongoing quality and agility, including unit testing and refactoring.
How we do: Testing, presented by Sarah Richey and Bruce McLeod (Managing Director and Principal Consultant at Devtest). Sarah and Bruce describe why testing is critical to success, and how we use metrics and automation to give us great coverage throughout the project.
How we do: Build and Deployment, presented by Emma Hanna and Simon Waight (Senior Consultant and Lead Consultant at Avanade). Emma and Simon describe how we use daily builds, CI builds and automatic deployments to ensure the solution is always in a known state for use and testing.
A Customer’s Journey, presented by Fiona Boyd (COO at Ticketek Australia). Fiona describes Ticketek’s experience as a customer working with the SDC.

I hope you find the videos interesting. Please feel free to post any questions on anything you see in the videos to the blog and I’ll try to get them answered (if not by me, then by someone!).

The Joy of Code Reviews

Tom Hollander — Tue, 06 Jan 2009 10:08:00 GMT

As I mentioned in my recent post about how my team does agile, one of the core ingredients of our process is that nobody is allowed to check in without having gone through a code review and a test review. No other team that I’ve worked on previously has had such a rigorous process around code reviews – while we’ve had ad-hoc pair programming and occasional code walkthroughs, there were no rules about all code being reviewed before check-in. So when I first joined my new team at the SDC, I was unsure what to expect or if I’d like it. But as you might guess from the title of the post, I’ve become a convert.

First, let me go into a bit more detail about how the process works. A developer prepares a change set which normally consists of one completed requirement, or one or more bug fixes. Once they believe they are ready to check in, they will shout out “Code Review!”, at which time any other developer who can spare some time will volunteer to be the reviewer. In some cases the “reviewee” will seek out a specific “reviewer” if they know them to be best qualified in the relevant technology or component.

A code review will typically take around 15 minutes, but they may be considerably longer or shorter. It takes place at the reviewee’s computer (we normally have our entire team working in the same room. For a while we did have one developer in another location - in this case we mimicked the “same computer” approach by using desktop sharing and speakerphones). Normally there isn’t any need to walk through the functional requirements or the high-level design in any detail, since the entire team is involved in the planning sessions and generally knows the critical details. However in some code reviews there may be some use of the whiteboard to communicate any design details that are needed to provided context to the code.

The review is performed by looking at the list of changed files in Visual Studio’s “Pending Changes” window, going through them one-by-one (sometimes from top to bottom, sometimes in whatever order the reviewee thinks is most logical), and performing a “Compare with Latest” operation on each file. Most of us have Beyond Compare or something similar installed to make this easier, but the Visual Studio text comparer works OK as well. We don’t have a specific checklist of things that need to be reviewed, but some typical areas for discussion include:

Quantity and quality of unit test coverage
Code readability, method and line length
Opportunities for reusing existing functionality, or merging new code with existing functionality
Naming conventions
Consistent application of patterns
Globalisation (appropriate use of culture classes, resource files etc.)
Hacks, shortcuts or other “smelly” code

If the reviewer is happy with everything in the change set, it’s ready for a test review (or if that happened first, it’s ready to be checked in). Alternatively, the reviewer can veto the check-in and insist that any uncovered issues are fixed first. In rare cases the reviewer may allow the check-in even if there are known issues, with TFS bugs or tasks created for tracking purposes. This option is most commonly used when the issues are minor and there are other developers waiting for the check-in before they can complete their own tasks.

So why did we choose to impose this additional workload across the development team? Well, it’s certainly not because the developers make a lot of mistakes or need close supervision – the team is as experienced and capable as any I’ve worked with. And in fact it is quite rare for a code reviewer to veto a check-in – I don’t have hard statistics but it probably only happens 1 time out of 10. Nevertheless I think the process extremely valuable for the reviewer, the reviewee and the quality of the codebase. First, each developer writes code with full knowledge that it will be scrutinised, so they take extra care to follow established patterns and avoid ugly hacks. Second, it helps “share the wealth” around who understands different parts of the solution. And finally it provides a very personal way for developers to learn from one another, whether it be a new Visual Studio shortcut key, a .NET API they didn’t know existed, or a new architecture, design or testing technique.

One more interesting observation about how this process works in my team: at our “retrospective” meetings that we run at the completion of each iteration, there have been a number of occasions where people have called out that it takes too long to check in code. However I’m not aware of any situations where anyone in the team has suggested abolishing (or even streamlining) the code review or test review processes to achieve this outcome. And having the support and confidence of the team is the ultimate measure of the success of any process.

How my team does agile

Tom Hollander — Tue, 16 Dec 2008 03:02:00 GMT

As you know, I’m a big fan of agile software development. But what exactly does “agile” mean? If you ask a room full of software engineers that question, you’re sure to get as many different answers as there are people. I’m not going to try to tell you what agile is, or what it should be – but a lot of people ask me how our team goes about implementing agile. We don’t practice any particular official brand of agile – like most good teams we’ve combined all of our past experiences to come up with something that works well for our project and team. So I’m definitely not claiming that this is the “right” way to do agile or that it will work for your team. However I’m also not going to accept any arguments that what we’re doing is “wrong” (since it works very well for us) – although constructive suggestions are always welcome!

To provide a bit of context, I’m working with a dedicated software engineering team that is part of Microsoft’s Solutions Development Centre in Sydney. We have been building a large new system for an external customer for the last 12 months or so. It’s largely a greenfield project, but there are a handful of external systems we need to integrate with. The application is in a new business area for the customer so their requirements have been evolving constantly, especially after development began. The project team consists of 2 project managers, 1 solution architect (me), 5-9 developers (the numbers have changed a bit over time), 4-7 testers, a build manager, a release manager (just for the last third of the project), and part-time SMEs focusing on infrastructure, security and database. We also have a full-time product manager from the customer, and around 3 other customer representatives that spend at least a day a week with our team.

The project began with an end-to-end analysis and estimation phase, which lasted a couple of months. While this does not fit in with many people’s idea of agile, I believe it’s a necessary step for commercial projects (and probably worthwhile even for internal projects). The goal of this period was to get as good an understanding as possible of the business requirements (as they were understood at the time) in a relatively compressed time period. During this period the team produced UI wireframes, flowcharts and very high level requirements and design diagrams. Doing this estimation allowed us to provide a reasonably good estimate of the schedule and budget for the overall project (even with the full awareness that many, many things would change throughout the project). It also helped us identify the most important and most risky areas so we could schedule them in the most appropriate iterations.

Once the estimation phase was out of the way we started our development iterations. We settled on 4 week iterations, involving 3 weeks of development and one week of stabilisation. Here’s how we tend to approach each iteration:

During the second half of the previous iteration, the project managers, architect and customer start planning what requirements should be candidates for the next iteration. For a large project like ours, strict stack-ranking of requirements or stories was not found to be practical. Instead we broke the project into larger “modules” of functionality that each fit roughly into a development iteration and prioritised those. This allows each iteration to have a single theme and vision for the entire team to focus on. Within each module we will prioritise the requirements to ensure we work on the most important features first, and often it will turn out that we don’t get time to complete the bottom-ranked requirements (although they may still be resurrected in future iterations). We also find that new, related requirements tend to emerge as we begin development. As long as they are higher priority than other candidates, we try schedule them for the current iteration.
As the architect, I will spend time in the last week of the previous iteration looking through the candidate requirements for the next iteration and come up with a high-level design. This normally involves a first-cut at a data model, 5-10 pages of documentation (usually with lots of pictures!) and often a “spike” or prototype. The project and product managers will also spend time updating or creating UI wireframes and clarifying the requirements stored in Team Foundation Server. The design documents and wireframes produced during this time are never final, but having a documented starting point has proven to help the development team get on the same page quickly and hit the ground running when the iteration begins.
On the first day of the new iteration, the entire team gets together to discuss the proposed requirements in detail. This will typically involve reviewing the UI wireframes and design documents on a projector as well as less formal whiteboard discussions. This is also the time when the development team loves to point out problems in my design and how they intend on fixing them :-). Once everyone has a good understanding of the requirements, we’ll start assigning work to the developers. For a while we tried variations of “planning poker”, but we found it rather time consuming and not overly helpful. So we’ve settled on a simpler system where developers will volunteer for requirements until they believe they have at least a week’s worth of work. They will then go away and break it into finer-grained tasks with estimates. We found that 3 weeks of development is too long to plan for in one go, so we do small planning sessions at the start of the next two development weeks where we “top up” any developers who have less than a week’s worth of work with new requirements. In essence, the development phase of each iteration is broken up into three one-week mini-iterations all with a consistent theme.
We don’t have any official design period within the iteration, but for the first couple of days the developers will usually spend a lot of time at whiteboards figuring out how best to approach their requirements, with actual development ramping up quickly after that. As soon as a developer (or sometimes several developers) is finished with a requirement, it is marked in TFS as “ready for test”.
Testing obviously goes on throughout the iteration, but for the first week the testers typically focus on analysing the requirements and writing test cases. By the second week things are usually in full swing, with the testers developing automated test cases as well as performing manual testing. Any bugs that are discovered are logged in TFS. Bugs are considered “blocking” if they preclude the requirement from being effectively tested, but lower priority “non-blocking” bugs are logged too. Any requirements without blocking bugs are marked as “tested”.
The product manager (who represents the customer to our team) gets the final say on whether a requirement is complete. They will go through each of the “tested” requirements and confirm that it meets their needs. If it does, the requirement will be closed. If not, they may raise bugs (if it is not implemented as requested) or new requirements (which generally means “that’s what I asked for, but not what I want” :-).
At the end of the third week we hit our “code complete” milestone for the iteration. This doesn’t mean all candidate requirements were completed (in fact we try to have more candidates than we can complete), but it means we have completed all of the highest priority requirements in the time available.
The final week is for stabilisation. This means the developers will not start any new requirements, and will work solely on fixing bugs. In order to prevent the bug count from getting unmanageable by the stabilisation week, we also impose a “bug cap” on each developer during the development weeks – once a developer’s personal bug cap exceeds this value they will need to temporarily stop work on new features to fix bugs. However even with the bug cap in place, there are always enough bugs to keep the team busy for the last week. Testing continues during the stabilisation week as well (in fact it’s usually the busiest week in the iteration for the testers). To be completely sure we don’t run out of bugs, we also schedule a “bug bash” event at the start of the stabilisation week. This involves the extended team (usually involving people from the customer who are not normally involved with our team day-to-day) spending an hour playing with the application with a goal to discover as many bugs as possible. To make this fun we put on food and drinks, and hand out prizes for the most bugs, the most severe bug and the most obscure bug.
At the end of the fourth week, the iteration comes to an end. We strive to exit each iteration with zero P1 and P2 bugs, and an agreed limit to the number of P3 and P4 bugs. This effectively means that everything we have built in that iteration is complete and stable and ready to be deployed. That said, for this project we don’t actually deploy the application to real users after each iteration. As we get ready for our final release there will be a “final stabilisation” iteration, as well as a number of processes that can only be completed at the time of the final release such as end-to-end security and performance tests, deployment and failover testing.

Hopefully this gives you an idea of the rhythm of our iterations. However we also have a daily rhythm within our team room:

At 9am we have our daily stand-up meeting. We’re lucky enough to have our entire team working in the same location, so this doesn’t involve any fancy technology like conference calls. The goal of the stand-up meeting is to make sure everybody knows what’s going on in the team. Each team member gets up to 2 minutes to describe what they did yesterday, what they are going to do today, and what impediments (if any) they might have.
After the stand-up everyone starts work – although it’s normally quite a noisy affair (well it is when I’m involved!). We have our own dedicated team room so we can make as much noise as we like, and we encourage face-to-face collaboration on problems as much as possible. Many times a day, someone will raise interesting and difficult questions that may require the customer to clarify requirements, project managers to reprioritise or modify requirements and plans, and surprise changes to the design and test cases.
We encourage our developers to check in at least once a day to prevent painful integration and long periods where the testers can’t make progress. Before anyone in the team can check in (yes, me included!) they must go through a code review and a test review. Any developer or tester can perform these roles for anyone else, and we encourage as many combinations as possible. For the code review, the reviewer and the reviewee will share a computer and go through each of the changed code files and discuss the changes. The reviewer has the right to veto the check-in if they find problems, which can include inadequate unit test coverage. The test review is much the same but normally involves running the application through whatever obscure scenarios the tester can think of to see if it holds up. Once both the code and test reviewer are happy, the developer can check in.
We run a continuous integration build after each check-in, where the application will be compiled and all of the unit tests run on the build server. If the build breaks, noisy sirens will alert everyone in the team to the problem, and whoever caused the problem needs to fix it immediately as everyone else is blocked from checking in. A good CI build will result in a much more cheerful “Yippee!” sound.
At 4pm we do our daily bug triage. This involves the test lead, one of the project managers, the customer product manager and the architect (me). We will go through each of the bugs raised in the last 24 hours, debate its priority, decide if and when it needs to be fixed, and (if it’s not closed) assign it to the most appropriate developer to fix.
Also at 4pm we commence our daily build. In addition to compiling the code and running the unit tests, this involves deploying the application to our test server and running a suite of automated Build Verification Tests. Again, if any of this fails, the appropriate people need to down tools and get it all fixed.
Once the build is declared “good”, we will start our nightly run of automated test cases. The next morning the testers will analyse the results of the previous night’s run and raise bugs for any regressions, or sometimes update the automated tests to deal with failures caused by changing requirements.

I hope this gives you some insight into one way of using agile methods to deliver a complex project. As I mentioned at the start of the post, this isn’t textbook agile, and we’ve needed to constantly evolve our process to address various challenges and to suit the personalities and experiences of our team. However I’m very happy to say that this is the most passionate, energetic and productive team I’ve ever worked with, and despite the fact that the requirements continue to evolve at what often seems like a frightening pace, we’ve continued to hit all of our milestones and targets. So I hope some of this will be of use to your team as well, and of course I’d love to hear about what you do differently in your team and how it’s working out for you.

Thoughts on being a Solution Architect

Tom Hollander — Tue, 29 Apr 2008 13:49:19 GMT

About a year ago I put together a post called Thoughts On Product Management, containing some random musings about my role at the time. The big reason I put together this post was because so few people had any idea what this job involved or why it is important.

Now that I've got "Solution Architect" on my business card again, I thought it would be worth putting together a sequel post. However the role of "architect" suffers from an entirely different problem to product management, which is that so many people interpret the role in so many different ways. Partly this is because there are so many different types of architects, such as enterprise architects, application/solution architects and infrastructure architects. But even within just one of these job titles, I've see a huge variation in different people's interpretation on the skills, tasks and responsibilities involved.

I'm not going to pretend to be able to answer the question of what a solution architect should do any better than anyone else can. But I did want to share some of my own experiences on what seems to work well - much of it learned from working with other architects on past projects, and hopefully still relevant and effective for my current team. I'd also like to hear your thoughts, whether you are a solution architect or someone who works with one, about where you think the most important responsibilities lie.

When I look at the work of a solution architect, I think it's interesting to separate the "what" from the "how". The "what" side, while definitely not simple, is probably the less contentious. Figuring out how to meet current and future business and non-functional requirements, making sense of new approaches and trends such as SOA, ESB, Web 2.0, federation, etc, and determining where and when to apply patterns are the bread-and-butter of an architect's job. But what does that really mean on a day to day basis? That's where the "how" comes in, and I'm sure that like me you've seen a huge variety of approaches. The stereotypical extremes are the "ivory tower architect", who has a three-ring binder full of Zachman matrixes and UML diagrams but is completely disconnected from reality, and the "developer in denial", who is really just a developer who wants a more fancy job title.

I'm sure you won't be surprised to hear my view that a real architect needs to lie somewhere in the middle. Certainly architecture is a distinct discipline from development. While moving from development to architecture is a common career path, just because you are a good developer does not mean that you are necessarily destined to become a good architect. And the reverse is also true - most architects (myself included) are not able to keep up with the coding skills of the developers on their teams, especially once they stop full-time development. That said, I personally do not see how someone can be an effective solution architect without some kind of development background and a thorough understanding of the application's ever-evolving code base.

So without further ado, here are some of my current thoughts on what being a solution architect (specifically one in an agile team) is all about.

Minimise the amount of code the developers need to write

Developers are paid to write code, and they are generally excellent at it. However once a developer is assigned a swag of requirements or stories they need to get down to work on those specific requirements, and it's not easy for them to keep up with what everyone else is doing in any level of detail. This can include discovering synergies between different requirements or opportunities for macro-level code reuse and refactoring. A big part of the architect's job is to pick up on these opportunities as they arise and ensure that developers aren't reinventing the wheel in their own worlds. Ideally this should result in patterns, components and frameworks that allow the developers to get their requirements done with less code, by concentrating on those parts that are unique.

Ensure an approach works before you ask the team to follow it

While it's nice to be able to replicate approaches that have proven successful in the past, new technologies and unique projects mean that it's often necessary to break new ground. Dealing with these challenges is a part of the job for everybody in the project team, not just the architect. However when there is a work coming up which will require the attention of multiple developers at the same time, those developers will be a lot more productive if they are given some initial guidance on how to approach the problem. Some architects like to communicate to developers with documentation, which can be fine when the approach is well understood. But when the solution has never been tried before, I believe it's critical for the architect (possibly with help from others) to do enough prototyping to ensure the solution is sound before it's handed over to the dev team for the real work to begin.

Make sure the requirements, and the technical implications are understood before it becomes a distraction

Business priorities and requirements are always in flux. The great thing about agile projects is that this fact is embraced rather than feared. Good developers and testers are able to cope with change well through high test coverage and continuous refactoring - but at the same time it's important that developers don't get too distracted while there is still considerable churn in requirements. Understanding how new requirements get injected into the project is largely the domain of product and program managers, but the architect has an important role to play too. In many situations I've found that the business stakeholders will modify, reprioritise or cut requirements based on advice on the technical implications. This process can take some time, and I believe it's important to shield the development team from most of these discussions so they can concentrate on implementing and testing the requirements that are better understood.

Put together just enough design docs and diagrams, just in time

I've seen a huge spectrum of opinions on how much, and what kind of documentation architects should produce. In my experience, face-to-face discussions over a whiteboard or computer terminal are normally a far better communications tool than any kind of documentation - however this does not mean that documentation has no value. In particular when there is a need to persist ideas or solutions for the team or external stakeholders, documentation is unavoidable. My strategy is to write documents only if I can identify the exact individuals who need it, and to hold off on writing the documents to the last responsible moment (to avoid rework when everything changes). In practice I tend to produce small design documents (typically about 5 pages long) throughout the project that each drill down on one specific topic. This may include a few block or UML diagrams, but I don't like to use any standard templates as each topic will require a different level of detail and will need to be explained differently.

Do enough development work to stay relevant

Whenever someone assigns me a requirement or a bug I often joke that if they want it done anytime soon they really should assign it to somebody else. There is quite a bit of truth to this - on any given day I'm randomised enough to make it difficult to make a lot of progress on development tasks. Still I believe it's very important to have these tasks on my plate (especially when I get the fun ones!). The main reason is that it forces me to stay familiar with the code base and the day-to-day issues facing the team (like how long it can take to check in!).

Help with questions, problems and code reviews at any time

I've left this one for last, but I think it's the most important part of my job (and definitely most time consuming!). On any given day most of the developers in my team will come up with unexpected issues and questions. In most cases they can be resolved with a quick chat or whiteboard session, and in almost all cases everybody involved contributes great ideas that help shape the solution. Since these kinds of issues are by definition unpredictable, helping people out with these can be quite a distraction. However if somebody isn't available then small issues can become major roadblocks very quickly. In addition, participating in these discussions helps me keep my finger on the pulse of what issues the team is facing, and can sometimes uncover a need for the overall architecture to be revised to minimise future issues.

So consider this one more opinion to add to an ever-growing collection of thoughts on this topic. As always, I'm always very interested in hearing yours as well.

Why your customers will love agile (even if they think they hate it)

Tom Hollander — Sat, 16 Feb 2008 10:36:00 GMT

Project management methodology has never been a particularly sexy topic. While it's always been necessary to employ some kind of process to guide any non-trivial software development project, in my experience most of the team will only expend the bare minimum effort to learn and follow the process (that is, just enough to get the project manager off their back). It definitely isn't a topic that's likely to create heated discussion at the team's Friday night drinks.

Well that was true, until the advent of agile development. For some reason, whether they love it or hate it, everybody loves talking about agile. For example, Microsoft's own internal discussion list on agile development often receives dozens of posts each day, and the company isn't even a particularly big follower of agile approaches. It seems project management is sexy again.

But one of the most fascinating aspects of this is that the interest and adoption of agile approaches tends to be driven mainly by developers. Not project management types, not management and not customers. I can't offer a good explanation of why this is the case, but I do think it's great to see developers taking a front seat in driving the evolution of our discipline. Still I find it strange that the people who probably have the most to gain from the benefits of agile are often the least willing to give it a go - the customer.

In some cases this may have been brought on by a bad experience with a team that used "agile" as a synonym for "having no process at all" or "not thinking ahead". These kinds of experiences are unfortunate, but hopefully will become less common as the growing interest in agile turns into practical experience and development teams start to understand that agile projects require at least as much discipline as waterfall projects, albeit of a very different type. However in my opinion a bigger reason why many customers fear agile projects is that they are so used to being promised a fixed scope on a fixed date with a fixed budget, but agile projects will never give that promise. On an agile project the customer does not know upfront precisely what they are going to get for their money. On a traditional project, they know exactly what they are going to get. Of course, most traditional projects blow out in schedule, scope, budget or all of the above, so the customer probably won't get what they wanted at all - but at the time the deal was signed, all was crystal clear and the ultimate failure of the project was clearly someone else's fault.

Some months ago I posted about the challenges with selling customers on agile processes, particularly in competitive tender situations. This post generated some fantastic discussion in the comments, and it's not my intention to go over that same ground again. Instead I want to talk about how customers who reluctantly or accidentally end up employing an agile development team will learn to love it, provided the team knows what they are doing. This all comes from my own experience, not just as an architect on agile projects, but as a "reluctant customer" myself. As I've probably mentioned before, when I joined patterns & practices as a Product Manager (aka "customer proxy") I knew practically nothing about agile development but found myself working with an agile team. After a few weeks of bewilderment, it dawned on me how empowering this development approach was for me, "the customer". Whenever new information changed my previous assumptions on requirements or priorities, the team was completely happy to allow for this. No change request forms. No complaining. No schedule blow out. These guys, thought I, are on to something here.

Let's explore some customer realities and how agile development can help:

No matter how detailed the original requirements, some things cannot be anticipated until you see the application in action.

I've worked on projects where requirements have ranged from 500 page specs and dozens of UML diagrams, to scribbles on the backs of coasters and random conversations. While the level of detail between these extremes is obviously very different, the one thing they have in common is that neither accurately reflected what was eventually delivered. Even though the whole point of requirements analysis is to get as deep an understanding of the business needs as possible, some problems or needs will only surface with real users interacting with real functionality. In some cases these problems can be overcome with minor tweaks, in others they will require significant parts of the system to be thought about differently. Either way, the ability to identify these problems early and inject new or changed requirements into the backlog is critical to ensuring that the product that gets delivered is what the business really needs.

A lot can change in the business during the life of a software development project.

Many software projects can span two years or more, even before their schedules slip. And of course, a lot can change in two years. Competitors can launch new products. Business strategies can change. Divisions can be reorganised. Mergers and acquisitions can occur. In short, almost all the business drivers that led to the project's existence are subject to change. Projects that are unable to course-correct are either cancelled, or will be essentially useless when they are delivered.

Development cost does impact prioritisation.

Like it or not, software development estimates have a pretty high error rate. Even if requirements stay relatively stable, the complexity and rate of change of software makes estimation very difficult. Many organisations have estimation methodologies that can prove quite good for a complete project, but even then it's likely that some requirements will take twice as long as expected and others half as long. Many feature prioritisation decisions are made with an explicit awareness or implicit assumption of development cost. If the costing changes, the prioritisation may change too - for example, if feature A is cheap then it's really important, but if it turns out to be really expensive then the business may decide that features B, C, D and E are more appropriate. Agile projects allow updated costing information to be made available, and decisions made, in real-time. In a waterfall project the scoping and prioritisation decisions are generally set in stone before accurate costing is possible, and a blow-out in one feature can blow out the entire schedule.

When you've gotta ship, you've gotta ship.

Every project has a budget. While some projects are in a better position to secure additional funding than others, no project stakeholder wants to go back to their superiors and explain that the money has run out and they have nothing to show for it. Imagine needing to report one of the following possible outcomes at the time that funding for a project runs out:

The project delivered everything we wanted, on schedule and on budget.
The project delivered the most important functionality that we wanted, on schedule and on budget. If we decide the remaining functionality is needed, we can get this done for approximately $x more.
The project has run out of time and money and they haven't delivered anything yet. However if we can secure $x in more funding, we should be able to get it finished.

Obviously everybody would want to go to the board with outcome 1, but if the estimates were inaccurate or the scope changes, outcome 2 will certainly look better on your performance review than outcome 3 - especially if the additional $x is not forthcoming.

So what's the bottom line? A customer working with a high quality agile development team:

Has more chance of getting business value delivered within an allocated budget,
Has more chance on getting what the business actually needs, not what they originally thought they wanted, and
Is empowered to make tough decisions throughout the project, and has the current project status information to inform these decisions.

Who could ask for anything more?

Code Generators: Can't live with them, can't live without them

Tom Hollander — Sat, 17 Nov 2007 12:34:46 GMT

I'm still not sure what I think about code generators. This may sound strange, coming from someone who has spent much of the last few years working on and talking up software factories, of which code generation is a significant part - but it's true. On one hand, I love the idea of eliminating manual coding of routine tasks and recurring patterns, improving productivity and minimising bugs. On the other hand, every code generator I've ever worked with has had problems, whether it is in the cost of maintaining the tool and templates, or issues with the generated code.

I like to divide code generators into two categories. The first is the "black magic" type, where you never change, or even look at the generated code. The good thing about this type is that you can re-run the code generator as often as you want without worrying about overwriting any of your changes. The bad thing is that if the generated code isn't exactly what you want, you're in trouble. There are a few ways you can tweak the code without actually changing anything written by the generator, such as using partial classes or inheritance, but your options are always going to be very limited.

The other category is the "one-time accelerator" type, which will spit out code which is hopefully pretty close to what you want, but which will need to be modified by hand to get it exactly right. The advantage of this approach is that you should always eventually be able to get what you want, but it means that you'll have to manually re-apply your changes every time you regenerate. It also means you need to fully understand the generated code, since you're ultimately responsible for maintaining it.

My main quarrel with code generators stems from the fact that we all want the "black magic" type, but in my experience they hardly ever deliver on their promise. The problem is that all too often the generated code just doesn't do what you want. This leads to a few possible outcomes:

You stubbornly stick with whatever the generator gives you, and consequently you are forced to engineer all sorts of hacks in your own code to work around the shortcomings in the generated code.
You modify the code generation templates, resulting in a vast array of additional configuration knobs and dials, so that the generator is able to build the code you need for your application. But chances are that these changes won't actually help for any future applications, as they will all bring a brand new set of idiosyncrasies and require yet more knobs and dials.
You bite the bullet and modify the generated code to meet your needs, dumping it into the "one-time accelerator" bucket and forcing you to live its implications.

One possible explanation as to why these problems are so common is that if you ever do find a problem that can be solved well by "black magic" code generators, you can probably codify the solution in a framework or component library, eliminating the need for any kind code generation whatsoever. The challenges with "black magic" code generation are the reason why patterns & practices software factories generally don't even try that approach. We tried to mitigate the "one-time accelerator" limitations by only generating a small amount of code in one go, but this brings its own set of problems.

This topic is at the front of my mind as code generators have caused a bit of angst in our team lately. We're using a generator (home-grown, but much the same as other solutions you've probably seen) to generate data access layers, stored procedures and business entities. The code that it generates is generally very good (otherwise we wouldn't be using it), but as always it isn't perfect. The biggest problem is that, for any given table, the generator will give you a complete suite of CRUD operations whether you want them or not. For many people this may not be a big deal - but I downright refuse to have code in my solution that is unnecessary and untested. My fear is that if we leave this code in the solution, at some stage some developer will be tempted to call it - and since nobody ever asked for it or tested it, it may be completely unsuitable for the application. So my rule is, if the generator builds something you don't need for your current task (even if it may be needed later), it's not allowed in the solution.

The problem is, since we're using a lot of agile development techniques, we tend to update our database schemas quite a lot. This means that we need to regenerate our data access artifacts a lot as well. To make matters worse, we've also found the need for the occasional tweak to the generated code to make sure it meets our requirements. So the combination of frequent schema changes, my rules about stripping out unneeded code, and the need to hand-tweak the code means that the generation process is fast becoming more trouble than it's worth. I know we could make changes to our generator or use an existing one with more features to get around some of these problems (such as being able to specify and save which operations are generated for each table), but I fear that we'll never get quite where we want to be. But on the other hand I'm concerned that if we stop using a generator for our data access artifacts then we'll face a swag of different problems, such as inconsistent implementations and increased development time.

This is where you can come in and save the day. The first person to explain how to make code generation work well in this situation (preferably without causing any disruption to our team or schedule) gets a six pack of the Aussie beer of their choice. Unfortunately due to customs regulations you'll need to come by to collect - but believe me it will be worth it.

Breaking the cycle of failed development projects

Tom Hollander — Sun, 03 Jun 2007 03:44:45 GMT

It's been a bit over two weeks since I arrived back in Australia, and things are slowly coming together. We're still waiting for all of our stuff to arrive, and for that matter a place to put it all - but at least it's sunny every day, and I'm enjoying my new role at Microsoft's Solutions Development Centre.

While it is strange going back to the world of consulting, I'm sure my tenure with the patterns & practices team will give me some new perspectives for the role. As an example, the subtle persuasion of people like Peter Provost has made me an agile development convert (although certainly not an expert!). While I believe there is a growing trend towards more agile approaches in many organisations, the p&p team is definitely ahead of the curve - but I'm very interested in applying what I've learned about agile development in my new role.

But I suspect it will be an uphill battle in many cases. One of the first tasks that was thrown at me was to read through a draft "request for tender" document from a customer. This document was essentially asking for a fixed scope, within a fixed delivery schedule, for a fixed price. I realise there is nothing particularly unusual about this request. But we all (hopefully) know that it can't be done, short of the odd fluke. Each project is governed by the "iron triangle" of scope, schedule and resources. You can choose which ones are fixed and which are variable, but you can't fix all three without sacrificing quality, unless you are extremely lucky and everything comes together within the expected parameters. Depending on which reports you believe (such as the Standish CHAOS reports), only 15-30% of projects finish on schedule and on budget, a similar proportion fail outright, and the remainder fall somewhere in between with the project delivered but not meeting expectations.

The thing I am trying to understand is why we (meaning both the development teams and the business groups) insist on playing this game by pretending it can all be done. Business, of course, wants the certainty on what they will get when and for how much. Most development and consulting shops will take a project with a relatively detailed (but never final) requirements list and fixed schedule, and happily provide a fixed price quote (probably with a hefty risk premium attached). But with such a poor project success rate, surely everybody should know that the emperor has no clothes? My cynical explanation is that this model makes it very easy for everybody to blame someone else. The business can blame the development teams for failing to meet the agreed deadlines or requirements. The development teams can blame the business that the initial requirements were not detailed or accurate enough. Everyone walks away vindicated that it wasn't their fault that the project failed.

This is where agile development approaches should really be able to help. I've had no formal training on agile so forgive me if I misrepresent anything, but as I see it one of the most fundamental concepts is that scope is never fixed, but is aggressively and continuously prioritised. Nobody should promise exactly what will be delivered at the end of the project, although it makes sense to do some upfront analysis on the minimum and preferred scope to get a reasonable idea of the how long it will take and whether this will fit into the proposed budget. Another key agile concept is the project will be broken into multiple iterations and releases, each of which will be deployable and provide real business value. This approach allows requirements questions to be addressed continuously based on customer feedback, allows scope to be reprioritised throughout the project, and mitigates "big bang" releases that often result in the business saying "this is just what we asked for, but not what we want". Finally, in theory at least, a well-run agile project should never ever run over time or over budget. It may not deliver all of the functionality the business wanted, but the functionality that was delivered should at least be usable and valuable, and more important than any scope that was not delivered.

Anyway, enough of Agile 101. My real question is more about psychology than methodology. What will it take to get people to understand, and act on the fact, that the way most projects are run doesn't work? Most of the agile success stories I hear about (including p&p, it turns out) have been grass-roots campaigns from developers who slowly convinced management teams in their own organisation to move to more agile approaches. This is a great start, and I guess it's gotten us to where we are now. But I'm not personally familiar with any examples of an external systems integrator or development shop successfully convincing a non-agile customer to think about their development approaches differently. Things get even harder in a tender situation, where the customer will have set ideas on how tenderers should respond to allow for an "apples to apples" comparison. If one response says "yes, we will deliver all of the scope in X months for $Y", and another says "We will deliver something in X months for $Y, but we can't tell you what it will be yet", you can bet the customer will choose the first option, even if the second has a far better chance of success.

I'd love to hear your thoughts on all of this, especially since I've been living in an alternate reality away from the consulting world for quite a while. Is the general state of software development really as bad as I make out? Is agile development a critical factor in addressing these problems? Have you been successful in introducing agile approaches in your own organisations or to external organisations? Is it possible to win a tender by proposing an agile approach when the customer wasn't expecting it? What do you think it will take to break the current cycle of blame and get everyone to think differently about software development?

Thoughts on Product Management

Tom Hollander — Mon, 26 Feb 2007 02:33:00 GMT

A lot has been written about the constantly evolving disciplines of development, test and project management. As software engineering matures and moves towards more agile approaches, many people have shared their thinking and experiences on how to do their jobs better. It occurred to me that I've seen practically nothing on the web or in books that describes the job that I do, which we call Product Management. I can think of a few reasons for this - many teams don't have anyone doing this job at all, some teams use the same title for something quite different, and since the patterns & practices is not a typical team, the requirements and approaches we follow are not always relevant to other teams. But that said, I think our team has enough in common with other development teams that it is worthwhile sharing a few things I've learned after doing this job for the last 3 years or so. I'm not claiming that this is the right or the only way of doing things - in fact it would be great to hear from other people who perform similar roles to find out what they do differently.

What is Product Management?

As I mentioned before, the term "Product Management" tends to mean different things to different teams, even in Microsoft. In many teams, the Product Manager is essentially the marketing person for a particular deliverable. I've got nothing against marketing people - it's an important and difficult job - but it's not the primary responsibility for Product Managers on our team. Our definition of Product Management is based heavily on the Microsoft Solutions Framework Team Model, which was defined a number of years ago in the days before MSF was so closely tied to VSTS. A paper on the MSF Team Model from 2002 nicely describes the role as follows:

Goal	Functional Areas	Responsibilities
Satisfied customers	Marketing Business Value Customer Advocate Product Planning	Acts as customer advocate Drives shared project vision/scope Manages customer requirements definition Develops and maintains business case Manages customer expectations Drives features vs. schedule vs. resources tradeoff decisions Manages marketing, evangelizing and public relations Develops, maintains, and executes the communications plan

I like to describe the role as being the primary interface between the engineering team and the customer. This means acting as an advocate or proxy for the customer for all important team decisions. It also involves acting as an a proxy or advocate for the team when dealing with customers. In our group we believe it's critical for everyone in the team to deal with customers, and the Product Manager should not be the only interface between the team and the customers - however for the Product Manager it is the primary responsibility.

How does this relate to other team roles?

One of the key properties of this team model is that there is no one "project lead". Instead, there are a number of key roles that are equally important, each addressing different aspects of the project. The MSF Team Model describes the roles as Product Management, Program Management, Development, Test, User Experience and Release Management. In the p&p team, we also have a dedicated Architect role.

Most of these roles should be pretty self explanatory, but many people are often confused about the differences between a Product Manager (PdM) and Program Manager (PM). In fact in many teams, both roles are played by the same person. The MSF guidance explicitly advises against this, and I strongly agree with this advice. To understand why, I'll give the MSF definition of the Program Manager's responsibilities (I greyed out the points that are covered primarily by the Architect in our teams).

Goal	Functional Areas	Responsibilities
Delivering the solution within project constraints	Project Management Solution Architecture Process Assurance Administrative Services	Drives development process to ship product on time Manages product specification—primary project architect Facilitates communication and negotiation within the team Maintains the project schedule and reports project status Drives implementation of critical trade-off decisions Develops, maintains, and executes the project master plan and schedule Drives and manages risk assessment and risk management

Goal

Functional Areas

Responsibilities

Delivering the solution within
project constraints

Project Management
Solution Architecture
Process Assurance
Administrative Services

Drives development process to ship product on time
Manages product specification—primary project architect
Facilitates communication and negotiation within the team
Maintains the project schedule and reports project status
Drives implementation of critical trade-off decisions
Develops, maintains, and executes the project master plan and
schedule
Drives and manages risk assessment and risk management

As you can see, the PdM's primary responsibility is to deliver the right product. The PM's primary responsibility is to deliver the product right. On a good day, these goals are completely compatible. However in all projects there are times when very difficult trade-off decisions need to be made, such as deciding whether to implement an important feature that may result in the project missing its deadlines. Having one person drive that decision results in an inherent conflict of interest. The PM/PdM division means that different people will be fighting for different things - the PM will fight for meeting the agreed plan, and the PdM for delivering necessary customer value. Even if a compromise is not possible, the role division results in increased transparency which normally results in a better decision.

Staying connected to customers

The most critical part of Product Management is figuring out who your customers are, and staying connected to them. I came from a consulting background, in which case it was usually obvious who your direct customer was, since they were the one that signed your timesheet. In a situation where your team is producing software for the masses (which is the case for p&p, and is also true for ISVs and even some framework or infrastructure teams inside enterprises or system integrators), things get much trickier. Even in a consulting situation, you probably have many indirect customers who are harder to identify and build relationships with.

As a new Product Manager in p&p, the hardest part for me was bootstrapping the entire process - even though we had a large installed base of people using the pre-Enterprise Library blocks, I had no systematic way of finding them. I literally started with people I knew, and gradually expanded my contacts with the people they knew, and so on. But the thing that really allowed us to find and communicate with customers were blogs and community sites (first GotDotNet, now CodePlex). To me, it's not only very cool, but critical to our group's success that we can have a many-to-many, many-to-one or one-to-one conversation with anyone in the world who uses our stuff. Now I know that our team is in a unique position, and it isn't realistic to expect communities of thousands of people around the types of projects you may work on. But the one thing I think you can take away is that, with few exceptions, being as transparent and open as possible with as many of your customers as possible will help you get a much better result.

While our broad community activities are the most visible, it certainly isn't the only way we work with customers. Data from a large community is extremely valuable in showing trends and getting answers to focused questions, but it's very difficult to get a detailed understanding of what any given person is trying to achieve or why they made a certain decision. To get this kind of data, it's necessary to form deeper, personal relationships with certain customers so you can really understand what makes them tick. But you can't be overly dependent on this kind of data either, as you can only have so many relationships at the level and it can't be taken for granted that these customers are representative of the broader customer base. As such, I've found that best way of understanding what customers need is to use a spectrum of "depth" to "breadth" techniques. In our team, this includes the following, ranked roughly from depth to breadth. Different kinds of teams will almost certainly warrant different approaches, but I'd say the breadth/depth concept should still apply.

Personal experiences
On-site visits to customers
Conference calls with "expert advisor" customers
Workshops
Meetings and conference calls with individual customers
Events and Webcasts
Discussions on community forums
Blog posts and comments
Online surveys

How does Product Management fit into an Agile team?

As you may know, the patterns & practices team takes a lot of pride in our success in implementing agile development processes. However there wasn't any one day when a decision was made to "go agile" - it kind of crept in over the last 3 years. I had no previous experience working in agile teams, so the whole process was extremely interesting but also quite difficult since I didn't really know what was going on or how I was meant to fit into this new process. I still don't know the official answer (if there even is one), but I can tell you want I've found.

There are many aspects to p&p's interpretation of Agile that make the Product Management discipline even more important than it would be in teams following different processes. Most importantly, there is a need to set the scope priorities up-front, and continually refine them throughout the project. The priorities need to be refined to keep them in sync with our changing view of reality as the project progresses, such as technical dependencies and complexity, and our ability to execute. An agile project treats dates and budgets as fixed (unless the person paying the bills decides they want to change them), so scope is the major variability point. Determining which features are in and out, and how deep to go into each feature, can only be done with extensive customer input throughout the project, so it's a core responsibility of the Product Manager.

Another key aspect of agile projects is that you don't pretend to know all of the answers up-front. You certainly need to have an idea on what you want to achieve, but detailed requirements and design questions can only be adequately answered once a lot of other pieces have fallen into place. From the Product Management perspective, this means you don't finish gathering requirements until the day you ship (and then you start gathering requirements for vNext :-). For example, in Enterprise Library 3.0 we decided very early on that a Validation Application Block was a top priority, and that it needed to provide a technology-independent way of specifying validation rules and validating data. We did not know which exact validator implementations we would provide, or which technologies we would build integration adapters for - these decisions were made only after we already had the core validation engine in place. In order to deal with these questions "just in time", the Product Manager needs to be deeply engaged with both the engineering team and customers throughout the project (which, in my view, is far more interesting than delivering a requirements specification paper upfront and leaving the team to figure out what it means and how to resolve inconsistencies).

Breaking the rules

One more important thing I've learned is that, while it's good to understand the formal division of responsibilities across different roles in a team, the best results can come when the boundaries are blurred. The "products" that our teams build consist of source code given to other development teams, rather than finished applications that address real business requirements, and practically everyone in p&p comes from an architecture/development background. We also have a culture that puts a lot of value in customer connection from everyone in the team. All of this means that many day-to-day tasks can be handled by many different individuals with different job titles. Oftentimes this means that key decisions, whether on scope prioritization, architecture or development are made jointly by different people in different roles. And sometimes certain tasks are done entirely by someone in the "wrong" role. I've personally completed many tasks that technically are owned by Architecture, Program Management, Development, Testing, User Experience and Release Management (yes, that's every role we have!). Many key Product Management tasks in our projects have been done with great success by people who aren't Product Managers. This flexibility is not just acceptable, it's often necessary as teams have differing workloads and individuals have their own strengths and weaknesses. All in all, the role responsibilities are better seen as a model for ownership, rather than a rigid rule on who does what. For example, if there is a need for a QuickStart sample to be developed and I'm willing and able to do it, then I should go ahead - but if the development lead isn't happy with my work then they get the final say on whether it's checked in.

I hope you found this valuable - I'd love to hear your feedback on which aspects of this are and aren't relevant to your teams, whether or not you are a Product Manager in name or responsibility.