The Tyranny of Waterfall in RFPs
When I was in consulting, we would routinely bid on RFP (Requests for Proposal) issued by government agencies and large corporations where the client would require the project to be performed in the worst possible model: Waterfall. (If you aren't familiar with the Waterfall Software Development Lifecycle Model, you are probably using it.)In this model, all analysis is done before any design... all design is done before any construction... all construction is done before any deployment... and nothing ever ships on time or within budget. These projects are very difficult to properly estimate. A project size grows, so does the risk of delivering a late project with major features snapped off.This was a problem as a consulting company, because we were routinely asked to deliver software for a fixed price with fixed requirements (fixed bid contracts). As a result, we were between a rock and hard place: asked to use a software development model that nearly guarantees failure, yet accept all the risk for failure.Is it any wonder so many consulting companies go out of business.
How did we get here?How does a bidder force a waterfall model in an RFP, you ask? It's easy! Simply tell your vendor that they must first deliver the analysis document, then the design document, then the code. In that order. Even better, require that each document must be formally approved by the agency before the next step may begin. (this is a recipe for suicide.)If this nearly always fails, why do it? It's part of a vicious circle that comes from failure and lack of trust. A project is described pooly and a consulting company, eager to get the bid, responds with a fixed price that is risky. The project fails to deliver value on time or within budget. So, as a way to cut risk, the business or agency asks for something of measurable value for each payment. That's the idea anyway.
The problem is: what artifacts of software development have measurable value? In the waterfall model, we would state that the analysis is valuable. We put a dollar figure on it, and manage it as a project.
The same with the design. It is documented to the hilt, with excessive details dealing with everything from architecture, to message definition, to class diagrams, to statements of how performance can be achieved.
I won't be the first person to tell you that this process doesn't work. So if the process doesn't work, how are these documents valuable? The idea is that you could have company A develop the analysis, company B develop the design, and Company C could use them to develop the application.
This is nonsense. There are two reasons why:
Therefore, if these documents fail to deliver a working project, then your controls don't work and the project fails. This engenders more distrust because the agency thought that the vendor knew what they were doing... so they increase oversight and set up payment terms based on delivery of these artifacts, in this order.
Setting up contract terms means assigning a dollar value to each phase. That creates an interesting question. If this process is a sure road to failure, what value should the signposts of failure really have? Not enough to pay for, I'll tell you that.
So why are most software development RFPs out there written to not only encourage this practice, but to create cash flow problems for companies that want to avoid it? It occurs to me that many of my friends in the government and large corporate circles may simply not know of another way to manage a vendor.
My response to darkness: light a candle
Now that I'm on the buying side (again), I've crafted a new list of requirements for a vendor to fulfill. Of course, this list will be vetted by my organization. So, I decided to put a vanilla list out here, on the web, as a way of providing information for my former clients and other interested parties to see. I fully expect that my organization will change this list, so this is my way of getting my opinion out unaltered.The message I want to share is this: provide these criteria to your vendors, instead of the arcane requirements of the waterfall model, and you will encourage your vendors to follow BEST practices, instead of WORST practices... that will make a much bigger difference than you can imagine.
First off, let me say that not all vendors perform all tasks. I am breaking down the tasks according to analysis, design, construction, and delivery, because it is the model that my former clients are familiar with, and because some vendor companies cannot do all of the steps. So, when reading each section, realize that each section stands alone. It can apply to a vendor hired to fulfill that role alone, or to a vendor that fulfills that role as part of a larger process.
However, if you do have one vendor creating requirements and another doing design, and potentially a third doing construction, you have to provide the facility (desks, computers, phones, server software licenses) for them to develop all of the artifacts in one place, because they will ALL be working AT THE SAME TIME.
This is absolutely key to success: the primary effort of analysis is not done until design has released an iteration and code is being delivered, and it has no positive impact if the developers don't meet, know, and trust the analyst(s).
Agile Requirements Analysis
If a vendor provides the analysis team, then add descriptions to this effect to your RFP.
The vendor will deliver a requirements document containing the items listed below. The requirements document will be delivered more than once, in iterations. With each delivery, it will be updated to reflect changes. Changes must also be tracked using a formal change management process. Note: The level of detail for each use case must match the use cases delivered. Therefore, if a code drop contains use cases 1-7 + 12 and 14, the we would expect that these nine use cases will be detailed and described in the requirements document re-delivered with that drop.The requirements document contains:
The use case survey (step 1) will be delivered before the design document. However, as each use case is detailed, it must be delivered to the design team as part of a weekly deliverable package. The design team is responsible for delivering the design components to the analysis team and the construction team in a joint weekly meeting.
The analysis team will meet regularly with members of the client staff, design team, and construction team. For larger projects or remote projects where the project manager may not be able to attend, minutes of these meetings must be produced and filed with the project office showing attendance and results.
If the analysis team begins a use case, it must deliver results for that use case every week until it is complete. The results must be understandable and useful, but do not need to be complete. The analysis team must be aware that design and construction will begin immediately after the first iterative delivery of a use case. Therefore, the use case should contain enough information to assist the design and development team as much as possible.
The vendor performing the analysis should expect that analysis activities will not complete until well after the first stable code drop is accepted by the customer. However, the headcount needs will diminish dramatically between the first proof of concept deliverable (at peak) and the release of the first stable drop, when a small fraction of the expected team would be required.
How not to do analysis as you would in a waterfall project:
If you notice from my text above, we don't require all of the analysis up front. In fact, don't require that any of it is delivered before the first code delivery. That's right... code first. Caveat: that code should be "proof of concept" code... completely disposable but eminently useful.
By letting your consultants write code before all analysis is done, they flush out further requirements, discover snafus in their ideas, get infrastructure in place, and push risk from the system, early. Wasn't that the reason we wanted all this stuff up front in the first place?
Agile Design by VendorsIf a vendor provides the design team, then add descriptions to this effect to your RFP.
The vendor will deliver a design document that contains the following items. The document will be re-delivered with each iteration, updated to reflect changes. Note: it is not important for this document to be 100% complete until the code complete milestone. It is, in fact, preferable, that only portions of the document are delivered with each iteration.
The goal of design is to progressivly develop an approach for describing the components of the system, their responsibilities, and how they interact. Therefore, with each delivery by the analysis team, a series of diagrams will be produced by the design team (within a week) that illustrates how the newly delivered requirements affect the current expected design. The first iteration of design begins after the use case survey is delivered.
The results of the design effort are presented at a weekly joint meeting with the developers, the customer and the analysis team. The goal is to flush out misunderstandings and see if the customer is likely to approve of the technical approach. After the most useful use case (as judged by the client) is delivered to the design team, and the design team develops diagrams to support construction, construction will begin.
The vendor performing the design should expect that design activities will not complete until well after the second stable code drop is accepted by the customer. However, the headcount needs will diminish dramatically between the first stable code drop (at peak) and the release of the second stable drop, when a small fraction of the expected team would be required.
How not to do a Waterfall design
Similarly to how we did the analysis, we will require that the vendor produce a design. However, we will not require all of it at once. In fact, we do not require more than the mere framework to be delivered until a period of coding is complete (often a month of coding). Then, ask for all sections to have some detailed content. With each iteration, EVERY section should become more detailed until that section is completely specified. (It doesn't matter which section is completed first, just that they are all completed along the way).
Also, require that the developers doing the code are updating this document, but that they are not spending more than 10% of their time on it. That means, in a typical month, they would spend no more than two days working on the document. The rest of the time is to be spent writing code.
What if that's not enough time to do a "good job" on the document?
Guess what? The document is supposed to be a communication mechanism to allow developers to speak to each other... not to you. If the developers finish writing code, and didn't need the document to have more in it, then the document is good enough.
Really. It's OK to write down only what you need to convey and communicate.
Of course, the closer your users can get to the development team, the fewer words you have to write down. Users nearly always do a better job of explaining what a system should do than developers do. After all, they are paying for it, using it, and counting on it.
What to require during Software ConstructionIf the vendor is involved in the construction of the system, then place text to this effect in your RFP.
During construction of software, the vendor will be required to adhere to the following practices:
How to avoid Waterfall constructionYou can see the iterations in this process, much more so than in the Analysis and Design phase. That's because we assume you have to do SOME analysis and SOME design before the construction team delivers usable "final state" code. This means that each iteration contains an update to those documents. Code, on the other hand, doesn't get done until it's done. As a result, the iterations are much more stark.All the same, each iteration must be time-boxed. There is a fixed (short) period of time in which to deliver. If you have to choose between shipping all features and shipping on time, drop a feature. Pick it up in the next iteration. Do Not Slip.
Even more importantly, each iteration must contain a representative of every deliverable artifact. In other words, if you are planning to deliver an online help document, you should deliver one with each iteration, even if the first three times you deliver it, it doesn't contain that many articles.Don't be tempted to say "it's a short project... we only have time to deliver once!" If the project lasts more than a week, you have time for multiple deliveries. Remember: deliver deliver deliver
Intelligent Software Stabilization
If your vendors are involved in the final delivery of the software system (nearly always true), then place text to this effect in your RFP.
The construction phase transitions to stabilization when all use cases required for the delivery of the system have been coded to completion. This determination is made at the sole discretion of the construction team in conjunction with the customer.
At this point, one more full code iteration will occur for the construction team to stabilize the code base and refactor out inefficiencies and correct mistakes made in the implementation of patterns and algorithms. The functionality at the end of this iteration is not expected to change much from the prior iteration. This is placed specifically for the construction team to improve quality, fill out remaining (boundary and exception) test case code, and make the system more resistent to events that could cause failure.
The delivery of the first drop in the stabilization phase will occur to the client systems at this point, and user acceptance testing will commence. The users understand that the system may not be stable at this point, and the users will have been involved in seeing code releases before this point. However, this phase will allow for minor (cosmetic) changes to the functionality as well as the repair of defects.
All testing is expected to be conducted on the client's site, and will involve client resources in individual contributor roles. It must occur on hardware and software that is a good model for the equipment and configuration that will be used in a production environment. The stabilization phase will include the following specific deliverables:
The system will be considered complete after the users accept the functionality as being "correctly interpreted from the user cases as delivered". Note that any expensive changes that are first raised during user acceptance testing will be considered a change request if the client organization could reasonably have raised the same issue during one of the demonstrations held during the construction phase.
The text of an RFP is wildly important to how the entire project will be run, since it drives financial terms, measurement criteria, and even decisions on the part of experienced consultant teams on whether to even bother bidding on the contract. If the description of the terms of software development are carefully described to encourage agile software development processes, then the client company and the vendor will both benefit from a more reasonable and more orderly process, where information is collected early and presented often in a valuable manner.