I will share a straightforward method to help development teams understand and communicate their ability to address the bugs in their areas. Not all product teams in Microsoft use a method such as the one I will describe below, but many do and there is much to be learned from this exercise.
The advantage benefits of this method:
[UPDATE 2010/12/18 – A draft version of the text was originally posted by mistake. The version below has been revised to bring it closer to the intended final version]
First, we need to pick a unit of time for this method. The most convenient is weeks instead of individual days. There are two benefits. First, there are fewer values to plot. Second, we don’t have to account for handling weekends in the calculations.
The number of bugs that are to be fixed at the start of a given week (Monday) is that week’s BUG COUNT. To compute the bug count for week n is simple. Take the bug count for week n-1 and add the net effect of all activities that affected the bug count.
How we compute the “net effect” (we’ll get to this later) is the strength behind the analysis.
We start with the simplest case. We have start with a bug count of 87 on week 0. Everyone is on vacation. Nobody is entering new bugs. Nobody is fixing them.
The chart below is the current model. It is nothing more than the BUG COUNT for each week.
As a convention, cells with red text with a gray background indicate calculated values. Cells not marked in this way are considered input that we can modify,
The table – its form will change throughout this post – is the numeric model of the bug count.
To assist in visualizing the predicted bug count, I will use a simple column chart with the table. Below, you see it matches the table – the bug count is static and does not change week-to-week.
Every week people enter new bugs. This is the INCOMING RATE.
For now I will simply assert that 40 new bugs are filed every week. I include it the model by adding a row called incoming. As you can see below, the bug count is growing linearly – 40bugs/week.
When a feature team looks at the new bugs filed, not every bug is kept. Some of them are dismissed immediately.
The reasons can be any of the following:
It’s not uncommon that 50% of incoming bugs are discarded for such reasons. This ratio of kept versus the incoming is the KEEP RATIO. For example, if we keep half of all bugs filed then then ratio is 0.5 and so we apply it to the weekly incoming number.
Instead of growing 40bugs/week, the BUG COUNT grows by 20bugs/week. The situation is still bad – it continues to trend up - but at least the trend is not as steep.
Now we account for the bugs fixing activity of the developers.
Although the number might seem low, we start by assuming that a developer can fix 1 bug each day. Thus the rate is 5 bugs/week for 1 developer. With a team of 6 developers, the capacity to fix bugs is 6 * 5 = 30 bugs/week. This is the FIX CAPACITY.
Now we include the FIX CAPACITY to the model. I’ve added a net change row to make the change per week obvious.
The model reveals the trend is now shrinking - down by 10bugs/day. This is good, the trend is desirable. Still, the team will not hit zero bugs by week 6.
To MODEL the bug count is to take into account the activities of the team and see how it affects the bug count. In practical terms by considering the behavior of the team – we will directly modify the weekly values for incoming, keep ratio, and fix capacity and then see how those modifications affect estimatin.
Some teams may schedule a “test pass”. In a test pass, the test team will formal run through every set of test cases and scenarios they can think of.
Suppose that the test pass occurs in week 4. Based on some historical data (which would be best) or some guesswork (which is acceptable to start) we can state that the incoming rate will be 2x the normal one and we will keep 75% of the bugs in that week. The reason for the higher KEEP RATIO is that we believe that the test team is catching more real issues with the product via their concerted effort during this week.
Accounting for the test pass, the team is further from the goal because of the influx of bugs from week 4.
The focus of dev and test in a phase shift. For example, after the test pass the test team may not be looking as actively for bugs – they may be moving on to planning activities for the next phase of the project. We adjust for that effect by reducing the incoming rate in week 5.
Likewise the dev team may be moving from fixing bugs to writing design documents for the next milestone so we need to account for that as well – in this case we’ll say that a single developer is moving from bug fixing to planning. Thus the FIX CAPACITY goes from 30 to 25.
We are still at 57 by week six!
The model reveals that the team is not going to hit zero bugs by week 6.
At this point the team requests for developers to be temporarily reallocated from other parts of the product to this one. The general manger will rightly ask how much of a difference will this make. And because we have a flexible way of modeling the FIX CAPACITY – we can show the effect and clearly explain the computation.
Suppose the team request for:
two additional developers for w0 and w1 only 1 additional developer for w2, w3, and w4.
two additional developers for w0 and w1
only 1 additional developer for w2, w3, and w4.
It would affect the numbers like this …
Look at week 6. Week 6 starts with 22 bugs, at the bottom the net change will be –20 bugs. This means week 7 begins with 2 bugs, So now we have a reasonable expectation that with additional help we will be done on early in week 7.
You can find the XLSX file used in the calculations above on my skydrive: