One of the things I spend a lot of time doing as a test manager is mining for interesting data in our work item tracking system (TFS of course).
For example, this morning, I went through a query of bugs that had been resolved as “won’t fix” but were also marked as regressions from previous releases. Most of these were correctly resolved because we’ve rewritten some features and the new features have different/improved functionality. But a few of them had truly worked in past releases and were now broken and for whatever reason our feature teams had decided not to fix them. Because we had added the “regression” field to our bug form and because people had correctly filled out these fields, we can easily identify such issues and dig into whether or not these decisions were correct.
Out of the box, the bug forms in the Agile and CMMI processes are fairly sparse (at least compared to our internal bug form). If they don’t have fields to track the information you need, you’ll have to customize the forms. Learn how to customize WIT forms here (TFS 2005, TFS 2008, TFS 2010 Beta)
Of course, there’s a fine balance to aim for between overloading your bug form with tons of fields and having a bug form that’s simple enough as to not discourage people from actually filling out all the fields properly. Here are some tips I’ve learned to help improve your chances of success:
Here are some other interesting queries I’ve run recently, the fields I use to gather data, and some follow-up questions I ask for each.
This one might take a little explaining. We use a parallel development model called “feature crews”. This means small teams work on new features in branches until the features are “complete”, then they merge those changes up with a reverse-integration into a higher level branch. We guide teams to fix all their feature bugs before integrating their work back into the main lines to avoid accumulating technical debt.
I hope this post helps ignite some conversations around how to identify opportunities for process improvement in your organizations.
Remember, you get what you measure, so measure what you get!