James Newkirk's Blog : Microsoft patterns & practices

The CodePlex Team and the patterns & practices Summit

jamesnewkirk — Sun, 21 Oct 2007 02:56:00 GMT

The CodePlex team will be well represented again at the patterns & practices Summit in Redmond, WA - November 5-9, 2007 . We will be presenting or co-presenting the following sessions:

"Yet Another Talk on Agility" - Brad Wilson and Peter Provost
Dependency Injection Frameworks - Brad Wilson and Scott Densmore
EntLib Devolved - Scott Densmore
Make It Your Own - Scott Densmore
Lessons Learned in Programmer Testing - Jim Newkirk
CodePlex is sponsoring an evening event on Tuesday titled "Open Source in the Enterprise".

Peter's not on the CodePlex team but we like him just the same. For additonal information on the summit and all of the sessions please click here.

xUnit Test Patterns and Better Software Development for Agile Teams

jamesnewkirk — Sun, 01 Jul 2007 02:30:00 GMT

These two books were recently published by Addison-Wesley. For a blog post on each book see the following on my new blog:

xUnit Test Patterns by Gerard Meszaros
Better Software Development for Agile Teams by Will Stott and James Newkirk

If you have any questions please let me know. Will and I look forward to your feedback on our book.

Moving Day

jamesnewkirk — Mon, 25 Jun 2007 18:44:00 GMT

I have decided for a number of reasons to move my blog to a new address. The new address is http://jamesnewkirk.typepad.com. For a few posts I will be cross-posting but about a month from now I will be at the new address exclusively. Please join me at my new blog home.

How to make sense of Code Coverage metrics

jamesnewkirk — Mon, 26 Feb 2007 02:14:00 GMT

(co-authored with Brad Wilson)

Even since the release of Visual Studio Team System 2005 with easy to use code coverage tools, people inside (and outside) of Microsoft have been talking about what it means. The most common approach we've seen is for a team leader to mandate a minimum code coverage number (like "all code must have at least 80% code coverage before being checked in"). As agile advocates, you'd expect that our team's TDD'd code would have 100% code coverage. And you'd be wrong.

We could all agree that test-covered code is better than not, so we probably also agree that a high coverage number is better than a low one. Why can't we come up with a hard and fast number that means "good"? The answer is that it's different for every project, and even for a single project, that number may change over time. It is inevitable that code is written which isn't covered by a unit test. A few examples of acceptable code without tests might include: web service wrappers generated by Visual Studio, views in a Model-View-Presenter system, and code for which failure is only possible because the underlying platform fails (like helper methods that pass default values into more complex .NET CLR methods).

So when you run that code coverage system for the first time and it pops up "73%", what next? Well, if it's the first time you've run the test, you're probably going to check out that other 27% and see how you categorize it. Does that code need coverage or not? If so, use the lack of code coverage to educate yourself on taking smaller steps in TDD. But if not, what then? Is 73% a magical number?

The answer lies in a metric that can be used to predict the weather, barometric pressure. Today, weather.com says the barometric pressure in Seattle is 29.66”Hg and falling. More important than the absolute measurement is the trend: it tells you that the overcast and drizzling rain outside is going to get worse, not better. The two pieces of information - value and trend - are used to predict what will happen next. The same can be said for code coverage. You can determine the relative health of your tests by using the value and the trend to determine the appropriate action. That’s the important bit: code coverage gives you relative measurements against itself, not an absolute measurement against a target value. If your coverage fell from 73% to 72%, would you be worried? What if it fell from 73% to 67%?

Rather than holding your team to an absolute goal, use the trend to determine when you should invest time to figure out where you're covered and not, and to determine what to do about it.