Test Driven Development (TDD) has three steps: write a unit test that fails, write some code that makes that test pass, and then refactor. It’s well-documented (http://en.wikipedia.org/wiki/Test_driven_development), and there are a huge number of tools like NUnit that support it very well. We have used it extensively on my team, and have come to the following conclusion: it works extremely well for internal implementation details and designs whose public API is not long-lived. However, those making APIs that they expect to support as-is for 7 or more years should be warned: if you rely only on TDD, you’ll end up with some very unhappy API consumers.

Public, supported, frozen APIs need to be designed, reviewed, and explicitly architected.

TDD is wonderful

My team has used a mix of plain ‘ole unit testing and TDD for the last several years. We’ve seen a slight increase in development time, but a dramatic decrease in defect density, as compared to other projects within our division. I personally believe the hype: getting ~80% code coverage at checkin time is worth the effort. Note that below 70% we didn’t see a decrease in defect density, and above about 95% - except for extremely critical pieces of code – there have been diminishing returns.

If you never need to write long-lived APIs, you can stop reading this article. Move along; there’s contentious to see here.

TDD isn't the bee's knees

Practically, we’ve found that the APIs produced by doing TDD – even after refactoring – are not up to the standards that we have for public APIs we intend to support for at least seven years. I’ve seen this on several teams that have used TDD (and one that even shipped the API – ick). I’ve seen this with the output from senior architects with > 15 years of development experience; I’ve seen it from new hires blessed with the Zen beginner’s mind.

It’s not due to a lack of skill, professionalism, or effort. Rather, it’s due to the laser-like focus that TDD encourages on a single set of code. Each piece of code will be individually well-designed, and factored nicely into patterns you could find in any book, but it tends to lack the following characteristics that we require of our long-lived APIs:

·         Support for scenarios that we plan to have in future versions
In our world, YAGNI expands to You Are Gonna Need It. To give an example from Visual Studio, I can’t think of a hook we exposed – many before we had anything but tests for them – that our customers don’t use. In fact, they wish that we had exposed more hooks. If you’re building a platform and your partners like it, you can bet that they will take it in directions you won’t be able to in-house. Part of having a great ecosystem is enabling that.

·         Building a system in a cleanly layered fashion
When projects get big, it’s nice to have some modularity and layering in the system. If you don’t, it’ll be unmaintainable down the road. I only like spaghetti. Unfortunately, there aren’t really any tools to enforce fuzzy concepts like “well layered” or “clean dependencies” unless you decide on them yourself, build tools, and audit the code like crazy. TDD and refactoring do not funnel these out in practice, and missing them early ends up having a huge impact on public APIs (and your schedule…) if you try to add them later.

·         Gradated complexity: simple things are simple and hard things are possible
Just supporting a set of requirements or even a set of pieces of sample code that need to work isn’t sufficient to have a truly great API. We’ve found that you need to also enable a progressive difficulty curve. Straightforward consumers of the API should be able to start a sample project up and use nothing but IntelliSense to figure out how to use it. The hardcore folks need to be able to find the documentation and stitch together pieces to support obtuse scenarios. It’s just part of being a good platform.

·         Resonance with domain-specific expectations
When you write .NET code, there are expectations that you have of any library that you link against: patterns it will use, naming conventions it will have, types it will accept and return, and types that they will derive from. In addition, in certain areas – data access in particular – there’s a huge legacy that has subtly colored the expectations of any experienced developer who touches a new data access API. Sheer luck is not sufficient to ensure you follow these conventions, and not even FXCop can validate that all of the fuzzy details

Again, I’m a fan of TDD personally. And we require unit testing before checkin across the board on my team. But realize that TDD alone isn’t going to be sufficient to ship a set of long-lived public APIs. Anybody who says otherwise is either inexperienced or trying to sell you something.