During the Christmas holidays I read about something I would like to call CUT (Continuous Unit Testing). CUT is a technique where you run your unit tests in the background all the time rather than after each time you compile (I'm assuming TDD/BDD is being used all the time). There are a number of reasons I'm skeptical toward this approach.
My first concern is why the tests are run in the background and not part of the regular build process. It could be because the build environment does not have the ability to run custom actions after each build. Hm... Let me rephrase that: The developer does not know how to add custom build steps to the build process. And that is not the only problem. Every time I write a new test I have to wait for the background process to finish, start a new test run and wait for that result. This means the average turn around time each time I write a test is not only the time it takes to compile and make a single test run, it is the compile time plus 1.5 times the test time.
Another argument might be that the use of a background process to run tests makes it possible to continue writing code while the tests run. But then you're not really following TDD practices, are you? You're just mixing in your test & code writing continuously running the tests.
But there is one relevant reason for running unit tests in the background. If it takes longer than you can wait to run all tests you will probably start running only a small subset of tests for the code you're actually working on and then run all tests just prior to check-in (or have the build server run all the tests for you). This will however give you feedback that you've changed something you didn't perceive quite late. Under these circumstances I think it makes perfectly sense to run all unit tests in the background and only have a small subset running each time you compile for that quick feedback in your TDD-loop.
IMHO, having the ability to run unit tests fast is important if you are to be successful with TDD/BDD. And sometimes you end up with a number of tests that are not fast enough (or you have so many tests that they all together takes too long). Under these circumstances the background test runner makes sense but not otherwise. Also, having unit tests that are slow is typically a sign of problems but I think I have to elaborate on that later.
When Agile Sweden had its Christmas party one of the lightning talks was on how some of the XP values seems to have been forgotten now that agile is becoming more and more mainstream. I think we all can agree that agile is much about communication and feedback which happens to be two of the XP values.
The next one is simplicity. Simplicity is kind of built into most agile practices but many people forget about it. For example agile practices more and more often tend to be initiated by management rather than the people actually doing the work. This also means much focus is on tools. Nothing bad with fancy tools but in my experience the teams performing best (with agile practices) are those with the simplest approach to everything. So in my opinion, simplicity is in the danger zone.
One XP value that I think definitely have been forgotten is courage. Especially since agile is more and more pushed down by management rather than pushed up by the developers. In my years as a consultant I've seen so many developers that just go ahead and do as they're told grumping about what a bad decision it is to do something this way or that way. I've also seen a fair amount of architects patching and patching a doomed design rather than throwing it away and do it the right way. This can also be seen in developers in a small scale where they stick to a bad design even after they see it is a bad design.
The last XP value is respect. Since a few of the other values are forgotten there is clearly no respect for the values so by definition this value is forgotten too...