Being Cellfish : TDD

Results of a simple TDD experiment

cellfish — Wed, 06 Jan 2010 09:06:00 GMT

Well, I wish it was my own report this time but it isn't. Instead I'd just link to this little experiment. As mentioned in the comments it is not a statistically proved result but I still think it's good that somebody took their time to add one more data point to the overall statistics. Hey, it inspired me to do that too. Well I probably can't do the same detailed story point per hour thing but I should be able to at least count the number of bugs found in different areas of the code and compare that to complexity and test coverage. No idea if that is interesting but it might be something. Just hope people with more data publish it even if not what they want.

Mocking framework comparison

cellfish — Sun, 03 Jan 2010 09:03:00 GMT

As you may know I think mocks should be used quite carefully. But as I've pointed out before I think you should use the best tool available and that means to use a mocking framework sometimes. One thing I've never really liked with the typical mocking framework is how it requires you to learn a new API to setup the test double. And if you don't know the API I think a typical mock setup can be very confusing if you see the code for the first time. However if you choose Stubs as your framework you get the benefit I'm looking for (primarily not having to hand roll my test doubles). You avoid a powerful framework that can mock things you shouldn't really mock (making your code easier to test by design rather than by use of framework). And also you do not require the user to learn a new API; Stubs uses standard language features (delegates and closures) to setup the stub.

There is actually a great project where you can compare different mocking frameworks (including Stubs) called mocking-frameworks-compare where you can study a number of different tests written with different frameworks. Here is one example from that project using Moq:

   1:  public void TouchHotIron_Yell()
   2:  {
   3:      var hand = new Mock<IHand>();
   4:      var mouth = new Mock<IMouth>();
   5:      hand.Setup(x => x.TouchIron(Match<Iron>.Create(x => x.IsHot))).Throws(new BurnException());
   6:   
   7:      var brain = new Brain(hand.Object, mouth.Object);
   8:      brain.TouchIron(new Iron { IsHot = true });
   9:   
  10:      mouth.Verify(x => x.Yell()); 
  11:  }

If you're not used to Moq I think there are a number of things that stand out as weird. First of all there is no assert in the test. Second the setup is not obvious. And it is not obvious that we are verifying Yell is being called or if Yell is going to be called by the verify method. Now, let's look at the same test using Stubs:

   1:  public void TouchHotIron_Yell()
   2:  {
   3:      var hand = new SIHand();
   4:      var mouth = new SIMouth();
   5:   
   6:      hand.TouchIron = (iron) =>
   7:      {
   8:          if (iron.IsHot)
   9:              throw new BurnException();
  10:      };
  11:      bool yelled = false;
  12:      mouth.Yell = () => yelled = true;
  13:   
  14:      var brain = new Brain(hand, mouth);
  15:      brain.TouchIron(new Iron { IsHot = true });
  16:   
  17:      Assert.IsTrue(yelled);
  18:  }

You might notice that the test is a little longer and feels more verbose. But that is not necessarily a bad thing. On the contrary I think that is a good thing. Also the only framework specifics you need to know is to use the stub-class and know it has a property to define behavior. Apart from that there is also a performance benefit from using Stubs which you can read more about here (note that the support for partial mocks that is mentioned as not yet supported is now supported). All this things considered I think Stubs is the best mocking framework choice you can do today.

Preemptive comment answer: But Stubs is not a mocking framework! Well I think it is because I can create mock objects, fakes and stubs (i.e. any kind of test double) with it. So the same thing as Moq or RhinoMocks for example. Same, same, but different...

Advent calendar 2009 - how to TDD thread safety

cellfish — Sun, 29 Nov 2009 07:28:00 GMT

Last year I wrote the same test in 24 different ways as a little advent calendar for you to enjoy. This year I'm going to do something similar but also a little bit different. I'm going to start with a simple piece of code and then show how this code can be tested for thread safety in different ways. I think it's pretty common to think that thread safety is one of those things that are really hard to test and something you typically handle by good practices and code reviews. Very similar to logging in an application. Most people don't TDD their logs. But why not? Personally I've experienced situations where fixing a bug means making something thread safe or adding a better log message. When I do this I want a test to reproduce the bug and verify the fix. These tests are typically hard to write since the code was not written to be easily tested in regards to logging or thread safety. But it doesn't have to be that way. For example there is a testing framework that basically works with verifying logs (TextTest) and thread safety can absolutely be designed using TDD. Remember; if you think it is hard, you're probably doing something wrong. Let's decide after Christmas if test driving thread safety is better than the standard way of just adding it where you know it is needed.

Team Coding Dojo 7

cellfish — Fri, 06 Nov 2009 14:11:00 GMT

This time we did a new Kata - variant of poker hands. Instead of being so specific about the interface the user story we used for the Kata was simply; Given two five card hands I want to know which hand is the winner. We also choose to use the object calisthenics rules. The biggest benefit from not having a specified interface was that we did not have to do any string parsing and could design our interface as we wanted. That definitely felt like a kick start for an object calisthenics session. In the retrospect we also noted that in some cases it is good to bend the object calisthenics rules for the sake of progress with the Kata. Sometimes it is just not worth following the rules strictly. But each breach of the rules should be carefully considered since the purpose of the rules is to flush out changes you would normally never see.

The poker Kata is definitely an interesting one and I can see how the solution can take many different paths. Looking forward to try it again.

Refactoring is not rewriting

cellfish — Thu, 05 Nov 2009 04:55:00 GMT

Today I got that feeling of how the use of one word, interpreted differently by different people result in misunderstandings. It was when I read this which IMHO is full of misunderstandings (including some of the comments). One example is that refactorings must break unit tests. No that can never happen because refactoring is about changing internal structure and design without changing the functionality. What is typically a pain with a bunch of unit tests in place is when you realize your design sucks and you want to rewrite something. The border when you go from refactor to rewrite may be gray but refactorings are always small and should never change the functionality (including the API). So when I say unit tests makes it easier for me to do refactorings that says nothing about when I need rewrite/change something.

There is a valid point that having unit tests means you have to change unit tests when you need to rewrite/change something. But if you've written your unit tests and code in a good way this impact should be small. Just because you feel something makes your life harder sometimes does not mean it is wrong, it might mean you're doing it wrong. And one way to prevent yourself from having to do these cumbersome rewritings is to make sure your code is very simple and that each entity only does one thing. Something that happens almost by itself if you're doing TDD... In my experience, every time I find a solution hard to test or hard to change because the tests breaks massively with a small change, it does not mean TDD is a bad idea. It has always meant that I should solve the problem in a different way where the code is easy to test and tests are not fragile.

Coding Dojo 6

cellfish — Tue, 06 Oct 2009 08:05:00 GMT

It was MineSweeper for the 6th MSFTCorpDojo today. We also applied the object calisthenics rules this time. We ended up with a lot of interesting discussions on how to design to follow the calisthenics rules and this lead us to not really driving the design using tests but rather to create tests following the design. In my opinion it was also unsatisfying that we only implemented the small building blocks we thought we needed and almost no part of the algorithm needed to solve the problem. After using object calisthenics a few times it feels like it leads to interesting design discussions at the expense of the basic TDD experience. It'll be interesting to see if we'll be able to get both in the next session.

Coding Dojo 5

cellfish — Wed, 02 Sep 2009 19:02:00 GMT

Yesterday was the fifth MSFTCorpDojo. MineSweeper as usual and we tried switching person at keyboard on each arrow in the TDD cycle (and not really pairing). As last time we tried BDD style tests. Didn't end up quite as neat as last time but we still learned something. I also noticed that everybody was very engaged this time. Don't know if it was the attendees or the fact that you could end up at the keyboard very soon all the time. Anyway I think we'll decide who goes next by random next time. Just to keep everybody on their toes... Will be interesting to see how that works. We'll also try to apply the object calisthenics rules next time.

This time it was also very clear how one person's decision on how to write a test or how to implement one thing may drive the complete solution in one direction unless you decide to do major refactorings. We also had to remind our selfs a few times not to do the smallest or dumbest change to make a test path but actually the simplest thing. I think it was nice how we sometimes tried to much, recognized that fast, backed out and did a simpler thing.

Team Coding Dojo 5

cellfish — Thu, 27 Aug 2009 09:24:00 GMT

This time we took the completed BankOCR Kata from last time and tried to refactor it using the object calisthenics rules. It was really fun and the dojo was quite different from the other dojos which has been more TDD focus. This time we had a lot of design discussions and we had to force our selfs to just do some refactoring and see where it took us. I think it was great to see how we refactored and created new classes just to later refactor these classes to nothing and removing them. It was a great experience in how refactoring in steps reveals the design for you. We also had the full test suite save us a bunch of times from stupid bugs which is also nice.

But refactoring existing code to follow the object calisthenics rules is very hard and takes time. So there is much more work to do. We decided that we maybe want to continue this exercise later but for our next team dojo we will try a new Kata and apply the object calisthenics rules from the beginning trying to keep the code base conformed to those rules at all times. Also we want a Kata with no string parsing of two dimensional things...

We actually did another fun thing. After the dojo we went to watch D9 at a local cinema. A great finale!

Object Calisthenics: First contact

cellfish — Sun, 16 Aug 2009 06:45:00 GMT

A few weeks ago I was introduced the the object calisthenics described by Jeff Bay in the book The ThoughtWorks Anthology. The object calisthenics is a way to practice writing object oriented code. The nine rules of are not intended to be used in your every day work. The rules are intended to be used on a small problem such as a coding Kata. The idea is that by applying these strict rules on a small problem you'll learn to write better code.

So I decided to try this on the MineSweeper Kata. In the beginning I decided to try to conform to the rules all the time, but pretty soon I changed my mind and wrote a working solution and then started to refactored to conform to the rules. I think this was a mistake. Some design decisions turned out to require very big refactorings when conforming to the rules and I actually never got all the way. But this doesn't mean I didn't learn anything. First of all I think I experienced a variant of you can't test in quality, you have to build it in from the start. I should have stuck with the initial strategy and make sure the code followed the rules all the time. I also learned that classes that I felt were really small and doing only one thing actually could be split up when I had to in order to conform to the rules. Reminds me of when people thought atoms were the smallest building blocks of the universe and then it turned out to be something smaller...

So all in all I think doing a coding Kata while applying the object calisthenics rules will improve my ability to write object oriented code. And it will be interesting to see how it works out in a coding dojo. By now you're probably wondering what the nine rules are. The rules are:

I'll go into more detail of these nine rules over the next few days.

What is a Coding Dojo?

cellfish — Sun, 09 Aug 2009 08:53:00 GMT

I've heard that some people don't think the CodingDojo site tells you a lot about what to expect in a coding dojo. It's more about practical things and listing a number of good problems to solve during the dojo. So I'll try to elaborate. In a coding dojo you will learn something. It might be TDD if you don't know a lot about it or it might be some smart trick in the development environment. You will also learn something about how others write their code. You will learn this since a typical dojo is setup with one computer desktop projected on the wall and all participants takes turns at the keyboard. You should not be intimidated by the fact that you will code in front of others for a few minutes. People attending dojos typically like to teach and learn so you will get help from the group.

You should not expect to see the announced problem being solved. In my experience it is very rare that a dojo ends with a complete solution. And completing is not the goal. Learning along the way is the important thing. A coding dojo is a safe environment where you can try new ideas and make mistakes without putting your real work at risk. A coding dojo is also a great opportunity to improve your development skills.

A typical dojo starts with a short introduction to the environment and a short design discussion. In most dojos two people then starts coding as a pair. After a few minutes (typically around five) one person in the pair gets substituted by somebody else in the group and so on. Even though the person at the keyboard is in command, the whole group typically participates giving suggestions on what to do. If more discussion is needed, that is done without stopping the timer counting down to the next switch. The dojo ends with a retrospect so that the dojo can be improved to the next time.

Coding Dojo 4

cellfish — Thu, 06 Aug 2009 08:00:00 GMT

Fourth MSFTCorpDojo today. MineSweeper and MicroPairing this time too.This time we, as a result of the retrospect from last time, we did BDD-style testing (I've added one of the test classes below as an example). We decided to start by implementing a parser that parsed the input and created an internal representation of the input that we could later use to generate the output. By the end of the session we had a pretty complete parser with error handling for all kinds of bad input. This is one of the few times I've seen any real error handling in a dojo.

This time in the retrospect we said that the next time we should implement the other part next time. That is; assume a parsed input in some internal representation and generate the output from that. We also talked about how we do MicroPairing and switch one person in the pair every seven minutes. We decided to next time switch one person in the pair every time the keyboard gets passed instead. Guess that will mean less pair programming since there is no real ping-pong but rather a circular queue. But in our session almost everybody participates all the time anyway so that might not necessarily be a problem.


   1:      public class Given_a_single_1x1_fieldset_with_no_bombs
   2:      {
   3:          private string input;
   4:          private Field field;
   5:   
   6:          public Given_a_single_1x1_fieldset_with_no_bombs()
   7:          {
   8:              input = "1 1" + Environment.NewLine +
   9:                      "." + Environment.NewLine +
  10:                      "0 0";
  11:              field = FieldParser.Parse(input).Single();
  12:          }
  13:   
  14:          [Fact]
  15:          public void It_should_have_one_row()
  16:          {
  17:              Assert.Equal(1, field.Rows);
  18:          }
  19:   
  20:          [Fact]
  21:          public void It_should_have_one_column()
  22:          {
  23:              Assert.Equal(1, field.Columns);
  24:          }
  25:   
  26:          [Fact]
  27:          public void It_should_have_an_empty_square_0_0()
  28:          {
  29:              Assert.False(field.IsBomb(0, 0));
  30:          }
  31:      }

Visual Studio Options for TDD revisited

cellfish — Fri, 31 Jul 2009 17:31:00 GMT

So I previously mention a neat way to set up visual studio to work with a console test runner so you don't switch to the error list all the time. There is a potential problem here. Sometimes you add a compilation error to the code. In this case the post build step will still be executed with the old binary and since we don't switch to the error list and just look at the output window we might miss the compilation errors. This is not a good thing...

The solution is however pretty easy. In your assembly with all your tests you have a post-build step to run the test runner. What you need to do is to add a pre-build step to your production assembly, i.e. the assembly you're testing looking like this: del $(TargetPath)

This way, if there is a compilation error in your production code you'll get an error when you try to run the tests since the production assembly will be deleted for sure.

Team Coding Dojo 4

cellfish — Thu, 30 Jul 2009 07:51:00 GMT

This time we diverted from what I think is a basic dojo rule; you start over each time. So we continued with our BankOCR variant from last time and actually finished it including extending the number of digits and having three different checksum variants. This time we made one mistake. Or maybe I did since I maybe should moderate the session harder. And that was that we got a little carried away to get things done rather than refactoring at certain times. In the end the code turned out pretty OK but the way there was shaky with pretty complex implementations at some times.

In the end it sounded like the participants were pretty happy with the session and we decided to actually continue the next dojo session with the same kata and code base but next time focusing on refactoring the code into something more object orientated using the nine Object Calisthenics rules.

Visual Studio Options for TDD

cellfish — Wed, 29 Jul 2009 06:35:00 GMT

When I first started using xUnit.net the only feasible way to run the tests for me was to use the console test runner as a post build event in visual studio. The only annoying thing about the default settings in visual studio with this setup however is that whenever there was a test failure visual studio switches to the error window which doesn't tell me more than the number of failed tests (since the return code is the number of failing tests). But there is a way to tweak your environment to always show the output window where all the interesting things about the tests are printed.

Take a look at the options dialog showed to the right. By disabling the "Always show Error List if build finishes with errors" and also checking "Show Output window when build starts" you'll make sure you'll have the console output handy if the build fails with unit tests. The drawback however is that for all compilation errors (and warnings) you'll have to switch to the error list window. But personally I see more test failures than actual build failures when I develop code so all in all it's a win for me with these changes.

Coding Dojo 3

cellfish — Tue, 07 Jul 2009 07:20:00 GMT

Third MSFTCorpDojo today and MineSweeper and MicroPairing again. We did almost no design this time and started with only end-to-end tests. In my opinion the purest way of executing a kata. We had very good progress in the first hour but started refactoring our tests quite hard, something I've never seen to this extent before in any dojo session I've ever attended. The result was that we did not complete the kata but we ended up with eleven really nice tests with a few nice helper methods. And if we've spent another 20 or 30 minutes we would probably have seen classes (or at least one) emerge from the code all by them selfs, just as I expect it to be. All in all a great session.

I especially like the fact that one of the participants did not do any coding in his daily work and had never tried TDD at all before the session. And he felt he really learned something and that he felt he could contribute and participate as any other person in the room. I guess that proofs how well the dojo format works for all kinds of skill levels.

I only really experienced one big setback today and that was when we did a 10+minute refactoring that involved more than one person to complete. The refactoring was in the test code and created a very general method to generate input and output data for our tests. I'm personally not sure that refactoring really added value in terms of readability. I would personally have sticked with a much less generic and simple way of implementing my tests even though it would have left me with a few more hard coded strings. But I think that is OK in the test code if it increases readability. On the other hand we got a really nice generic method to use in future tests so I guess it's a trade off as usual and you should do what you think is best in each situation.