Being Cellfish : BDD

Advent calendar 2009 - how to TDD thread safety

cellfish — Sun, 29 Nov 2009 07:28:00 GMT

Last year I wrote the same test in 24 different ways as a little advent calendar for you to enjoy. This year I'm going to do something similar but also a little bit different. I'm going to start with a simple piece of code and then show how this code can be tested for thread safety in different ways. I think it's pretty common to think that thread safety is one of those things that are really hard to test and something you typically handle by good practices and code reviews. Very similar to logging in an application. Most people don't TDD their logs. But why not? Personally I've experienced situations where fixing a bug means making something thread safe or adding a better log message. When I do this I want a test to reproduce the bug and verify the fix. These tests are typically hard to write since the code was not written to be easily tested in regards to logging or thread safety. But it doesn't have to be that way. For example there is a testing framework that basically works with verifying logs (TextTest) and thread safety can absolutely be designed using TDD. Remember; if you think it is hard, you're probably doing something wrong. Let's decide after Christmas if test driving thread safety is better than the standard way of just adding it where you know it is needed.

Coding Dojo 5

cellfish — Wed, 02 Sep 2009 19:02:00 GMT

Yesterday was the fifth MSFTCorpDojo. MineSweeper as usual and we tried switching person at keyboard on each arrow in the TDD cycle (and not really pairing). As last time we tried BDD style tests. Didn't end up quite as neat as last time but we still learned something. I also noticed that everybody was very engaged this time. Don't know if it was the attendees or the fact that you could end up at the keyboard very soon all the time. Anyway I think we'll decide who goes next by random next time. Just to keep everybody on their toes... Will be interesting to see how that works. We'll also try to apply the object calisthenics rules next time.

This time it was also very clear how one person's decision on how to write a test or how to implement one thing may drive the complete solution in one direction unless you decide to do major refactorings. We also had to remind our selfs a few times not to do the smallest or dumbest change to make a test path but actually the simplest thing. I think it was nice how we sometimes tried to much, recognized that fast, backed out and did a simpler thing.

Team Coding Dojo 5

cellfish — Thu, 27 Aug 2009 09:24:00 GMT

This time we took the completed BankOCR Kata from last time and tried to refactor it using the object calisthenics rules. It was really fun and the dojo was quite different from the other dojos which has been more TDD focus. This time we had a lot of design discussions and we had to force our selfs to just do some refactoring and see where it took us. I think it was great to see how we refactored and created new classes just to later refactor these classes to nothing and removing them. It was a great experience in how refactoring in steps reveals the design for you. We also had the full test suite save us a bunch of times from stupid bugs which is also nice.

But refactoring existing code to follow the object calisthenics rules is very hard and takes time. So there is much more work to do. We decided that we maybe want to continue this exercise later but for our next team dojo we will try a new Kata and apply the object calisthenics rules from the beginning trying to keep the code base conformed to those rules at all times. Also we want a Kata with no string parsing of two dimensional things...

We actually did another fun thing. After the dojo we went to watch D9 at a local cinema. A great finale!

What is a Coding Dojo?

cellfish — Sun, 09 Aug 2009 08:53:00 GMT

I've heard that some people don't think the CodingDojo site tells you a lot about what to expect in a coding dojo. It's more about practical things and listing a number of good problems to solve during the dojo. So I'll try to elaborate. In a coding dojo you will learn something. It might be TDD if you don't know a lot about it or it might be some smart trick in the development environment. You will also learn something about how others write their code. You will learn this since a typical dojo is setup with one computer desktop projected on the wall and all participants takes turns at the keyboard. You should not be intimidated by the fact that you will code in front of others for a few minutes. People attending dojos typically like to teach and learn so you will get help from the group.

You should not expect to see the announced problem being solved. In my experience it is very rare that a dojo ends with a complete solution. And completing is not the goal. Learning along the way is the important thing. A coding dojo is a safe environment where you can try new ideas and make mistakes without putting your real work at risk. A coding dojo is also a great opportunity to improve your development skills.

A typical dojo starts with a short introduction to the environment and a short design discussion. In most dojos two people then starts coding as a pair. After a few minutes (typically around five) one person in the pair gets substituted by somebody else in the group and so on. Even though the person at the keyboard is in command, the whole group typically participates giving suggestions on what to do. If more discussion is needed, that is done without stopping the timer counting down to the next switch. The dojo ends with a retrospect so that the dojo can be improved to the next time.

Coding Dojo 4

cellfish — Thu, 06 Aug 2009 08:00:00 GMT

Fourth MSFTCorpDojo today. MineSweeper and MicroPairing this time too.This time we, as a result of the retrospect from last time, we did BDD-style testing (I've added one of the test classes below as an example). We decided to start by implementing a parser that parsed the input and created an internal representation of the input that we could later use to generate the output. By the end of the session we had a pretty complete parser with error handling for all kinds of bad input. This is one of the few times I've seen any real error handling in a dojo.

This time in the retrospect we said that the next time we should implement the other part next time. That is; assume a parsed input in some internal representation and generate the output from that. We also talked about how we do MicroPairing and switch one person in the pair every seven minutes. We decided to next time switch one person in the pair every time the keyboard gets passed instead. Guess that will mean less pair programming since there is no real ping-pong but rather a circular queue. But in our session almost everybody participates all the time anyway so that might not necessarily be a problem.


   1:      public class Given_a_single_1x1_fieldset_with_no_bombs
   2:      {
   3:          private string input;
   4:          private Field field;
   5:   
   6:          public Given_a_single_1x1_fieldset_with_no_bombs()
   7:          {
   8:              input = "1 1" + Environment.NewLine +
   9:                      "." + Environment.NewLine +
  10:                      "0 0";
  11:              field = FieldParser.Parse(input).Single();
  12:          }
  13:   
  14:          [Fact]
  15:          public void It_should_have_one_row()
  16:          {
  17:              Assert.Equal(1, field.Rows);
  18:          }
  19:   
  20:          [Fact]
  21:          public void It_should_have_one_column()
  22:          {
  23:              Assert.Equal(1, field.Columns);
  24:          }
  25:   
  26:          [Fact]
  27:          public void It_should_have_an_empty_square_0_0()
  28:          {
  29:              Assert.False(field.IsBomb(0, 0));
  30:          }
  31:      }

Visual Studio Options for TDD revisited

cellfish — Fri, 31 Jul 2009 17:31:00 GMT

So I previously mention a neat way to set up visual studio to work with a console test runner so you don't switch to the error list all the time. There is a potential problem here. Sometimes you add a compilation error to the code. In this case the post build step will still be executed with the old binary and since we don't switch to the error list and just look at the output window we might miss the compilation errors. This is not a good thing...

The solution is however pretty easy. In your assembly with all your tests you have a post-build step to run the test runner. What you need to do is to add a pre-build step to your production assembly, i.e. the assembly you're testing looking like this: del $(TargetPath)

This way, if there is a compilation error in your production code you'll get an error when you try to run the tests since the production assembly will be deleted for sure.

Team Coding Dojo 4

cellfish — Thu, 30 Jul 2009 07:51:00 GMT

This time we diverted from what I think is a basic dojo rule; you start over each time. So we continued with our BankOCR variant from last time and actually finished it including extending the number of digits and having three different checksum variants. This time we made one mistake. Or maybe I did since I maybe should moderate the session harder. And that was that we got a little carried away to get things done rather than refactoring at certain times. In the end the code turned out pretty OK but the way there was shaky with pretty complex implementations at some times.

In the end it sounded like the participants were pretty happy with the session and we decided to actually continue the next dojo session with the same kata and code base but next time focusing on refactoring the code into something more object orientated using the nine Object Calisthenics rules.

Visual Studio Options for TDD

cellfish — Wed, 29 Jul 2009 06:35:00 GMT

When I first started using xUnit.net the only feasible way to run the tests for me was to use the console test runner as a post build event in visual studio. The only annoying thing about the default settings in visual studio with this setup however is that whenever there was a test failure visual studio switches to the error window which doesn't tell me more than the number of failed tests (since the return code is the number of failing tests). But there is a way to tweak your environment to always show the output window where all the interesting things about the tests are printed.

Take a look at the options dialog showed to the right. By disabling the "Always show Error List if build finishes with errors" and also checking "Show Output window when build starts" you'll make sure you'll have the console output handy if the build fails with unit tests. The drawback however is that for all compilation errors (and warnings) you'll have to switch to the error list window. But personally I see more test failures than actual build failures when I develop code so all in all it's a win for me with these changes.

Coding Dojo 3

cellfish — Tue, 07 Jul 2009 07:20:00 GMT

Third MSFTCorpDojo today and MineSweeper and MicroPairing again. We did almost no design this time and started with only end-to-end tests. In my opinion the purest way of executing a kata. We had very good progress in the first hour but started refactoring our tests quite hard, something I've never seen to this extent before in any dojo session I've ever attended. The result was that we did not complete the kata but we ended up with eleven really nice tests with a few nice helper methods. And if we've spent another 20 or 30 minutes we would probably have seen classes (or at least one) emerge from the code all by them selfs, just as I expect it to be. All in all a great session.

I especially like the fact that one of the participants did not do any coding in his daily work and had never tried TDD at all before the session. And he felt he really learned something and that he felt he could contribute and participate as any other person in the room. I guess that proofs how well the dojo format works for all kinds of skill levels.

I only really experienced one big setback today and that was when we did a 10+minute refactoring that involved more than one person to complete. The refactoring was in the test code and created a very general method to generate input and output data for our tests. I'm personally not sure that refactoring really added value in terms of readability. I would personally have sticked with a much less generic and simple way of implementing my tests even though it would have left me with a few more hard coded strings. But I think that is OK in the test code if it increases readability. On the other hand we got a really nice generic method to use in future tests so I guess it's a trade off as usual and you should do what you think is best in each situation.

Why it is worth adding tests even when the investment is large

cellfish — Sat, 27 Jun 2009 16:27:00 GMT

Have you ever found yourself fixing a bug or adding a simple feature to some code that have no tests and since you usually do things test driven you know you should get the code under test but you decide not to because the change only takes a few minutes and getting the code under proper tests would take days? The first time it might feel OK but then you find yourself having to touch that code again later and sooner or later you end up regretting you didn't get the code under test the first time.

I've been in that situation and each time I make the same mistake since the trade off feels reasonable. But in hindsight almost all the time it would have been better getting the code under test right away. And a few days I found somebody who actually took the time to get some numbers proving why you should get the code under test as fast as possible. The interesting fact is that in two large projects (gcc and python) the probability of touching the same code twice in the same week is 35%. The number might be a little on the high side since touching the same file twice in the same week does not necessary mean you're working on two different things. It might be incremental check-ins. But if they're incremental, then it's probably worth getting the code under test anyway since you're spending days on the code anyway. It is also interesting to see that there is almost a 50% chance of touching the same code twice in a month.

So now we have great statistics and a gut feeling telling us to add tests the first time even if it feels costly at the moment because the probability of the investment to pay off within a few weeks is very high. And you'll be much happier the second time around this way...

Team Coding Dojo 3

cellfish — Thu, 25 Jun 2009 06:40:00 GMT

We basically did two changes this time compared to last time. We adopted the micro pairing previously discovered and we tried a new Kata; BankOCR. I had prepared a variant of BankOCR where we added one user story at the time so that the group didn't really know what was coming later. Since we only got the initial parsing and check-sum calculations done we decided to bend a dojo rule for next time and actually continue where we ended this time. It'll be interesting to see where that takes us.

We also made an interesting observation that is worthy a post of it self.

Test readability

cellfish — Fri, 12 Jun 2009 07:23:00 GMT

I read this excellent post on test readability a few days ago. I like the way he shows how he refactors a test for readability but personally I would have written the example test a little bit differently. First of all I would have added another extension method:


   1:      public static void PushOkButtonMultipleTimes(this ILoginView view, int times)   
   2:      {   
   3:          for (int i=0; i<times; i++)
   4:              view.PushOkButton();   
   5:      }

With that extension method my version of the test would have looked like this:


   1:      [Test]
   2:      public void only_three_login_attempts_are_allowed()
   3:      {
   4:          presenter = given_a_login_presenter_with_bad_credentials();
   5:   
   6:          view.PushOkButtonMultipleTimes(3);
   7:   
   8:          view.ShouldBeClosed();
   9:      }
  10:      
  11:      public ILoginView given_a_login_view_with_bad_credentials()
  12:      {
  13:          ILoginView view = CreateLoginView("BadUser", "BadPassword");
  14:          ILoginService loginService = CreateLoginService(false);
  15:   
  16:          LoginPresenter presenter = new LoginPresenter(loginService);
  17:          presenter.Initialize(view);
  18:          return view;
  19:      }

I personally prefer to move as much as possible of test setup to separate methods that are called explicitly by the test. The use of the extension method to remove the three times duplicated PushOkButton call is not something I'd always do. I was considering having the for-loop in the test method but I think an in place for-loop is less readable than making the calls three times. The extension method wrapping the for-loop however removes the duplication without decreasing the readability of the test.

Coding Dojo 2

cellfish — Thu, 04 Jun 2009 08:50:00 GMT

Today was the second MSFTCorpDojo I organized. Naturally we did the same kata as the first time. Three very interesting things happened this time. I've done this kata in about ten different dojos now and there are usually two different paths the design takes. But this time we had a third design suggestion that sounded promising. Instead of parsing the input and then transform the input to output we decided to aim for a solution where we updated a data structure while parsing. Might not sound that very different but believe me, it is.

The second thing that happened was that instead of doing PingPong we did Micro Pairing. Usually you only switch keyboard after writing a failing test, but now we tried switching keyboard each time we've written a test (passing or failed, preferably failed), made a test pass or refactored. This had two interesting side effects the group liked. First it meant we switched more often getting a better pair programming experience I think. Second it added a fun kind of competition where we almost competed to not have to write a new failing test.

Last, but not least, this was the first time I've ever finished this kata in a two hour dojo. Finishing the kata is not important but an interesting experience. So why did we finish the kata? Three things; We did not always make baby steps implementing. Or at least it is questionable. Sometimes we did a solution to make all tests pass that logically was more complex than the trivial change but it was less number of key strokes... I'm personally leaning toward that an important part of the session is to do baby steps and see the refactorings appear obviously even if it means doing things I know I'll delete in 30 seconds. The second thing that made us complete in two hours was probably that the design/algorithm we decided on turned out to be pretty simple to implement. The last thing is that we ended up with only one of 16 tests that actually tested edge cases that should cause an error so our implementation was very straight forward and solved the problem as long as we didn't get any bad input.

Is BDD/TDD and pair programming a waste of time?

cellfish — Fri, 29 May 2009 08:45:00 GMT

As usual the answer is it depends. If you insist on looking at short term effects of TDD for example there is no doubt in my mind it is a waste of time. Why deliver great software in six months when you can deliver decent software in four months? Personally I believe there is too much software out there that started as "a little temporary solution we will only use a few times" and that later turns out to be used, tweaked, fixed and updated for years and years to follow. You never know what will happen except that you probably have to maintain the software you write for a longer period than you really want. And the best way to not spend time on old stuff you wrote ages ago is to make sure it works in the first place and that changes/updates can be made quickly (and preferably by someone else without asking you for help). So in the long run, TDD is definitely not a waste of time in my book. There are also some research on this if you like to crunch numbers.

So what about pair programming? As pointed out here, you do much more than just write code when you develop software. In certain situations such as spreading knowledge pair programming is a no-brainier. One experienced person showing someone else how things work at the same time as new features gets created is probably more cost effective than having somebody sit for days by them selfs trying to figure out how things work in the code. And two skilled developers probably solve any problem faster together than if they try alone. Since two people also look at the code all the time you will probably end up with a better design because it is the work of two rather than one. And you always have (at least) two people that understands each part of the code so if somebody gets seriously ill it is not catastrophic. But all this (except maybe teaching novices) is definitely a waste of time if you insist on looking at the short term consequences. But once again, software always survives longer than you think. And each thing you do in your development process must be measured against the total life cycle of the software. Not just until the next release (or on a day to day basis).

I think all agree that quality costs but it pays in the long run. And there is always a breaking point. Some things are just not worth the price of quality but before you can make that call you must know the real costs and the real benefits. As provided above there is some research on TDD and developers don't just write code so two developers pairing is not half as fast as two working independently. And if you really need numbers then you should try TDD and pair programming and actually measure the cost vs benefit rather than focusing on the short term cost in time. Remember, quality typically pays of in the long run. Both in terms of less bugs and happier staff that stays on the job.

Feeling awkward when writing tests?

cellfish — Wed, 27 May 2009 15:27:00 GMT

I recently observed a discussion where one person hesitated to add new dependencies to one class since it meant all calls to the constructor had to be changed. At first glance this only looks as if the person was very strict about constructor dependency injection. But there is nothing wrong in having a constructor that uses all the default dependencies in your production code.

Or maybe the person felt this about the test code where he couldn't use the default constructor. But in this case we're talking about code duplication in some form. You probably want a common function to set up your test environment. Or maybe the person just thought it was ugly adding yet another dependency creating a constructor parameter list marathon.

Regardless of which the real reason for feeling it's awkward to add new dependencies the annoying answer is: "you're probably doing something wrong". And this applies not only to dependencies but to all kinds of behavior/test writing. As soon as something feels awkward, back-wards or just wrong to write it is a huge warning bell for me. When this happens I try to stop and think about what I'm doing questioning myself to see if there is a simpler way to do what I'm about to do. Sometimes something is wrong in the design and sometimes there is something more simple to test instead of the thing I thought of first.