Being Cellfish

Stuff I wished I've found in some blog (and sometimes did)

Change of Address
This blog has moved to
  • Being Cellfish

    Why slow unit tests are a bad smell:


    Earlier I promised to elaborate on why slow unit tests are a sign of problems (or a smell if you like). So here it goes.

    The first thing I would like to look at is when the complete test suite takes too long to run to be part of a tight TDD-loop. If it is just the number of tests that makes the difference and each single test is super fast you will have a very large number of tests. Probably several thousands of tests. This can only mean two things. The least bad thing is that you have a very large application that really need all these tests. But you could probably split things up into different modules and test each module by it self thus reducing the number of tests that need to be run when you're working on a single module. Any dependencies between the modules can be faked in your day to day work. A much worse problem is that you're over-testing your code. That means testing the same thing in several similar (but not identical) ways. While over-testing it self should not be a problem it impacts TDD bad in two ways. First of all writing more tests takes more time so you get the feel that TDD is much slower than "developing the old way". It also makes the feedback slower since tests takes longer to run and that means to loose focus.

    The second reason for slow tests is when a single test actually takes to long to execute just by itself. This is a sign of trouble since it probably means you're having badly written tests or even worse; badly designed production code. The reason for the test to be slow is typically because it takes some time to set up the circumstances for the test. The reason for this is typically a bad design since it is difficult or impossible to fake/stub/mock dependencies away in your test code. If you're having trouble faking dependencies you probably have a too tightly coupled design which most people I know tend to think is a bad design.

    So whenever you feel the complete test suite takes too long to run, don't start looking at what is wrong with the tests - start looking at what is wrong with your production code...

  • Being Cellfish

    CUT - Continuous Unit Testing


    During the Christmas holidays I read about something I would like to call CUT (Continuous Unit Testing). CUT is a technique where you run your unit tests in the background all the time rather than after each time you compile (I'm assuming TDD/BDD is being used all the time). There are a number of reasons I'm skeptical toward this approach.

    My first concern is why the tests are run in the background and not part of the regular build process. It could be because the build environment does not have the ability to run custom actions after each build. Hm... Let me rephrase that: The developer does not know how to add custom build steps to the build process. And that is not the only problem. Every time I write a new test I have to wait for the background process to finish, start a new test run and wait for that result. This means the average turn around time each time I write a test is not only the time it takes to compile and make a single test run, it is the compile time plus 1.5 times the test time.

    Another argument might be that the use of a background process to run tests makes it possible to continue writing code while the tests run. But then you're not really following TDD practices, are you? You're just mixing in your test & code writing continuously running the tests.

    But there is one relevant reason for running unit tests in the background. If it takes longer than you can wait to run all tests you will probably start running only a small subset of tests for the code you're actually working on and then run all tests just prior to check-in (or have the build server run all the tests for you). This will however give you feedback that you've changed something you didn't perceive quite late. Under these circumstances I think it makes perfectly sense to run all unit tests in the background and only have a small subset running each time you compile for that quick feedback in your TDD-loop.

    IMHO, having the ability to run unit tests fast is important if you are to be successful with TDD/BDD. And sometimes you end up with a number of tests that are not fast enough (or you have so many tests that they all together takes too long). Under these circumstances the background test runner makes sense but not otherwise. Also, having unit tests that are slow is typically a sign of problems but I think I have to elaborate on that later.

  • Being Cellfish

    Forgotten XP values


    When Agile Sweden had its Christmas party one of the lightning talks was on how some of the XP values seems to have been forgotten now that agile is becoming more and more mainstream. I think we all can agree that agile is much about communication and feedback which happens to be two of the XP values.

    The next one is simplicity. Simplicity is kind of built into most agile practices but many people forget about it. For example agile practices more and more often tend to be initiated by management rather than the people actually doing the work. This also means much focus is on tools. Nothing bad with fancy tools but in my experience the teams performing best (with agile practices) are those with the simplest approach to everything. So in my opinion, simplicity is in the danger zone.

    One XP value that I think definitely have been forgotten is courage. Especially since agile is more and more pushed down by management rather than pushed up by the developers. In my years as a consultant I've seen so many developers that just go ahead and do as they're told grumping about what a bad decision it is to do something this way or that way. I've also seen a fair amount of architects patching and patching a doomed design rather than throwing it away and do it the right way. This can also be seen in developers in a small scale where they stick to a bad design even after they see it is a bad design.

    The last XP value is respect. Since a few of the other values are forgotten there is clearly no respect for the values so by definition this value is forgotten too...

  • Being Cellfish

    Reader survey

    I thought I start off the new year with a reader survey so please take the time and complete this one question survey.
  • Being Cellfish

    The 2008 Advent Calendar wrap up


    Writing all these tests I sometimes felt that the changes between two days was not significant. And sometimes I felt there were versions that did not make it into this advent calendar. But I stuck with the versions you have seen since I think they represent a quite natural evolution of the initial test and hence I have not presented any side tracks that eventually turn into a dead end because I wanted to progress toward the final version which is the one I like most of all these versions. Also I had to be a little bit puristic and let even the smallest changes result in a version by it self rather than merging a number of small changes into a single version change in order get 24 different versions. There are however a few of the versions that I think are worth a special mentioning.

    • December 3rd: I think this is a typical example of how people write their tests when they're starting out to use TDD. In my experience TDD novices tend to put too much into each test.
    • December 5th: The second most common mistake is to forget that standard software engineering practices and common sense still apply to the test code. They tend to forget code reuse in their tests.
    • December 11th: I think this is the first acceptable version in this series. Simple, easy to understand tests with code reuse. TDD novices that take TDD seriously typically end up with this kind of tests pretty soon.
    • December 15th: The use of an object that extends the object under test and adds functionality needed to simplify testing is a great step forward toward writing good tests (IMHO). I think this version of the test basically is the first peek at BDD-type test-writing.
    • December 20th: Now we have easy to understand tests and each method have really descriptive names that are easy to read. I think this is the first really good version of the test in this series.
    • December 24th: In my experience people tend to put many different kinds of tests into the same test class ending up with tear-down code that has to take take the state of several tests into account. This makes the test class more complex than it has to be. The use of several specialized test classes for different types of test scenarios is definitely something you start thinking of after writing several unit tests. I think it is a common misconception that you should have one test class for each production class. Some test classes will test several production classes while others production classes will need several test classes to complete the test suite.

    Please leave a comment and let me know what your favorite version in this series of test is!

  • Being Cellfish

    2008 Advent Calendar December 24th

    1: namespace Advent24 2: { 3: public class FileUtil_Tests 4: { 5: public FileUtilWithDelete Given_A_Readable_File(string content) 6: { 7: FileUtilWithDelete file = new FileUtilWithDelete("SomeFile.txt"); 8: file.Create(content); 9: return file; 10: } 11: } 12:   13: public class FileUtil_Tests_With_Readable_File : FileUtil_Tests 14: { 15: [Fact] 16: public void Reading_A_Readable_File_Returns_File_Content() 17: { 18: using (FileUtilWithDelete file = Given_A_Readable_File("CONTENT")) 19: { 20: string content = file.Read(); 21: Assert.Equal<string>("CONTENT", content); 22: } 23: } 24: } 25:   26: public class FileUtil_Tests_With_Unreadable_File : FileUtil_Tests 27: { 28: private FileUtilWithDelete Given_An_Unreadable_File() 29: { 30: FileUtilWithDelete file = Given_A_Readable_File("SHOULD NOT BE ABLE TO READ THIS"); 31: file.Readable = false; 32: return file; 33: } 34:   35: [Fact] 36: public void Reading_An_Unreadable_File_Throws_Correct_Exception() 37: { 38: using (FileUtilWithDelete file = Given_An_Unreadable_File()) 39: { 40: Assert.Throws<AccessViolationException>(() => { file.Read(); }); 41: } 42: } 43: } 44: }

    This is the final version of this test. Tomorrow you'll get my final thoughts on this advent calendar.

  • Being Cellfish

    2008 Advent Calendar December 23rd

    1: namespace Advent23 2: { 3: public class FileUtil_Tests_With_Readable_File 4: { 5: private FileUtilWithDelete Given_A_Readable_File(string content) 6: { 7: FileUtilWithDelete file = new FileUtilWithDelete("SomeFile.txt"); 8: file.Create(content); 9: return file; 10: } 11:   12: [Fact] 13: public void Reading_A_Readable_File_Returns_File_Content() 14: { 15: using (FileUtilWithDelete file = Given_A_Readable_File("CONTENT")) 16: { 17: string content = file.Read(); 18: Assert.Equal<string>("CONTENT", content); 19: } 20: } 21: } 22:   23: public class FileUtil_Tests_With_Unreadable_File 24: { 25: private FileUtilWithDelete Given_An_Unreadable_File() 26: { 27: FileUtilWithDelete file = new FileUtilWithDelete("SomeFile.txt"); 28: file.Create("SHOULD NOT BE ABLE TO READ THIS"); 29: file.Readable = false; 30: return file; 31: } 32:   33: [Fact] 34: public void Reading_An_Unreadable_File_Throws_Correct_Exception() 35: { 36: using (FileUtilWithDelete file = Given_An_Unreadable_File()) 37: { 38: Assert.Throws<AccessViolationException>(() => { file.Read(); }); 39: } 40: } 41: } 42: }

    Once again we've ended up with the same code in different places. That should be fixed.

  • Being Cellfish

    2008 Advent Calendar December 22nd

    1: namespace Advent22 2: { 3: public class FileUtil_Specification 4: { 5: private FileUtilWithDelete Given_A_Readable_File(string content) 6: { 7: FileUtilWithDelete file = new FileUtilWithDelete("SomeFile.txt"); 8: file.Create(content); 9: return file; 10: } 11:   12: private FileUtilWithDelete Given_An_Unreadable_File() 13: { 14: FileUtilWithDelete file = Given_A_Readable_File("SHOULD NOT BE ABLE TO READ THIS"); 15: file.Readable = false; 16: return file; 17: } 18:   19: [Fact] 20: public void Reading_A_Readable_File_Returns_File_Content() 21: { 22: using (FileUtilWithDelete file = Given_A_Readable_File("CONTENT")) 23: { 24: string content = file.Read(); 25: Assert.Equal<string>("CONTENT", content); 26: } 27: } 28:   29: [Fact] 30: public void Reading_An_Unreadable_File_Throws_Correct_Exception() 31: { 32: using (FileUtilWithDelete file = Given_An_Unreadable_File()) 33: { 34: Assert.Throws<AccessViolationException>(() => { file.Read(); }); 35: } 36: } 37: } 38: }

    So now we have a better name for our specification. But why should everything that has to do with FileUtil be in the same specification?

  • Being Cellfish

    2008 Advent Calendar December 21st

    1: public class Advent21 2: { 3: private FileUtilWithDelete Given_A_Readable_File(string content) 4: { 5: FileUtilWithDelete file = new FileUtilWithDelete("SomeFile.txt"); 6: file.Create(content); 7: return file; 8: } 9:   10: private FileUtilWithDelete Given_An_Unreadable_File() 11: { 12: FileUtilWithDelete file = Given_A_Readable_File("SHOULD NOT BE ABLE TO READ THIS"); 13: file.Readable = false; 14: return file; 15: } 16:   17: [Fact] 18: public void Reading_A_Readable_File_Returns_File_Content() 19: { 20: using (FileUtilWithDelete file = Given_A_Readable_File("CONTENT")) 21: { 22: string content = file.Read(); 23: Assert.Equal<string>("CONTENT", content); 24: } 25: } 26:   27: [Fact] 28: public void Reading_An_Unreadable_File_Throws_Correct_Exception() 29: { 30: using (FileUtilWithDelete file = Given_An_Unreadable_File()) 31: { 32: Assert.Throws<AccessViolationException>(() => { file.Read(); }); 33: } 34: } 35: }

    Now let's consider the name of the class containing all tests. A pretty bad name if you ask me. And since we've used the "given" pattern from BDD we can use BDD to inspire us into a better "test class name".

  • Being Cellfish

    2008 Advent Calendar December 20th

    1: public class Advent20 2: { 3: private FileUtilWithDelete Given_A_File(string content, bool readable) 4: { 5: FileUtilWithDelete file = new FileUtilWithDelete("SomeFile.txt"); 6: file.Create(content); 7: file.Readable = readable; 8: return file; 9: } 10:   11: [Fact] 12: public void Reading_A_Readable_File_Returns_File_Content() 13: { 14: using (FileUtilWithDelete file = Given_A_File("CONTENT", true)) 15: { 16: string content = file.Read(); 17: Assert.Equal<string>("CONTENT", content); 18: } 19: } 20:   21: [Fact] 22: public void Reading_An_Unreadable_File_Throws_Correct_Exception() 23: { 24: using (FileUtilWithDelete file = Given_A_File("SHOULD NOT BE ABLE TO READ THIS", false)) 25: { 26: Assert.Throws<AccessViolationException>(() => { file.Read(); }); 27: } 28: } 29: }

    The "Given_By" pattern has been "stolen" from BDD and the whole purpose is to make the test as readable as possible. Reading a test should more or less be like reading a paragraph. So "Given_A_File" is not really that great. It is probably better to have different Given-methods depending on what we want.

Page 40 of 49 (482 items) «3839404142»