Being Cellfish

Stuff I wished I've found in some blog (and sometimes did)

Change of Address
This blog has moved to blog.cellfish.se.
Posts
  • Being Cellfish

    Selenium the test tool

    • 3 Comments

    I almost forgot that before the BDD/DDD presentation, there was a presentation about Selenium. I didn't forget it because it was a bad presentation but because the following presentation was so good...

    I've never had any good experiences with tools for automatic testing but I must confess that Selenium looks nice.  At least if you want to add regression tests to an existing GUI. And it is a web based GUI. So if those two fact apply to you, you should take a look at Selenium I think.

    Selenium is good at recording actions and create test code. And it creates test code for unit tests frameworks such as NUnit and a whole bunch of other languages such as Python, Java, Ruby, PHP and more. However there seems to be some stability problems at the moment. According to the presenter (I have not found a link to verify this); Google have more than 51000 selenium tests in their projects. 96% of the tests run with no problem, 2% have problems due to confirmed bugs in Selenium and the last 2% are tests failing where the reason  is unclear (Selenium vs the code tested).

    This is a common problem I see when I work with open source. Open source applications are typically quite useful and easy to use 99% of the time. But I always tend to end up needing the last percent which is never implemented or have bugs. And the reason for this is usually that no one have bothered to implement the last tricky advanced feature I end up needing. You probably wanna know why I do not sit down and implement that feature and contribute it back to the open source community. Well that is a whole other topic and I will write about that it the near future.

  • Being Cellfish

    Readable SQL

    • 3 Comments

    I read this article on SQL Code Layout and Beautification and can only agree with the author that other people's SQL often is hard to understand because I'm so used to how I write my SQL statements. One of the links is to an on-line tool formatting your SQL according to a number of rules. I was happy to find that my way of writing SQL was supported:

    SELECT a,
           b,
           c AS d
      FROM x,
           y,
           z AS w
     WHERE a = 2
       AND b IN (3,4,7)

    And another example:

      SELECT obj.run,
             obj.camcol,
             STR(obj.field,3) AS field,
             STR(obj.rowc,6,1) AS rowc,
             STR(obj.colc,6,1) AS colc,
             STR(dbo.FOBJ(obj.objid),4) AS id,
             STR(obj.psfmag_g - 0 * obj.extinction_g,6,3) AS g,
             STR(obj.psfmag_r - 0 * obj.extinction_r,6,3) AS r,
             STR(obj.psfmag_i - 0 * obj.extinction_i,6,3) AS i,
             STR(obj.psfmag_z - 0 * obj.extinction_z,6,3) AS z,
             STR(60 * distance,3,1) AS d,
             dbo.FFIELD(neighborobjid) AS nfield,
             STR(dbo.FOBJ(neighborobjid),4) AS nid,
             'new' AS 'new'
        FROM (SELECT obj.objid,
                     run,
                     camcol,
                     field,
                     rowc,
                     colc,
                     psfmag_u,
                     extinction_u,
                     psfmag_g,
                     extinction_g,
                     psfmag_r,
                     extinction_r,
                     psfmag_i,
                     extinction_i,
                     psfmag_z,
                     extinction_z,
                     nn.neighborobjid,
                     nn.distance
                FROM photoobj AS obj
                     JOIN neighbors AS nn
                       ON obj.objid = nn.objid
               WHERE 60 * nn.distance BETWEEN 0 AND 15
                 AND nn.mode = 1
                 AND nn.neighbormode = 1
                 AND run = 756
                 AND camcol = 5
                 AND obj.TYPE = 6
                 AND (obj.flags & 0x40006) = 0
                 AND nchild = 0
                 AND obj.psfmag_i < 20
                 AND (g - r BETWEEN 0.3 AND 1.1
                      AND r - i BETWEEN -0.1 AND 0.6)) AS obj
             JOIN photoobj AS nobj
               ON nobj.objid = obj.neighborobjid
       WHERE nobj.run = obj.run
         AND (ABS(obj.psfmag_g - nobj.psfmag_g) < 0.5
               OR ABS(obj.psfmag_r - nobj.psfmag_r) < 0.5
               OR ABS(obj.psfmag_i - nobj.psfmag_i) < 0.5)
    ORDER BY obj.run,
             obj.camcol,
             obj.field

  • Being Cellfish

    Code Coverage

    • 3 Comments

    Measuring code coverage is often perceived as a good measure of test quality. It is not. Good tests finds problems when changes are made to the code. But if you just want to have large code coverage you can easily make a number of tests calling all your methods but not checking the results. The only thing high code coverage values really tell you is that the code is at least not crashing with the given input.

     

    If you however are using BDD/TDD, code coverage values might be of interest. For example if you do not have 100% function coverage (i.e. not 100% of the functions are called) then you aren't really using BDD/TDD are you? Because how could you write a method that is not called by anyone? Well actually you might have created methods that are never called by your tests. Many TDD practitioners use a rule "to simple to test" which applies to real simple, property like methods. I don't really like that philosophy, but more on that in a later post.

     

    So now you think with 100% function coverage you will also have 100% line coverage with BDD/TDD, right? Well, yes and no. Typically you don't since you will add error handling with logging methods that will never occur since the errors never occur in your tests. It might be that you open a file and if that fails you log an error and exit your application. This never happens in the tests since you always manage to open the file in your tests. So how did those lines get in there if never called? Well one other rule often used by TDD practitioners is that you "should not do stupid things". If a system call may fail, you check the result regardless of tests or not. With dependency injection you'll probably get close to 100% but there is no point in bending back-wards in order to achieve high code coverage.

     

    Does this mean there is no point in measuring code coverage? I think it is great to measure code coverage if you use the result correctly. You should not add more tests just to increase coverage since test added just to increase coverage tend to just exercise code and not really testing something interesting. But low code coverage when using BDD/TDD is definitely a warning signal that something is wrong. The team is not using the methodology correctly. So what is considered OK coverage levels? From personal experience I think anything below the levels listed in the table below should be considered bad since type without injectiontype withoutinjection with with you should have no problem at all achieving the given values.
     

    Coverage type without dependency injection with dependency injection
    Function 90% 99%
    Line 75% 95%

    But sometimes there is someone (usually a manager) that thinks coverage should be above some level. So even though you know those tests will not really be useful you have to add more tests for coverage. What do you do? Either you can try to ignore the coverage fact and just try to add more tests, testing interesting things. Or you could try using Pex. Pex is a tool from Microsoft Research that is an automated exploratory testing tool. It can be used to make a small test suite with high code coverage and with only a few simple examples I get the impression it is quite good at finding border cases in your code. This will not however replace your traditional tests/specifications written as part of your TDD/BDD process. But it can help you test some cases you did not think of and that way increase code coverage even more without any extra effort from you. At least it is better than adding coverage tests by hand.

  • Being Cellfish

    C++ pointer basics for kids

    • 3 Comments

    A little video to explain pointer basics in C++ to your kids... If you for some weird reason want to do that...

  • Being Cellfish

    Mobile phone costs in the US

    • 3 Comments

    So today I finally got my mobile phone service set up. And how mobile services are billed in the US is quite different than how it works in Sweden. The basic rule in Sweden is that if you do not make the call, you don't pay. That is, you pay nothing to receive calls and you pay nothing to receive text messages. The exception from the rule is if you're roaming outside Sweden. In that case you pay the costs for transferring the call/text to wherever you are.

    This is not how things work in the US. Here you pay for incoming text messages as well as incoming calls... Always... So if I call/text my wife we have to pay for the communication twice... Actually that only applies to text messages since we have unlimited minutes because of the service plan we have but still... I don't get the logic behind this scheme. Maybe it's because I'm Swedish...

  • Being Cellfish

    Can code reviews really change design?

    • 3 Comments

    The other day I wrote about effective code reviews. Once problem I've seen with check in reviews are that once the review take place it is common that one or two days of work have been put into the change so any comments on how the design of the code can be changed are typically too late once we reach a check in review. A bad design is usually changed but an OK design is rarely changed into a great design basically since the design is OK so both reviewer and author does not feel it is worth another day just to improve the design. So check in reviews are not the silver bullet for better designs. You have to do something else too.

    On our team we have design reviews. A design review is something each developer calls out (just like a check in review is called out) and then the developer goes over a proposed design or a few test cases with another developer before he starts implementing things in code. These design reviews have proved to be valuable and has resulted in less design comments during check in reviews.

    As of yet we have not tried pair programming which would replace both the design reviews and the check in reviews in a natural way.

  • Being Cellfish

    The 2008 Advent Calendar wrap up

    • 3 Comments

    Writing all these tests I sometimes felt that the changes between two days was not significant. And sometimes I felt there were versions that did not make it into this advent calendar. But I stuck with the versions you have seen since I think they represent a quite natural evolution of the initial test and hence I have not presented any side tracks that eventually turn into a dead end because I wanted to progress toward the final version which is the one I like most of all these versions. Also I had to be a little bit puristic and let even the smallest changes result in a version by it self rather than merging a number of small changes into a single version change in order get 24 different versions. There are however a few of the versions that I think are worth a special mentioning.

    • December 3rd: I think this is a typical example of how people write their tests when they're starting out to use TDD. In my experience TDD novices tend to put too much into each test.
    • December 5th: The second most common mistake is to forget that standard software engineering practices and common sense still apply to the test code. They tend to forget code reuse in their tests.
    • December 11th: I think this is the first acceptable version in this series. Simple, easy to understand tests with code reuse. TDD novices that take TDD seriously typically end up with this kind of tests pretty soon.
    • December 15th: The use of an object that extends the object under test and adds functionality needed to simplify testing is a great step forward toward writing good tests (IMHO). I think this version of the test basically is the first peek at BDD-type test-writing.
    • December 20th: Now we have easy to understand tests and each method have really descriptive names that are easy to read. I think this is the first really good version of the test in this series.
    • December 24th: In my experience people tend to put many different kinds of tests into the same test class ending up with tear-down code that has to take take the state of several tests into account. This makes the test class more complex than it has to be. The use of several specialized test classes for different types of test scenarios is definitely something you start thinking of after writing several unit tests. I think it is a common misconception that you should have one test class for each production class. Some test classes will test several production classes while others production classes will need several test classes to complete the test suite.

    Please leave a comment and let me know what your favorite version in this series of test is!

  • Being Cellfish

    Why slow unit tests are a bad smell:

    • 2 Comments

    Earlier I promised to elaborate on why slow unit tests are a sign of problems (or a smell if you like). So here it goes.

    The first thing I would like to look at is when the complete test suite takes too long to run to be part of a tight TDD-loop. If it is just the number of tests that makes the difference and each single test is super fast you will have a very large number of tests. Probably several thousands of tests. This can only mean two things. The least bad thing is that you have a very large application that really need all these tests. But you could probably split things up into different modules and test each module by it self thus reducing the number of tests that need to be run when you're working on a single module. Any dependencies between the modules can be faked in your day to day work. A much worse problem is that you're over-testing your code. That means testing the same thing in several similar (but not identical) ways. While over-testing it self should not be a problem it impacts TDD bad in two ways. First of all writing more tests takes more time so you get the feel that TDD is much slower than "developing the old way". It also makes the feedback slower since tests takes longer to run and that means to loose focus.

    The second reason for slow tests is when a single test actually takes to long to execute just by itself. This is a sign of trouble since it probably means you're having badly written tests or even worse; badly designed production code. The reason for the test to be slow is typically because it takes some time to set up the circumstances for the test. The reason for this is typically a bad design since it is difficult or impossible to fake/stub/mock dependencies away in your test code. If you're having trouble faking dependencies you probably have a too tightly coupled design which most people I know tend to think is a bad design.

    So whenever you feel the complete test suite takes too long to run, don't start looking at what is wrong with the tests - start looking at what is wrong with your production code...

  • Being Cellfish

    2008 Advent Calendar December 17th

    • 2 Comments
    
    
    1: public class Advent17 2: { 3: private FileUtilWithDelete SetUp(string content, bool readable) 4: { 5: FileUtilWithDelete file = new FileUtilWithDelete("SomeFile.txt"); 6: file.Create(content); 7: file.Readable = readable; 8: return file; 9: } 10:   11: [Fact] 12: public void TestReadReadableFileReturnsFileContent() 13: { 14: using (FileUtilWithDelete file = SetUp("CONTENT", true)) 15: { 16: string content = file.Read(); 17: Assert.Equal<string>("CONTENT", content); 18: } 19: } 20:   21: [Fact] 22: public void TestReadUnreadableFileThrowsCorrectException() 23: { 24: using (FileUtilWithDelete file = SetUp("SHOULD NOT BE ABLE TO READ THIS", false)) 25: { 26: Assert.Throws<AccessViolationException>(() => { file.Read(); }); 27: } 28: } 29: }

    Starting all test methods with "Test" does not really add any value to the name either. Naming the tests something that sounds like a real sentence is something I prefer.

  • Being Cellfish

    ALT.NET Sweden unconference summary

    • 2 Comments

    Yesterday the Swedish chapter of ALT.NET held its second unconference. It all started with six lightning talks (of which I held one) followed by an opens space session. It was a great experience to meet so many dedicated developers at once. And even though an unconference tries to be a single long coffee break (since most interesting conversations at a regular conference typically happens during the breaks) it was during the breaks I had the most stimulating conversations.

    My lightning talk was on "should we not be mocking" where I talked about the good and bad things that might happen if you're using mock objects and/or mock frameworks. Other talks covered MSpec, OpenTK, db4o and continuous integration (CI). A funny quote from the CI-session is "you can argument against the SOLID principles and TDD but you can't argue against CI".

    The open sessions I attended covered TDD of WPF applications, Conventions vs configurations and working with legacy code where CloneDetective was mentioned. I also facilitated an open session on BDD without BDD frameworks. The day ended with a session where we discussed what ALT.NET should be doing. I'm looking forward to see what is going to happen with the "mentor list" that was mentioned.

Page 3 of 49 (481 items) 12345»