Being Cellfish

Stuff I wished I've found in some blog (and sometimes did)

I don't like Mock objects

Change of Address
This blog has moved to blog.cellfish.se.

I don't like Mock objects

  • Comments 8

When looking at BDD and TDD examples it is very common to see the use of mock objects in the code. This however strikes me as a little bit strange since I think you should use stubs and not mocks when working with BDD/TDD. A mock is basically a stub with the added knowledge that a mock object knows how it is expected to be used. In my opinion this fact disturbs the focus when  applying BDD/TDD. Additionally I think maintainability is decreased since the tests will now have a dependency on implementation internals and you risk having to change your test not because the API has changed but how things are done internally.

Let me explain what I mean with an example. Consider a data object used to handle transactions and the ability to read and modify the amount money on a given bank account. Now assume this data object is used as a parameter to a transfer method that is used to transfer funds between two accounts. One way to implement the transfer method could be:

  1. Start a transaction.
  2. Get amount from the to-account.
  3. Get amount from the from-account.
  4. Set new account amounts for both involved accounts.
  5. Check that the from-account does not have a negative balance.
  6. End the transaction.

From a maintainability point of view I think mocking the data object will be risky since the mock will verify that the correct number of calls are made in the correct order. It wouldn't surprise me if we some day saw a new transfer implementation looking something like this:

  1. Start a transaction.
  2. Get amount from the from-account.
  3. Check that the amount is enough.
  4. Get amount from the to-account.
  5. Set new account amounts for both involved accounts.
  6. End the transaction.

The example may seem a little far-fetched but still I think it points toward a maintainability risk involved when using a mock. But OK, let's assume that maintainability will not be a problem. We're still shifting focus from what is important I think. As soon as a mock is involved when I write my test/scenario I do not only have to think about the behavior I expect. I must also think about how my mock will be used since that may or may not affect my test code. Using stubs instead of mocks will reduce that focus shift since stubs are just doing what they're told without regard of context.

Sure one might argue that using stubs will also involve having to think about how the stub is used. That is true but without the risk of having order or number of calls to the stub mess up your test/scenario setup. And the less time you spend on thinking about setup and the more you spend on defining behavior, the better I think. And thinking about behavior of internals and/or dependent objects should just be kept to an absolute minimum if you ask me.

  • Yea... I tent to agree that you SHOULD use stubs instead... good point!

  • First of all I must remind you that I don't like mocks and mocks aren't stubs . But recently I read a

  • "From a maintainability point of view I think mocking the data object will be risky since the mock will verify that the correct number of calls are made in the correct order. "

    Why would it do that?

    Setting the expectations up this way is a common failure mode of folks new to mocking. It leads to what some call "md5" tests.

    As with any technique, mocking done badly will give poor results. And as with any technique, doing it well doesn't guarantee good results—but you will be in with a chance.

    In this case, the expectation doesn't need to be any stronger than that accounts are asked for amounts, and do have new amounts set. This records the dependencies, which is useful, and nothing else.

    In fact, a hard-core mocker wouldn't want those getters and setters, as you know.

    One advantage over stubs is that the day someone changes the implementation of this transaction and it no longer does the gets the test will fail, even if the (now invalid) calculation still gives the right answer.

    And this is where I agree with you that using mocks has the advantage of moving the focus of attention while writing tests: attention is moved from whether or not a calculation happens to give the right answer (which does need to be tested, but elsewhere) to whether or not our (weakest) beliefs about the object interactions are true.

  • Since I'm not a fan of mocks I guess my prayers have been heard. Microsoft Research will soon release

  • @Keith:

    Sure, I agree that verifying order is optional and something you should not do. Still that is in most beginner examples I've seen so beginners tend do do that and sooner or later realize that is bad.

    I guess we can agree on disagreeing however. You think it is a good thing to shift focus from verifying calculation is correct to verifying calculation is implemented correctly. I think the opposite.

    Looking at it from a BDD perspective where you typically work "outside-in" (and using the example above) you would start with something like a service object that can be used to perform a number of services. One service is "transfer money". Being strict here - the only thing I can say about that method is that it should reduce one account's balance with the same ammount that is added to the other account. If I at this point think too much about the internal implementation details of this method I might end up with an interface that reflects the underlying implementation rather than an interface of what I want to do. In this simple example this may sound like a ridiculus statement since there is not many ways we can implement this correctly. But my point is that we should try to not think about implementation when writing the test/spec. And using mocks forrces us to do so.

    But I can however agree that mocks makes it harder to write bad tests where you fake to much (for example if we just fake the accounts and return the new balance whatever the test expects). But since we don't know in what context the mock is called I guess we can do the same thing there (calling remove first and commiting it and then calling add on the other account). Whatever type of object you choose good tests will mean having a number of tests working together in order to verify the correct behavior.

  • "start with something like a service object that can be used to perform a number of services. One service is 'transfer money'"

    So, that's an illuminating difference: I don't think I would start with that. It seems to me like a premature assignment of responsibilities.

    If BDD is to mean anything in particular (and it's not clear that it does) then I would understand it to mean structuring the system in a way closely bound to the user/customer's idea of their domain. I would be surprised to learn that the users in this world of accounts and payment would think first of any sort of "service object". That strikes me as very much an implementer's concept.

    I absolutely wold want to capture a functional test that gave examples of accounts and payments and new balances. Such a test indeed should say nothing whatever about how the post-condition might be achieved. And separately from that I would, when designing the interaction of the objects that fulfil that contract, and when assigning responsibilities to them, want unit tests that talk about those things. Mocks allow that in a way that assertions and stubs just don't.

    This approach has a surprisingly large impact on design thinking. Mocks as we have them today in jMock and such were invented to help answer the question: How would you unit test objects that had no setters or getters?

    Getterless code is hard to get to if your tests have to interrogate objects from the outside to see if post-conditions have been met. And yet it turns out to have lots of desirable properties.

    This is the real advantage of using Mocks for me: within the context of TDD they open the door to a design discipline based on roles and interactions rather than state.

  • I realize I didn't explain why I chose to start with a service object. I thought of the application as a service (ex: web service) so the entry point for any user would be the service interface, hence the service object. I could have started with something closer to a GUI ifit was a banking application with GUI I guess.

    On BDD I want to point out that BDD is a development practice comparableto TDD. Do not confuse BDD (last D is for development) with DDD (last D for design) even though many feel that DDD and BDD workwell together. One of the key properties of BDD (compared to TDD) is that BDD typically start from the outside working inwards (as opposed to TDD where work typically is done from the inside out). That is why I cannot know anything about the objects I use to solve a problem when writing the first specs (tests). If I start thinking about that I start thinking about implementation too early (IMHO). In both BDD and TDD my personal experience is that interfaces will evolve as will implementation. I'm human so I have some kind of design in my head when I write my first spec/test, but since I force myself to think about the interface instead of the implementation, I ususally change my design and implementation endingup with something different (and better) than what my first idea was.

    I agree mocks helps you test an object without getters, but it does so at a cost if you ask me. The cost of not really getting allthe benefits of BDD and TDD.

    So how do we test objects with no getters? Well at the first level (same example as above) I'm not really tesing the behaviour of the accounts. So I can stub them and make sure (using getters on the stub) that they end up with the correct value. But what we're really interested in testing is what happens when the account objects generate exceptions due to insuffient funds (or other errors) and when they work just fine.

    At some point however we'll be drilling down to the last object in the chain, i.. the account object in the example above. In its simpliest form we must test that it behaves correctly when adding and removing money. How do we do that if the account have no getters and we don't use mocks? We could sub-class the account object getting access to the getters (unleass they're private). We could use some other mechanics to verify the account state. If there is no way to know the account object's state it might as well be stateless. This can be as simple as having an account printer that prints the balance of the account.

    I admit both these approaches are workarounds because of the forbidden getters and the refusal to use mocks. Not really great. But mocks wouldn't help here either because we're testing such a basic object. Unless you have an ammount object which only pushes the problem further. But the problem of testing the basic account object which have no getters isnot solved by mocking the account at a higher level. A stub is just as useful since the stub may have getters even though the object being stubbed does not.

    So I'm not saying mocks aren't useful. I'm just saying that by using mocks you typically include behaviour of dependant objects intoyou tests. So you're not only testing the behaviour of the object being tested. You're also testing that the objects on which it depends are used in a certain way. IMHO that is bad. You risk loosing one benefit of TDD/BDD, namly the one that your interface and implementation design is driven by your tests as they are written. If you create your tests as "set state, execute, verify state" you have complete freedom ofwhat to do during "execute". As opposed to testing "set state, verify that execute actually does X with A". Now you haev locked your design. "Execute must do X with A" or the test must be rewritten. And if the test has to be rewritten because of refactoring it is not a good test I think (or rather refactoring is impossible since refactoring by definition is done without changing the tests).

  • The day started with Ken Schwaber talking on the topic: It is not Scrum if... Basically he tried to correct

Page 1 of 1 (8 items)