Progressive Development : test-driven development

Motley says: "The only thing I'm going to mock is you"

James Waletzky — Tue, 07 Aug 2007 20:11:00 GMT

Summary

Motley: I can't mock something that I have no control over (like third-party or old legacy code)

Maven: Design to interfaces, create a wrapper, and use tools for mocking. These are all great practices for leveraging mock objects.

______________________________

[Context: Motley is struggling with creating mock objects for his unit tests]

Motley: Arrrrggggghhhh. You know, I'm really trying to do this TDD thing but it is quickly becoming a real pain in the butt. You said that tests have to execute quick. Trust me, I totally believe you on that front. I'm doing my best to make them execute quickly but some things just are not made to mock.

Maven: What's the problem?

Motley: I'm interfacing with Marty's code. I "new" the object up and call it. Unfortunately Marty's stuff takes quite a few seconds to execute. It's making my TDD experience as painful as looking at that shirt you're wearing.

Maven: What's wrong with my shirt? Anyway, the problem is that you are hard coding the instance creation of Marty's object. You are depending on an implementation, which is a violation of one of our key design principles we talked about.

Motley: Yeah, yeah, design to interfaces. But Marty has defined the interface and I have no control over it. No stubbing here, bubba!

Maven: Ah, but you do have some control! The best thing to do would be to collaborate with Marty and get him to expose a well-defined and documented public interface to his component. Then you can simply couple yourself to the interface and not the implementation.

Motley: Right! Then I can plug and play different implementations of the component, replacing Marty's version with my own stub.

Maven: You betcha! You got it, Mot!

Motley: But let's say Marty's component is actually a third party component that I have no control over.

Maven: You can solve any software problem with an extra layer of indirection. I'm kidding of course, but it does help here. In this case, wrap that component with a façade design pattern that provides an interface that you define. Then your core code depends on that façade interface instead of directly on the third party component.

Motley: Ok, fortunately that makes sense. You used the word "mock" previously but it just seems like a stub to me. What is the difference?

Maven: Don't worry too much about the nomenclature. A stub provides some sort of canned response to a call. Usually the response is hard-coded and there is no real logic associated with the result that is returned. A mock is similar to a stub, but it also verifies some kind of behavior. Did the client of the mock call it appropriately? Is the state of the object as we expect? Are return values from methods that we expected to be called set appropriately? The term "mock" goes along with TDD quite frequently, but you can get by effectively just using a stub if you wish.

Motley: So a mock does extra work on top of a stub. How do I get the mock to do this extra work of behavior verification? Do I have to write all the code myself?

Maven: You can, but you can also use a mock object framework. There are a few out on the Internet for various platforms. NMock is an example for .NET, but they're out there for other languages too. Generally the name is xMock, where the "x" is replaced by some kind of letter or phrase corresponding to the language or platform for which it is built.

Motley: I'm still not sold - what about all the wrappers I'll have to write to mock out bits of legacy code that were not designed with interfaces in mind?

Maven: Check out a free tool called TypeMock.NET. TypeMock allows you to mock any class in your system, including concrete classes. No changes are required to the production code. TypeMock automatically intercepts calls to the class(es) of your choosing and injects whatever behavior you specify in its place. It does this in one of two ways: (a) by recording expectations of the code in a similar way that you would record a macro; and (b) using reflection to look for calls to expected methods and replace them with alternatives. The key is running an executable that wraps your test harness such that it is able to hook the desired method calls.

Motley: Holy crap - that sounds incredibly cool!

Maven: About as cool as you are, with all the windows open in the winter.

Motley: Ouch. A dig from Mave. Actually, the only thing you have dug is your grave once we leave the office and I get my hands on you. In all seriousness, I can understand the mock story, but I'm still not sure about the best technique for mocking something out. I figured I would just hack in an "if" statement to give me the right object given the scenario - release or test.

Maven: Good question. A hacked-in hook could work, but it's not the best option. Why don't we go over some methods of injecting mock objects into your design. Let's go write some code on the whiteboard...

______________________________

Maven's Pointer: To take advantage of the full power of mock objects, try a mock object framework like NMock. The Quickstart documentation for NMock contains a simple example of how to use the framework and what is meant by behavior verification.

Maven's Resources:

Mocks Aren't Stubs, by Martin Fowler;
TypeMock.Net
Working Effectively with Legacy Code, by Michael Feathers, Prentice Hall PTR, ISBN: 0131177052, Sept. 2004.

Motley says: "Refactoring means no more up-front design"

James Waletzky — Tue, 10 Jul 2007 18:08:00 GMT

Summary

Motley: Refactoring keeps my design clean from the start, so no more up-front design!

Maven: Up-front design is still necessary to achieve clarity on the overall approach (preventing rework later) and needs to be documented to allow others to review your thinking.

______________________________

[Context: Motley is trying to educating himself on agile development and is reflecting on some reading with Maven]

Motley: This is really cool - I've been doing some reading on agile development and because I am doing refactoring continuously, that means I can do away with all that up-front design stuff we used to do.

Maven: Do away with it? Hardly! Maybe for a small couple-hundred line program, but any bigger than that, there is still value in thinking through the problem before you code anything - at least, if you want to prevent major rework later in the cycle when mistakes are more costly to fix.

Motley: But refactoring is going to keep my design clean. What's the point of all the up-front thinking?

Maven: That's true - refactoring helps you continuously keep quality high. However, if you make an investment in a bad high-level design and architecture, it becomes extremely hard to refactor yourself out of it. Then you are talking about reengineering, which can be a huge task. Refactoring is for small design changes without changing external functionality. Large changes are not as well suited to refactoring as it may lead to drastically changing the way requirements are implemented.

Motley: But I want my design to emerge based on the learning I get while coding!

Maven: You want the details to emerge, but you need clarity on the overall approach before you flesh out the detail. A design phase is needed to understand the problem, investigate various high-level solutions and their trade-offs, and document your overall approach.

Motley: I'm not sure I buy that - the agile manifesto states that we should favor "working software over comprehensive documentation." No design document needed!

Maven: Not exactly. You stated the keyword: favor. It does not say there is no need for documentation whatsoever. We need that documentation to keep the stakeholders in our design happy and to provide something concrete for people to review.

Motley: And I thought I was done with 100 page design documents…

Maven: You are! You want to document "just enough" in your design. Documents of 100 pages in length are generally overkill and violate the lean principles we talked about previously.

Motley: But how do I know what is "just enough"? If I have 3 days on my schedule for design, "just enough" usually just means I hit the end of day 3.

Maven: Let's save that for another conversation. I have a great way to measure "just enough", but I am going to leave you in suspense.

Motley: Fine. I don't want to be here all day anyway.

Maven: Of course, the other topic of interest is once you have "just enough", you want your design to be "good".

Motley: Great. I want "just enough" - I have no idea what that really means - and I want it to be "good" - I have no idea what that really means either. Judging "good" on a design is all in the eye of the beholder.

Maven: You're comparing judging a design for "goodness" to judging looks?? Hah. Fortunately, we can get a little more scientific than that and focus on the principles that generally make one design better than another. Let's do that...

______________________________

Maven's Pointer: Be careful with the term "emergent design". Many developers equate the phrase "emergent" design with jumping into Visual Studio and coding. What "emerges" at the end is some kind of design (usually junk). Truly "emergent design" uses a feedback technique to provide you continuous confirmation that you have a clean and working design. An example of an emergent design technique is test-driven development (TDD). Tests drive your design, refactoring keeps it clean, and test automation validates it all works.

Maven's Resources: Agile Development Manifesto, Wikipedia entry on Continuous Design

Motley says: "Test both private and public methods"

James Waletzky — Tue, 12 Jun 2007 10:41:00 GMT

Summary

Motley: To get high code coverage, we should be testing both public and private methods

Maven: Only test public methods. Testing private methods gets in the way of refactoring

______________________________

[Context: Maven checks in on Motley's unit testing practices and notices something odd]

Maven: Hey Mot - how is the unit testing thing going?

Motley: Pretty good, I must say. I'm leveraging Test-Driven Development (TDD) as much as possible and it has really improved my code coverage. Check out all these tests!

Maven: Wow - that is pretty good! In fact, that's more tests than I was expecting. Are you literally testing everything?

Motley: You betcha. I have test cases around every method I write.

Maven: Even the private methods?

Motley: Duh. Of course! I gotta get that 100% code coverage, man! That was your idea don't forget.

Maven: Okay, I am willing to bet you are doing TDD without the refactoring part of the process. Am I right?

Motley: Well, um, I guess. I'm trying to do a bit but it's such a pain in the butt. But wait, how can you tell just by looking at my tests???

Maven: Well, if you are testing private methods that typically leads to a slightly larger number of tests, because a good practice is one test (or more) per method that you are testing. If you just test public methods, you can typically get the same code coverage with a slightly lower number of tests.

Motley: So what? So I have a few more tests. It doesn't harm anything.

Maven: Ah, but it does. Testing private methods makes it really hard to refactor. You see, refactoring does not usually affect a public interface but instead the implementation behind that interface, i.e. the private methods. With TDD, you need to be free to refactor, or you defeat one of its primary purposes - to drive a better design. If you need to update tests every time you make a refactoring change to your implementation because you are testing private methods directly, it really gets in the way and you are not going to do it. I am willing to bet that's why you are skipping an extremely important part of the TDD process.

Motley: The private methods do get in the way of refactoring, but code coverage is everything!

Maven: With TDD you don't write a line of code without having a test in place first. When you do refactoring, you rerun your tests to ensure you haven't broken anything. You should theoretically end up with the same coverage even just testing via public methods. If you can't hit your private methods through your public methods, why are they even there? Besides, the code coverage number in itself is not super interesting. What is more interesting is the analysis of why coverage numbers are low. This can lead you to more tests that you may otherwise miss. Don't go overboard trying to get to 100% - it's not worth it. Some C# language constructs, like foreach, have extra compiler-generated code in the intermediate language, which makes it real tough to get 100%. It's not worth the effort. Oh, and by the way, when I say "public", I typically also mean protected methods (public for subclasses) and even internal methods (public within the assembly).

Motley: So you're saying I should just test public methods and code coverage will just come if I use TDD. You're also saying that getting to that magical 100% code coverage number is often not worth the extra effort, and although code coverage is a good measure of your tests, the analysis is more important.

Maven: You should write a book, Mot. I could not have put it better myself.

Motley: I'm a quick learner, as you can see from my stellar deliverables. I can even teach you a few things, like how to be likeable in a social atmosphere.

Maven: What are you saying?!?

______________________________

Maven's Pointer: You may ask, "How do I test private methods anyway? They are private!". In .NET, there are several ways to test private methods should you have a rare case where you definitely need to test a private:

Leverage a test framework that uses reflection to call methods. The System.Reflection APIs can access private members of a class
Use Visual Studio 2005 (and later) for unit testing. It will generate a test context object when you use the wizard to create a test that lets you access privates.
Create a friend assembly using the InternalsVisibleTo attribute that lets you access internal methods at least.
Create/Run a tool that flips some bits in the .NET assembly metadata to change the scope of classes and methods all to public (you are not testing the original compiled assembly in this case though).
Create/Run a tool that generates a proxy for your class and accesses members via reflection. Your tests then use the proxy instead of the original class.
Use preprocessor definitions to change the scope of the private methods to public in a special build.

Maven's Resources: How to Test Private and Protected Methods in .NET provides similar arguments for/against testing private methods, and talks about a subset of the above list for testing private methods.

Motley says: "I don't have time to Refactor - I need to get it right the first time"

James Waletzky — Thu, 19 Apr 2007 16:34:00 GMT

Summary

Motley: Refactoring takes too much time - I have to ship you know!

Maven: Refactoring leads to much more maintainable code, and if you have tests in place, the return on investment far outweighs the cost.

______________________________

[Context: Motley has just embarked on his first adventures with TDD, but Maven wants to check-in to ensure Motley really understands the process]

Maven: So how are things going with TDD, Motley?

Motley: I've only been at this for a couple of hours - why are you bugging me already?

Maven: It's good to start off on the right path. Just want to make sure you understand the TDD process.

Motley: Yeah, yeah, yeah. Write a failing test. Write code to pass it. Move on to the next requirement.

Maven: Whoa there, Nelly! You forgot one of the most important steps - refactoring.

Motley: Not really sure what you mean by that. If I have syntax errors I fix them. Duh.

Maven: Refactoring is all about improving the design of your code without changing any existing functionality. You mentioned previously that one of your values is writing great software. Well, refactoring is not just about writing it great the first time, it's about being realistic with yourself that you won't get it right the first time. You make the necessary changes to prevent it from starting just OK and turning into a big turd later as you maintain it. Obviously it takes more time than just writing the code once, but the pay-off down the road is huge.

Motley: You've been sniffing too much glue, Maven. I barely have enough time to write my code once let alone keep redoing it over and over. I have to ship. If I don't ship, the business suffers, and most important, I get fired, and I suffer.

Maven: What about when someone has to maintain that code moving forward? Do you want to make it difficult for them? How much do you think that costs the business?

Motley: Like everything in life, there is a trade-off. I have to do my best given that one shot is all I get. I vote ship and deal with maintenance issues later.

Maven: That's what has gotten us into all this trouble with our legacy code! How do you like fixing bugs in all that old stuff?

Motley: Well… it sucks. Our legacy code is a "turd" - as you so eloquently put it.

Maven: Exactly. We need to make it easier for the next person to maintain. Refactoring is a way to get us there. Write your code, improve it, and then move forward with the next requirement. It's actually quite natural once you get in the rhythm.

Motley: But I'm afraid that once I have it working that if I make changes I'll break something.

Maven: Ah, but you're doing TDD, right? That means you have tests in place so that if you do happen to break something you know about it right away. It's quite beautiful, really. Give it a shot and I'll check in with you later.

Motley: Why do I keep letting you do this to me? Fine, I'll try it. But if I'm late on my deliverables, it's your paycheck. Deal?

Maven: I'm so confident that you're going to rock that yeah, I'll take that bet.

______________________________

Maven's Pointer: Many times the primary reason that developers are afraid to refactor isn't because they don't have the time, but because they don't have the confidence to make a change without breaking existing behavior. Having unit tests in place increases that confidence and makes refactoring easier and fun. Avoid refactoring without first having tests in place!

Maven's Resources: Refactoring: Improving the Design of Existing Code, by Martin Fowler et al, Addison-Wesley Professional, ISBN: 0201485672, 1999.

Motley says: "If I write code to pass tests I'm cheating!"

James Waletzky — Sun, 15 Apr 2007 09:44:00 GMT

Summary

Motley: Tests need to validate code that has already been written - not the other way around.

Maven: Writing tests before you code allows the tests to drive your code while providing more sound requirements validation.

______________________________

[Context: Motley is thinking through test-driven development before he goes off and tries it]

Motley: I've been thinking about this TDD thing and something is weird - if I write code to pass tests that's cheating! Tests need to validate code that has already been written.

Maven: Cheating? Hardly. Your tests are written around the requirements. What's wrong with writing code to execute the requirements? You're actually cheating if you write tests after the code. Now the tests are exercising the code but you are an extra step away from the requirements. Validation that you met the requirements is loose at best.

Motley: But I write my application code to satisfy the requirements. What's the difference?

Maven: The difference is that tests validate that the requirements are met, which is a logical layer between the requirements and the code. It forces you to think about the requirements in greater detail before implementing them. Plus, you get all the other benefits we talked about with TDD.

Motley: Well, maybe, but…

Maven: Just do me a favor and try out TDD for a couple of days, okay?

Motley: Fine, fine, fine. If it gets you off my back. Give me a couple of days to play. Now go make yourself useful and clean up the kitchen or something.

______________________________

Maven's Pointer: "Requirements" as described in TDD are usually described at a lower level than you may be used to. A "requirement" is one small nugget of functionality that you want to implement. It may be an entire (simple) method, or one positive or negative path through the method.

Maven's Resources: Test-Driven Development in Microsoft .NET, by James Newkirk and Alexei Vorontsov, Microsoft Press, ISBN: 0735619484, 2004.

Motley says: "I can't write tests before application code - there's nothing to test."

James Waletzky — Mon, 09 Apr 2007 18:32:00 GMT

Summary

Motley: You can't possibly write tests before code - there's nothing to test.

Maven: Writing tests for a method before coding it has all kinds of design advantages.

______________________________

[Context: Motley has been writing unit tests for a week. Maven wants to take the next step]

Maven: Hey. Looks like you're getting pretty good at this unit testing thing.

Motley: Yeah, I can definitely see the benefits of having a thorough suite of tests. I can make sure that with every code change I don't break anything.

Maven: Want to take it even farther? Do you want to make an even bigger difference in quality?

Motley: You really are a poop disturber, Maven. Fine. Let me hear about your hair-brained scheme.

Maven: How about this? Try writing your tests first before you write your code. It's called test-driven development, or TDD for short.

Motley: I can't write tests before I write any code! There would be nothing to test! You've been sniffing to much glue for your own good.

Maven: No, I am serious. Thinking about how you are going to test your code before you write it has all kinds of advantages, including driving your design. In fact, TDD is more of a design technique than a test technique when you really think about it.

Motley: So how could this possibly drive my design? What difference would it make if I write my tests before my code - which still seems crazy to me - or after? It's like saying that I should edit my book before I write it.

Maven: Maybe if you thought more carefully about how your book was to be edited, you'd do a better job writing the book the first time. With TDD, since you write your tests first, your design automatically is testable, which the test team will love. You can also view your APIs from the perspective of a client even before the API is implemented, so you know right up front whether the API is usable. In addition, coupling and cohesion problems in your code come to light immediately. I am sure you hate programming to difficult APIs. Well, this way you make sure that your APIs are clean before you start because you are using them. TDD essentially forces a good design to emerge based on your tests.

Motley: I'm still not sure about this. So all I have to do is write my tests before the code?

Maven: Well, to truly do TDD, here are the steps to follow:

Start with a requirement. A requirement could be one method you want to implement.
Write a test for that requirement.
Write just enough stub code to make the test and code compile.
Write just enough code to fail the test.
Execute your tests and watch them fail.
Write just enough code to pass the test.
Execute your tests and watch them pass.
Refactor.
Execute your tests to ensure you have not broken anything.
Go to step 1.

Motley: That's a lot of steps. Seems too complicated to really be useful. And what's the point of writing a failing test first?

Maven: Actually, it's not complicated. Once you get in the groove, it actually feels quite natural. Oh, and the point of writing a failing test is essentially to test your test. If you write a test and expect it to fail but it passes, you know you have a problem with either your test or your code. By starting with a failing case and then watching it pass, you are more confident that your test is good.

Motley: Let me finish what I am doing with my current set of modules and then maybe I'll give this a shot. No promises. You'll also have to enlighten me a bit on the refactoring step, as I'm not sure what to do there.

Maven: No problem, Motley. I applaud you for having an open mind to try this stuff out.

Motley: Whatever. The boss says you were hired for your sound engineering ideas, so I guess I should humor him and listen. Hehehe. Kidding, of course. Maybe.

Maven: Stay tuned. I have a lot more to say about TDD. It's a great developer technique.

Motley: Can't wait. Ugh.

______________________________

Maven's Pointer: A precursor to effectively doing test-driven development is understanding how to write good unit tests. Use a proven test framework to make it easy to write tests, and ensure your tests execute quickly. After all, with TDD, you are perpetually running your tests. It's such a great way to develop code that I don't think I'll ever go back to writing tests after code.

Maven's Resources: Test Driven Development: By Example, by Kent Beck, Addison-Wesley, ISBN: 0321146530, 2002.