It's OK Not to Write Unit Tests
Don't feel bad, it's okay not to write unit tests.
But Unit Tests Work For Me!
First, are you sure you're really unit testing? Unit testing is all about testing “units”—independent pieces of logic. According to Michael Feathers, a real unit test cannot talk to a database, communicate across the network, touch the filesystem, run concurrently with another test, or require extensive setup. If they are any dependencies, they are mocked away. Do your tests do that?
That's what I thought. They're not unit tests, they're just tests.
But okay, who cares if you follow all the rules, at least you write tests. You probably work on a project that lends itself to easy testing, and your platform likely has great tools for the job. You also have great intuitions where the right level of testing is—or if not, you're at least at that stage in your career where you make lots of little mistakes and the tests you write actually help you catch them.
Realize, however, that not everyone has that same context as you.
College students, for some reason, seem especially enthusiastic about unit testing. Maybe it's that unit testing was that teaching tool that got them from unconscious incompetence to conscious incompetence all the way up to conscious competence. For some, this experience can be the start of a love affair that lasts many years into their adult career. No one forgets their first crush.
But once somewhere around level four of the Dreyfus model of skill acquisition, unit tests start to lose their effectiveness. In fact, they might even be what's holding you back from further growth.
Training wheels are a best practice—for learning how to ride a bicycle. They're not a best practice for riding the tour de France.
Unit Tests Give Me Confidence When Refactoring!
Yeah, right. Let's be real now.
There's this theory out there that Agile projects can refactor fearlessly because there's this immaculate suite of tests that can sound the alarm the second the smallest regression gets introduced. But anyone who's actually tried it knows that it's mostly just a fantasy. Unit tests tend to overspecify behavior—they test implementation details that don't matter, rather than fixed contracts that do.
And they're too fine-grained. The larger component may preserve its behavior (or not), but the comprising sub-units will be radically transformed. Many of them will not even survive the process at all. That's what refactoring means. I've never known a significant refactoring which didn't require the tests to be majorly reworked or even rewritten.
A test that needs to be updated every time the product changes is not really a test at all. Think about it.
Unit Tests Catch Bugs!
Really? When was the last time?
Let's be honest. Your tests mostly follow the “happy path”. Sure, on occasion you remember to test “the failure case”—the caller passed in null or a negative integer as an argument—mostly because you just got done writing that check and wouldn't it be a waste not to write a test to show how clever you were? Never mind that null or a negative argument is an assertable precondition that could never happen in production anyways.
If you're a Java programmer and want to have a rude awakening, go download Jester. Jester is an automated mutation testing tool—it goes in and replaces “<=” with “<”, “&&” with “||”, “!” with whitespace. And then it re-runs the tests. And then you get to watch in horror as your tests still all pass, regardless of what the product code actually does.
It's also not uncommon for the tests to have the same bugs as the product. And why not? The same person wrote both. If the programmer writes code to give a discount to all orders above $1,000, when he writes the test is he going to double-check the spec to find out that the threshold is really $10,000? This is the so-called “ugly mirror” problem, where the code looks just like the code it supposedly tests.
Unit testing is no substitute for adversarial testing.
There are so many kinds of failures unit testing can't find for you. Memory leaks. Intermittent issues. Stress failures. Unexpected behavior in third-party components. Subtle interactions between features. Usability issues.
Code you forgot to write.
Yes, unit tests can only catch problems in the small—and the majority of your bugs are bigger than that, aren't they?
Unit Tests Improve My Design!
No, they don't. If anything, unit testing encourages some pretty questionable practices, like making private methods public just for so they can get the code under test, or creating zillions of interfaces for mocking purposes, interfaces that leak implementation details like water leaks through a sieve.
Unit testing isn't all horrible, of course. For one, it's big on writing clean function and object contracts, which is a good thing. Its heart is in the right place when it comes to its rabid stance on decoupling—although a more sober analysis on sources and causes of design change is usually more helpful.
But there's a lot of things unit tests don't teach. You'd never get to Tell Don't Ask with unit tests alone—in fact a great number of tests involve introspection of state, often in encapsulation-breaking ways. Many unit testers are just fine with deriving Circle from Ellipse as long as they thoroughly test the arbitrary, idiosyncratic way Circle handles the case where you set unequal major and minor axes. And who cares about Open Closed Principle as long as the tests still pass after you make an invasive change?
At best, unit testing is a weathered signpost saying “good design practices are somewhere over there”. But it's no substitute for actually knowing those practices.
Have you ever watched someone try to solve even a moderately complex problem with TDD alone? It's painful. Really, really, really, really, really painful. Maybe it can be done, but it really just amounts to a random walk across the problem space, and you'll have to stub your toe a good many times before you finally get to where you want to go. To make any reasonable progress, you already need to know a little something about good design principles from the outset.
Yes, unit testing is just a signpost—and not a very good one either. Many are beginning to discover that functional programming teaches far better design principles than unit testing ever will. Academic esoterica like currying and monads and the lambda calculus aside, it's shocking the number of code construction bugs which relate somehow towards mutable state—which is what functional programming teaches you to avoid.
Strangely enough, functional programs often turn out to easily testable anyways, even when no deliberate effort has been spent to make them so.
If You Can't Test It, You Can't Ship It!
Unit testing is not the only sort of testing out there. Don't make the assumption that if a product isn't unit tested, then it's not tested at all.
Testing is an economic activity like any other. Every minute spent writing tests is a minute not doing something else. There's always another test that can be written, and if you had infinite time you would write them all. But you don't. Somewhere out there is a point of diminishing returns. There's a point where you say, “we're confident enough; let's ship”.
Now I'll give you that some tests pay for themselves. Some of them are so good they even pay a never-ending stream of dividends. But then again, tests can have negative ROI. Not only do they cost a lot to write, they're fragile, they're always broken, they get in the way of your refactoring, they're always having you chase down bogus failures, and the only way to get anything done is to ignore them.
Don't be suckered by the trite dictum that “no test is worse than having no tests”.
But It's The Corporate Standard!
Then your organization is dysfunctional.
Sorry to be so blunt, but there it is. If it's any consolation, a lot of organizations have the same dysfunction: they don't trust their employees to do the thinking. They think they know better than you, and they don't.
Have you ever worked with a good manager? They're full of ideas. They'll suggest, they'll coach, they'll recommend. They'll cajole and beg and plead if they have to. But they won't tell you how to do your job, not unless it's a real emergency, anyways. They hired you, they gave you a task they know you can handle, and they trust you to make the details happen.
Good organizations know there are no best practices that apply regardless of context. They also know that every employee has a unique working style, many of them equally valid, and so practices that work well for one person or one group may not work as well with the next. Particularly enlightened organizations know that many so-called “best” practices are really just a gang of superstitions and prejudices in disguise. What else could account for the prevalence of thoroughly-discredited Waterfall methods in this day and age? It's a short walk from “the only practice we know” to “the best practice, period”, and before you know it, you've blinded an entire organization to better ways of doing things.
It's a wonder how any innovation occurs in such companies—but if it happens, you can be sure it happens by treasonously unconventional means.
So What Are You Saying, Man?
Don't get me wrong. I'm not saying that all unit tests are worthless, nor that you should never write one ever again. But I am suggesting that you take a good hard look at the time you're spending and ask yourself what benefit you are really deriving.
What I'm saying is that it's okay if you don't write unit tests for everything. You probably have already suspected this for a long time, but now you know. I don't want you to feel guilty about it any more.
There are so many other ways you can find bugs with less effort. Like asserts, to pick a random example. Maybe you already write a lot of them—but do you really leverage them? Or do you instead routinely ignore them like so many others do? If the latter, don't worry—you're probably not using them as effectively as you could. Just remember that if an assert goes off, it means you need to fix something. Don't write asserts to “notify” you of interesting events which your code already handles correctly. Assert is not a logging mechanism.
Acceptance tests are great too. Like, let's say I was writing a SHA-1 hash implementation. That's a lot of code. And I wouldn't write little tests all the way down. But I would have a few tests validating that it works at a very high level, absolutely.
Agile dogma says that tests should be fine grained, but really, what's the point? Debugging is easy, at least in comparison to writing all those tedious tests. If you think about it, all you really need is something that alerts you that something, somewhere, has gone wrong—a “tripwire” test, so to speak.
Test what can fail. Test stuff that's easy to test. But don't beat yourself up trying to get 100% code coverage. 100% code coverage doesn't mean all that much, anyhow.
Hope this helps,
- cashto
