A wise test lead once said to me, “do as little as possible while still ensuring quality”. He wasn’t giving me tips on how to be a slacker =) I think what he wanted to say was there always seems to be more work than there are people so it pays to be as efficient as possible, especially when prolonged testing can hold up shipping a product.
One thing I’ve seen at Microsoft is sometimes the tendency to want to test everything, which isn’t a bad thing as long as you don’t have to ship =) Hence the title – “just because you can test it doesn’t mean you should.” If say you have several components on a stack calling each other, it makes sense to just test at the highest layer to automatically get coverage for lower components. Some people also call this end to end testing. When you test each component separately, you may have the following issues:
So my recommendation?
Of course there are times when component testing is called for.
What about MVVM (Model View ViewModel) you say? This is where you have a layer (ViewModel) that encapsulates data and functionality needed by the UI allowing you have to have a very thin UI layer, even making it easy to replace the UI layer if needed. Some folks prefer testing against the ViewModel as opposed to going through the UI. As somebody who has done both, I can tell you that testing against the ViewModel is much easier and UI testing can be a pain. But in my opinion, the easy way is not worth the risk. We saw several bugs slip through because the thin UI layer that supposedly didn’t have any bugs of course did. Going through the ViewModel has its uses like expediting certain operations but I don’t recommend exclusively testing against it.
So to close, test end to end first, and component test only if you need to. Lazy testers are efficient testers, unless they’re trying to outsource their work on craigslist =) Feel free to comment if you have any thoughts on this topic.