A portion of the material from How We Test Software at Microsoft comes from some of the classes we teach to testers at Microsoft. Bj’s chapters on functional and structural testing, for example, cover much of what we teach to new testers at Microsoft. I recently taught a class to a group of experienced testers at Microsoft and had a chance to reflect on some of the material that exists in both the book and this particular course.
Testability and patterns of software test are two topics a recent group of senior testers and I discussed (both topics are discussed briefly in chapter 4 of HWTSAM). When I first started talking with testers about testability a few years ago, there wasn’t a whole lot happening with the concept around the company. Today, just about every team understands the importance of the concept and we have a huge amount of material available internally for guidance and education. There’s still more to do, but it’s nice to see the growth and maturation.
The use of patterns for software testing is also gaining some traction. Design patterns, of course, have been popular among software developers for years, and most developers recognize the value of communicating common solutions to software design problems. One of the things that has always bugged me about software testing is that we don’t have a common language to communicate testing ideas. Testers throughout the industry often refer to the same concept by different names. Worse, we don’t have a common way to teach new testers about testing. Most of the time we sit testers in front of an application and say, “here test this”. Describing testing as patterns is one way to communicate common testing approaches to new testers. In the book – and at Microsoft, we use a variation on Robert Binder’s patterns described in Testing Object Oriented Systems. The simplified pattern includes eight elements:
In the book, we use the example of boundary-value analysis – or more simply stated, the technique of testing around the edges of valid input values (e.g. testing for “off by one” errors). Another example of a pattern could be the “All Pairs” test pattern (another technique discussed in-depth in hwtsam) . Using the template above, that pattern may be described like this:
I find test patterns to be a great way to communicate and educate testers about design techniques, heuristics and approaches to solving software testing problems. Students in our classes and I have created dozens of patterns using the template above. Some are merely “good”, but others are outright fantastic. Once we get them all polished up, I’ll make sure we share them somewhere (perhaps even in another MSPress book) so that other testers can learn test design through patterns.
"Testers throughout the industry often refer to the same concept by different names."
or worse... refer to different concepts by the same name (e.g. integration testing, test plan.) I know Cem Kaner and others have done work on this, but you can't really impose a dictionary on people, especially if the only way to do it is to get them to memorize the dictionary.
I think I agree - a dictionary isnt' the solution to communication problems, but I don't think patterns are a dictionary. I see them as conceptual building blocks; or a taxonomy rather than strict dictionary definitions.
If we do this right, patterns can be more than dictionaries, like algorithms are more than data types, and recipes are more than the measuring cups.
Done right, I think talking in the language of patterns is actually an escape from having to impose on others.
Grade school textbooks around the world can share that a useful "pattern" for computing the circumference of a circle is 2*pi*R, even though one student may be required to use inches and the other has to use centimeters.
Exactly. Great - no, fantastic(!) analogy. Thanks for the comment.