Treating test automation as a product changes the way we design and implement test code and helps to ensure that the tests provide cost-effective feedback for their entire lifespan. The time a test spends being used for the verification of new functionality is relatively short, but the same tests must continue to run for many years as a guard against regressions during the creation of service packs for a released version of the software or during development of newer versions.
While it is essential to not lose sight of the importance of the software that is being tested, considering “test automation as a product” provides some interesting additional perspective on what constitutes good test automation. Some of the implications are:
Attention to crafting the test automation product involves more than just sound engineering. There should be a conscious effort to limit the portfolio of frameworks and libraries used in test development. In many cases, ownership of tests is transferred to other teams for long-term support of the released software. The down-stream teams are often smaller than the development teams and cannot afford the extra effort that is required to understand and maintain tests running on a diverse set of test tools. When confronted with the choice of creating a new test framework or test library, think hard about the users of the automation product you are creating. Do they benefit from the new tools or is it just creating an additional barrier to the uptake and long-term maintenance of these tests?
Page, A., Johnston, K. and Bj Rollison. 2009. How we test software at Microsoft. Microsoft Press, Redmond, WA.