Update: this blog is no longer active. For new posts and RSS subscriptions, please go to http://saintgimp.org.
Before I get into technical subjects, I should briefly outline the context in which I do my work.
Microsoft Game Studios publishes games written by internal MGS studios and by third-party game developers, for both the PC and Xbox platforms. MGS has a shared testing organization called the GTO that’s responsible for testing most of the games that MGS publishes.
The GTO has a testing philosophy that’s somewhat unique in the game industry; rather than just throw hordes of bodies at an almost-finished game to find all the bugs, we apply the engineering-oriented testing approach that’s found in the rest of Microsoft. That means we have a strong emphasis on driving quality early in the development cycle, on preventing bugs from being created rather than just finding them after the fact, and on using tools and technologies to reduce the amount of tedious manual testing that we have to do. We want to automate as much of the testing process as possible, and make the rest of it as efficient as possible.
Of course, trying to do these things as a game publisher presents some big obstacles. As I noted in my intro post, I came to MGS nine years ago thinking it would be a pretty easy task and it turned out to be no such thing. It’s an interesting, multi-faceted topic that I may go into detail on later, but for now I’ll just outline the highlights:
In the face of all of those challenges, how do we focus on engineering-oriented testing? First, we hire really smart engineers who have the skills to solve those challenges. A position in the GTO isn’t merely entry-level or dead-end; it’s a serious and rewarding career and lots of us have been around for a long time because we love what we do and we’re good at it.
Second, we have a team of programmers, of which I am a member, who are dedicated full-time to making our games testable and to building testing tools. This team takes on a wide variety of jobs depending on the needs of each game title but the major themes include:
From a software development standpoint, whenever we work with our development partner’s source code (like when adding test hooks to a game engine) we strictly hold to their coding standards and ways of doing things. That work is usually in C++. However, when we work on standalone tools that don’t get compiled into the game engine, we work in C# where possible and we’re adopting more and more XP and Agile practices to improve the way we design and write our code so we can deliver more business value.
I should note that while the GTO has had several full-time tools programmers for a number of years, we came together as a united tools team within the GTO less than a year ago and we’re still working on meshing our individual ways of doing things into a cohesive whole. Obviously, I’ll write my blog posts from my own perspective which doesn’t necessarily represent the position of anyone else in the GTO tools team, the GTO, or MGS.