Don’t tell anyone, but it is spring here in Seattle. The daffodils in the patch of dirt outside my front door have started peeping out. A blossom tree on my way to work is in full bloom. And I saw an e-mail thread yesterday trying to get a crowd together to go to the nearest ski-able snow - in Tahoe. All this feels strange, because all the Valentine’s Day books I have been reading with my son recently have had pictures of snow in them.
Maybe it is only adults who see these incongruities. After all, during my childhood in the tropics I read my fair share of American and British children’s books (I won’t date myself by telling you which ones), but I don’t recall ever being perturbed that I was not likely to ever see any real snow in my life.
Segue to the other big sign of new beginnings in the Puget Sound. At work, we recently concluded our first Test Day – an in-house, one-day conference devoted to Testing. It was like any regular conference, except you did not have to catch a plane and stay in a hotel. It even had a little trade show. However, it was different from a regular conference (like the one I attended in Baltimore back in Dec.) in that there were some very good career track sessions where senior Testers talked about career paths and options. Another difference was that you did not get any time to stand around and chat with people who are total strangers, yet with whom you share this special bond. (Like being on the slopes or the greens, I guess.)
Of late, I’ve also noticed a lot more blog activity from the Testing community here are Microsoft. (Of course, maybe it was always there, but, as usual, I wake up to the party when it is just finishing.) (No, I am not going to waste any more time searching for a bunch of links) A lot of this blogging seems to be about what it means to be a Tester at Microsoft. It is almost as if this community has hit adolescence – the outward swagger along with the internal nerve-wracking quest for self-identity.
This raising of the public profile of Testing is interesting, because I normally think of Testers as being on the dour end of the spectrum. Of course, I work daily with all sorts of people - the chaotic, the orderly, the quiet, the flamboyant, those who rush into action, and those who plan excessively – so I should not be tarring them all with the same brush. But as a breed I think we Testers are conservative, and a bit pessimistic. We are not the creative energy of an organization. Rather we have the reputation of being the naysayers. We don’t build anything, yet constantly argue that it is broke. It is our business to know what can go wrong (and like Tom Ridge we even color code the status of our features). We are not people who can do a Steve (Ballmer or Jobs, your pick) in front of the faithful.
The spurt of blogging is positively uplifting when compared with the reading I have been doing recently about Software metrics, and Defect Analysis: 6σ, CMM, Blackbelts, TSP, PSP, ODC, QGM, TQM, and on and on and on. So much process! Of course, if you are Boeing (you forget, I live in Jet City), or GE (the six sigma kings who make the engines for Boeing) all this matters a great deal. It is easy for you and me to see what an airplane falling out of the sky can do to their bottom lines. (Mind you, with each passing Windows security bug, Microsoft moves one step closer to that. And if Air New Zealand takes delivery of the first 787 - yes, the 7E7 got its official name last Friday - before we ship Longhorn, then maybe we should consider changing things around so we become more like Boeing.)
All this - the Baltimore conference, Test Day, Testing blogs, Quality Processes, Boeing & GE – all this finally kinda gelled in my mind this morning as I drove in to work, We – the Testing community at Microsoft - are different. We do live in a different land – where the trees are in bloom and there is no snow in February. We do feel out of place amidst the regular Testing literature and conferences. Maybe we just need to be children again, and not be so bothered by the difference.
Here’s why we are different. In my mind, software projects at Microsoft are bit chaotic. (you may detect the lingering after-taste of Paul Graham’s Made in USA essay here. I myself did not realize it until many hours after I first wrote this para.) The product grows bit by bit around bursts of intense creativity by individuals or very small teams. Designs change for technical reasons. Market pressures come to bear. Stuff gets ripped out. New stuff gets put in. Things that were previously not possible, suddenly become. Stuff that was carefully planned gets canned. The mass of features, both old and new, aggregate in to the final package, with interconnections to the rest of our product line. Almost from the first sketch on a napkin, to the final shrink wrap on the glitzy box, we in Test are there -hovering over every process, like clucking mother hens worried about what almost surely is going wrong right now. We are constantly trying every thing we can to make sure that there is nothing wrong in the product, as well as everything right in it.
If you think of a software project as akin to building a large, new public garden, then we are the groundsmen in the ever-shifting landscape. If we don’t choose the seeds, and we don’t plant or water them, and we don’t lay down the pathways through which the users will enjoy our garden – since there are other people (maybe even non-dour ones) who do this, and who do it extremely well – then what is it we do then?
Well, when we first see the plans we might remark that service trucks might not be able to get to the back of the buildings. We might observe that the sidewalk is not wide enough for the crowd on a busy day. We might point out that it is not intuitive as you round the first corner that you should keep going to the right, rather than cross over to the left. Maybe that tree is going to leave a lot of leaves on the ground in the fall. And if you dropped in a paved path just here, it would make testing much quicker easier.
As construction gets under way, we are there every day, sitting in the benches, walking down the pathways, reading the signs. And each time we hope we see it just the way the first-time visitor would, and just as the old-timer would too. Where is the nearest rest-room? Is it sign-posted? What if you came at the sign from the other side? What if you did not read English. What if you could not see? Why do I always have to go all the way around to get to it? Can a wheelchair go on this path? Does that tree block the view of the lake from this bench? Is the bench back in the same place after they ripped it out to lay the new plumbing? Did they put the sign back too? Did they paint it just the right color? What would happen to the lake if it rained four days straight. What if it rained four days straight with one foot of snow on the ground? What if the rain was warm? What if the power went during the snow and did not come back when it started to rain? What if someone held a banquet at night? Or a rock concert?
When the first alpha and beta visitors come in, we wait with bated breath to see what they will find. Oops, no one caught the possibility of splinters on those benches. Hey, we used that same wood for the picnic table across there. The signage is not working, we are going to have to re-think this. Note to self: Sign got repainted and repositioned - Did they stick with the correct color? Is the post blocking any views in the new place? Damn, they extended the parking lot without extending the sidewalk; too late to fix that now. And so on and on it goes, with more and more tension until the last service truck (and Program Manager, and Developer, and Architect, and Usability Expert and User Educator, and Setup Team member) leaves before opening day. Then, it is just us and the final product, and a few short hours for the final walk-through. It is almost like the mother of the bride, proud of what waits on the other side, yet reluctant to let her baby go. (For a blow-by-blow account of the last week of our first VS 2005 Beta, see this blog by Soma, ex-tester, now VP of our Division.)
That’s what we do – except we ‘walk’ and ‘look’ and ‘read’ and ‘sit’ and ‘cause storms’ in a real software application, using little code golf carts we write, and little code binoculars we write, and little code rainmakers we write, and big harnesses we write that let us do this again and again for months without going crazy.
But if you’ve ever worked in QA - as is very likely if you are this far into my ramble - then all this is old hat for you. “Tell me something new” you say.
The key is - and this came to me only today – the key is that while we are inextricably linked into each part of the process, we Testers at Microsoft also need to be as transparent and frictionless as possible. We can’t weigh down the creative, and at times chaotic, flow with layers and layers of process that bring the whole creaking machine to a halt.
It is our role to apply only just as much pressure as necessary at each step to ensure a high quality product, on time and on budget. Some times this could be the barest of nudges. Sometimes we might have to refuse to sign-off. When we do the former, we our doing our job well – we’ve been looking in the right places, at right time, in the right way, and we know exactly where, when and how hard to nudge. If we find ourselves doing the latter, we obviously failed to intervene adequately at all the previous points.
So, … do we know how to do this process-free, just-in-time QA? Heck, no! You never know how to do such things. You just constantly, constantly, constantly try to get better, and then one day it looks to others like you know what you are doing. It’s then you realize that you are no longer an adolescent – You know what music you like, what clothes you like, what food you like, and what having fun means – especially while at work.
Of course, since this is Microsoft and the landscape is an application, you have to understand software. Nay, it has to be in your blood and at your every synapse. You have to be able to design and write code like the best. Otherwise you would not know where to look, how to look, and when to look for bugs. You would not be able to get it all done in time, with confidence. But in Test it is a lot, lot more than that. It’s also about people, about processes, about organizations, about customers. It’s both about keeping bugs out of the product and about getting those that are in fixed.
It’s about being transparent. It’s about leaving your mark, with not a line of shipping code to your name.
It’s not sexy. But it sure is fun.
[I owe this blog to Suma on the C# IDE team. This piece began as a conversation with her, and she urged me to blog it, and then stuck around to provide very useful comments on a draft version.]