Rededication and retargeting of this blog to a pure QA focus
The original incarnation of my blog went fallow for a simple reason. At the same time that I launched it, numerous other blogs focusing on Visual Studio Team System for Database Professionals were created by management, PM, marketing, etc. Most topics that I would have covered were being covered repeatedly elsewhere. This fact gave me the excuse to leave the blog behind and focus on driving testing of our product as our V1 RTM data rushed toward us.
Why resurrect the blog now? Well, we learned a lot of lessons about doing QA in a Scrum development environment in V1. While we achieved good product coverage, as measured by code coverage numbers for our test automation, in our V1 cycle, it was not always pleasant. The problem was that we were using 'old' approaches to QA in a new, highly iterative development environment. Here were some lessons we learned:
- Scrum is well-suited to 'greasing the wheels' of dev, but it is a methodology that is not naturally QA-focused. Yes, I know that the methodology stresses quality upstream. I've read all of the books and been through a full product cycle under the methodology. Does this mean that you cannot deliver a high-quality product under Scrum? Of course not. What it does mean, though, is that QA must interact with Dev in new ways and re-prioritize how it executes. The Scrum methodology suggests this, of course, but now how it is to be accomplished in the real world. We've made a lot of progress here.
- QA must be much more agressive in its communication than under non-Agile methodologies. Not snarky, mind you, just more productively agressive :)
- Teams used to puting a primary emphasis on system-level testing must shift their priorities in Scrum or other Agile environments. You will still write system-level tests, but your automation efforts will need to 'grow' to these tests rather than starting there.
- Inefficient internal tooling that is merely annoying in a non-Agile environment can present a major productivity hurdle under Scrum
- In highly iterative development there is a greater need for metadata associating specific test automation with specific Dev work items so that QA can quickly identify specific scenarios that are and are not covered and quickly create automation runs that are targeted to specific changes.
- If you think that communication between QA feature owners and their Dev counterparts is 'OK', it's not good enough.
- If you want your QA team to engage in customer-scenario-centric, extended, 'deep dive' testing, you must plan it out sprints in advance, defend it vociferously to Dev and to management, and be ready to show off the results that justify the efforts.
- A lot of teams in Microsoft are placing bets on model-based testing (MBT). We learned a lot about how MBT helps and where it's full of hype. Many aspects of MBT do not fit the Scrum environment in my view; nevertheless, there are some key uses of MBT tooling and methodologies that can pay off for QA in an iterative development environment
As a result of lessons learned in our V1 product cycle we are radically changing how we do our work in QA for our V2 cycle. We were successful in V1, but we are confident that we now know how to significantly improve our effectiveness in V2. The approaches I'll discuss in this blog are already in place and already showing productivity gains.
Most of what I discuss in this blog will be specific to the work our team is doing, not an academic discussion of what testing should be like. Further, I'll be providing details on customer scenarios that we're testing, database schema constructs we're testing, input data we're using for our tests, and verification approaches we're using. It is my sincere hope that customers of our product, real or potential, will see what we're doing and provide feedback that validates where we're doing the right thing and correcting us where we're making incorrect assumptions about how customers build their databases. Such feedback will help us to get more and more into the customers' shoes.
We really do work very hard in the Developer Division to provide quality tools to our customers. I've spent a decade of my life, so far, trying to maximize customer satisfaction and I take that responsibility very seriously, as do the excellent members of my team.
