[Rob]

You've probably had enough time to read the last post...Ok, so maybe we were reallllly busy getting our beta product release squared away so you can all get your free Visual Studio 2005 Express Beta1 downloads!  Grab it, have fun, win an Xbox, tell me about what was good and what needs more! (There are also versions focusing on other programming languages and even web development!). http://lab.msdn.microsoft.com/express/default.aspx

***

Tactical thoughts

Thinking about all the things that go into testing feature...is the API working? Are there any attack paths through this feature (is it secure)? Does it integrate with existing features? Does it depend on features (our group or another?!) that might change before lock down? Is the UI compliant with suite requirements?  Is the UI clear?  Are UI redundancies appropriate? What will our users think about our answers to these questions?  What happens to the UI when the product is localized (translated to another language)? What happens to the behavior of the feature when the product is localized? (Add a number of other questions from networking to user permission levels depending on the feature category...).

The list appears endless, but luckily we have limited time (some sarcasm on that use of 'luckily', if you didn't notice) to do our testing so we work with a finite set that makes sense for the feature in question.

The highest urgency task is to get the feature tested for main stream functionality.  What will our customers need this to do?  What will our customers want this to do?  What assumptions may be made that could get this feature into trouble (and put the user into a confused or frustrated state)?

We also want to see that the feature discoverable and useable.  The UI needs to indicate what will happen and let users know about progress if this takes time.  Keeping UI consistent so different tasks at least 'feel' familiar in the welcoming sense is valuable...but difficult when taking legacy features into a new arena (a challenge for many still useful feature sets now housed in a building of .NET splendor).  Is there a point to upgrading old UI to match the new style if the old features are stable as is (and use the resources instead for new features and to fix bugs)?  Is old UI style a bug or an acceptable artifact of evolution?

These questions are what test cases are all about.  Each test case seeks an answer. Customer feedback, experience, and research help us to ask the right questions for our current product.  Once we have a base set of questions (Test Case Planning is a cyclical process, just as implementation is) we can move forward to measure the status of product drops based on the answers we get on test passes.   We build our automation library which allows us to gather answers faster and feel more secure against regressions that could be introduced every time developers touch the product.  At the end of the cycle, we have an automation set that allows us to have a level of confidence that sustained engineering will be able to do service pack work while maintaining a high level of regression test coverage (while the front line team moves to the next generation product).

[This posting is provided "AS IS" with no warranties, and confers no rights. Use of any included script samples are subject to the terms specified at http://www.microsoft.com/info/cpyright.htm.]