The p&p Web Client team has now shipped the Contextual AutoComplete, Validation, Composite Web Client Library, Composite Web Client Automation, and Responsive Composite Web Client Reference Implementation Guidance Bundles. Both Glenn and Mike mentioned these in previous blogs. I want to give you insights into how we built quality into these bundles.

As the PM, it feels good when I can say that we shipped something customers want and we feel good about putting our name on it. With the bundles that shipped, I feel good about shipping guidance on creating responsive composite Web clients. As Glenn mentioned in his Blog, adding AJAX support to the Web Client Software Factory was the highest requested item on Codeplex.

Ok, but what did you do to ensure it is a quality set of bundles. For us to do these smaller pieces of guidance, and still be able to release them somewhat regularly, we have made some minor changes to our quality bar for shipping.

For our past deliverables like WCSF v1 and v1.1 and SCSF we have done a lot to ensure high code quality, including (but not limited to):

From development

  • Having as much of the team in a collaborative environment as possible
  • Lots of spiking
  • Example Driven Development
  • Continuous Integration
  • Tracking Code Coverage of Unit Tests
  • Periodic reviews of code quality metrics gathered from nDepend
  • Aggressive Refactoring
  • Use of good architecture
  • Usability testing

From test

  • Code reviews
  • Acceptance testing
  • Functional testing
  • Whitebox testing
  • Deployment testing
  • Disaster testing
  • Security testing
  • Globalization testing
  • Localization testing
  • Stress testing
  • Scalability testing (scale up and scale out)
  • Usability testing
  • Overhead testing
  • Threat modeling
  • Revew with SWI (Secure Windows Initiative) team
  • Documentation testing

This is not an exhaustive list and I am sure I left off a lot of things.  Needless to say, we do a lot to make sure the code we ship is good.

As you can guess, all of this takes time.  As an example, we shipped the June 2007 release of WCSF.  This had minimal code changes for a few bug fixes and an update of several dependencies (Enterprise Library DLLs, really).  A full test pass took over three weeks.  Due to the delay, we took a lot of flak from some members of the community who claimed they could (and some did) ship a new version quicker than we did.  With the Microsoft and patterns & practices names on the deliverable, we needed to do a high level of testing to be confident we were shipping high quality code that our customers. 

This brings us to how we test the Guidance Bundles which are smaller and more frequent releases.  To do this, we are being very thoughtful on what testing we do for a bundle.  Each bundle and the risks for each bundle are carefully considered.  We may decide that for a certain bundle that we do not need to do Globalization, Localization, and Scalability testing.  When the factory is released, we will consider what additional testing is required to call it the factory.

For example with the AutoComplete bundle we:

  • Created spikes
  • Example Driven Development
  • Continuous Integration
  • Tracking Code Coverage of Unit Tests
  • Periodic reviews of code quality metrics gathered from nDepend
  • Conducted white box test for the QuickStart and Extender
  • Performed Block box test
  • Created test automation
  • Created Threat Model
  • Conducted stress test
  • Conducted Globalization test
  • Reviewed the CHM documentation

When we do this, we will be as transparent as possible, and let the community know by including what was done and what was not done as part of the bundle description or details. 

Please give feedback on this idea over at the WCSF CodePlex Discussion board.