If we find Whidbey quality is not ready for a Nov 7 ship we should and will delay shipping. We feel, however, that the product has reached that key inflection point where shipping the huge progress we’ve achieved brings far greater benefit to customers than further delaying to fix a few remaining issues that won’t impact most users.
We opened the Product Feedback Center last year with the release of Beta 1 to empower customers and as a result have received a lot of high quality feedback. We’ve fixed a very high percentage of reproducible customer reported code defects. As VS Client Dev Manager Shawn Burke writes “if you're filing bugs on MSDN Feedback that you really, truely, honestly believe are ship-stopping bugs, reactivate the bug and add your justification as to why you believe that. Write the scenario that's broken and why it's so important, or ask for more information about why it doesn't meet the bar. State that well, and the right things will happen.” See Shawn Burke’s full commentary on this on his blog - http://www.shawnburke.net/Default.aspx?document=264&userinterface=9
Over the course of the Whidbey product cycle we have modified shipping schedules and put new processes in place, to make sure our customer needs are addressed. There is always more to be done and no release is 100% bug free, especially with such a complex product as Visual Studio.
Visual Studio is over 43 million lines of code, there are over 30 teams working on different pieces, with roughly 700 developers checking-in code to 11 different virtual build labs that are then integrated on a rotating schedule producing over 100 different builds of the product daily. In addition we have interdependencies with SQL and MSDN. When we ship an official release like a Beta or RTM (release to market) we lock down and are code complete several months before the actual release date to allow for a final test pass, to stabilize and hit stress goals, then get the best bits to fulfillment for mass production of media. Before code complete there is a long list of exit criteria for the product that must be met, ensuring all key features and scenarios meet expectations. We have customer bug exit criteria to ensure high priority issues are fixed.
Why are we postponing so many bugs? As Shawn says “It's a massive balancing act: how do I ship quality software that will do the right thing for my users and still close it down and get it out the door with known issues? Is this paradoxical? For large software projects, at some point this becomes a treadmill. You could literally keep at it forever if you kept fixing all the bugs. Multiply this by the complexity factor…If you keep perturbing the system, you'll never get there, especially when it comes to sensitive things like stress results.”
Bug triage, where we determine priority and resolution of bugs, can be an unbelievably difficult process in which teams must reach consensus on issues where there are often conflicting considerations. Security, usability, accessibility, integration, would it be a breaking change, is it blocking or a build break… etc. Over 30 people from across the Division meet daily to discuss bugs and fixes ensure the right calls are made in team triage. The Product Feedback Center gives customers a new visibility into this process. You can see how your bug has been assessed against the triage bar and why Microsoft has made the decision to resolve it as Postponed, Fixed, or Won’t Fix. Decisions have been reversed based on customer comments and community vote. Bottom line is we are committed to shipping the right product and through the Product Feedback Center we have an even better sense of what that is.
Thanks,
Marie Hagman
Visual Studio
Program Manager
