Milestone Quality & Dogfooding (Matt Gertz)

Milestone Quality & Dogfooding (Matt Gertz)

  • Comments 10

After some hard but fruitful work, we’ve just finished our “milestone quality” (MQ) efforts, the goal of which was to get ourselves ready for developing the next versions of Visual Studio, Team System, and .NET.  Although not specific to Visual Basic, I’ve opted to use this “bully pulpit” to let you know about these efforts, which certainly do impact Visual Basic along with other Visual Studio and .NET products. 

As I mentioned in a previous blog post, the MQ work doesn’t involve new features or any such thing – what it does is get our tools and processes ready to go for the next cycle.  It’s important to get this work done before we start coding, because it’s very disruptive to change engineering processes mid-stream.  This MQ was very challenging for us, given some of the big engineering changes that we made, and I’m pleased to day that it resolved itself very well. 

Like I said in that earlier post, we decided to spend a lot of MQ focusing on “dogfooding” – using our own code to get our development work done – and although we did other set-up work in MQ as well, I’m going to focus on our dogfooding efforts in this post.   We dogfood our code for three reasons:

1.       Improved stability for *you*:  we find the bugs in day-to-day usage before the product ships to you

2.       Practicing what we preach:  If we say it’s good enough for you, then we should be using it too

3.       It’s the right way to do things:  We firmly believe that we offer the best foundation for product development, all the way from hobbyist code to enterprise solutions work.  The best tool for developing Visual Studio is Visual Studio.

Now, we’re no strangers to dogfooding, but we’d never done it so early in the past.  Instead, we generally made the switch during the first or second beta, after the coding was mostly done.  In earlier versions (such as Visual Studio .NET 2002, where we rewrote everything) that made sense because the code really didn’t come together until that time.  The downside of that was that late-start dogfooding just doesn’t give us the same coverage, because in the betas we’re winding down and focusing solely on fixing bugs instead of developing new features, and developing quality features is the whole point of VS, TS, and .NET.  However, now that we’re building on a platform of stable code, we can leverage that to do early dogfooding, finding the development bugs early (or avoiding them altogether) and making adjustments to features before they become too “set in stone.”  I’m really excited about being able to do this, as I’ve always envied the Windows and Office teams their ability to adopt dogfooding early.

Our dogfooding prep work this time was in two areas: Team Foundation Server and Side-by-Side support.

Team Foundation Server (TFS)

In earlier product cycles, we’d used a proprietary system for tracking bugs, another one for storing code, and some combination of bug reports and spreadsheets to track feature progress.  None of these things “talked” to each other at all, and reporting status on any of them was far more complicated than it needed to be.  A codebase the size of ours requires careful lifetime management handling, and it therefore has long been our goal to switch to our own product, Team Foundation Server (TFS), which handles all three (and a lot more besides).

Despite being a vocal proponent of this change, I will be very honest and say that I went into this switch to TFS with more than a bit of nervousness.  Visual Studio, Team System, and .NET together comprise a really, really big division.  If you just took that division (the Developer Division) alone without considering the rest of Microsoft, it would still be one of the biggest enterprise-size software vendors in the world, easily in the top five.  We’ve been doing things a specific way for a long time, and that’s hard to abandon, even for more efficiency, given the number of processes that would need to change along with it.

Fortunately, these fears have turned out to be groundless.  Certain of our product units (about 25%) already had pioneered TFS usage during the VS2008 timeframe, and they’d used that experience to make improvements to it.  Because of their early diligence to dogfooding, the rest of the division switched over very easily a couple of months ago – we had all of our source code moved into the new project (and building!) within a couple of days, literally – and that’s a lot of code.  While doing so, we identified and implemented several scaling improvements to better enable our hundreds of developers to interact with that code storage – improvements that will be ultimately benefit our customers as well.

Furthermore, we are also now tracking our bugs and features using TFS, leveraging the Team Explorer piece that ships as part of the Visual Studio family of products.  This is another area where some pioneer dogfooding occurred in the previous version of VS development, so the ultimate transition for the division was also similarly smooth.   The upshot of this is that we now can strong linkage between code, feature planning, and bug tracking, all with reporting services that are just far and away the best that I’ve ever seen.  The savings for us (and to me, personally) in managing the lifetime of the upcoming product cycle is going to be phenomenal – I’m super-excited to be using TFS!

Side-by-side (SxS)

Of course, if we’re going to develop the next product using the product itself, we need to be able to install it as our customers do and not have it interfere with any previous versions of the product which may also be on our machines.  This goal is known as “side-by-side,” or “SxS” for short.  It involves a lot of re-versioning, changing registry entries, dealing with the few necessarily hard-coded strong assembly references, and so on, followed by lots and lots of testing.  There are literally thousands of components that need changing between product cycles, all coming in from various teams.  In the past, it’s something we generally didn’t complete until we reached a beta milestone, as people were hard at work writing features before beta and all of their time was very focused on that.  However, deferring SxS work is not something we ever liked doing – it meant that when our customers installed our betas, there were sometimes problems lingering which required a “repair” to earlier products after the beta was uninstalled.  Furthermore, as we added features in the early parts of the product cycle, we’d add that much more “SxS debt,” often resetting any SxS work that might already be going on. 

So, to avoid those pitfalls, we decided to tackle this issue head-on in MQ.  To be frank, it was quite a challenge to do it so quickly – there were definitely some long nights and weekends for all of us, and we ran into a lot of processes that needed retooling to accelerate our work.  But by setting aside time to do this SxS work earlier, before we start coding features for the next version, not only will we be able to dogfood our own product without stomping on VS2008 installations we might already have, but your beta installation/uninstallation experience should be of much higher quality, leading to a better beta experience for you and better feedback for us. 


That’s it for this post; a brief discussion on a lot of hard but good work which should ultimately lead to big wins for all of us.  BTW, I’m already working on another blog post focusing on coding (which hopefully will cover something not already in covered a book this time, unlike last time J), and hope to post that up in a few days.

‘Til next time…


Leave a Comment
  • Please add 2 and 1 and type the answer here:
  • Post
  • Matt Gertz, Developer Division's manager responsible for our internal development tactics and process,

  • Критики Microsoft порой ссылаются на такую "народную мудрость", как "Не используйте те

  • How can I capture mouse click events in webbrowser control, using VB .Net 2005. Please suggest a way for the same. I could not find anything on this on the net, so I approched here on your blog. Thank you in advance !

  • To see MS focus on total developer productivity is good.  It helps greatly that quality bug prevention and code quality tools (i.e, static analysis and code metrics) are in the new VS tool-set.  These tools greatly reduce our development time and, more importantly, our support costs given that our systems need to have a lifespan of 5-7 years at low support costs.

    These tools have been greatly beneficial based on my previous experience in other environments (C/C++) with embedded systems I've been involved in.  They have save me many days of bug finding.

  • Vinod:  I've actually not done much web app development since the old ActiveX days (as can be seen from the examples I typically post); I'll see what I can dig up.

    Greg:  Thanks for the feedback.  I agree with what you say, and am really happy that we're moving in that direction internally.  I should also mention that my division is not the only one adopting TFS for life-time management; it's starting to take hold in a number of areas in Microsoft, having been gaining a lot of momentum.

  • Vinod:  Sorry, I didn't read your question carefully enough -- I thought you were asking about web apps (for some reason -- I'm brain-dead today), and I was trying to figure out if this was a trick question!  Very silly of me.  Anyway, you're right, there is no click event for the WebBrowser Control -- this is because the control has to handle it itself to properly deal with links and such.  (It's essentially a full-blown browser, after all.)

    So, what do you do if you really need to start tracking clicks and such?  Well, you have to leverage the Document() property of the Web browser control.  See for a pretty throrough description of this.  Note that, for step (3), where it tells you to add a reference to Microsoft.mshtml in the project, the actual reference you'll be adding from the COM tab of the Add Reference dialog will be the Microsoft HTML Object Library.

    Hope this helps,


  • I think I encountered a compiler error situation in both VS 2005 and VS 2008.

    How can I report this to you people?

    My email address is

  • Hi, Shiguo,

     Please post your bug at -- that will allow you to track the VB team's progress on it.



  • ハードで実り多い仕事の結果、次期バージョンの Visual Studio 、 Team System 、および .NET の開発に備える目標を掲げたマイルストーン品質 (MQ) を達成しました。 Visual

  • [原文作者]:Matt Gertz [原文链接]: Milestone Quality & Dogfooding 在辛苦又有成果的工作后,我们完成了质量控制的里程碑。这个里程碑的目的是让我们准备好开发下一版本的

Page 1 of 1 (10 items)