Larry Osterman's WebLog

Confessions of an Old Fogey
Blog - Title

What's the big deal with Vista Betas?

What's the big deal with Vista Betas?

  • Comments 16
In this somewhat silly Channel9 post about unsubstantiated rumors of Microsoft canceling Vista Beta 2, "Sabot" wrote the following question:

What's all the fuss about a Beta? I don't actually get it? It comes when it's ready and I'm happy with that.

I'm one of those guys that's not really all that fussed about knowing the release date of any product, I always architect systems using existing technology, it's a school-boy error to do otherwise because the plain truth is, there is a 50/50 risk it won't be released by the time you need it in the project plan, and that in terms of risk is to high a percentage to be acceptable

As a consumer of Microsoft's operating systems, I'm 100% behind him.  I don't get the hype (although I think seeing the purported screen shots is cool).

As a developer who is working on Microsoft's operating systems, I have a rather different view.

You see, the betas of Microsoft's operating systems are really the only opportunity that Microsoft's developers have to see the "real world" outside the ivory towers of Microsoft.

Within Microsoft, you see, we all tend to have more-or-less the same types of machines.  They come from maybe a dozen different manufacturers, and they all have reasonably different capabilities (for example, many don't have any audio).  But they tend to be pretty homogenous - 1.5 to 3 GHz machines with between 256M and 2G of RAM, an "ok" video card, and an on-the-motherboard audio and network solutions.  The thing is, they don't even come close to representing the diversity of systems that our customers have.  We typically don't have the $200 low-end computers in our office, neither do we have $7,000 watercooled ultra high-end gaming rigs.

We also don't have all the myriad applications that our customers have.  We've got an amazingly huge library of apps to test (I know, I've been looking at appcompat failures), but we still get reports of problems in apps that we've never heard of.  We also don't have the applications that haven't yet shipped, so we have no ability to test them.

As a result, the OS developers within Microsoft LOVE betas.  We love it when real customers get a chance to use our stuff, because their environments are so much more diverse than any we could possibly have.  In reality, we do a pretty darned good job of weeding out the bugs before we ship (for instance, we've only found a handful of new customer reported audio bugs since Beta1), but there's no substitute for honest-to-goodness in-the-field tests.

It's kinda interesting - Platform software is very different from just about every other engineering endeavor, because the compatibility matrix is SO complicated.  You can build an automobile totally in isolation - you can solve every engineering problem you're likely to ever find before it leaves the plant, because you can pretty accurately gauge how your customers are going to use your product.  But you can't do that with a software platform, because the number of potential use scenarios is essentially limitless.  So for software platforms, betas are indispensable tools that provide critical feedback for the developers of the platform.