Since we started doing CTP with the prerelease of SQL Server 2005 I've been a huge fan. It allows the product team to get a build of the product in the hands of users quicker and more often. It means the feedback loop from user to product group is shorter. This is a great situation for everyone. But there are taxes to pay on both sides. On the product group side it means more overhead with turning the release crank. Builds, test runs, the release dance, etc, the exhaustion of always feeling like you're in ship mode. But the benefits absolutely outweigh the cons – for the product group.
What about customers? CTPs don't always have the cleanest install and uninstall experience. Upgrades between CTPs aren't always supported. Customers are instructed to not install CTPs on their main machine. Everyone has multiple machines, right? It's these user taxes that are really bugging me lately. Isn't there a better way to deliver CTPs? Is there something that will lower the impact on users? Is delivering CTPs as virtual images the answer? What about dealing with the size of the virtual image (>2GB)? You're not going to download that puppy!
Tell me, am I just making up a problem? I don't think so, but tell me if I'm being a nut. I lived through all the customer install/uninstall issues with each and every Yukon CTP. It wasn't fun. And we're still getting the random post asking for magical uninstall script. So what's the answer? Do the benefits to users beat out the taxes?
Well, the two biggest hurdles to a installing a CTP is trying to get the previous one completely uninstalled and waiting through the install of the new one. I think if the install/uninstall process were streamlined (faster and cleaner), it would go a long way toward making CTPs easier.
As for the virtual machine idea, I tried that with the current Orcas CTP's, but found that the VirtualPC instance to run much more slowly than the normally installed one, so I would prefer a normal setup whenever possible.
BTW, I can't wait for the first Katmai CTP to come out - I'd like to see all of the new stuff that's been under development over the past year!
If you decide to use virtual machine images, you could share a common base image with other product groups, for example the VS Orcas team.
CTP's can be good or evil- it depends.
-reduce the amount of ctp's (eg AJAX as bad example)
-make public and non-public CTP's. The private community must sign NDA's and have NOT to publish anything. One of the biggest issues today is, what ctp is the author in this article using I found on google.
-focus on quality, of code, of setup, of documentation. A product never looses a negative start image