Everything you want to know about Visual Studio ALM and Farming
Brian Harry is a Microsoft Technical Fellow working as the Product Unit Manager for Team Foundation Server. Learn more about Brian.
More videos »
Up until Christmas, I was blogging almost every day about the progress we’ve made on addressing the Beta 2 feedback that VS 2010 had significant performance issues. Since then, our efforts have hardly taken a break. I saw active investigations all the way up to Christmas eve and then a bit of a lull until Jan 4th when activity resumed in full force.
The improvements are so many and so dramatic that I really couldn’t enumerate them all here. I think the best way to summarize it is to talk about the feedback we are getting from people as they use new builds. Since our renewed performance effort started, we have produced 3 Super Limited Community Technology Preview releases (SLCTPs – yeah, a mouth full :)). These have been delivered to virtually anyone who has expressed performance dissatisfaction, and for whom we could find contact information. With each one we contact as many recipients as we can and solicit feedback on the changes.
About a week ago, we delivered SLCTP3. A few hundred people (outside Microsoft) have received it and so far we have close to a hundred survey responses from them. We also made a huge push to get everyone in Developer Division upgraded to a January build. I think, as of late this week, we managed to get over 80% of the division upgraded. Lastly, we’ve reached out to partner teams like Blend, Exchange, CodePlex and Oslo to upgrade and give us feedback on the new builds. The difference between now and Beta 2 is staggering.
Before I go into the details, let me remind you of the results we got from our initial Beta 2 surveys.
Our initial external customer survey said that approximately 30% of Beta 2 testers where somewhat or very dissatisfied with Beta 2 performance (the earler version of the “Performance” chart below). After seeing that and running an internal survey of dogfooders, we found that about 70% of internal users were somewhat or very dissatisfied with Beta 2 performance.
Now, at SLCTP3, it looks very different. Our external LCTP3 results look like this. And remember, all of these results are from people who complained about Beta 2 performance (not the 70% of people who didn’t). We didn’t give any of the LCTPs to or survey people who were already happy.
And our internal dogfooder survey looks like this. The “improved” questions are relative to Beta 2. The Ready To Ship and Feature Satisfaction questions are absolute. The performance feature satisfaction question was 70% somewhat or very dissatisfied after Beta 2.
As you can see, the picture looks totally different. The very dissatisfied numbers are nearly gone and the total dissatisfied are way better. I have no illusion that we can make everyone happy – we could work on it for 2 more years and still have a few people who would like even more but I feel like we have really turned the corner. We aren’t done but we are now ready to go out with a broad Release Candidate and solicit feedback from thousands rather than hundreds.
Beyond the statistics, some of the verbatim customer comments tell an even more compelling story…
And not to ignore any negative feedback:
We are following up on all of these with survey respondents.
We are not done with performance or functional improvements but I believe we have addressed all of the major issues and are now ready to make sure we’ve hit all of the combinatorics, edge cases, etc. That’s what a Release Candidate will help us do. In parallel with getting an RC ready, there are a few performance areas where we are still working diligently. Probably the biggest one is solution build performance. Many build scenarios are faster than VS 2008 but we’re also still tracking a number of applications where it is slower (generally between 15 and 20%). We are also putting some finishing touches on some remaining editor/WPF painting and responsiveness issues. I think even without them you’ll find the RC performance to be quite good but we’ll continue to polish it while we wait for RC feedback.
We’re working on getting the release candidate ready now and I’ll let you know as soon as it is available.
It’s been a hectic couple of months working through all of the feedback and resulting issues. It’s not quite what I was expecting to be doing but that’s the way it goes. I’ve got more fodder for posts on things we’ve learned along the way and I’ll try to get a few of them written over the next month or so.
Thanks for helping us work through this and deliver a product that you are going to love,