Everything you want to know about Visual Studio ALM and Farming
Brian Harry is a Microsoft Technical Fellow working as the Product Unit Manager for Team Foundation Server. Learn more about Brian.
More videos »
We've heard loud and clear that the WPF designer performance in Beta 2 was unacceptable. Well, in a good kind of way, we knew that before we shipped Beta 2. This was one case where our performance tests were telling us the right thing. We discovered some issues late in the Beta 2 cycle (partially a regression, I think) and didn't have time to fully test a solution to address the performance issues. We did code it up and include it in the Beta but you have to use a registry key to enable it. Here's thread on it if you want to give it a try:
http://social.msdn.microsoft.com/Forums/en-US/vswpfdesigner/thread/4511d43f-c134-4329-a970-e374252a620e
This change will often address slowness and random pauses in the WPF designer. It's not the only cause but we've found most people who try it see good results. We've since had time to finish it and it is on permanently from here on out.
Today I saw some perfmon graphs that demonstrate how WPF designer overhead is improving over time.
This first graph is from Beta 2 without the reg key flipped. The very first spike is the CPU hit to open a C# file. the rest is the CPU hit to open an empty XAML file.
Here's the state with the reg key flipped. Don't be suprised that the reg key didn't make a huge difference in this scenario. It makes a much bigger difference in other scenarios.
And here's the current graph for the same thing with current bits. As you can see, it continued to improve.
Brian
Hi Brian,
The links to the graph images appear to be broken. It's great to hear MS are paying attention to the amount of pain the WPF designer is causing. VS2010 in beta 2 is a huge tease, I want to much to enjoy it (I love the v4 CLR and BCL), I appreciate what the IDE is *try* to do when it comes to moving forwards, the current implementation as embodied by beta 2 is sadly unusable in-anger.
Tom
Images in your post don't show up
Thanks for the warnings about the links. I've gotten into the habit of checking the post when I do it from how on Outlook Web Access but I forgot this time. My wife was bugging me to come to dinner :)
I've fixed them now - sorry for the inconvenience.
great to see the perf push - for those of us having Beta2 crash/hang for us, any chance you could share some data on Watson reports, fix rates, etc?
IOW, please try to disprove this :)
http://graphjam.com/2009/12/08/funny-graphs-error-microsoft/
I will be sharing lots of data in the coming weeks. I'll try to include some of this kind of data. But as an anecdote, just yesterday I asked that we go look at all performance bugs filed via Connect and ensure that they were being handled appropriately. Here is the analysis I got:
"I just reviewed all of the 87 Connect issues reported in the last 45 days with the word "performance" anywhere in the report. A vast majority of these are not related to VS performance. Of those that were, nearly all were marked as fixed. Of those that weren't marked as fixed (we're now talking small single digits), most were cases where the customer didn't reply to our request for more into. There was one 'no repro' case where the customer did actually reply afterwards and no action has yet been taken (the response was late last week). I sent mail to the PM on the bug making sure he'll follow up."
In addition we have broad goals around addressing issues filed through Connect but I explicitly wanted to make sure we checked on all performance related ones.
One of the issues that we have around Connect "bugs" is that many of them are actually "feature requests" rather than bugs and by the time we get them, the feature set is locked for the current release unless it's something that's really a gaping hole. In principle those feature suggestions are supposed to go into planning for the next release cycle but it doesn't happen as well as I'd like and that's the area I'm hoping to see us do better when planning the Dev 11 cycle.
There are only 3 bugs in Connect that were submitted in the last 45 days with the word "performance" that are about performance that are closed as "Fixed", according to the Connect search tool (504384, 504705, 505142). A quick scan through the "Active" bugs with the same criteria found 6 (514849, 513130, 509076, 508914, 504538, 504376), only 2 of which were waiting on the user (508914, 504538). So I don't think it is accurate to say that "nearly all" of the recently reported performance related bugs have been marked as fixed.
More generally, while the graph is highly cynical, it is hard to argue with the basic premise. I have been a regular Connect user for years, and it is very discouraging to see the percentage of bugs and suggestions that are completely ignored, closed without any response, closed with a canned (and often irrelevant) response, closed after a cursory response that does not address the issue, or in many other ways not addressed satisfactorily.
That said, it is good to see that at least most of the performance related bugs have some kind of activity.
David -
I did the analysis of the Connect bugs that Brian mentioned above. I should have been clearer in my description...my analysis did not include the currently active bugs. Instead, I only looked at those that had been resolved one way or another. I was looking to make sure that our teams were behaving appropriately as they triage the performance issues coming in from the beta release and not trying to make a more general statement about the state of our overall response.
Sorry for any confusion or consternation my miscommunication might have caused.
jeff
I can't decide which I'm more pleased about - that VS perf is finally getting some serious attention, or that people within MS are starting to get a clue about the insulting abomination which is 'Connect'.
At least perf improvements with VS will benefit us throughout the product's life - but I'm 100% certain that as soon as this push is over, Connect will revert to having bugs summarily closed - the non-English speaking minions in some distant low-budget corner of MS will be back with their bland pidgin dismissals.
Here's a suggestion - why not make the entire VS team file *every* bug through Connect? A product which is good enough for the paying customers to file bugs with you for free ought to be good enough for the guys who are being paid to do it, shouldn't it? ;-)
I don't want to get into an argument over this. If you want me to say that we don't handle Connect as well as I'd like, I'll say it. "We don't handle Connect as well as I'd like". Yes, we have a team that filters and routes Connect issues to feature teams. Yes their English isn't alway exemplary (although, mine isn't always either). Yes too many Connect bugs are resolved with no useful feedback (including no feedback at all). Yes, we don't incorporate enough feedback into our future product plans. Yes, the navigation of the site itself has been pretty ugly. Did I cover everything?
But that doesn't mean it's useless, insulting, etc. We use Connect quite a lot. Above I mentioned checking Connect for the outcome of performance issues. This isn't the first or only time. First of all, the majority of Connect bugs get routed to the right team and incorporated into our TFS database that we use to manage our project. We have release criteria around % of "fixable" Connect bugs fixed. We've worked on the user experience of Connect to make it better. We've tried to educate the team on how to interact respectfully with customers through Connect (it's a big team and everyone is under pressure to get things done fast). We data mine Connect in situations like this performance one to help ensure we are getting a holistic picture of things.
I've watched the dynamics around Connect for a few years now and my conclusion is that we need to do better around aligning our expectations with customer expectations. For instance, we've left the Connect database open for previous releases - long after we are done with them. The truth is, after we've shipped the first SP for a release, we dont actively monitor the Connect feedback for any longer. We've talked about closing out feedback for prior releases and redirectiing it to the "next" release.
There's also a constant tension about what constitutes a "bug". As I mentioned above, much of the Connect feedback is actually feature requests and we respond to a much smaller portion of that than we do on bugs (we could do better at this but it will always be true).
I also think people struggle with how to disagree with customer feedback politely. Let's admit that not every suggestion submitted through Connect is going to be something that we think makes sense in the product. I think some people, fearing conflict, are more likely to thank someone for their feedback and then forget it rather than disagree with them. What's hard to understand is that an active disagreement is actually more satisfying to both parties than avoiding the argument.
Lastly, no, I don't think we'll be publishing our entire bug database through Connect :) All kinds of crap makes it into our bug database and the issues with exposing it all externally are numerous - including overload of useless info, offending people with off hand comments, premature disclosure, potential IP issues and more.
Brian, I don't really want to start an argument either, especially as it seems that we agree, but it is an aparent precondition for change at MS that there's a public admission of problems, and I'm happy to do my bit to force the admission!
At PDC08, people like Hanselman and Hejlsberg were openly mocking the dreadful performance/footprint of VS (2008) in high-profile presentations - and this is the point which probably has to be reached before resources (and I understand that they're finite) can be directed at a particular problem - and I am just SO pleased that they finally are.
At PDC09, there were still Chan9 people making self-congratulatory noises about Connect. They're the wrong side of the looking-glass, because out here I think you'll struggle to find anyone who has anything good to say about it.
One voice saying something's OK is going to drown-out dozens of voices saything there's a problem - that probably chimes with your experience of sudden perf-related disillusionment post the B2 release which you blogged about recently. *You* may not be fooled about the success and popularity of Connect, but I bet there are plenty who are - I should probably be whining on their blogs instead..
Though of course they're the kind of people who don't have blogs.
Is WPF actually some kind of a sophisticated joke?
Or is the idea that it will work at an acceptable speed some time in the next five to ten years?
The 2010 version is madly enough far slower than the 2008 version; which was already like wading though glue.
Would the Microsoft development team please spend some time using something else; like delphi 2010 perhaps; to get a idea about what kind of performance is acceptable in a modern Windows IDE and also in a deployed application..
No, it's not a joke. It's a very powerful UI framework with which you can create some stunning applications. I'll have to admit, we didn't exactly make stunning use of it in VS but using it in the editor, at least, I think will prove out to have been a very good choice. Other places like the architectural tool, branch visualization, etc I think will also turn out to have been good choices over time.
There's no question that going into the release we over estimated how ready WPF was for an application the size, breadth and complexity of VS. We've worked VERY hard over the last 9 months or so to both improve WPF and our use of it. I think when you see the VS 2010 Release Candidate you'll feel like that work is paying off. It won't be perfect. There are somethings I wish would could do better on, but in the aggregate, it's a huge step forward over VS 2008 - even on performance.