Using PerfWatson to report performance issues to Microsoft

Using PerfWatson to report performance issues to Microsoft

  • Comments 36

By Selma Ikiz

Would you like Visual Studio 2010 to be even faster? Would you like any performance issue you see to be  reported automatically without any hassle? Well now you can, with the new Visual Studio PerfWatson extension! Install this extension and help us deliver a faster Visual Studio experience.

We’re constantly working to improve the performance of Visual Studio and take feedback about it very seriously. Our investigations into these issues have found that there are a variety of scenarios where a long running task can cause the UI thread to hang or become unresponsive. Visual Studio PerfWatson is a low overhead telemetry system that helps us capture these instances of UI unresponsiveness and report them back to Microsoft automatically and anonymously. We then use this data to drive performance improvements that make Visual Studio faster.

Here’s how it works: when the tool detects that the Visual Studio UI has become unresponsive, it records information about the length of the delay and the root cause, and submits a report to Microsoft. The Visual Studio team can then aggregate the data from these reports to prioritize the issues that are causing the largest or most frequent delays across our user base. By installing the PerfWatson extension, you are helping Microsoft identify and fix the performance issues that you most frequently encounter on your PC.

To enable PerfWatson perform correctly, please make sure that Windows Error Reporting (WER) is enabled on your machine. PerfWatson employs WER service to send the collected data to Microsoft. For details on WER and how to enable it, please refer to PerfWatson blog.

Download and Installation

Following are the pre-requisites for installing Visual Studio PerfWatson:

  • Windows Vista/2008/2008 R2/7 (Note: PerfWatson is not supported for Windows XP)
  • Visual Studio 2010 SP1 (Professional, Premium, or Ultimate)

The extension can be downloaded from the Visual Studio Extension Manager or at the VS Gallery.

Limitations
  • The tool doesn’t work on Windows XP
  • The tool needs WER be enabled. For details on how to enable WER, please see this blog post

Please let us know what you think! Thanks!

Leave a Comment
  • Please add 1 and 8 and type the answer here:
  • Post
  • No entry for PerfWatson (or "Visual Studio PerfWatson") in Tools\Extension Manager.  Hmmm.  Can I disable PerfWatson by deleting a file?  Or setting an environment variable?

  • I've been using it for a while and it's been reporting like crazy.  I'd love to see some results being posted from this now that it's been out for over a month.

  • @JohnC: I agree. On average >400 files are submitted a day from just one instance of VS 2010 on my dev box, but what actually happens to these?!

  • @JohnC:  They are collected into a database and paired with other reports that involve the same callstack.  Reports are generated on the top slow downs in VS and performance bugs are filed and teams fix these issues.  Obviously, like normal Watson reports, the likelihood of something being fixed depends on the overall bug load, the number of users it is affecting and whether the fix is low-cost vs. high cost and low risk vs. high risk.  

    It isn't a simple process where all perf bugs are fixed (though I wish it was), but I think we do a good job internally directing resources towards the biggest perf hotspots, and the more customer data we have the better we can weed out those problems and add internal performance tests to cover missed scenarios / bottlenecks to ensure they don't regress after the fix is made.

    Ryan

  • In other words, JohnC, no results for you (or me).

  • @Miria,

    What exactly do you mean by 'results'?  Are you looking for access to the database that holds all of this info?  I don't think there is any way to share that out, and having the team invest time in exposing the information, dealing with symbol scrubbing (most public PDBs Microsoft issues are 'scrubbed' but we deal, interally, will private symbols to get full callstack resolution), getting the OK from third parties who have shared their PDBs with us so that, internally, we can get fully resolved stacks, etc... would be a non-trivial amount of work.  I don't know if all that would be justified to put up a website that 2 people would query once and then never look at again.

    There was a recent Channel 9 piece done on this very topic by someone on the team, you can take a look at that: channel9.msdn.com/.../Visual-Studio-Toolbox-Reporting-Performance-Issues-with-PerfWatson

    If you were interested in more specific details short of a queryable website feel free to post ideas on that front here or on Connect (it 'kind of' a bug, more of a request I suppose).

    Ryan

  • Do you really need a tool to find out where VS2010 is slow? I can tell you that for free...The reason is WPF, if you remove it (i.e. re-write the VS properly/native) it will speed up.

  • @ML

    Large amounts of collected data conflict with your prescription. There are areas in WPF and our usage of WPF that can be improved but we arre seeing many more bottlenecks/responsiveness issues outside of WPF than inside of it.

    Ryan

  • "What exactly do you mean by 'results'?  Are you looking for access to the database that holds all of this info? I don't think there is any way to share that out, and having the team invest time in exposing the information, dealing with symbol scrubbing (most public PDBs Microsoft issues are 'scrubbed' but we deal, interally, will private symbols to get full callstack resolution), getting the OK from third parties who have shared their PDBs with us so that, internally, we can get fully resolved stacks, etc... would be a non-trivial amount of work. I don't know if all that would be justified to put up a website that 2 people would query once and then never look at again."

    Ryan, stop this, please. Nobody is asking you to provide access to your internal databases or create complex web sites with lots of functions. Stop pretending.

    I am asking for patches and service packs, or at least a list of problems that you identified and fixed. A blog post. Anything.

  • @Miriam

    I wasn't pretending, it honestly wasn't clear (to me) what you were asking for.  Your post consisted entirely of 'In other words, JohnC, no results for you (or me).' which was in response to JohnC asking 'what happens to all this data'. I wasn't able to infer from that you meant 'patches and service packs', I interpreted it as meaning 'I would like to see this data', sorry for the confusion.

    Performance data collected from the extension is being rolled into active bugs which are assigned to area owners. Whether those result in patches or service packs is 'above my paygrade', sorry, I don't make those decisions and can't speak to them.

    A list of fixed problems is an interesting idea, perhaps one could be put together, though of course the caveat always exists that a fix may not apply to all scenarios as our user base is diverse and complex.  For instance if PerfWatson indicates solution load slow downs due to network I/O becuase people have network mapped drives holding projects in their solution then we can do things to mitigate that slowdown, it won't however increase solution load speed across the board so to speak, so it may or may not positively effect your specific situation, which is why we need as many people to install the extension as possible so we can get a broader cross section of data, more data == more fixes.

    As far as 'anything', there is the link I already posted:

    channel9.msdn.com/.../Visual-Studio-Toolbox-Reporting-Performance-Issues-with-PerfWatson

    There are no blog posts on the topic to my knowledge.

    Ryan

  • Ryan.

    Me, JohnC and others have been running your add-in for more than two months.

    We are asking a simple question.

    Did you identify any of the performance problems you were looking for? Did you fix any of these problems? When do you plan to deliver the fixes?

    This is not rocket science.

  • @Miriam

    I am attempting to answer your questions, sorry, I am a little slow on the uptake apparently.

    >Did you identify any of the performance problems you were looking for?

    >This is not rocket science.

    This indeed is not rocket science but it also is not trivially simple to answer as 'did you identify any of the performance problems you were looking for' is not the right question. We weren't looking for any problems in particular, in fact we didn't KNOW where the performance issues were as all of our in house perf measurements and day to day use don't hit these poorly performing scenarios, or we would have already fixed them.

    As I previously stated the sheer diversity of use across the VS user base is staggering. What you consider 'main stream' in terms of number of projects, number of files, number of classes, languages used, third party extensions, third party libraries, target platforms / environments, build process, etc.. is likely far different than what huge numbers of our other users consider main stream. We strive to have good performance across ALL use cases, but if we are deficient in some of them we need to data to understand why and how to fix it, hence the extension.

    >Did you fix any of these problems?

    There have been a number of bugs filed from the extension and fixes made. I don't have exact numbers but perhaps a PM or someone with a 'higher level' view of the work would be able to post such data. I have personally made and/or code reviewed a number of perf related fixes, I would estimate at least 4-5, but I am a single developer, there are LOTS of people working on VS, across many different teams, so I don't know the 'all up' number.

    >When do you plan to deliver the fixes?

    Any bugs fixed are fixed in the vNext product which we are working on at the moment, as I said previously if these fixes also go out as hot patches for 2010 or a follow on SP is far above my paygrade. I don't make those decisions nor can I speak to them as I have no info on that matter.

    Ryan

  • Thank you.

    You keep making things more complex than they are. This makes me think I should have been more specific in my requests. I apologize if I was not being specific enough.

    "Did you identify any of the performance problems you were looking for?" was meant in the sense "did you identify any of the performance problems of the kind you were looking to identify when you were creating this add-in?". No hidden bombs. The answer to the question is apparently "yes".

    The answer to "did you fix any of these problems?" is apparently "yes" as well. Good.

    I understand that you don't have info on the patches. Whom should we ask for that info?

    Finally, could you or anyone else provide any info on what performance problems you actually fixed? The video in your link contains some talk about maybe someday posting a list of top X issues you identified with info on what you are going to do for each such issue. This is exactly what we are asking for. Are you going to post such a list? If so, when? If you don't know, whom should we ask?

    Thanks again.

  • So, any hope of answer?

  •  As for a list of problems fixed, I personally don't have one.  I could point a PM at this conversation if they aren't already aware of it but I don't know how much time they may have at the moment to make such a list (if they don't already have it), nor can I speak for anyone else's intention of posting this kind of information or the timing around such an occurance if it does happen.  I am not trying to be difficult, btw, this just isn't in my spehere of day-to-day work so I really don't have any answers, dates, lists, etc...

     As for general announcements around things like patches you wouldn't find them in the comment section, at least not anything authoratative.  It would either be in a blog post here, or more likely, somewhere like Soma or JasonZ's blog.

    Ryan

Page 2 of 3 (36 items) 123