February, 2010

  • Kirill Osenkov

    How to investigate and report Visual Studio issues

    • 7 Comments

    With the Visual Studio 2010 RC released, here are some tips to help you help us find and fix the remaining issues.

    We want Visual Studio to have no bugs. If you want the same and are willing to help us a little, read on! And thanks so much for trying it out!

    Help –> Customer Feedback Options

    First off, enabling the Customer Experience Improvement Program is a very helpful step!

    image

    image

    This will enable sending detailed error reports when Visual Studio crashes or hangs that GREATLY help us to identify the root cause of a problem.

    Send Error Report

    When Visual Studio crashes or hangs, you will be presented with the so-called "Watson dialog" – a choice to "Send Error Report" or "Do Not Send":

    image

    By clicking "Send Error Report" you greatly contribute to making Visual Studio better – this is the one most important thing that will help us the most.

    Watson Bucket ID

    After the crash has been submitted, two error entries are created in the Application Event Log in Control Panel –> Administrative Tools –> Event Viewer:

    Event 1000 and Event 1001 (may also be others, like Event 1023).

    The most interesting one is Event 1001, because it will contain the Watson Bucket ID:

    Bucket 1040873707, bucket table 1, faulting application devenv.exe, version 10.0.30128.1, stamp 488f2b50, faulting module cslangsvc.dll, version 10.0.30128.1, stamp 488f2dea, debug? 0, fault address 0x000ceedd.

    Knowing this bucket, Microsoft employees who are helping you with the problem can find the information you submitted in our tracking system. This is helpful during investigations.

    ActivityLog.xml

    If the error you're seeing reproduces consistently, you can usually get more information by starting Visual Studio with the /log switch like this: devenv.exe /log

    This MSDN page contains a nice description of the Visual Studio activity log: http://msdn.microsoft.com/en-us/library/ms241272.aspx

    You can find the log file in %APPDATA%\Microsoft\VisualStudio\10.0\ActivityLog.xml

    This log is especially useful if the error you're seeing happens at Visual Studio startup when it's loading packages.

    Search for the error text

    If you're getting an error message, try using your favorite search engine to search the internet for it – maybe someone has already encountered it before?

    Log a Connect bug

    To actually let us know that you've found a bug, the central location to do this is the Microsoft Connect website: https://connect.microsoft.com/visualstudio/feedback

    By opening a bug there it gets directly inside our bug tracking system.

    You Can Do More: minidump and call stack

    A while ago I wrote an article called How to debug crashes and hangs. This article explains in great detail what steps you need to do to create a minidump with heap of Visual Studio when it has crashed or is hanging.

    The simple way is, if you're running Windows Vista or later, just open Task Manager, click Processes, find the Visual Studio process (devenv.exe), right-click and select Create Dump File.

    That's it! It will create the large minidump file in your Temp directory and show a message box with the path where it saved the file.

    The original article can help if you're running Windows XP or in a more advanced case, when you also want to get the call stack, load symbols and so on. But usually a minidump file (.dmp) should be sufficient.

    How to send us the minidump

    The minidump file is usually huge (hundreds of megabytes), because it contains the exact snapshot of the full process memory of devenv.exe. To send us the minidump, open a bug on Connect and mention that you have a dump file. The MS employee who looks at the bug should give you a path to the temporary FTP website where you can upload the dump using FTP. If they don't give you this site or you have any other problems, just let me know.

    Debugging errors that are not crashes using First Chance Exceptions

    You might encounter an error which does not fully crash Visual Studio, and does not show the Watson dialog. In this case, there is still a good chance of finding what's going on and providing us with useful information about the error.

    For this you'll need to attach a debugger to the Visual Studio process (usually open another Visual Studio side by side). Again, a detailed tutorial is here. Just before the error is about to happen, break into debugger, and enable first chance exceptions (check everything in Debug –> Exceptions). Then resume debugging and let the error happen. Chances are that the exception will be thrown, but it will be subsequently suppressed. Enabling first-chance exceptions helps you to preview that exception before it gets swallowed. Once the exception happens, copy the Call Stack from the debugger Callstack toolwindow (Ctrl+A) and paste it into notepad. Sending us this call stack will be invaluable. Also, you can go to Debug –> Save Dump As and create a minidump with heap file (.dmp) from the current debuggee process state. Again, sending us this dump file and the call stack should really help a lot.

    One warning here is that there might be "red herrings" along the way – false first-chance exceptions which are normal and do not lead to an error. If you can't tell whether the first-chance exception you're seeing is normal or is the actual root cause of the error, send us the call stack for this and we'll be able to help further.

    devenv.exe /ResetSettings

    If you believe that you've gotten Visual Studio into a weird state with weird settings, one last resort is trying to start Visual Studio with the /ResetSettings switch:

    devenv.exe /ResetSettings

    This will reset the Visual Studio settings to the defaults and maybe help you solve the problem.

    devenv.exe /ResetSkipPkgs

    If you've had a Package Load Failure, running devenv.exe with the /ResetSkipPkgs flag will re-enable those failed packages and will try loading them again.

    devenv.exe /ResetUserData

    As a last resort, you can also try /ResetUserData.

    devenv.exe /SafeMode

    Or /SafeMode.

    devenv.exe Command Line Switches

    Here are other Visual Studio command line switches: http://msdn.microsoft.com/en-us/library/xee0c8y7.aspx

    New: Visual Studio Setup Log Collection Utility

    Check out the instructions here on how to use collect.exe to collect the logs: http://blogs.msdn.com/heaths/archive/2008/05/09/visual-studio-and-net-log-collection-utility.aspx

    Thanks!

    Thanks again for making Visual Studio a better product!

  • Kirill Osenkov

    Stress testing Visual Studio 2010

    • 18 Comments

    In the past several months Visual Studio and I have been really busy stress testing each other. This post is a general overview on what we've been up to and what kind of testing we're doing. I've learned a lot about stress testing and I have to say it's actually a lot of fun, so I guess it's worth sharing. I'll try to make this a series of several posts, diving into more technical details in the upcoming posts.

    Background

    During Beta 1 and Beta 2 it became painfully obvious that the new VS had an obesity problem: it was slow, consumed a lot of memory and the worst thing, with enough modules loaded it stopped fitting into the 2GB address space on 32-bit machines. There were several reasons for this, which Rico Mariani, Brian Harry and others have extensively blogged about. In a nutshell, with a lot of new functionality a lot more modules were loaded into memory. Besides, we now had to fully load the CLR and WPF at application startup. Moreover, there were all kinds of memory leaks all over the place.

    Making performance a top priority

    Of course this wasn't good, so our management made the right decision to make performance our top priority. Jason really took it seriously and we dedicated a lot of people to work fulltime to make Visual Studio fast and lean. As part of this effort I became a member of a virtual team called "Perf SWAT". This team is responsible for essentially three things: performance, memory consumption and design-time stress.

    Performance is clear: we need to be fast. Memory consumption is clear too: when we load, we need to take as little memory as possible, and avoid things such as double-loaded modules, loading both NGEN and IL versions of an assembly, and so on.

    Design-time stress on the VSL team

    As for design-time stress, the goal is once we're loaded into memory, jitted, warmed up and all the caches are filled, we should not continue to grow in consumption. This means find and eliminate all memory and resource leaks. Run-time stress means finding leaks in the CLR and BCL, design-time stress means finding leaks in the VS and tooling. I am responsible for design-time stress testing for the VSL team (managed languages). I need to make sure that there are no significant leaks in 4 areas:

    1. C# IDE and editor integration (C# code editor, navigation, refactorings and other core C# areas)
    2. VB IDE and editor integration
    3. F# IDE
    4. Hostable Editor (Workflow Designer in VS 2010 is essentially hosting a full-blown language service to show IntelliSense in the expression editor on the workflow diagram)

    Progress

    The good news is that we've made tremendous progress since Beta 2 and have brought the product into a much better state: it is much faster, more responsive, takes up much less memory and we also hope to have eliminated all major known memory leaks. A common complaint was that VS was growing in memory during usage and you had to restart it after a certain time. Right now we hope that you can mostly keep Visual Studio open for days (even weeks) without having to restart it.

    8 hour stress tests

    The official sign-off criteria is that the end user needs to be able to keep VS open for an entire work week without any noticeable performance degradation (this means 5 days times 8 hours a day). We've calculated that in average continuous human usage of 40 hours is equivalent to running our tests for 8 hours (tests are doing things faster than a human).

    We have identified and implemented a series of 22 tests for all the 4 language teams mentioned above. Each test covers one continuous kind of activity, e.g. CSharpStressEditing, CSharpStressNavigation, CSharpStressIntelliSense, CSharpStressDebugging, CSharpStressUI, VBStressEditing, VBStressProjectSystem, FSharpStressEditing, and so on.

    Each test runs for 8 hours on a machine in the lab and VS memory usage details are automatically logged. We've also developed tools to automatically analyze the stress logs and produce Excel spreadsheets and charts for analysis and reporting.

    Several months ago a typical test would start at about 300 MB ProcessWorkingSet and crash after several hours with OOM (Out-Of-Memory exception). None of the tests would even be able to run for 8 hours. After finding and fixing a lot (a lot!) of bugs, we were able to get it running for 8 hours – VS memory usage grew from about 300-400 MB of WorkingSet to over 1 GB over the period of 8 hours (that was anywhere from 200-500 stress iterations).

    Right now a typical test starts at about 150-200 MB and finishes 8 hours later at 200-300 MB. Also, instead of 500 iterations, it is able to do 3000-5000 iterations during 8 hours on the same hardware. Which means we made it considerably faster and also reduced the leaks in major feature areas to a minimum (right now a feature is considered not leaking if there is average increase of less then ~5KB per iteration).

    I'll try to continue blogging about our stress testing and dive more into the technical details: what we measure, how we measure, how we find bugs and how we'll know when we're eventually done.

Page 1 of 1 (2 items)