Visual Studio Profiler Team Blog

Write Faster Code with VS 2010 Profiler

Well, Visual Studio 2010 Beta 1 is finally here!!! WooHoo. As you may have seen in Brian Harry’s blog , we have added quite a few new features to Profiler to help you write faster code. The following is a list of the new features. In this list, I have also included a note on which features work for managed versus native (where applicable, of course). By the way, this is just an overview and we’ll be drilling into the details on these features in the weeks ahead:

Virtualization support for sampling (VPC, Hyper-V, VMware)

You can now use VS 2010 Profiler to sample profile an application (both native and managed) on a virtual machine, irrespective of whether the virtual machine was created using VPC, Hyper-V or VMware. The timers used for profiling are different on virtual machines than raw hardware, and I will elaborate more on this on a future blog post. By the way, all the screenshots you see below have been gathered on a Hyper-V virtual machine while sample profiling the Dinner Now and Beer House applications.

New Summary and Function Details reports

The Summary page has gotten a “total makeover” and is a lot more actionable than the summary page in VS 2008. The data on the summary page is going to look different depending on the method of profiling you choose, but regardless of the method of profiling, you can visually identify the performance spikes in your application and navigate to the top 5 methods by just a click.

VS Profiler 2010 includes a new page called the Function Details page. The following is a screenshot of the Function Details page, which will depict the caller/callee information in a graphical way along with the performance metric. There is also an embedded, editor which includes the line-level performance measurements.

Both Summary and Function Details pages are available for managed and native applications.

Tier Interaction Profiling (TIP)

Tier Interaction Profiling (TIP), enables you to see the number of times a query (or stored procedure) was executed in your application, and to see the amount of time you spend inside of ADO.Net for that query. This feature is turned on via the session and target property pages, and once it is run, you will see an additional profiler report called Interactions (see below). As this feature only works with ADO.Net at the moment, it is only supported for managed applications. And, by the way, if your application is a web application, TIP also shows you the number of times a page was called and the processing time for that page:

Client Side JavaScript Profiling for IE 8

When incrementing a web application, you can now turn on JavaScript profiling via the session or target property pages to see the performance of your client-side JavaScript (including Ajax calls back to the web server). JavaScript functions will be included in your report alongside the methods from your web application.

Concurrency Profiling

Concurrency profiling is a new method of profiling added to VS Profiler 2010 enabling you to profile your multi-threaded applications to see how your threads are behaving inside the application. There are two sub-modes for Concurrency profiling, which I will elaborate more on during a future blog post. Screenshot below shows the place in performance wizard where you can select concurrency profiling. Suffice to say for now that there is lightweight “Resource Contention” profiling and a more comprehensive “Thread Execution” profiling, each of which adds their own reports to profiler results. This feature works for both native and managed applications.

Just My Code

Have you ever wanted to exclude framework related information from your profiler reports so that you can look only at the data related to your code? You can now, with Just My Code feature. Just My Code reduces the noise in your profiler reports by removing framework related measurements. Just My Code works for both native and managed and applies only to sampling mode (in Instrumentation you specify the binaries you wish to profile, so it is “Just My Code” by default). There will be more detailed information on Just My Code to come in future blog posts from Profiler team.

CLR 4.0 Attach

When profiling applications written using .Net Frameworks 3.5 and below, some environment variables need to be set in the process. The Visual Studio IDE does this for you, but when profiling with the command line tools, you need to set up the environment variables using VSPerfCLREnv.cmd. In Dev10, you can attach directly to any v4.0 managed process to do Sample profiling using VSPerfCmd.exe /attach:PID.

Rich integration with the editor

Rather than me writing about what it does, why don’t we just look at the screenshot below:

As you can see, the code is color coded depending on the performance data collected (in this case, the hottest line). Editor integration is available for both native and managed applications.

Built-in Rules and Guidance

VS 2010 analyzes the performance data collected during the performance run and will use pre-defined rules to recommend a few areas for further investigation. These recommendations will be displayed in the “Error List” in Visual Studio (double clicking the warning will bring you to the profiling data that triggered the rule or a help document about the rule). You can modify and expand upon these “Rules and Guidance” by writing your own rules (another blog post to be provided in the near future). The rules are available for both managed and native applications.

[Daryush Laqab]

Visual Studio 2008 AJAX Profiling Extensions Power Tool

Originating in Microsoft Research as Ajax View, the Visual Studio 2008 AJAX Profiling Extensions Power Tool was released today (download here).  It’s a JavaScript performance profiler with a novel approach: it modifies the code sent by your server to include diagnostic code to trace and time function execution.  The client browser then sends profiling data back to your server for aggregation and analysis as the JavaScript executes.  Once you’ve collected your data, you can install the power tool extensions for VS2008 Team Developer or Team Suite to view and analyze the results using the Visual Studio Profiler’s UI.

The benefit of the server-side instrumentation approach is that it works with any client browser, letting you see how various browsers affect your performance.  On the server, you must be running IIS7 with an integrated pipeline.  Currently, running this power tool on production servers is not recommended.

Between this power tool, the built-in IE8 JavaScript profiler, and the integration with Visual Studio, we hope to shine much needed light on client-side browser performance.

[Chris Schmich]

Website Performance Talk at MIX09

John Hrvatin, lead program manager on the IE team, recently gave a talk at MIX09 entitled “Building High Performance Web Applications and Sites” (see the presentation key below).  John goes over a number of common performance pitfalls encountered when developing websites with CSS and JavaScript as well as some issues you might encounter at the HTTP level.  Some of the presentation’s content can be found in Steve Souders’ High Performance Web Sites book, but John goes more in depth on JavaScript issues and where IE8 has addressed specific problems.

John also gives a demo of IE8’s new JavaScript profiler which gives you absolute timings and call counts for your JavaScript functions.  This is a great first step, giving you the raw data you need in order to see where time is being spent in the browser.  To jump right in with IE8’s JavaScript profiler, check out the “How Do I: JavaScript Profiler” video.

Integrated JavaScript profiling will also ship with Visual Studio 2010, though you’ll get a richer analysis and reporting experience with tools like hot path, call tree trimming and folding, .csv and .xml exporting, VS IDE integration, and the function details and caller/callee views.  A future blog post will go more in depth on this.

Presentation Key

[Chris Schmich]

Patch Released To Fix Sampling On Intel Core i7 Processors

A hotfix has been issued for those of you trying to do sample profiling on Intel’s Nehalem-based processors, the most recently released being the Intel Core i7 family.  Previously, trying to do so would result in a blue screen of death (i.e. Windows crash, machine lockup and reboot).  Not very useful.  With this patch, sampling should work as usual.

If you’re running Visual Studio 2008 SP1…

• Download the hotfix
• Read about the issue

If you’re running Visual Studio 2005 SP1…

• Download the hotfix
• Read about the issue

[Chris Schmich]

See The Profiler At PDC2008

Steve Carroll, development lead for the Visual Studio Profiler, will be at PDC2008 next week giving a presentation on the profiler:

• What: TL24: Improving .NET Application Performance and Scalability
• When: Wednesday, October 29th, 1:15pm - 2:30pm
• Where: Room 153
• Abstract: “Performance must be considered in each step of the development lifecycle. See how to integrate performance in design, development, testing, tuning, and production. Work with tools and technologies like: static analysis, managed memory profiling, data population, load testing, and performance reports. Learn best practices to avoid the performance pitfalls of poor CPU utilization, memory allocation bugs, and improper data sizing.”

If you’re going to be there, it’d be worth checking out some of the new features we’ve been working on as well as seeing how the profiler and load testing tools in Visual Studio Team System can help you improve your software.

UPDATE: The video of this session is now available online. It covers a bunch of new profiler features including Javascript profiling, contention profiling, 'hot lines' source code highlighting, summary page and other UI improvements, load-test integration and numerous other changes that will be shipping in Visual Studio 2010 (Team System).

[Chris Schmich & Colin Thomsen]

Walkthroughs: Using VSTS Test and Profilers to Find Performance Issues

Tess Ferrandez recently wrote a couple of articles that extend her popular series of debugging labs to focus on measuring and improving performance.

• In 'Using VSTS Test and Profilers to troubleshoot performance issues (low cpu hang)', Tess introduces a performance problem into an ASP.NET application and uses the Visual Studio Profiler with a Load Test to track down the problem.
• In 'Using VSTS Test and Profilers to troubleshoot a High CPU in GC issue', Tess tracks down a string concatenation issue that is causing high CPU usage due to garbage collection.

The walkthroughs include sample code, lots of screenshots and analysis.

[Colin Thomsen]

Tip: VS2008 – Finding and Setting Properties (Right-Click)

The Visual Studio Profiler has many properties and options and this tip shows you where to find most of them. Future posts may cover some of the specific properties in more detail.

Performance Session:   Select an existing Performance Session in the Performance Explorer to see properties in the Properties Window. If the Properties Window is hidden:  Press ‘F4’ or go to ‘View->Properties Window’.

Performance Report: Select a Performance Report in the Performance Explorer to view many properties including Collection, ETW, General, Machine Information, Performance Counters, Process, Thread and Version Information.

Performance Session Properties (and Options):

To adjust Performance Session properties: Right-click on the Performance Session (Performance1 in this example). Select ‘Properties’.

Properties for Performance1 are shown below. There are different categories of properties on the left (e.g. General, Launch, Sampling, …).

Performance Targets:

To adjust Performance Target properties: Right-click on the Target (ConsoleApplication3 in this example). Select ‘Properties’.

Adjust the properties for the Performance Target as required. These properties do not often need to be changed, with the possible exception of the Instrumentation property ‘Exclude small functions from instrumentation’.

Tools –> Options –> Performance Tools:

Some global options can be configured using the Visual Studio Options dialog, which is accessed via:
Tools –> Options –> Performance Tools

That’s all the properties I can think of but I’m probably missing some still. Probably the most important aspect to this tip is to emphasize that right-clicking with the mouse is often the way to access important contextual information.

[Colin Thomsen]

Visual Studio Profiler: Found A Bug? Have A Suggestion?

The profiler is not flawless.  We work hard every day to improve it, however, and we’re always listening to and considering your feedback.  The profiler exists to help you, so if you feel it has shortcomings or if it fails to fully address your scenario, please let us know.  The more we hear from you, the more we know what’s important, what’s lacking, and what we should tweak.  Your feedback informs our daily decisions and sets the tone for future development.

Luckily, using Microsoft Connect, you can easily file a bug or give us suggestions for improvement.  After you’ve signed into Connect (Live ID required), you can go through the Visual Studio Feedback Center to submit a bug or a suggestion.  The higher quality bug or suggestion you file, the more likely we’ll be able to help you.  There’s a lot to be said about how to file an actionable bug, but please try to include the following:

• If applicable, the smallest project possible with which you can reproduce the issue
• A concise but complete set of steps to reproduce the issue
• If you’re profiling from the command-line, include the commands verbatim
• Is your OS 32- or 64-bit?  Is your target application 32- or 64-bit?
• What anti-virus/-spyware/-malware applications do you have installed?
• If it’s an instrumentation issue, include the binary (.exe, .dll) that you’re trying to instrument along with the matching symbols (.pdb) for that binary
• Include as much CPU information as possible including the number of sockets and cores as well as details for each CPU
• Do you have any custom build steps, build scripts, pre-/post-build events, or custom output locations?
• Do you have any special deployment for your binaries?  GAC?  NGEN?
• What Visual Studio edition are you using?  If any, what other Visual Studio editions do you have installed?
• What language is your application written in?  C#?  C++?  Managed C++?  Any assembly?
• Include any error strings, error codes, or screenshots of errors

Alternatively, if Connect feels too heavyweight or impersonal, you can contact us directly or post a question or suggestion in the profiler forums.

[Chris Schmich]

VS2008 SP1 Website Profiling Bug And Workaround

Visual Studio 2008 Service Pack 1, while fixing many profiling bugs, has also unfortunately introduced an issue with website profiling.  If you have a website project created with a pre-SP1 version of Visual Studio and then try to profile it using instrumentation with VS2008 SP1, you might run into a rather vague error: “Value does not fall within the expected range.”

Thankfully, there is a simple workaround: open the Configuration Manager from the Build menu, and for each of your website projects listed there, change its Platform to “Any CPU”.  Be sure to make these changes for each configuration type defined for your solution (e.g. Debug, Release).  You should now be able to successfully profile your website using instrumentation.

[Chris Schmich]

Visual Studio 2008 Service Pack 1 Released To The World!

After months of development and some grueling bug fixes, we released the VS2008 SP1 patch to the web this morning.  You can check out more details in the readme.  For the profiler, we fixed a number of bugs including:

• Adding support for instrumenting 64-bit managed C++ applications
• Improved instrumentation experience with precompiled websites
• Shipping the 64-bit performance SDK (VSPerf.h, VSPerf.lib)
• Ability to load a previously saved filter on non-English VS installations
• Numerous fixes around comparing profiling reports
• Numerous fixes around function name (symbol) resolution
• Numerous IDE fixes

The standalone profiler has also been updated to include the profiler’s SP1 fixes.  If you do profiling on production machines or other environments where minimal footprint is a virtue, you’ll want to use this updated, full installer.

As always, if you have questions or feedback, feel free to contact us or post a message in our forums.

[Chris Schmich]

Walkthrough: Profiling With Automated Tests

When doing performance investigations, we recommend having some well-defined scenario in mind in order to increase the reproducibility of your issue and increase the consistency of profiling data collected.  This approach is similar in spirit to creating the smallest self-contained repro possible when debugging.  When diagnosing any complex system, limiting the variables involved is a key step to discovering a solution.

Automated testing can often lead to naturally self-contained scenarios.  Integration tests ensure the correctness of a subsystem as its separate components work together.  If you test large portions of your application in this manner frequently enough, it is possible to catch performance issues and regressions before they become too serious.  With the profiler and test integration added in VS2008, once you notice a regression, you can create a performance session directly from the slow test via its context menu:

This will lead you to the Performance Wizard where you must decide between instrumentation and sampling.  Which method you should use depends upon the type of application you’re profiling and the kinds of questions you’re trying to answer.  Once you’ve completed the wizard, you’ll get a shiny new performance session ready to be launched.  You’ll see below that, in this case, our primary project and profiling target, Intergalactic.exe, is marked for instrumentation while the unit test is what we’ll be launching.

After launching the profiler, the test will execute as normal, exercising your code under the profiler.  The benefit of this approach is that reproducibility is maximized since the scenario is defined in code.  After the test completes, you’ll get a performance report to assist you with further investigations.

Performance sessions can be created from unit tests, web tests, database tests, and load tests containing web tests.  Load test and web test profiling will only work if you’re running against a web site or web service hosted locally in IIS or Cassini.  The reason for this limitation is that the profiler must run on the same physical machine as the target process in which you’re interested.  If you must profile a web site or web service running remotely, try using the command-line tools included in the standalone profiler, referring to this blog article for guidance.

[Chris Schmich]

Sample Profiling And Stdin

Some command-line applications can read from stdin for use as program input.  The canonical example of this is sort.exe which sorts the list of strings given to it and prints it back out to the console:

C:\Temp> type letters.txt
m
z
a

C:\Temp> sort < letters.txt
a
m
z

Sample profiling an application that reads from stdin is a bit trickier than typical sample profiling.  It is not easily possible from the VS IDE, so you have to drop down to the profiler’s command-line tools.  The command-line tools support passing arguments to the target program via vsperfcmd /args, but a stdin file parameter is not considered to be a program argument.  In other words, the following does not work as you might expect:

vsperfcmd /start:sample /output:profile.vsp /launch:sort.exe /args:”< letters.txt”

This will behave as if you launched sort.exe “< letters.txt”.  Unfortunately, this is not the intent here.  In order to get around this, we can use cmd.exe’s built-in start utility and its background feature.  The start utility is the Swiss Army knife for launching programs.  Its background feature lets you launch a command in the background without creating a new cmd window while still passing through any stdin parameters.  Knowing this, and assuming we’re profiling a native application, we can create a simple batch file to get the job done:

start /b sort.exe < letters.txt
vsperfcmd /start:sample /attach:sort.exe /output:profile.vsp
vsperfcmd /shutdown

Because we are attaching after launching the application, we will probably miss a few samples at the beginning.  Having these commands in a batch file, however, ensures that we miss as few samples as possible.  For any real workload, losing a few samples should not be an issue.

Interestingly, instrumentation (trace) profiling does not suffer from this issue at all because of the way in which instrumented binaries are launched.  To achieve the above with instrumentation, you can simply do the following:

vsinstr sort.exe
vsperfcmd /start:trace /output:profile.vsp
sort.exe < letters.txt
vsperfcmd /shutdown

If all else fails, or if your scenario is more complicated than above, you can always just modify your application to use a hardcoded input path.

[Chris Schmich]

Tip: VS2008 - Understanding Performance Targets

If you have a solution that contains multiple projects it is important to know what the 'Targets' group in the Performance Explorer is used for. The PeopleTrax solution shown on the right has 4 projects, with 3 of them compiling to managed DLLs and 1 compiling to an executable.

After running the Performance Wizard to create a Performance Session the Performance Explorer contains a single target as shown below.

Only the project that compiles to an executable is listed in the 'Targets' folder (for other project types like websites it would include the default launch project). What about the other 3 projects? As this tip explains, it depends upon the type of profiling you wish to do.

Sampling

With sampling there is no need to add the additional projects to your targets list. We do not modify assemblies when sampling and we will automatically attempt to collect data for any assemblies loaded by the PeopleTrax target. The only exception to this requirement is if you wish to collect data for multi-process scenarios and therefore need to launch multiple targets.

Instrumentation

For instrumentation, if you wish to collect data for the additional projects they should be added to your targets list as follows:

1. In the Performance Explorer, right-click on the 'Targets' folder:
   
   
2. Choose 'Add Target Project' to display a dialog:
    
   
3. Select the assemblies you wish to collect Instrumentation data for and choose OK.

The selected projects will now be modified (instrumented) when you start profiling. You can selectively disable instrumentation for certain projects by right-clicking on the target and unchecking the 'Instrument' option.

Instrumentation properties for a specific target.

  [Colin Thomsen]

Excluding Small Functions From Instrumentation

In Visual Studio 2008, we added a feature to exclude small functions from instrumentation.  A small function is a function that has very few instructions and makes no function calls (i.e. it’s a leaf).  The idea behind this feature is that small functions are unlikely to be significant in the overall profile of your application, so we can skip collecting data for them altogether to avoid the overhead of otherwise doing so.  This feature is meant to exclude property getters and setters, but it will also exclude very simple functions that are typically inlined anyway.

The first benefit of this feature is reduced instrumentation probe execution overhead.  Typically, small property getters and setters incur the time cost of executing two probes: the enter and exit probes.  The execution time of these probes dwarfs the execution time of the getter or setter itself.  To top it off, this cost is paid on every single property access which can be frequent.  By not instrumenting these property getters and setters, we can reduce execution overhead.

The second benefit is reduced profile data file (.vsp) size.  Fewer probes means less data collected.  If you’ve done instrumentation profiling before, you probably know that the data files can grow quickly.  Keeping file sizes smaller facilitates sharing and faster analysis.

The third benefit is reduced noise in your profile.  Since we don’t collect any data for these insignificant functions, they won’t even show up in your reports.  This is similar to, but independent of, the new noise reduction feature in VS2008.

With this feature enabled, it’s important to note that the time spent executing a small function does not just disappear.  Instead, the time gets attributed to the callers of the function excluded.  For example, if Foo calls SmallFunc, the exclusive time of SmallFunc when Foo called it is added to Foo’s exclusive time.  It’s as if the function was inlined at all of its call sites.

Small function exclusion is enabled by default which can, on occasion, cause issues.  Really simple projects such as a default C# console application or a default WCF service, are essentially infrastructure code in addition to (nearly) empty functions.  Because of this, profiling one of these default project types immediately after creation will typically yield “Error VSP1734 : File contains no data: ...” since nothing gets instrumented (they’re all small functions, after all).

You can disable small function exclusion by opening an instrumentation target’s properties, selecting the Instrumentation section, and unchecking “Exclude small functions from instrumentation”.  Each instrumentation target has its own value for this property.

If you’re instrumenting your binaries on the command-line using vsinstr.exe, you can specify /excludesmallfuncs to exclude small functions from instrumentation.  The default command-line behavior is to instrument everything.

[Chris Schmich]

Quick Tip: VS2008 - Compare Reports Quickly

While investigating a performance problem you may need to collect many Performance Reports and compare them. You can use the Performance Explorer to quickly compare two reports by:

1. Selecting two reports.
2. Right-clicking and choosing 'Compare Performance Reports...'
   

The oldest report will be used for the 'Baseline' report and the other report will be used for the 'Comparison' report, as shown below:

[Colin Thomsen]