Welcome to MSDN Blogs Sign in | Join | Help

See The Profiler At PDC2008

Steve Carroll, development lead for the Visual Studio Profiler, will be at PDC2008 next week giving a presentation on the profiler:

  • What: TL24: Improving .NET Application Performance and Scalability
  • When: Wednesday, October 29th, 1:15pm - 2:30pm
  • Where: Room 153
  • Abstract: “Performance must be considered in each step of the development lifecycle. See how to integrate performance in design, development, testing, tuning, and production. Work with tools and technologies like: static analysis, managed memory profiling, data population, load testing, and performance reports. Learn best practices to avoid the performance pitfalls of poor CPU utilization, memory allocation bugs, and improper data sizing.”

If you’re going to be there, it’d be worth checking out some of the new features we’ve been working on as well as seeing how the profiler and load testing tools in Visual Studio Team System can help you improve your software.

UPDATE: The video of this session is now available online. It covers a bunch of new profiler features including Javascript profiling, contention profiling, 'hot lines' source code highlighting, summary page and other UI improvements, load-test integration and numerous other changes that will be shipping in Visual Studio 2010 (Team System).

[Chris Schmich & Colin Thomsen]

Posted by profiler | 0 Comments

Walkthroughs: Using VSTS Test and Profilers to Find Performance Issues

Tess Ferrandez recently wrote a couple of articles that extend her popular series of debugging labs to focus on measuring and improving performance.

The walkthroughs include sample code, lots of screenshots and analysis.

[Colin Thomsen]

Tip: VS2008 – Finding and Setting Properties (Right-Click)

The Visual Studio Profiler has many properties and options and this tip shows you where to find most of them. Future posts may cover some of the specific properties in more detail.

Performance Session:
session_properties 
Select an existing Performance Session in the Performance Explorer to see properties in the Properties Window. If the Properties Window is hidden: 
Press ‘F4’ or go to
‘View->Properties Window’.
  Performance Report:
report_properties

Select a Performance Report in the Performance Explorer to view many properties including Collection, ETW, General, Machine Information, Performance Counters, Process, Thread and Version Information.

Performance Session Properties (and Options):

session_properties_1 To adjust Performance Session properties:
  1. Right-click on the Performance Session (Performance1 in this example).
  2. Select ‘Properties’.

Properties for Performance1 are shown below. There are different categories of properties on the left (e.g. General, Launch, Sampling, …).
session_properties_2

Performance Targets:

target_properties_1 To adjust Performance Target properties:
  1. Right-click on the Target (ConsoleApplication3 in this example).
  2. Select ‘Properties’.

Adjust the properties for the Performance Target as required. These properties do not often need to be changed, with the possible exception of the Instrumentation property ‘Exclude small functions from instrumentation’.

target_properties_2

Tools –> Options –> Performance Tools:

Some global options can be configured using the Visual Studio Options dialog, which is accessed via:
Tools –> Options –> Performance Tools

tools_options

That’s all the properties I can think of but I’m probably missing some still. Probably the most important aspect to this tip is to emphasize that right-clicking with the mouse is often the way to access important contextual information.

[Colin Thomsen]

Posted by profiler | 0 Comments
Filed under: , ,

Visual Studio Profiler: Found A Bug? Have A Suggestion?

The profiler is not flawless.  We work hard every day to improve it, however, and we’re always listening to and considering your feedback.  The profiler exists to help you, so if you feel it has shortcomings or if it fails to fully address your scenario, please let us know.  The more we hear from you, the more we know what’s important, what’s lacking, and what we should tweak.  Your feedback informs our daily decisions and sets the tone for future development.

connectLuckily, using Microsoft Connect, you can easily file a bug or give us suggestions for improvement.  After you’ve signed into Connect (Live ID required), you can go through the Visual Studio Feedback Center to submit a bug or a suggestion.  The higher quality bug or suggestion you file, the more likely we’ll be able to help you.  There’s a lot to be said about how to file an actionable bug, but please try to include the following:

  • If applicable, the smallest project possible with which you can reproduce the issue
  • A concise but complete set of steps to reproduce the issue
  • If you’re profiling from the command-line, include the commands verbatim
  • Is your OS 32- or 64-bit?  Is your target application 32- or 64-bit?
  • What anti-virus/-spyware/-malware applications do you have installed?
  • If it’s an instrumentation issue, include the binary (.exe, .dll) that you’re trying to instrument along with the matching symbols (.pdb) for that binary
  • Include as much CPU information as possible including the number of sockets and cores as well as details for each CPU
  • Do you have any custom build steps, build scripts, pre-/post-build events, or custom output locations?
  • Do you have any special deployment for your binaries?  GAC?  NGEN?
  • What Visual Studio edition are you using?  If any, what other Visual Studio editions do you have installed?
  • What language is your application written in?  C#?  C++?  Managed C++?  Any assembly?
  • Include any error strings, error codes, or screenshots of errors

Alternatively, if Connect feels too heavyweight or impersonal, you can contact us directly or post a question or suggestion in the profiler forums.

[Chris Schmich]

Posted by profiler | 0 Comments

VS2008 SP1 Website Profiling Bug And Workaround

out-of-range Visual Studio 2008 Service Pack 1, while fixing many profiling bugs, has also unfortunately introduced an issue with website profiling.  If you have a website project created with a pre-SP1 version of Visual Studio and then try to profile it using instrumentation with VS2008 SP1, you might run into a rather vague error: “Value does not fall within the expected range.”

Thankfully, there is a simple workaround: open the Configuration Manager from the Build menu, and for each of your website projects listed there, change its Platform to “Any CPU”.  Be sure to make these changes for each configuration type defined for your solution (e.g. Debug, Release).  You should now be able to successfully profile your website using instrumentation.

build-menu

configuration-manager

[Chris Schmich]

Posted by profiler | 0 Comments

Visual Studio 2008 Service Pack 1 Released To The World!

After months of development and some grueling bug fixes, we released the VS2008 SP1 patch to the web this morning.  You can check out more details in the readme.  For the profiler, we fixed a number of bugs including:

  • Adding support for instrumenting 64-bit managed C++ applications
  • Improved instrumentation experience with precompiled websites
  • Shipping the 64-bit performance SDK (VSPerf.h, VSPerf.lib)
  • Ability to load a previously saved filter on non-English VS installations
  • Numerous fixes around comparing profiling reports
  • Numerous fixes around function name (symbol) resolution
  • Numerous IDE fixes

The standalone profiler has also been updated to include the profiler’s SP1 fixes.  If you do profiling on production machines or other environments where minimal footprint is a virtue, you’ll want to use this updated, full installer.

As always, if you have questions or feedback, feel free to contact us or post a message in our forums.

[Chris Schmich]

Posted by profiler | 1 Comments

Walkthrough: Profiling With Automated Tests

When doing performance investigations, we recommend having some well-defined scenario in mind in order to increase the reproducibility of your issue and increase the consistency of profiling data collected.  This approach is similar in spirit to creating the smallest self-contained repro possible when debugging.  When diagnosing any complex system, limiting the variables involved is a key step to discovering a solution.

Automated testing can often lead to naturally self-contained scenarios.  Integration tests ensure the correctness of a subsystem as its separate components work together.  If you test large portions of your application in this manner frequently enough, it is possible to catch performance issues and regressions before they become too serious.  With the profiler and test integration added in VS2008, once you notice a regression, you can create a performance session directly from the slow test via its context menu:

test-results

test-results-context-menu

This will lead you to the Performance Wizard where you must decide between instrumentation and sampling.  Which method you should use depends upon the type of application you’re profiling and the kinds of questions you’re trying to answer.  Once you’ve completed the wizard, you’ll get a shiny new performance session ready to be launched.  You’ll see below that, in this case, our primary project and profiling target, Intergalactic.exe, is marked for instrumentation while the unit test is what we’ll be launching.

performance-wizard

performance-explorer 

After launching the profiler, the test will execute as normal, exercising your code under the profiler.  The benefit of this approach is that reproducibility is maximized since the scenario is defined in code.  After the test completes, you’ll get a performance report to assist you with further investigations.

summary

Performance sessions can be created from unit tests, web tests, database tests, and load tests containing web tests.  Load test and web test profiling will only work if you’re running against a web site or web service hosted locally in IIS or Cassini.  The reason for this limitation is that the profiler must run on the same physical machine as the target process in which you’re interested.  If you must profile a web site or web service running remotely, try using the command-line tools included in the standalone profiler, referring to this blog article for guidance.

[Chris Schmich]

Posted by profiler | 0 Comments

Sample Profiling And Stdin

Some command-line applications can read from stdin for use as program input.  The canonical example of this is sort.exe which sorts the list of strings given to it and prints it back out to the console:

C:\Temp> type letters.txt
m
z
a

C:\Temp> sort < letters.txt
a
m
z

Sample profiling an application that reads from stdin is a bit trickier than typical sample profiling.  It is not easily possible from the VS IDE, so you have to drop down to the profiler’s command-line tools.  The command-line tools support passing arguments to the target program via vsperfcmd /args, but a stdin file parameter is not considered to be a program argument.  In other words, the following does not work as you might expect:

vsperfcmd /start:sample /output:profile.vsp /launch:sort.exe /args:”< letters.txt”

This will behave as if you launched sort.exe “< letters.txt”.  Unfortunately, this is not the intent here.  In order to get around this, we can use cmd.exe’s built-in start utility and its background feature.  The start utility is the Swiss Army knife for launching programs.  Its background feature lets you launch a command in the background without creating a new cmd window while still passing through any stdin parameters.  Knowing this, and assuming we’re profiling a native application, we can create a simple batch file to get the job done:

start /b sort.exe < letters.txt
vsperfcmd /start:sample /attach:sort.exe /output:profile.vsp
vsperfcmd /shutdown

Because we are attaching after launching the application, we will probably miss a few samples at the beginning.  Having these commands in a batch file, however, ensures that we miss as few samples as possible.  For any real workload, losing a few samples should not be an issue.

Interestingly, instrumentation (trace) profiling does not suffer from this issue at all because of the way in which instrumented binaries are launched.  To achieve the above with instrumentation, you can simply do the following:

vsinstr sort.exe
vsperfcmd /start:trace /output:profile.vsp
sort.exe < letters.txt
vsperfcmd /shutdown

If all else fails, or if your scenario is more complicated than above, you can always just modify your application to use a hardcoded input path.

[Chris Schmich]

Posted by profiler | 0 Comments

Tip: VS2008 - Understanding Performance Targets

default_wizard_output_slnexplorer

If you have a solution that contains multiple projects it is important to know what the 'Targets' group in the Performance Explorer is used for. The PeopleTrax solution shown on the right has 4 projects, with 3 of them compiling to managed DLLs and 1 compiling to an executable.

After running the Performance Wizard to create a Performance Session the Performance Explorer contains a single target as shown below.

default_wizard_output_perfexplorer 

Only the project that compiles to an executable is listed in the 'Targets' folder (for other project types like websites it would include the default launch project). What about the other 3 projects? As this tip explains, it depends upon the type of profiling you wish to do.

Sampling

With sampling there is no need to add the additional projects to your targets list. We do not modify assemblies when sampling and we will automatically attempt to collect data for any assemblies loaded by the PeopleTrax target. The only exception to this requirement is if you wish to collect data for multi-process scenarios and therefore need to launch multiple targets.

Instrumentation

For instrumentation, if you wish to collect data for the additional projects they should be added to your targets list as follows:

  1. In the Performance Explorer, right-click on the 'Targets' folder:
    add_target_project_rightclick
  2. Choose 'Add Target Project' to display a dialog:
    add_target_project_dialog 
  3. Select the assemblies you wish to collect Instrumentation data for and choose OK.

The selected projects will now be modified (instrumented) when you start profiling. You can selectively disable instrumentation for certain projects by right-clicking on the target and unchecking the 'Instrument' option.

targets_launchable_trace_properties_crop
Instrumentation properties for a specific target.

  [Colin Thomsen]

Posted by profiler | 0 Comments
Filed under: , , ,

Excluding Small Functions From Instrumentation

In Visual Studio 2008, we added a feature to exclude small functions from instrumentation.  A small function is a function that has very few instructions and makes no function calls (i.e. it’s a leaf).  The idea behind this feature is that small functions are unlikely to be significant in the overall profile of your application, so we can skip collecting data for them altogether to avoid the overhead of otherwise doing so.  This feature is meant to exclude property getters and setters, but it will also exclude very simple functions that are typically inlined anyway.

The first benefit of this feature is reduced instrumentation probe execution overhead.  Typically, small property getters and setters incur the time cost of executing two probes: the enter and exit probes.  The execution time of these probes dwarfs the execution time of the getter or setter itself.  To top it off, this cost is paid on every single property access which can be frequent.  By not instrumenting these property getters and setters, we can reduce execution overhead.

The second benefit is reduced profile data file (.vsp) size.  Fewer probes means less data collected.  If you’ve done instrumentation profiling before, you probably know that the data files can grow quickly.  Keeping file sizes smaller facilitates sharing and faster analysis.

The third benefit is reduced noise in your profile.  Since we don’t collect any data for these insignificant functions, they won’t even show up in your reports.  This is similar to, but independent of, the new noise reduction feature in VS2008.

With this feature enabled, it’s important to note that the time spent executing a small function does not just disappear.  Instead, the time gets attributed to the callers of the function excluded.  For example, if Foo calls SmallFunc, the exclusive time of SmallFunc when Foo called it is added to Foo’s exclusive time.  It’s as if the function was inlined at all of its call sites.

Small function exclusion is enabled by default which can, on occasion, cause issues.  Really simple projects such as a default C# console application or a default WCF service, are essentially infrastructure code in addition to (nearly) empty functions.  Because of this, profiling one of these default project types immediately after creation will typically yield “Error VSP1734 : File contains no data: ...” since nothing gets instrumented (they’re all small functions, after all).

You can disable small function exclusion by opening an instrumentation target’s properties, selecting the Instrumentation section, and unchecking “Exclude small functions from instrumentation”.  Each instrumentation target has its own value for this property.

target-props exclude-small-funcs

If you’re instrumenting your binaries on the command-line using vsinstr.exe, you can specify /excludesmallfuncs to exclude small functions from instrumentation.  The default command-line behavior is to instrument everything.

[Chris Schmich]

Posted by profiler | 3 Comments
Filed under: ,

Quick Tip: VS2008 - Compare Reports Quickly

While investigating a performance problem you may need to collect many Performance Reports and compare them. You can use the Performance Explorer to quickly compare two reports by:

  1. Selecting two reports.
  2. Right-clicking and choosing 'Compare Performance Reports...'
    comp_reports

The oldest report will be used for the 'Baseline' report and the other report will be used for the 'Comparison' report, as shown below:

comp_reports_2

[Colin Thomsen]

Posted by profiler | 0 Comments

What code path is allocating the most bytes for a type?

In Visual Studio 2008, we added the ability to quickly determine the most expensive call stack in your application.  In profiler parlance, the “most expensive call stack” is known as the “hot path”.  What do we mean by “expensive”?  “Expensive” is whatever measure you want it to be: number of samples in a function, time spent in a function, or even number of bytes or allocations in a function.

If you profile the managed memory usage of your application (also available in VS2008), you will get a new view in your profiling report called the Allocation View:

allocation-view

For each managed type allocated in your application, this view will show you the call stack where each allocation took place as well as the number of bytes and instances allocated in each function.

You can explore these call trees as you do normally, including using hot path to find the expensive allocation paths through your code for a specific type.  This is accessible through either the toolbar or the context menu by right-clicking on the function from where you want to start hot path:

allocation-hot-path-menu

In this example, we know we’re allocating about 741KB of memory for System.Drawing.Graphics objects.  Using hot path, we can quickly determine that the majority (67%)  of the bytes are being allocated in System.Drawing.Graphics.FromHwndInternal.  Because we do not control this code, we can look up the stack until we see code that we can change: MainForm.SetCell.  This would be a great place to begin investigation.

allocation-hot-path

It’s important to note that the allocation hot path is not the single function allocating the most bytes in your application (you can use the Function View for that).  Instead, it is the specific path through your code that is allocating the most bytes for a type.

[Chris Schmich]

Posted by profiler | 0 Comments

Very Quick Tip: Getting the most out of your screen for viewing call trees

I spend a lot of time in the profiler's Call Tree view when I use the profiler.  When you are really digging deep into the tree, screen real estate becomes a big issue.  There is one very simple Visual Studio trick that really helps out so you can get more of the call tree on the screen at one time without scrolling around.  Set the focus on the call tree view and hit Shift-Alt-Enter.  It will bring up a full screen view of just the document you are working on which makes it much easier to do the data analysis (especially on wide screen monitors).  This works in all VS documents. 

[Steve Carroll - scarroll]

Posted by profiler | 1 Comments
Filed under: , ,

Quick Tip: "Save Analyzed" for large VSP files

One cool new feature in the Visual Studio 2008 Profiler is the ability to take a large VSP file that has been analyzed once and save the report data into a VSPS file (Visual Studio Profiler Summary files).  Saving the analyzed data has several major benefits:

    1. The analyzed data is orders of magnitude smaller than the VSP file.  We've seen 4GB trace VSPs get compressed into less than 100K VSPS files.
    2. Because the data saved in the VSPS files has already been processed by our data analyzer, it will load up much faster. 
    3. Symbol data is automatically packed into the VSPS file.

VSPS files because of their small size, quick analysis, and saved symbols are perfect for use as saved baselines with the new performance comparison functionality in VS2008.  A best practice is to regularly save off VSPS files for your performance scenarios so you can look for performance regressions from one milestone to the next.

This feature is not particularly discoverable.  Here's a quick map of where to find it (highlighted in yellow):

SaveAnalyzed

One caveat: because the data has already been analyzed in the VSPS file, you can't use filters on VSPS files.

[Steve Carroll - scarroll]

Performance: Find Application Bottlenecks with Visual Studio Profiler

A couple of members of the profiler team recently wrote an article for msdn magazine, which describes how to use the Visual Studio 2008 profiler to improve the performance of an application. They examined a Mandelbrot fractal drawing program in some detail and isolated and fixed several performance problems in the code, speeding up program execution time approximately tenfold.

[colinth]

Posted by profiler | 1 Comments
More Posts Next page »
 
Page view tracker