Colin Thomsen's Microsoft Blog : profiler

VS2010: Attaching the Profiler to a Managed Application

colinth — Sat, 05 Dec 2009 23:52:00 GMT

Before Visual Studio 2010, in order to attach the profiler to a managed application, certain environment variables had to be set using vsperfclrenv.cmd. An example profiling session might look like this:

vsperfclrenv /sampleon
[Start managed application from the same command window]
vsperfcmd /start:sample /output:myapp.vsp /attach:[pid]
[Close application]

If the environment variables were not correctly set, when attempting to attach you would see this message:

The profiling environment for ConsoleApplication2 is not set up correctly. Use vsperfclrenv.cmd to setup environment variables. Continue anyway?

The generated report would typically look something like the report below. The warning at the bottom of the page indicates the problem and the report itself would typically not be useful since no managed modules or functions would be resolved correctly.

Report with 'CLRStubOrUnknownAddress and Unknown Frame(s) and the warning ‘It appears that the file was collected without properly setting the environment variables with VSPerfCLREnv.cmd. Symbols for managed binaries may not resolve’.

Fortunately the Common Language Runtime (CLR) team provided us with a new capability to attach to an already running managed application without setting any environment variables. For more detailed information take a look at David Broman’s post.

Caveats:

We only support attach without environment variables for basic sampling. It will not work for Allocation or Object Lifetime data collection and Instrumentation attach is not possible. Concurrency (resource contention) attach is supported.
The new attach mechanism only works for CLR V4-based runtimes.
The new attach mechanism will work if your application has multiple runtimes (i.e. V2 and V4 SxS), but as noted above, you can only attach to the V4 runtime. I’ll write another post about the profiler and Side by Side (SxS).
The old environment-variable-based attach still works, so you can still use that if you prefer.

The new procedure for attaching the profiler to a managed application in Visual Studio 2010 goes like this:

Launch your app (if it isn’t already running)
Attach to it, either from the command-line or from the UI.
When you’re finished, detach or close the app to generate a report.

If you want to diagnose any issues with attach, the CLR V4 runtime provides diagnostic information via the Event Log (view with Event Viewer) and the profiler also displays information there:

Event Log: ‘Loading profiler. Running CLR: v4.0.21202. Using ‘Profile First’ strategy’

There are two .NET Runtime messages regarding the attach, the first indicating that an attach was requested and the second that the attach succeeded. The VSPERF message describes which CLR is being profiled.

PDC 2008 - See the Sessions

colinth — Sun, 02 Nov 2008 00:07:00 GMT

This year if you didn't get a chance to go to the Professional Developer's Conference (PDC), there is still a wealth of information available to you. The most valuable resource I think are the videos of all the PDC sessions. Here are a few of the sessions that I've viewed and found most interesting:

Improving .NET Application Performance and Scalability, starring my boss Steve Carroll and Ed Glass, this session covers a bunch of new Visual Studio 2010 Profiler features.
Visual Studio Debugger Tips & Tricks, with speaker John Cunningham who is a Microsoft Development Manager (and Steve's boss), covering features in Visual Studio 2008, 2008 SP1 and features to look forward to in Visual Studio 2010. Note to self 'if you ever ship without symbols, I would fire you'.
Microsoft Visual Studio Team System: Software Diagnostics and Quality for Services, featuring Habib and Justin, who are also folks from the diagnostics team. The most exciting demo from this talk shows off the cool new Historical Debugging feature. It also features the new Test Impact Analysis feature, which can tell you which tests you should run after changing your code.
Framework Design Guidelines, by the guys who wrote the book of the same name, Krzysztof Cwalina and Brad Adams. If you write managed code this is a must-see session.

If you'd like to try some of the Visual Studio 2010 features for yourself, you can download the newest CTP here.

Tip: VS2008 – Finding and Setting Properties (Right-Click)

colinth — Sat, 20 Sep 2008 04:47:29 GMT

The Visual Studio Profiler has many properties and options and this tip shows you where to find most of them. Future posts may cover some of the specific properties in more detail.

Performance Session:

Select an existing Performance Session in the Performance Explorer to see properties in the Properties Window. If the Properties Window is hidden:
Press ‘F4’ or go to
‘View->Properties Window’.

Performance Report:

Select a Performance Report in the Performance Explorer to view many properties including Collection, ETW, General, Machine Information, Performance Counters, Process, Thread and Version Information.

Performance Session Properties (and Options):

To adjust Performance Session properties:

Right-click on the Performance Session (Performance1 in this example).
Select ‘Properties’.

Properties for Performance1 are shown below. There are different categories of properties on the left (e.g. General, Launch, Sampling, …).

Performance Targets:

To adjust Performance Target properties:

Right-click on the Target (ConsoleApplication3 in this example).
Select ‘Properties’.

Adjust the properties for the Performance Target as required. These properties do not often need to be changed, with the possible exception of the Instrumentation property ‘Exclude small functions from instrumentation’.

Tools –> Options –> Performance Tools:

Some global options can be configured using the Visual Studio Options dialog, which is accessed via:

Tools –> Options –> Performance Tools

That’s all the properties I can think of but I’m probably missing some still. Probably the most important aspect to this tip is to emphasize that right-clicking with the mouse is often the way to access important contextual information.

Tip: VS2008 - Understanding Performance Targets

colinth — Fri, 18 Jul 2008 23:48:43 GMT

If you have a solution that contains multiple projects it is important to know what the 'Targets' group in the Performance Explorer is used for. The PeopleTrax solution shown on the right has 4 projects, with 3 of them compiling to managed DLLs and 1 compiling to an executable.

After running the Performance Wizard to create a Performance Session the Performance Explorer contains a single target as shown below.

Only the project that compiles to an executable is listed in the 'Targets' folder (for other project types like websites it would include the default launch project). What about the other 3 projects? As this tip explains, it depends upon the type of profiling you wish to do.

Sampling

With sampling there is no need to add the additional projects to your targets list. We do not modify assemblies when sampling and we will automatically attempt to collect data for any assemblies loaded by the PeopleTrax target. The only exception to this requirement is if you wish to collect data for multi-process scenarios and therefore need to launch multiple targets.

Instrumentation

For instrumentation, if you wish to collect data for the additional projects they should be added to your targets list as follows:

In the Performance Explorer, right-click on the 'Targets' folder:
Choose 'Add Target Project' to display a dialog:
Select the assemblies you wish to collect Instrumentation data for and choose OK.

The selected projects will now be modified (instrumented) when you start profiling. You can selectively disable instrumentation for certain projects by right-clicking on the target and unchecking the 'Instrument' option.

Instrumentation properties for a specific target.

Quick Tip: VS2008 - Compare Reports Quickly

colinth — Sat, 21 Jun 2008 03:52:00 GMT

While investigating a performance problem you may need to collect many Performance Reports and compare them. You can use the Performance Explorer to quickly compare two reports by:

Selecting two reports.
Right-clicking and choosing 'Compare Performance Reports...'

The oldest report will be used for the 'Baseline' report and the other report will be used for the 'Comparison' report, as shown below:

Tech-Ed 2008 Demos

colinth — Tue, 03 Jun 2008 06:25:50 GMT

Last year my boss took a trip to sunny Orlando to present at Tech-Ed and to offer help and suggestions in the Technical Learning Center (TLC). This year I'm lucky enough to be attending with a couple of other folks (Habib and Tim) and since I'm not an official Speaker I'll be spending most of my time hanging out in the Application Lifecycle Management (ALM) demo station for Visual Studio 2008 Team System, Development Edition.

We've prepared a few demos covering things like:

Profiling using Instrumentation Mode on a Virtual PC image.
Collecting Allocation and Object Lifetime information.
Analyzing Performance Reports.
Using Code Analysis to improve your code.
Enabling Code Analysis Check-In Policies.

We're also looking forward to discussing your specific scenarios so if you're at Tech-Ed and interested in diagnostic tools and solving performance problems we'd love to chat with you.

Performance: Inserting Marks Using Code

colinth — Thu, 01 May 2008 17:55:00 GMT

Ian previously covered using the VS 2008 Data Collection Control to choose when to collect data. The Data Collection Control can also be used to insert marks into the performance report, but sometimes it is convenient to modify the source code to do this automatically.

Consider a typical application (PeopleTrax) where I am interested in gathering profiler data only between when a button is clicked and when data is displayed. The application is shown below.

After the 'Get People' button is clicked, data is displayed after just over 6 seconds. This seems a little excessive so I want to focus my performance investigation in this area.

To filter the data so that it only shows information collected between those two points, I could use the Data Collection Control, but maybe I'm planning to run a complicated scenario and don't want to have to remember to insert the marks manually. Instead, it is possible to modify the original code to request the profiler insert marks in the required locations.

The Profiler API is available for managed code in an assembly that can be added directly to the project from \Program Files\Microsoft Visual Studio 9.0\Team Tools\Performance Tools.

After adding the reference it shows up in the 'References' list for the PeopleTrax project.

I can then use functions in the Profiler API to control the profiler. This might include starting or stopping data collection or in this case, inserting marks into the datastream. This is easily achieved as shown below.

I can then profile the application and when I open the Performance Report and switch to Marks View I see that the marks have been correctly inserted. We can also see that the time elapsed between the marks is about 6.5 seconds, which corresponds with the measurement that is already displayed in the PeopleTrax UI.

I can use the marks to filter the report to only show profiling data for the time between the two inserted marks and then start my performance investigation.

Performance: Find Application Bottlenecks With Visual Studio Profiler

colinth — Thu, 13 Mar 2008 20:39:00 GMT

If you're a subscriber to msdn magazine, take a look at the article in the March 2008, Vol 23, No 4 issue on Page 81 which describes how to use the Visual Studio 2008 profiler to improve the performance of an application. A couple of members of the profiler team examine a Mandelbrot fractal drawing program in some detail. They isolate and fix several performance problems in the code, speeding up program execution approximately tenfold.

UPDATE: You can read the article here.

Noise Reduction in the VS2008 Profiler

colinth — Fri, 01 Feb 2008 04:36:00 GMT

One of the new profiler features in Visual Studio Team System (VS2008) is called Noise Reduction. This feature is intended to make it easier to review the Call Tree view by reducing the amount of data that is displayed, while still showing the most important functions.

To illustrate this new feature I wrote a very simple native C++ application that utilizes TR1 in the Feature Pack Beta. In this simple app I create some shared_ptrs in a for loop after calling a recursive function a few times. If you're not familiar with TR1 take a look at the VC blog for more information.

   1: #include <tchar.h>

   2: #include <memory>

3:

   4: class A

   5: {

   6: public:

   7:     A(int v, int w) : b(v), c(w)

   8:     {}

   9: private:

  10:     int b;

  11:     int c;

  12: };

13:

  14: void recurse(int v)

  15: {

  16:     if (v > 0)

  17:     {

  18:         recurse(--v);

  19:     }

  20:     else

  21:     {

  22:         A someA(1,2);

  23:         for (int i=0; i < 100; ++i)

  24:         {

  25:             std::tr1::shared_ptr<int> a(new int(2));

  26:             std::tr1::shared_ptr<A> b(new A(2,3));

  27:             std::tr1::shared_ptr<A> c(b);

  28:         }

  29:     }

  30: }

31:

  32: int _tmain(int argc, _TCHAR* argv[])

  33: {

  34:     recurse(5);

  35:     return 0;

  36: }

I profiled the application using Instrumentation Mode as shown below. Shifting to 'Call Tree' view and expanding I see my main function, which calls recurse 6 times (the initial call from main plus 5 recursive calls from inside the if statement in recurse). There are also a lot of other calls that have very little inclusive time (e.g. __RTC_CheckESP).

To reduce the noise in the call tree I choose to enable noise reduction by clicking on the icon on the far right of the toolbar (it looks like a checklist). In the dialog I enable trimming, using a threshold of 3%.

After enabling this option I expand the call tree again and many of the extra calls are gone and the calls to recurse are much easier to see as shown below.

I still think I can do a little better so I open up the Noise Reduction dialog again and 'Enable Folding'.

The resulting call tree can now be completely expanded and shown without scroll bars. I can see that functions that have significant exclusive time include: __security_init_cookie (28.40%), shared_ptr setup for A, (10.23%), shared_ptr refcount destructor (11.30%) & the last call to recurse (7.58%).

Using Noise Reduction in a larger application should make it easier for you to find performance problems.

Details:

Trimming removes leaf nodes in the call tree that have Elapsed Inclusive Time % (or Inclusive Sample % if you use sampling mode) lower than the threshold.
Folding combines (folds) a child node up to its parent node if it is the only child and the Elapsed Inclusive Time % (or Inclusive Sample % if you use sampling mode) is within Threshold of its parent. The intention is to fold simple forwarding functions that don't have much influence on the performance of your code.

Trimming is applied first and then folding, which is why the calls to recurse are all folded.

Visual Studio Team System Chat – December 5th

colinth — Wed, 21 Nov 2007 22:04:00 GMT

Join members of the Visual Studio Team System product group to discuss features available in Team Foundation Server, Team Suite, Architecture Edition, Development Edition, Database Edition, and Test Edition. In addition, discuss what's new for these editions for Visual Studio 2008.

We will be holding two sessions:

Join the chat on Wednesday, December 5th, 2007 from 10:00am - 11:00am Pacific Time. Add to Calendar | Additional Time Zones

-and-

Join the chat on Wednesday, December 5th, 2007 from 4:00pm - 5:00pm Pacific Time. Add to Calendar | Additional Time Zones

---

I'll be in the Wed 4 pm - 5 pm chat to answer any questions related to profiling. Another member of the profiler team will be online for the earlier chat.

---

Microsoft Blogs I Read

colinth — Tue, 16 Oct 2007 21:50:00 GMT

There are a lot of Microsoft bloggers, literally thousands of them. When I first joined Microsoft I wasn't sure who to read. I've gradually built up a list based on interesting product and feature announcements and people I've met. Here they are:

Profiling

Our Team Blog
IanWho's Blog. Written by a fellow dev on the profiler team, Ian has probably written the most about profiling across the team.
joc's bLog. Written by my bosses' boss.
mgoldin's blog. Written by a senior dev on my team. Find out about the difference between different types of samples etc.
My Code Does What?!. A relatively new blog about profiling by another fellow dev.
scarroll's Blog. Written by my boss.

Technical

bharry's WebLog. Written by a Technical Fellow (read more about this) with a huge amount of experience who has a big focus on TFS.
Greggm's Weblog. Written by a senior dev on the Debugger team. Has many advanced debugger tips.
Mark Russinovich. Mark wrote some cool Sysinternals tools and now blogs some fascinating posts about his investigation into problems he finds everyday just using his PC.
Rico Mariani's Performance Tidbits. Written by a senior Microsoftie who has been here for a long time. Gives tips for analyzing performance and provides guidelines to use in writing .NET code.
ScottGu's Blog. Find out about LINQ, ASP.NET AJAX etc. etc. This blog has many examples including screenshots and source code.
Somasegar's WebLog. As the corporate VP of DevDiv, Soma covers a lot of Visual Studio features and other developer tools.

Other

Microsoft International Tech Jobs. I keep an eye on Microsoft's international opportunities here.
Microsoft's JobsBlog. Discusses many aspects of Microsoft's recruiting. Covers the Vancouver office developments and has tips for prospective Microsofties.

That's just some of the Microsoft blogs I read. Are there other 'must-reads' that I'm missing?

Visual Studio 2008, Beta 2 (now with some of my code)

colinth — Fri, 27 Jul 2007 09:14:00 GMT

Today we released Beta 2 of VS2008. This is the first public release from Microsoft that contains a nontrivial amount of code that I wrote (even though I haven't written too much code just yet). I had barely synched up the source tree and only fixed a couple of bugs when we released Beta 1 but now I've found my feet and am contributing more.

The major release announcements have focussed on the flashier (and admittedly very cool) aspects of the Beta like LINQ and some of the HTML editing and Javascript debugging features. However, us Profiler folks have also been toiling away adding new features and fixing bugs. Look out for things like (and some of these already featured in Beta 1, but they just keep getting better):

A promotion to the new Developer menu
Hot path - find the critical path/paths through your call trees
Noise reduction - trim and/or fold your call trees so that they are easier to examine. See above for folding example.
Comparison reports - compare subsequent profiler runs to determine if code changes are improving performance
x64 OS support - profile on x64 Vista or W2K3 server

If you can, please download it and let us know what you think. If you don't have the time at least take a look at the overview video showing some of the major features. You should also check out Ian's entry about controlling data collection while profiling. Hopefully I'll have time to go through some of the new profiler-specific features soon.

Tech-Ed 2007

colinth — Sun, 03 Jun 2007 19:54:00 GMT

Tech-Ed 2007 is starting tomorrow and the Profiler Team is sending a few people to sunny Orlando for the event. This is great news for me because my boss, Steve Carroll, is away for the week (just kidding Steve), but it is really great news for folks at Tech Ed because he'll be there presenting with Marc Popkin-Paine:

DEV313 - Improving Code Performance with Microsoft Visual Studio Team System [N210 E]

June 07

9:45 AM

11:00 AM

I believe they'll be demoing a few new Orcas features and giving a pretty good introduction to profiling. If you didn't know Visual Studio Team System has a profiler, or you don't think performance is important, you should definitely check this out.

If you're not lucky enough to be able to make it to Orlando this year, be sure to take a look at Virtual Tech Ed, which will include webcasts and other content from some of the sessions. One that jumps out at me is MSDN Webcast: A Lap around Microsoft Visual Studio Code Name "Orcas" (Level 200).

UPDATE: Steve is already helping people at Tech Ed. If you're there and you're interested in performance go and have a chat with him in the Technical Learning Center.

Learning to Profile

colinth — Tue, 22 May 2007 01:53:00 GMT

I went to a meeting with Rico the other day and he showed us a few approaches he uses when solving performance issues. He is a performance engineer with many years of experience so it really was a case of watch and learn. This got me thinking about how people can best learn to use performance tools.

One starting point in this process is to consider my own experience learning a more mature dynamic code analysis tool - the debugger. Think back to the first time you ever started up a program running under a debugger. What was the first thing you did? My first debugging experience went something like this:

Set a breakpoint at the beginning of main() - this was C/C++ afterall.
Run the code in the debugger. Hey, it stopped. cool.
Step through a few lines of code and inspect the values of some local variables.

Sit back and think that's pretty cool - maybe I'll have to use a few less printfs to work out what's going on with my program. That's pretty much it. Gradually I learnt more and more about things like:

The difference between Step In, Step Over, Step Out, Run to Cursor
The value of different types of breakpoints like conditional breakpoints, data breakpoints etc.
The value of the Watch window. I'm still surprised by how much you customize the output to make it easier to find issues.
The various other windows - threads, memory, etc. etc.
Etc.

It took a long to discover some of these features. It took even longer to use them almost automatically while debugging.

Obviously the learning curve depends a lot upon the tool you use. Visual Studio tries to be more intuitive and easy to use than something like WinDbg, which is a command-line tool. Even with the ease of use of the visual debugger, you still need to know the typical debugging pattern (using breakpoints) before you can use the tool effectively.

Fewer people have used code profilers than debuggers and the tools are still less mature than their debugger equivalents, so it is harder for new programmers to profile their code than to debug it. In an ideal world we might have a 'fix my code' button or at the very least a 'highlight problem code lines' feature, but for now we need to develop patterns that developers can use to do this themselves.

What features would make profiling easier for you? Are we missing a fundamental concept (the equivalent of 'set breakpoint' in debugging land) that would make things so much easier?

Basic Profiler Scenarios

colinth — Mon, 07 May 2007 23:28:00 GMT

This post was going to cover some basic scenarios discussing the differences between sampling and instrumentation and when you would choose to switch methods, but then I found there is already something like that in MSDN. If you haven't already, go and take a look. See if you can improve the performance of the PeopleTrax app.

Instead I'll discuss sampling and instrumentation from a user's perspective. There are already many definitions of sampling vs instrumentation so I won't repeat them.

For some background reading on the sampling aspect, take a look at David Gray's post. There are a few things that he hasn't covered in that post. The main question I had was should I use sampling or instrumentation?

A generic answer to that would be:

If you know your performance problem is CPU-related (i.e. you see the CPU is running at or near 100% in task manager) then you should probably start with sampling.
If you suspect your problem may be related to resource contention (e.g. locks, network, disk etc), instrumentation would be a better starting point.

Sometimes you may not be sure what type of performance issue you are facing or you may be trying to resolve several types of issues. Read on for more details.

Sampling

Why use sampling instead of instrumentation?

Sampling is lighter weight than instrumentation (see below for reasons why instrumentation is more resouce intensive) and you don't need to change your executable/binaries to use sampling.

What events do you sample with?

By default the profiler samples with clock cycles. This should be familiar to most users because they relate to the commonly quoted frequency of the machine. For example, 1 GHz is 1 billion clock cycles / second. If you use the default profiler setting for clock cycles that would mean 100 samples every second on a 1 GHz machine.

Alternatively, you could choose to sample using Page Faults, which might occur frequently if you are allocating/deallocating memory a lot. You could also choose to profile using system calls or some lower level counter.

How many samples is enough to accurately represent my program profile?

This is not a simple question to answer. By default we only sample every 10000000 clock cycles, which might seem like a long time between samples. In that time, your problematic code might block waiting on a lock or some other construct and the thread it is running in might be pre-empted allowing another thread to run. When the next sample is taken the other thread could still be running which means the problematic code is not included in the sample.

The risk of missing the key data is something that is inherent in any sample-based data collection. In statistics the approach is to minimize the risk of missing key information by making the number of samples large enough relative to the general population. For example, if you have a demographic that includes 10000 people, taking only 1 sample is unlikely to be representative. Taking a sample of 1000 people might be considered representative. There are more links about this on Wikipedia.

Won't this slow down my app?

No, not really. When a sample is taken the current thread is suspended (other application threads continue to run) so that the current call stack can be collected. When the stack walk is finished, execution returns to the application thread. Sampling should have a limited effect on most applications.

Sounds good, why use instrumentation?

See below.

Instrumentation

Why use instrumentation?

As discussed above, sampling doesn't always give you the whole picture. If you really want to know what is going on with a program the most complete way is to keep track of every single call to every function.

How does instrumentation work (briefly)?

Unlike sampling, with instrumentation the profiler changes the binary by inserting special pieces of code called probes at the start and end of each function. This process is called 'instrumenting the binary' and it works by taking a binary (dll or exe) along with its PDB and making a new 'instrumented binary'. By comparing a counter at the end of the function with the start, it is easy to determine how long a function took to execute.

What if I call other people's code?

Usually you don't have access to the PDB files for other people's code which means you can't instrument it. Fortunately as part of the instrumentation process the profiler inserts special probes around each call to an external function so that you can track these calls (although not any functions that they might call).

Why not just use Instrumentation all the time?

Computers execute a lot of instructions in 10000000 clock cycles, so using instrumentation can generate a LOT of data compared with sampling. The process of calling the probe functions in an application thread can also degrade performance more than sampling would.