Angry Richard's WebLog

Memory Allocation Profiling is Managed Only!

AngryRichard — Wed, 13 Dec 2006 18:24:00 GMT

People have been discovering that the VS Team System profiler can collect allocation data for an application. It isn't long after that they discover that it only works on managed code, not native. Sadly, the documentation is not clear on this.

The memory alloction profiling support in VSTS uses the profiler API provided by the CLR. This gives us a rich set of information that allows us to track the lifetimes of individual objects. There is no such off-the-shelf support in native memory management, since there are nearly as many heap implementations as there are applications in the world.

Memory allocation profiling is a much bigger deal for managed code, as the CLR has effectively turned what used to be a correctness issue (leaks, double frees, etc) into performance issues (excessive GCs and memory pressure)*. This does not mean that some kind memory allocation profiling wouldn't benefit the world, but the combination of it being less important and more difficult keeps it out of the product for now.

If you think you are facing memory issues in native code, there is at least one utility I can offer up: In the server resource kits, the vadump utility can give you information about your virtual address space, including memory allocated by VirtualAlloc.

http://www.microsoft.com/downloads/details.aspx?FamilyID=9d467a69-57ff-4ae7-96ee-b18c4790cffd&DisplayLang=en

Unfortunately, this is a pale shadow (if that) of what you can get from the managed side.

* The idea that correctness issues become performance issues as we develop more advanced runtimes was something I heard David Detlefs mention somewhere.

How to shoot yourself in the foot with const (or the lack thereof)

AngryRichard — Wed, 24 May 2006 21:00:00 GMT

It has long puzzled me why various Win32 functions take non-const string parameters, when clearly they have no business manipulating the string. Take, for instance, GetNamedSecurityInfo:

DWORD GetNamedSecurityInfo(
  LPTSTR pObjectName,
  SE_OBJECT_TYPE ObjectType,
  SECURITY_INFORMATION SecurityInfo,
  PSID* ppsidOwner,
  PSID* ppsidGroup,
  PACL* ppDacl,
  PACL* ppSacl,
  PSECURITY_DESCRIPTOR* ppSecurityDescriptor
);

I can't imagine why this function should ever need to write to pObjectName. In fact, the SDK documentation explicitly lists it as an [in] parameter. Also note that pObjectName is an LPTSTR. This is important to the shooting of the foot.

A tidy, const-aware developer might want to do something like this:

const LPWSTR SomeFile = L"\\somefile.txt";

[...]

GetNamedSecurityInfo(Somefile, ...)

Of course, this will fail to compile, because SomeFile is const. But casting solves this problem, right?

GetNamedSecurityInfo((LPTSTR)Somefile, ...)

Yay, this compiles. Why, it even appears to return the correct data.

Fast forward a year or two, when suddenly the code stops working on Vista. What could be wrong? Why, in the process of casting away const, we've also turned an LPWSTR into an LPTSTR. In this case, TCHAR resolved to char, so the code above was actually returning a security descriptor for "\\". Well, that is until Vista came along.

So while I hang my head in shame for being lame enough to cast between incompatible types, I will also caution the rest of you to consider whether your string parameters really need to be non-const, and avoid making your clients play casting games to use your API. And yes, using const_cast<> might have prevented this issue, but I didn't know about it at the time. You, having read this far, have no excuse.

DataCollection at the Global Level

AngryRichard — Mon, 10 Oct 2005 20:41:00 GMT

If you are using the DataCollection API (either from native, by linking agains VSPerf.lib, or from managed, by importing Microsoft.VisualStudio.Profiler.dll), and wish to enable or disable collection at the global level, you must pass CurrentId for the elementId parameter:

using Microsoft.VisualStudio.Profiler;

[Not so interesting code here]

DataCollection.StartProfile(
ProfileLevel.Global,
DataCollection.CurrentId);

[Interesting code here]

Similarly, for native code:

#include "VSPerf.h"

[Not so interesting code here]

StartProfile(PROFILE_GLOBALLEVEL, PROFILE_CURRENTID);

[Interesting code here]

Note that both the native and managed versions return zero on success, so it is possible to detect whether the call succeeded:

C#	c++	Description
ProfileOperationResult.ErrorNotYetImplemented	PROFILE_ERROR_NOT_YET_IMPLEMENTED	API level, id combination not supported
ProfileOperationResult.ErrorModeNever	PROFILE_ERROR_MODE_NEVER	Global mode was "never" when called
ProfileOperationResult.ErrorLevelDoesNotExist	PROFILE_ERROR_LEVEL_NOEXIST	Profile level does not exist
ProfileOperationResult.ErrorIdDoesNotExist	PROFILE_ERROR_ID_NOEXIST	Element id doesn't exist

Response Files

AngryRichard — Mon, 08 Aug 2005 05:12:00 GMT

On the forums, someone was using the /INCLUDE option in VsInstr.exe. It is possible to use multiple instances of this option to include different sets of functions. For a big chunk of functions, you might want to use dozens of function specifications. Who the heck wants to do all that typing? You could make a batch file, but a response file would be better.

Response files are text files where each line in the file is a single command line option. Since each line holds exactly one option, quotes are not necessary. They are much easier to edit than a batch file (which would have single, really long line). To use a response file, simply use @filename on the command line of the tool. For example:

VsPerfCmd /start:sample "/output:c:\Documents and Settings\AngryRichard\foo.vsp" "/user:NETWORK SERVICE"

can be turned into a response file like this:

Startup.rsp:

/start:sample
/output:c:\Documents and Settings\AngryRichard\foo.vsp
/user:NETWORK SERVICE

VsPerfCmd @Startup.rsp

All of the command line tools for the profiler accept response files. It beats all that error prone typing, and if you run scenarios from the command line a lot, it can pay to have some response files laying about for common scenarios.

Off-road Profiling Windows Services

AngryRichard — Thu, 28 Jul 2005 22:36:00 GMT

I've just posted an article on the pitfalls of profiling services with the Visual Studio profiler. It includes a sample service with a quick walkthrough. Enjoy.

Profiling Windows™ Services with the Visual Studio Profiler

Limiting the Data Crunch from Trace Profiling

AngryRichard — Mon, 17 Jan 2005 03:41:00 GMT

Typically, one can use the sampling profiler to nail down the hot spot in an application. Having done that, what does one do when the sampling data doesn't provide enough information? The trace profiler can offer up more detail, particularly if the issue revolves around thread interaction. However, if you profile a heavily CPU bound application, you may find that you are getting huge trace files, or that the profiler is significantly impacting the performance of your application. The VisualStudio profiler offers a mechanism to stem the avalanche of data.

I'll illustrate the general idea with an example.

Suppose we'd like to run trace profiling on the following highly useful piece of code. We've decided that we only care about profiling in the context of the function "OnlyProfileThis()"

using System;

public class A
{
private int _x;

public A(int x)
{
  _x = x;
}

public int DoNotProfileThis()
{
  return _x * _x;
}

public int OnlyProfileThis()
{
  return _x + _x;
}

public static void Main()
{
  A a;
  a = new A(2);

  Console.WriteLine("2 square is {0}", a.DoNotProfileThis());
  Console.WriteLine("2 doubled is {0}", a.OnlyProfileThis());
}
}

The VisualStudio profiler provides an API for controlling data collection from within the application. For native code, this API lives in VSPerf.dll. A header (VSPerf.h) and import library (VSPerf.lib) provided in the default Team Developer install allows us to use the profiler control API from native code. For managed code, this API is wrapped by the DataCollection class in Microsoft.VisualStudio.Profiler.dll. We can update the example to the following to control the profiler during our run:

using System;
using Microsoft.VisualStudio.Profiler;

int x;
Console.WriteLine("2 square is {0}", a.DoNotProfileThis());

  DataCollection.StartProfile(
      DataCollection.ProfileLevel.DC_LEVEL_GLOBAL,
      DataCollection.PROFILE_CURRENTID);

x = a.OnlyProfileThis();

  DataCollection.StopProfile(
      DataCollection.ProfileLevel.DC_LEVEL_GLOBAL,
      DataCollection.PROFILE_CURRENTID);

Console.WriteLine("2 doubled is {0}", x);

}
}

We still need to instrument the application as normal. We also have one additional step. When running the code above, data collection will be enabled by default, so the API won't appear to do anything beyond stopping data collection after the call to OnlyProfileThis. We need to disable data collection before running the application. The profiler control tool,
VSPerfCmd, has options to do this.

To run this scenario from the command line:

csc /debug- /r:Microsoft.VisualStudio.Profiler.dll Sample.cs
VSInstr Sample.exe

VSPerfCmd /start:trace /output:SampleTrace.VSP
VSPerfCmd /globaloff
Sample.exe
VSPerfCmd /shutdown

VSPerfReport /calltrace SampleTrace.VSP

There are several control functions in the DataCollection class. Here is a brief rundown.

StartProfile(ProfileLevel level, UInt32 id)
StopProfile(ProfileLevel level, UInt32 id)

This unconditionally starts or stops profiling. The level parameter indicates whether profiling should be stopped for the current thread, process, or globally. For thread and process level changes, the id specifies thread id or process id to control (respectively), or PROFILE_CURRENTID to control the current thread or process.

SuspendProfile(ProfileLevel level, UInt32 id)
ResumeProfile(ProfileLevel level, UInt32 id)

This works very much like StartProfile and StopProfile, however, calls to these functions are reference counted. If you call SuspendProfile twice, you must call ResumeProfile twice to enable profiling.

MarkProfile(Int32 markId)
CommentMarkProfile(Int32 markId, String comment)
CommentMarkAtProfile(Int64 timeStamp, Int32 markId, String comment)

Inserts a 32-bit data value into the collection stream. Optionally, you can include a comment. With the last function, the mark can be inserted at a specific time stamp. The id value and the optional comment will appear in the CallTrace report.

The profiler control tool provides similar functionality through a command line interface, though the notion of a "current" process or thread id is obviously not relevant.

VSPerfCmd /?

[...]

-GLOBALON Sets the global Start/Stop count to one (starts profiling).

-GLOBALOFF Sets the global Start/Stop count to zero (stops profiling).

-PROCESSON:pid Sets the Start/Stop count to one for the given process.

-PROCESSOFF:pid Sets the Start/Stop count to zero for the given process.

-THREADON:tid Sets the Start/Stop count to one for the given thread.
Valid only in TRACE mode.

-THREADOFF:tid Sets the Start/Stop count to zero for the given thread.
Valid only in TRACE mode.

-MARK:marknum[,marktext]
                Inserts a mark into the global event stream, with
                optional text.

[...]

If you find yourself buried under a ton of trace data, investigate the profiling API to help focus on the important parts of your application.

Trying out the profiler on the VPC

AngryRichard — Fri, 06 Aug 2004 21:19:00 GMT

I've frequently heard the question asked, "Can I use the profiler on a Virtual PC?" It has even come up on the blog feedback a few times. My answer has always been, "Theoretically, yes." I didn't want to post this answer externally until I'd actually gotten around to trying it myself.

I've finally been nagged into it.

In my limited experience with our VirtualPC product, it has quite impressed me with its functionality. However, it does not emulate the hardware performance counters upon which the profiler implicitly depends. For this reason, you can not run the sampling profiler using a performance counter based interrupt. My collegue Ishai pointed out^* that you should be able to use page-fault or system-call based sampling, but the VPC has a different problem with these modes that is still under investigation.

Instrumentation based profiling will work on the VPC. However, as I've already mentioned, there is a bug check issue with the driver when it unloads. Fortunately, instrumentation based profiling doesn't rely on the presence of the driver.

By renaming the driver to prevent the profiling monitor from installing it at startup, I was able to use instrumentation based profiling on the VPC. This is obviously just a workaround, but I hope this will allow you to investigate some of our tools in the comfort of a VirtualPC environment.

Here is how you can prevent the driver from loading on your VPC installation.

Use regedit to delete the key HKLM\CurrentControlSet\Services\VSPerfDrv.
Find C:\Program Files\Microsoft Visual Studio\Enterprise Developer Tools\Performance Tools\VSPerfDrv.sys, and rename it to something silly like __VSPerfDrv.sys__, or delete it.
Reboot your VPC to ensure that the driver isn't installed

Happy hunting!

^*I'm pretty sure Ishai was hired to point out things I do wrong. Fortunately, he usually points out solutions as well.

VPC and the BSOD (part 2)

AngryRichard — Fri, 04 Jun 2004 03:00:00 GMT

I had a nice long email chat with members of the Virtual PC team.

The good news: The Virtual PC emulates the host processor well enough that our kernel-mode driver can detect what features are enabled.

The bad news: The Virtual PC does not emulate an APIC or performance counters.

So, if you were planning on running the profiler inside a Virtual PC, the best you can hope to do is get function trace data on an instrumented app. Sampling will not work at all, and collecting perfomance counter data in the instrumentation will fault the application.

Bummer. I wish I had better news.

VPC and the BSOD

AngryRichard — Wed, 02 Jun 2004 03:14:00 GMT

I'm so pleased. Someone did something exciting and dangerous with the profiler. In case you're not reading the newsgroups, an intrepid customer tried to profile on a Virtual PC, and discovered that it only leads to pain and misery via the BSOD.

So don't do that.

Seriously, is this something people want to do? I mean, VPC is about the coolest thing ever, but we do use hardware performance counters by default, and VPC is not exactly a real life environment for performance analysis and measurement. Still, maybe y'all have good reasons for this.

At the very least, we'll fix that BSOD thing. I mean, how 1990's is that?

This is an excellent time to point out that the profiler does in fact install a kernel-mode device driver in order to play with the hardware counters on your Intel and AMD processors. There are some fun implications from this:

You need to be running as a user who can install a kernel-mode device driver. We currently install the driver dynamically when starting the profiling monitor, and we uninstall it when the monitor shuts down.
The kernel-mode device driver is not signed. If you're one of those cheeky folks who has unsigned drivers shimmed through verifier, you may discover the joy of a forced BSOD if you exercise the profiler in new interesting ways we hadn't thought of. This is not all bad; just make sure you send us the error report after you've rebooted.
If you're running on Intel Pentium I or AMD K6* (or anything older), expect no love from the sampling based profiler, and some uphill battles with instrumentation. I may post hints on this latter point if a few people speak up about legacy hardware.

C'mon, all your Friends are Doing it

AngryRichard — Thu, 27 May 2004 22:15:00 GMT

A bunch of the guys on the team I work for have been starting up blogs. I started feeling left out,
which made me very angry.

It appears all blogs start with "Hi, I'm a developer who does X and I'm going to talk about Y and maybe Z."
It's all part of Microsoft's new image -- we're transparent now.

Transparency's good, right?

Go down to your local German car dealer, or, if you own a newer German car, go out to your driveway. Shiny, pretty, isn't it? Open the hood. Look at that -- a big, fat, intake pipe that goes into a big box that says "BMW" in a 4" Century Gothic font. Cool. Now pop that plastic thing off the top. Go ahead, I dare you. Not so pretty now. Look at all those wires and tubes. Look at all that stuff you could cut yourself on. That's why that plastic thing is there; it makes owning all that power a little less scary.

Some of us spend all our time under the shiny plastic thing, hands full of wires and tubes and spark plugs. That'd be me and some of my less 'transparent' friends, working on the instrumentation and data collection engine in the profiler.

Of course, with great power, comes great potential for disaster. Some day you'll have your turn with the profiler. Trust me, something's always too slow. If it works, great. If you find yourself upside down in a ditch, tell us, we want to know where we need to cover up the sharp edges.