Steven Pratschner's .Net CF WebLog : Diagnostics

The CLRProfiler for the .Net Compact Framework, Part IV: The Call Tree View

stevenpr — Tue, 12 Feb 2008 21:10:23 GMT

This series of posts provides an introduction to using the CLRProfiler for the .Net Compact Framework. In previous posts (part 1, part 2, and part 3) we've looked at various profiler features as we try to solve a performance problem with a sample application. So far, we've learned that our performance problem is due to excessive boxing of value types. We've also learned what type of objects we are boxing and what methods in the sample are causing the boxing to occur.

In some cases, knowing which method is causing the problem is all you need, especially if the method is small and straightforward. However, there are cases where even more detail is needed. In this post I'll use the Call Tree View to determine the exact line of source code that is causing my performance issue!

After you've finished profiling, you can bring up the Call Tree View from the summary page:

The Call Tree View

The Call Tree View shows every allocation made as a result of every method call. For all but the simplest programs the Call Tree View contains a huge amount of data. Fortunately, navigating through all this data isn't very hard once you get used to a few tricks.

Method calls are shown in black and object allocations are shown in green. For example, the following picture shows the initial state of the Call Tree View. You can see that only one method (the program's Main method) has been called at this outermost level. You can also see that several allocations have already been made. These initial allocations are made by the CLR just before it starts executing a program. As you can see, these initial allocations represent several of the common exception types as well as the initial Application Domain.

Along with the individual method calls and object allocations, the Call Tree View also shows summary statistics such as the number of method calls that result from a given call, the number of bytes and objects allocated by a given call and so on. You can sort the data either by number of calls or by the amount of memory allocated.

As you start to expand the tree, you'll notice that one child node at each level is highlighted in bold. The highlighted node represents the method that has allocated the most memory (or generated the most calls, depending on your sort preference). This is the key to drilling into the data quickly - just follow the highlighted nodes until you find the method you are looking for.

Back to the Example

In previous posts we've established that the objects we are boxing are of type Box.Block and that the boxing is occurring in calls to RotateGameBlocks and InitializeGameBlocks. By drilling through the data in the Call Tree View we can find out exactly which line of code in RotateGameBlocks (or InitializeGameBlocks) is causing the boxing to occur.

The following picture shows the result of following the highlighted notes until I have found the RotateGameBlocks method. Some of the methods in the tree look familiar, like Main, and Application.Run, but many methods don't. Method names that you don't recognize are likely part of NetCF's internal implementation. For example, methods that contain "AGL" in their name are part of NetCF's implementation of Windows Forms.

When looking at the child nodes of RotateGameBlocks, we see a green line representing the allocations of Box.Block. The Objects column tells us that a call to RotateGameBlocks causes 14,400 instances of Box.Block to be allocated. Furthermore, we can see that RotateGameBlocks calls ArrayList.set_Item 14,400 times. The fact that the number of calls to ArrayList.set_Item and the number of instances of Box.Block that are allocated are the same indicates that the calling ArrayList.set_Item is the line in my sample that is causing our boxing to occur.

In my next post I'll describe a set of managed APIs you can use to control profiling programmatically.

Thanks,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights.

The CLRProfiler for the .Net Compact Framework, Part II: Histograms and "Show who Allocated"

stevenpr — Thu, 18 Oct 2007 20:22:31 GMT

Yesterday I started a series of posts on how the use the CLRProfiler for the .Net Compact Framework. The first post contained the basic information you need to get started. I described how to install the profiler, launch an application on the device, and collect profiling data.

In order to direct the discussion, I've written a sample application that exhibits a performance problem that is surprisingly easy to fall into. Throughout these posts I'll show you how to use the profiler to diagnose the problem. To refresh your memory, the sample application is a basic game and the performance problem is that the main windows paints way too slowly.

After I stopped profiling the game in the first post, following summary page was displayed.

In this post I'll use some of the histograms to begin diagnosing our performance problem.

Histograms

The first thing that stands out at me when looking at the summary form is the amount of managed data I'm creating. While profiling the painting portion of my application I generated over 6MB of managed objects. That's clearly way too much for a relatively simple operation like painting my main window. My first step in determining what's going on is to get some basic statistics about the objects my application is using. For example, I'm interested in which objects I'm creating, how many of them there are and how long they live. This data can be obtained by looking at some of the histograms the profiler offers.

I can choose to view a histogram for all objects created as my application ran or only for those objects that were in the GC heap when my application exited. In my scenario I need to look at all objects. If I were to only look at the objects alive at the end of the run I may miss some important trend that occurred earlier on.

Clicking the "Histogram" button next to the "Allocated Bytes" value displays the following graph:

The histogram form has two panes. The pane on the right describes how many instances of each type of object were created and the total size of those instances. The pane on the left graphs type instances by size. The color coding next to the types in the right pane matches the bars in the left pane which show the relative amounts of objects created.

A quick glance at this form helps narrow my suspicions about what's causing my performance issue. As you can see, about 97% of the objects I created were of type Box.Block as indicated by the red box on the right hand pane and the red bar in the left hand pane. I can also see that each instance of Box.Block is relatively small at an average size of 136 bytes (see the right hand pane).

Who Allocated all those Objects?

Now that I know the majority of my objects are instances of Box.Block, I'd like to see where in my application those instances are getting created.

To determine the source of my allocations I can right-click on the bar that represents Box.Block in the histogram and select "Show Who Allocated" (the bar turns black when selected):

Doing so brings up a window referred to as an Allocation Graph:

The Allocation Graph traces the flow of every call that allocated an instance of Box.Block. I typically interpret this graph starting with the rightmost node. This node represents all instances of Box.Block in the system. Stepping back one level to the left we see two nodes representing methods that created instances of Box.Block: Form1.RotateGameBlocks and Form1.InitializeGameBlocks. The data in these nodes tell us that 75% of the Blocks were created in RotateGameBlocks and 25% were created in InitializeGameBlocks. Notice that the width of the lines connecting the nodes represents the percentage of instances that call created.

Now that I know where my objects are coming from I can dig into my code to see what's going on.

In some scenarios, the information we've learned so far may be all that we need to fix the problem. However, there are a few more pieces of data that may be required in some cases. For example, it may be useful to know the times at which Blocks were created and destroyed. Also, if RotateGameBlocks and InitializeGameBlocks are long, complicated methods, we may need to know the exact calls within those methods that caused the allocations. I'll describe how to get this information in future posts.

Thanks,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights

The CLRProfiler for the .Net Compact Framework, Part 1: Getting Started

stevenpr — Thu, 18 Oct 2007 04:28:00 GMT

Version 3.5 of the .Net Compact Framework contains a new memory profiler called the CLRProfiler. The CLRProfiler is a great tool for looking into the details of how your application is allocating and using managed objects. For example, the profiler allows you to look at the contents of the GC heap at any point in time, provides a historical record of what's going on in the heap, let's you see which calls in your application are allocating which objects and so on. This level of detail is often needed to diagnose memory-related issues in your device applications. The Compact Framework's version of the CLRProfiler is an adaptation of the profiler that has been available for the full .Net Framework for some time.

In this series of posts I'll walk you through the primary features of the CLRProfiler for the .Net Compact Framework. The profiler contains numerous ways to analyze data about the GC heap so instead of briefly touching on all of them, I'll go into depth on the views I've found most useful.

We'll be looking at the profiler by way of an example, as I've found that learning a new tool is often easier if you have a specific problem to solve rather than just looking at the tool's features without any context. If you'd like more information on a view that I don't cover here you can read the document that ships with the full Framework's version of the profiler.

Our Sample

The sample I'll use to describe the profiler is the beginnings of a game I started to write using the Compact Framework. All the game currently does is allow you to start a new game and to rotate a set of blocks on the screen. Here's a simple view of the application:

The performance problem I'm having with this game involves my drawing logic. The blocks on the screen draw very slowing. On my Dell Axim I can literally see each column of blocks paint individually. Throughout these posts I'll use the CLRProfiler to figure out what I can do to make my game paint more quickly.

Launching an Application with the CLRProfiler

The CLRProfiler ships in the .Net Compact Framework Power Toys package. After installing the Power Toys you can find the CLRProfiler executable (NetCFClrProfiler.exe) in the bin directory of the .Net SDK. On my machine that directory is c:\Program Files\Microsoft.Net\SDK\CompactFramework\v3.5\bin. The Power Toys setup program also adds a menu item for the profiler to the Windows Start menu.

The main window of the profiler is strikingly simple:

Through this main window you can start and stop applications, take snapshots of the GC heap, and control various profiling options. These options include control over whether profiling is currently active and whether allocations, calls or both are logged.

Clicking the "Start Application…" button displays the following form:

On this form you enter the name of the device you'd like to connect to, the fully qualified path to the device executable you'd like to profile, and any command line parameters to be passed to the application. The CLRProfiler supports profiling applications on devices connected either over ActiveSync or TCP/IP. You can also profile applications running on emulators. The profiler supports the same types of devices that the .Net Compact Framework Performance Monitor does(in fact, the device selection and connectivity mechanisms are shared between the two tools).

In my example I have a device connected over ActiveSync and the name of my application is box.exe. Selecting "Connect" from the launch dialog starts the application on device and begins profiling. At this point just run your application as you normally would. The CLRProfiler causes your application to run much slower than normal. In order to speed up the debugging process you can turn profiling on and off for different sections of your application using the "Profiling active" checkbox on the profiler's main form.

When you're done profiling your application, you can stop it either by selecting the "Kill Application" button on the main form of the CLRProfiler or by just closing the application directly on the device.

The Summary Form

After your application exits the CLRProfiler displays the following summary form:

The summary form provides some general statistics about the use of managed memory as your application ran:

Heap Statistics. The statistics displayed in this group box describe the total size of the objects in the managed heap. The "Allocated Bytes" value counts the total size of allocations made as the application was running. When the Compact Framework garbage collector detects significant fragmentation in the heap, it will compact it. The "Relocated Bytes" value shows how many bytes the garbage collector moved around during compaction. "Final Heap Bytes" shows the size of the managed heap when the application exited and "Objects Finalized" is what you'd expect: the number of objects that had finalizers to run. Note that the Heap Statistics group also provides a count of critical finalizers run. The .Net Compact Framework doesn't have the notion of critical finalization so this value will always be 0.

There are several buttons in this box that launch viewers that allow you to analyze data in various ways. You can view data in histograms based on object size, age, address and so on. I'll describe most of these views in detail in subsequent posts.

Garbage Collection Statistics. This group box tells you how many garbage collections occurred while your application was being profiled and how many of those collections were induced by calls to GC.Collect. The "Timeline" button enables you to see how the contents of the GC heap changes over time. I think this is one of the coolest views. I'll describe it in great detail in a subsequent post.

GC Handle Statistics. The GC Handles group box shows you how many handles were created and destroyed while your application ran. Keep in mind that the Compact Framework CLR creates GCHandles under the covers as it executes your application so all of the handles you see here aren't likely to have been created explicitly by you.

Profiling Statistics. The CLRProfiler enables you to take snapshots of the GC heap as your application is running. I didn't do this while profiling my application, but if I would have I would be able to choose a snapshot from the dropdown and view it.

Getting back to our example, there are several pieces of data on this form that concern me, or are unexpected. The key to analyzing performance data such as this is not just looking at the raw values, but in interpreting the values in the context of what your application was doing as it was being profiled. For example, I profiled the time my application spent painting, yet I see that 5 garbage collections occurred and I created over 6 MB worth of objects. This data leads me to believe that I'm making the garbage collector work harder than it should for my scenario. Clearly there shouldn't be so much activity going on in the managed heap while I'm painting boxes on the screen!

In my next post, we'll use some of the histogram views to see how many objects my application is creating and of what type they are.

Thanks,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights.

.Net Compact Framework: “Hey, what happened to the diagnostic tools?”

stevenpr — Thu, 13 Sep 2007 03:33:00 GMT

If you’ve installed the Beta2 version of Orcas you may have noticed that the NetCF diagnostic tools (RPM, CLRProfler, ...) are missing. Don’t worry, these haven’t been cut from Orcas, they will just be distributed via the web in a separate “power toys” pack. A CTP of these tools is now available at: http://www.microsoft.com/downloads/details.aspx?familyid=C8174C14-A27D-4148-BF01-86C2E0953EAB&displaylang=en . This CTP works with Orcas Beta2.

We intend to distribute the final Power Toys release at the same time that Orcas ships.

Thanks,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights.

Using the .NetCF Remote Performance Monitor to find memory leaks: A real world example

stevenpr — Fri, 23 Mar 2007 20:32:00 GMT

A few weeks ago I posted an entry describing how to use the .Net Compact Framework Remote Performance Monitor to find managed memory leaks. The other day I ran across a post from Rabi Satter describing how he used to tool to solve a critical leak for one of his customers. His real world experience provides a much more complete memory leak example than my original post did. Check it out: . http://www.satter.org/2007/03/thank_god_for_c.html

Thanks,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights.

Using the .Net Compact Framework Remote Performance Monitor to Optimize your application's memory usage

stevenpr — Wed, 03 Jan 2007 02:51:00 GMT

The November issue of .Net Developers Journal includes a new article on using the Remote Performance Montior. In the article I describe how to interpret the various memory-related counters to optimize how your Compact Framework application uses memory on the device.

Thanks,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights.

Auto-Deployment of Remote Performance Monitor Device-side Components

stevenpr — Wed, 01 Nov 2006 02:35:00 GMT

In my earlier post on using the .Net Compact Framework Remote Performance Monitor, I describe that you must manually copy two files (netcfrtl.dll and netcflaunch.exe) to your device before you can launch an application to receive performance statistics.

Chris Tacke of OpenNetCF.org recently got tired of this manual process and wrote a program you can use to automate the deployment of these two files. Check out: http://blog.opennetcf.org/ctacke/PermaLink,guid,793b6c03-496c-4187-a27c-bf369c67d713.aspx

Thanks for your help Chris,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights

Analyzing Device Application Performance with the .Net Compact Framework Remote Performance Monitor

stevenpr — Mon, 17 Apr 2006 20:15:00 GMT

Service Pack 1 of the .Net Compact Framework version 2 (see http://blogs.msdn.com/netcfteam/archive/2006/04/21/580901.aspx) includes a new utility called the .Net Compact Framework Remote Performance Monitor (RPM). The RPM helps you diagnose performance problems in your application by providing a dynamic, graphical view of various runtime performance statistics. The performance statistics displayed by the RPM are those that are present in the ".stat" files in previous versions of the Compact Framework. See David Kline's blog entry "Monitoring Application Performance..." for a thorough description of all the counters viewable through the RPM.

You have two choices when viewing data using RPM. First, you can view the counters in a textual table format using the RPM tool itself, or you can view all the data the RPM gathers using the standard Windows Performance Monitor.

This post provides instructions for installing and using the RPM. This tool is a big step forward for .Net Compact Framework developers. For the first time, you'll be able to to use standard performance tools to gain insight into your application's performance as it runs!

Installation

The .Net CF Remote Performance Monitor includes some files that reside on the desktop machine and some files that must be present on the device.

Installing the desktop components is easy - just run the SP1 setup program. After the installation completes, the RPM executable (netcfrpm.exe) is placed in the bin directory of Compact Framework SDK. On my machine, this directory is C:\Program Files\Microsoft.NET\SDK\CompactFramework\v2.0\bin.

Installing the device-side components involves manually copying two files from the desktop machine to the device. The SP1 setup program places the device-side files in the same directory as the cab file that matches your processor type and operation system version. I have a Pocket PC 2003 SE device, so my device-side components are installed in C:\Program Files\Microsoft.NET\SDK\CompactFramework\v2.0\WindowsCE\wce400\armv4. The two files you must copy are netcfrtl.dll and netcflaunch.exe. Both of these files must be copied to the \windows directory of your device.

Installation Notes for Windows Mobile 5.0 Devices

There are two issues you may run into when installing the device-side RPM components on Windows Mobile 5.0 devices.

- Depending on the security configuration chosen by the device manufacturer, you may see a security prompt on the device the first time you launch the RPM. This prompt appears because netcfrpm.dll is not signed.

- An additional installation step is necessary on Windows Mobile 5.0 devices to provision the device so the RPM can run. Provisioning involves copying the following XML text into a file and using the rapiconfig utility to send the XML file to the device.

- <wap-provisioningdoc>
   - <characteristic type="Metabase">
      - <characteristic type="RAPI\Windows\netcfrtl.dll\*">
         <parm name="rw-access" value="3" />
         <parm name="access-role" value="152" />
         
      </characteristic>
   </characteristic>
</wap-provisioningdoc>

For example, if you pasted the above XML text into a file named rpmprov.xml you'd issue the following command from your desktop machine to provision your device:

rapiconfig /p rpmprov.xml

Launching the Remote Performance Monitor

To view dynamic performance statistics for your application, launch netcfrpm.exe from the desktop machine and select the "Live Counters...." option under the File menu. Doing so displays the following window:

Before you can view live statistics, you must connect the RPM to your device and specify the application you'd like to run. The instructions for connecting the RPM to your device vary based on whether the connection between your desktop machine and your device is via Active Sync or an Ethernet connection. If your connection is over ActiveSync, your device will automatically appear in the Device drop down in the RPM user interface. If your desktop machine is connected to the device through a direct network connection, you must know the IP address and port number through which the RPM can connect to your device. This information can be obtained by running netcflauch.exe from the \windows directory of your device. Running netcflaunch.exe displays the following:

Using the data displayed by netcflaunch.exe, enter the IP address and port number in format <IPAddress><single space><port number> in the Device drop down in the RPM user interface.

Before connecting to the device, you must specify the application you'd like to monitor using the Application text box. Be sure to type the fully qualified path name to the executable you wish to run on the device. For example, \Program Files\poomcominterop\poomcominterop.exe. If your applications takes command line parameters, you can specify them in the Parameters text box. If you need to specify more than one parameter, be sure to separate them with a space.

After you've selected the device to connect to, and specified the application to launch, click the Connect button in the lower right corner of RPM's user interface. The RPM will send a command to the device to remotely launch the application and start pulling back performance statistics. RPM displays the statistics in tabular form as shown in the following picture:

Again, a description of the various counters can be found on David Kline's blog.

Viewing Performance Statistics using Perfmon

While viewing performance statistics in RPM is useful, it's often easier to spot performance trends by viewing the data in a graphical form. The RPM enables you to view the statistics graphically by sending all the data it collects to the Windows Performance Monitor. After opening performance monitor, select the "Add Counter" option. In the dialog that is displayed you'll see counters corresponding to each category of performance statistic gathered by the RPM as show below:

After selecting the counters you'd like to view, the values for those counters appear as a line in the Perfmon graph. You may need to adjust the scale Perfmon uses to display a counter depending on the expected range of values. In the picture below I've chosen to view counters representing the size of the GC heap and the total number of bytes allocated while my application runs.

Other RPM Features

In addition to viewing performance statistics, the RPM also allows you to set various configuration options for your device. Selecting the Logging Options item from the Device menu displays the following dialog:

As you can see, the RPM allows you to configure the various types of logging done by the Compact Framework and to put your device into a state in which you can attach the debugger to any managed process. The following links provide more information about the various logging options:

Loader logging: http://blogs.msdn.com/stevenpr/archive/2005/02/28/381744.aspx
Interop logging: http://blogs.msdn.com/stevenpr/archive/2005/06/22/431612.aspx

Thanks,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights.

Diagnosing Marshaling Errors using Interop Logging

stevenpr — Wed, 22 Jun 2005 20:46:00 GMT

In a previous post I described how the Loader Log can be used to help diagnose problems such as assembly load failures, version mismatches and so on (http://blogs.msdn.com/stevenpr/archive/2005/02/28/381744.aspx ). Another critical element of our new diagnostics feature set in Whidbey is Interop Logging.

Interop logging was added in response to the huge number of questions and support calls we’ve seen over the last few years related to PInvoke. If you’ve used PInvoke to call a native function from your managed application, you’ve likely seen how easy it is to get the managed definition of your native function incorrect, leading the marshaling errors at runtime. In fact, the difficulty in matching native and managed definitions is a significant enough issue that an entire site now exists to help you get this right. Check out www.pinvoke.net for the managed definitions of many of the Win32 APIs. Help with both WindowsCE and “desktop” definitions is available.

In this post, I’ll show you how our new Interop logging feature makes it much easier to diagnose errors encountered when using PInvoke or COM interop to call native code from your .Net Compact Framework application. In many ways, Interop logging is the Compact Framework’s equivalent to the Interop Customer Debug Probe (aka Managed Debug Assistant) offered by the full .Net Framework (http://blogs.msdn.com/adam_nathan/archive/2003/05/13/56680.aspx ).

Enabling the Interop Log

Interop logging is enabled and configured by setting a series of registry keys. At a minimum, you must set two keys: one to enable the CLR’s overall logging feature and one to specifically enable interop logging.

To enable CLR logging, set the following DWORD value to 1:

HKLM\Software\Microsoft\.NETCompactFramework\Diagnostics\Logging\Enabled

To specifically enable interop logging, the following DWORD value must be set to 1:

HKLM\Software\Microsoft\.NETCompactFramework\Diagnostics\Logging\Interop\Enabled

When these two keys are set, the CLR will create a file named netcf_Interop.log in the same directory as your application.

There are a few other values you can set to customize how the log file is named and how eagerly it is written to. If you saw my earlier post on Loader logging, these values should look familiar as they apply to all logging done by the Compact Framework. These values (all DWORDs) are:

· HKLM\Software\Microsoft\.NETCompactFramework\Diagnostics\Logging\UseApp

The UseApp value causes the name of the application to be included in the name of the log file. For example, if I’m running an application called BankClient.exe, setting the UseApp value to 1 will cause the log file to be named netcf_BankClient_Interop.log. The UseApp value is useful if you enable logging and have several applications installed in the same directory.

· HKLM\Software\Microsoft\.NETCompactFramework\Diagnostics\Logging\UsePid

The UsePid value causes the process’s ID to be included in the name of the log file. As with UseApp, the UsePid value is useful if you enable logging and have several applications installed in the same directory.

· HKLM\Software\Microsoft\.NETCompactFramework\Diagnostics\Logging\Flush

Setting the Flush value to 1 will cause the CLR to write log events to the log file as they occur instead of buffering them up. Flushing eagerly negatively affects performance, but it can be useful in case you’re not getting the data you need in the log file because of application crashes or other errors.

What’s in the Interop Log?

The Interop log contains two primary pieces of information:

· The managed and native function signatures for all calls that cross an interop boundary. That is, both calls from managed code to native code and calls from native back into managed.

· Errors returned to native callers from managed code. This category of logging is particularly useful when diagnosing errors you get when calling a managed object from native code using Com Interop.

In addition to these two general categories of information, the Interop log also contains data about scenarios the Compact Framework doesn’t support which would otherwise be very hard to diagnose. For example, interfaces implemented in managed code cannot be called from Com if the interface contains any generic methods. Trying to call methods on such an interface will generate error text in the Interop log.

The Interop log contains three lines for every call that uses either Pinvoke or Com Interop. Here’s an example:

[pinvokeimpl][preservesig]

int Sample.Form1:: NativeFunction (int , string);

int (I4_VAL) NativeFunction (int (I4_VAL) , WCHAR * (STRING_LPWSTR) );

The first line is a set of flags that describes the call. In the example above, the [pinvokeimpl] flag indicates that this call is a PInvoke. A value of [delegate] indicates a call from native code to managed code through a delegate that was marshaled to native code as a pointer to a function. The absence of a value altogether signifies a call through Com Interop.

The second line is the managed definition of the native function being called. In this case, you can see that the native function was declared in managed code as taking two parameters, an int and a string, and returning an int.

The third line is the native definition of the function as understood by the Compact Framework. In this case, we can see that the managed string argument is being marshaled to native code as a pointer to a Unicode string, for example.

For calls that include error information, the text describing the error, including the error value is written either just before or just after the call information.

Diagnosing Marshaling Errors

The native function signature (the 3^rd line from our description above) is the most useful when diagnosing marshaling errors that occur when you are calling native code either through PInvoke or Com Interop. The Compact Framework does not have intrinsic knowledge of the native function you are calling – it simply takes the managed definition you’ve provided and creates an equivalent native signature. If this signature doesn’t match the actual signature of the targeted native function, a marshaling error will occur. These errors are fixed by changing your managed definition to match the actual native function signature.

Here’s an example. Let’s assume we’re using PInvoke to call a native function with the following signature (in C):

int NativeFunction(long value);

In managed code, we declare NativeFunction as follows:

[DllImport("Sample.dll")]

static extern int NativeFunction(long value);

Calling NativeFunction from managed code, gives us the following entries in the Interop log:

[pinvokeimpl][preservesig]

int Sample.Form1:: NativeFunction (long);

int (I4_VAL) NativeFunction (INT64 (I8_VAL) );

Can you see the marshaling error? The C definition of NativeFunction takes a single long as a parameter. Variables of type long is C/C++ are 32 bits (I’m on x86). However, the long type in C# is 64 bits. By looking at the 3^rd line of our Interop log, we can see that the .Net Compact Framework is marshaling the long parameter as a 64 bit value, when the native function is expecting only a 32 bit value. In this case, an extra 4 bytes will be pushed on the stack as the call is made.

Fixing this problem involves defining the parameter to NativeFunction as an int in managed code like so:

[DllImport("Sample.dll")]

static extern int NativeFunction(int value);

Now everything lines up at call time according to the interop log:

[pinvokeimpl][preservesig]

int Sample.Form1:: NativeFunction (long);

int (I4_VAL) NativeFunction (int (I4_VAL) );

Even with this simple example it’s easy to see how easy it is to make mistakes when writing the managed definitions of unmanaged functions. These problems are easier to diagnose now that the interop log tells you exactly how the data is being marshaled.

COM Error Information

As described, the other major purpose of the Interop log is to help diagnose errors returned from calls made through COM interop. This is particularly useful when calling from native code into managed code mostly because the Compact Framework implements several COM interfaces on your object’s behalf.

The following log entries show the result of a failed called to QueryInterface on a managed object. As you can see, the log includes the GUID of the interface that the Compact Framework couldn’t return.

[no flags]

void BankObjectsLib.IAccount::Withdraw(BankObjectsLib.IAccount , int );

HRESULT Withdraw(IAccount *(INTF_VAL) this, int (I4_VAL) );

ERROR: CCW IUnknown::QueryInterface Error: 0x80004002 {06FAF1B7-66B8-4FFC-9396-A0FD3C01E75C}

This case is perhaps oversimplified if you own the native code that is making the call. In that case, you presumably know which interface you are querying for can just check the HRESULT in code. However, this logging really pays off in situations where you are using a COM object or ActiveX control that you don’t own. In scenarios like this, it’s not uncommon to find the object querying for interfaces or using other COM features that you didn’t expect.

Logging information about COM errors also helps you in case you attempt to use a feature that the Compact Framework doesn’t support. For example, we don’t support the ability to get type information by calling IDispatch::GetTypeInfo. Attempts to use GetTypeInfo will be clearly displayed in the Interop log. By the way, I’m currently gathering a list of all such unsupported features and hope to post it soon.

Comment or email with questions as always. Thanks,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights.

Better Diagnostics in Whidbey: Loader Logging

stevenpr — Mon, 28 Feb 2005 18:51:00 GMT

If you’ve spent much time developing applications with version 1 of the .Net Compact Framework, you’ve likely been frustrated at times by the lack of tools to help you diagnose problems in your applications. Even common errors, like the failure to find an assembly are hard to diagnose, much less more complex problems like memory leaks or performance issues. Admittedly, the lack of a good diagnostics infrastructure in version 1 of our product has been one of the greatest sources of customer frustration and support cost. Fortunately, we’ve listened to the feedback and have invested a significant amount of time and resources in version 2 to help the situation. Some of the new features you’ll find in version 2 that make it easier to understand what’s going on in a running application include:

Stack traces for all exceptions,
Descriptive error messages for “common” exceptions, such as the failure to load a type, failure to find a PInvoke DLL and so on,
More performance counters (see http://blogs.msdn.com/scottholden/archive/2004/12/28/339007.aspx ),
Remote performance monitoring (see http://blogs.msdn.com/scottholden/archive/2004/12/27/332998.aspx ),
Logs for understanding how types are marshaled between managed and native code (and vice versa),
Logs for network traffic, and
Loader logging.

In this post, I’m going to focus on how the new loader logging feature can help you diagnose failures encountered by the CLR’s class loader. As we’ll see, loader logging helps you determine why assemblies couldn’t be loaded, why types couldn’t be found and so on. It also provides information about the version of .Net CF used to run your application and whether or not your application is being run in “compatibility mode” (see http://blogs.msdn.com/stevenpr/archive/2004/12/30/344540.aspx for background on our approach to application compatibility).

Enabling the Loader Log

Loader logging is enabled and configured by setting a series of registry keys. At a minimum, you must set two keys: one to enable the CLR’s overall logging feature and one to specifically enable loader logging.

To enable CLR logging, set the following DWORD value to 1:

HKLM\Software\Microsoft\.NETCompactFramework\Diagnostics\Logging\Enabled

To specifically enable loader logging, the following DWORD value must be set to 1:

HKLM\Software\Microsoft\.NETCompactFramework\Diagnostics\Logging\Loader\Enabled

When these two keys are set, the CLR will create a file named netcf_loader.log in the same directory as your application.

There are a few other values you can set to customize how the log file is named and how eagerly it is written to. These values (all DWORDs) are:

· HKLM\Software\Microsoft\.NETCompactFramework\Diagnostics\Logging\UseApp

The UseApp value causes the name of the application to be included in the name of the log file. For example, if I’m running an application called VW.exe, setting the UseApp value to 1 will cause the log file to be named netcf_VW_loader.log. The UseApp value is useful if you enable loader logging and have several applications installed in the same directory.

· HKLM\Software\Microsoft\.NETCompactFramework\Diagnostics\Logging\UsePid

The UsePid value causes the process’s ID to be included in the name of the log file. As with UseApp, the UsePid value is useful if you enable loader logging and have several applications installed in the same directory.

· HKLM\Software\Microsoft\.NETCompactFramework\Diagnostics\Logging\Flush

What’s in the Loader Log?

Now that we’ve seen how to enable loader logging, let’s take a look at the type of information the CLR’s class loader writes to the log files. Loader logs consist of two sections: a header and a body.

The header of the log file contains the following data:

The name of the application’s main executable
The process ID as assigned by the operating system
The date and time the log file was created
The version of .Net CF used to run the application
Information about the platform on which your application is running

The data in the body of the log file includes descriptive information about each assembly as it is loaded, as well as diagnostic data to help you find errors encountered by the class loader as your application is executed. Specifically, the body of the log file contains:

· The “coercion” state. That is, whether or not your application was run in backwards compatibility mode.

· Tracing for each assembly load, including where the assembly was loaded from and which version was loaded.

· The trust level assigned to each module as it is loaded

· Whether or not the CLR found a configuration file associated with your application

· Failures to find methods, types, assemblies and modules.

· Failures to find a PInvoke DLL or a function within a PInvoke DLL

Here’s a sample log file from the run of an application that encountered no errors (I’ve added the line numbers for the line-by-line summary that follows the sample). This sample gives us a good chance to get a general idea of the type of information logged by the CLR class loader:

========================================

1 | Process [\Program Files\VW\VW.exe]

2 | ProcessID [0x4d9585d2]

3 | Date [2005/02/25] (yyyy/mm/dd)

4 | Time [18:33:14] (hh:mm:ss)

5 | NETCF [2.0.5035.00]

6 | Platform [Windows CE v4.20.1081 (PocketPC) WinCE4ARMV4 release Beta2 ARMV4 IJITv2]

========================================

7 | Compatibility mode [0.0.0.0]

8 | Loading module [\Windows\GAC_mscorlib_v2_0_0_0_cneutral_1.dll]

9 | Loaded [mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=969DB8053D3322AC] from [\Windows\GAC_mscorlib_v2_0_0_0_cneutral_1.dll]

10 | Loading module [\Program Files\VW\VW.exe]

11 | Policy file not present. [\Program Files\VW\VW.exe.config]

12 | Loading module [\Windows\GAC_System.Windows.Forms_v2_0_0_0_cneutral_1.dll]

13 | Loaded [System.Windows.Forms, Version=2.0.0.0, Culture=neutral, PublicKeyToken=969DB8053D3322AC] from [\Windows\GAC_System.Windows.Forms_v2_0_0_0_cneutral_1.dll]

14 | Loading module [\Windows\GAC_System_v2_0_0_0_cneutral_1.dll]

15 | Loaded [System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=969DB8053D3322AC] from [\Windows\GAC_System_v2_0_0_0_cneutral_1.dll]

16 | Loading module [\Windows\GAC_System.Drawing_v2_0_0_0_cneutral_1.dll]

17 | Loaded [System.Drawing, Version=2.0.0.0, Culture=neutral, PublicKeyToken=969DB8053D3322AC] from [\Windows\GAC_System.Drawing_v2_0_0_0_cneutral_1.dll]

18 | Loading module [\Program Files\VW\Cars.dll]

19 | Failed to load [Cars, Version=5.0.0.0, Culture=neutral, PublicKeyToken=null] from GAC

20 | Loaded [Cars, Version=5.0.0.0, Culture=neutral, PublicKeyToken=null] from [\Program Files\VW\Cars.dll]

21 | Loading module [\Windows\GAC_SUVs_v10_0_0_0_cneutral_1.dll]

22 | Loaded [SUVs, Version=10.0.0.0, Culture=neutral, PublicKeyToken=1EF9E65A8D4F2CD6] from [\Windows\GAC_SUVs_v10_0_0_0_cneutral_1.dll]

==================(end)=================

Lines 1-6 are the header of the loader log.

Line 7 states that the application is not being run in compatibility mode. If this application were built with .Net CF version 1.0 and was run by .Net CF version 2.0, the version number displayed in line 7 would be 1.0.0.0. Again, refer to http://blogs.msdn.com/stevenpr/archive/2004/12/30/344540.aspx for a description of how compatibility mode works in .Net CF 2.0.

Lines 8 and 9 show the CLR loading the mscorlib assembly. As you can infer from this trace, the CLR loads mscorlib before loading any other assemblies. The loader log has (at least) two lines for every assembly that is loaded while the application is running. These two lines describe identity of the assembly the CLR is looking for and the location from which that assembly was loaded.

Line 10 shows the name of the module containing the application’s main executable.

Line 11 states that the CLR could not find a configuration file for this application.

Lines 12 and 13 show the CLR loading the System.Windows.Forms assembly from the GAC.

Lines 14 and 15 show the CLR loading the System assembly from the GAC.

Lines 16 and 17 show the CLR loading the System.Drawing assembly from the GAC.

Lines 18, 19 and 20 show the CLR loading an application assembly called Cars. In this case, the log shows that the CLR loaded Cars out of the application directory after failing to find it in the GAC. By the way, searching the GAC is extraneous in this case because Cars is not strongly named.

Lines 21 and 22 show the CLR loading an application assembly called SUVs. As the log shows, the CLR found SUVs in the GAC.

As described, the primary goal of the loader log is to help you diagnose failures that occur while running your application. The following two examples show how the loader log makes it easy for you to figure out what’s happening when errors are encountered. This first example shows the log entries that are written when the loader fails to find an assembly:

Loading module [\Program Files\VW\Golf.dll]

Attempt to load [\Program Files\VW\Golf.dll] has failed (err 0x80001000).

Loading module [\Program Files\VW\Golf.exe]

Attempt to load [\Program Files\VW\Golf.exe] has failed (err 0x80001000).

Failed to load [Golf, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null]

As you can see, the loader logs the identity of the assembly it is looking for, along with the names of the files and locations it probed while looking for the assembly.

This example shows the log entries that are written when the loader fails to find a specific type:

Missing Type. Type [Cars.Jetta], Assembly [Cars].

Missing Type. Class [Cars.Jetta], Assembly [Cars, Version=5.0.0.0, Culture=neutral, PublicKeyToken=null].

As expected, the CLR logs the name of the type it couldn’t find along with the assembly in which it expected the type to be found.

We hope that the data provided in the loader log will make it much easier to help debug applications running on .Net CF 2.0. As described, the loader log is just one of the many new debugging tools we’re providing in Whidbey. As always, feedback is appreciated. If there are specific loader-related errors you repeatedly see that we’re not logging, be sure and let us know.

Thanks,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights.