Steven Pratschner's .Net CF WebLog

The .Net Compact Framework Configuration Tool

2008-05-14T00:24:51Z

The .Net Compact Framework 3.5 Power Toys include a new utility called the NetCF Configuration Tool.

The Configuration Tool is a diagnostic tool. You won't typically use it in the course of everyday development, but it will come in handy for tasks like diagnosing failures related to device configuration and authoring configuration files.

While the tool isn't targeted at your application's end users, they will be able to use it with your help should you need them to provide you with information needed to debug a problem remotely. For this reason, the Configuration Tool runs directly on the mobile device instead of on a desktop machine connected to a device.

After describing how to install the tool, I'll discuss its four main functional areas:

Displaying which versions of NetCF are installed
Displaying the contents of the Global Assembly Cache
Authoring device.config files
Authoring application configuration files

Installing the Configuration Tool

The Configuration Tool isn't automatically installed on the device when you install the Power Toys. Instead, you must copy it to your device manually.

Fortunately there's only one file to copy: NetCFcfg.exe. This file is OS and processor-specific so you'll find it under the appropriate WindowsCE directory in the .Net Compact Framework SDK. I'm running a WindowsMobile 5.0 device, so the Configuration Tool executable for my device is in: C:\Program Files\Microsoft.NET\SDK\CompactFramework\v3.5\WindowsCE\wce500\armv4i on my desktop machine.

It doesn't matter which directory you put the file in on your device. I usually put it in the \windows directory but you can deploy the file to any location you please.

The About tab

The "About tab" describes which versions of NetCF are installed on a device. This is the tab that is displayed when you first launch the Configuration Tool:

It's not unusual to have more than one version of NetCF installed on a device. Most all Windows Mobile and WindowsCE devices come with a version of NetCF in ROM. Another version is often installed in RAM to support an application that requires a newer version of NetCF than the version that came with the device.

When multiple versions of NetCF are installed, it's not always obvious which version is being used to run your application. I've seen several cases where confusion results because an application is running with a different version than expected (see "promoting an application" for a general description of the rules used to determine which version of NetCF will be used to run an application). For example, remote tools like the CLRProfiler and Remote Performance Monitor launch an application on device in order to profile it or gather other performance statistics. If your application is launched with a version of NetCF other than the one the tool is expecting, the application may start but the tool won't be able to gather the diagnostic information it is looking for.

Knowing which versions of NetCF are installed on the device, which version was used to build your application, and the rules used to determine which version will be used to run your application can help you determine whether any unexpected behavior you are seeing results from the "wrong version of NetCF" problem.

The GAC tab

The GAC tab lists the assemblies installed in the Global Assembly Cache:

It helps to know the contents of the GAC when diagnosing assembly load failures. It may be you think an assembly is in the GAC when it isn't, or NetCF may load an assembly from the GAC when you don't expect it to. If you encounter a failure to load an assembly, use Loader Logging to find out why, then check the contents of the GAC using the Configuration Tool if the failure looks related to the GAC.

The Device Policy tab

In a previous post I described how to use device.config to cause all applications on a device to run with a given version of NetCF. The Configuration Tool enables you to edit device.config using a GUI rather than having to modify the XML by hand.

The Device Policy tab let's you choose a version of NetCF to run all "unconfigured" applications on the device. By "unconfigured" I mean all applications that do not have an application configuration file that contains a supportedRuntime element.

In addition to the installed versions of NetCF, you can also select the value "Default". Doing so will remove all supportedRuntime elements from device.config causing the device to revert to its default behavior for selecting a runtime.

Keep in mind that not all values you select on this tab are "valid" for all applications. For example, if I chose 1.0.4292 in the example above, no applications built with later versions of NetCF would run (unless they had configurations files of their own that overrode the device-wide setting).

The Application Policy tab

In addition to specifying device-wide policy using the Device Policy tab, you can also specify which version of NetCF should be used to run a specific application by using the Application Policy tab.

The top combo box on the Application Policy tab is pre-populated with all NetCF applications that are installed on the device. After selecting an application, use the lower combo box to select a version of NetCF to run the application. Under the covers this dialog is adding (or removing) supportedRuntime elements from your application's configuration file.

Any value chosen using the Application Policy tab overrides values chosen using the Device Policy tab.

Thanks,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights.

The CLR Profiler for the .Net Compact Framework Series Index

2008-05-09T00:04:17Z

I've completed all that I had planned to write (at least for now) about how to use the CLRProfiler with NetCF. Here's a brief explanation and a link to each post the series:

Part I, Getting Started. Describes how to install the profiler, launch an application, and begin collecting profiling data.
Part II, Histograms and "Show Who Allocated". The profiler displays various histograms you can use to determine the types and number of objects your application is allocating. You can also determine what methods in your application cause which types to get allocated.
Part III, The Timeline View. The timeline view shows the state of the GC heap over the lifetime of your application.
Part IV, The Call Tree View. Detailed information about every method call and every object allocation is shown in the Call Tree View.
Part V, Controlling the Profiler Programmatically. The CLRProfiler generates a large amount of data and significantly slows down your application. You can programmatically control when the profiler runs using an API that you call from your application. This API also allows you to insert comments into the profile data and to take a snapshot of the GC on demand.

Thanks,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights.

The CLRProfiler for the .Net Compact Framework, Part V: Controlling the profiler programmatically

2008-04-23T03:58:46Z

If you've used the CLRProfiler for NetCF you've probably noticed that your application runs much slower when being profiled. You likely have also seen the huge amount of data the profiler generates, even for relatively simple applications. The time it takes the profiler to run coupled with the volume of data it creates can sometimes make the profiler impractical to use. For example, I've had customers tell me that it can take several minutes to start an application while profiling. If it takes you several more minutes just to navigate to the portion of the application you'd like to profile, it can quickly become extremely frustrating.

The CLRProfiler ships with an additional assembly you can reference and call from your application to control the profiler while your application is running. This assembly, called NetCFCLRProfilerControl contains APIs you can use to start and stop profiling, insert comments into the profiling log, dump the contents of the GC heap on demand, and so on. The ability to control when the profiler collects data enables you to profile only the sections of your application you want to, rather than having the profiler collect data all the time. In this way, your application will start faster and there will be less data to sift through later.

In this post I'll describe how to use the APIs provided by the NetCFCLRProfilerControl.

Using the NetCFCLRProfilerControl Assembly

The following steps are required to use NetCFCLRProfilerControl from your application:

Reference NetCFCLRProfilerControl in your Visual Studio project.
Add code to your application to call the APIs.
Deploy NetCFCLRProfilerControl along with your application.
Start your application using the CLRProfiler.

Referencing NetCFCLRProfilerControl

NetCFCLRProfilerControl is installed on the desktop machine in same directory as the NetCFCLRProfiler executable. On my machine the files are in C:\Program Files\Microsoft.NET\SDK\CompactFramework\v3.5\bin.

You'll need to add a reference from your application to NetCFCLRProfilerControl to use its APIs. If you're using Visual Studio you can add this reference using the standard "Add Reference..." dialog.

Public APIs

NetCFCLRProfilerControl contains one public class called CLRProfilerControl which contains the following public static members:

Member	Notes
bool ProcessIsUnderProfiler { get; }	Returns true if your application was launched using the CLRProfiler. Note that this is independent of whether the profiler is actually logging allocations or calls at the time.
bool AllocationLoggingActive { set; get; }	A boolean property that controls whether the profiler logs managed allocations.
bool CallLoggingActive { set; get; }	A boolean property that controls whether the profiler logs method calls.
void LogWriteLine(string comment) void LogWriteLine(string format, params object[] args)	Enables you to insert comments into the profiling log. These comments show up in the various profiler views to help you narrow down exactly where in your application certain allocations or calls were made. See the example later in this post.
void DumpHeap()	Causes the profiler to display a graphical view of the GC heap. Individual views will also show up on the profiler's summary form when the application exits.

Deploying your application

NetCFCLRProfilerControl is not installed on the device automatically. You'll need to deploy it along with your application. It's probably easiest to deploy NetCFCLRProfilerControl to the same directory as your application but you can also install it in the GAC if you prefer.

Launching your application

The APIs provided by NetCFCLRProfilerControl have no effect if your application isn't started using the CLRProfiler. However, assuming you are going to use the APIs to control when the profiler logs allocations and/or calls, be sure to clear the relevant checkboxes on the profiler's main form before launching your application (otherwise you'll be logging all the stuff you wanted to prevent in the first place!).

After clearing the checkboxes, start your application using the profiler as you always have. The actions of the profiler will now be controlled by your application instead of the CLRProfiler's user interface.

Turning profiling on and off

Assuming you've started your application with the "Profile" checkboxes on the profiler's main form cleared, no logging of allocations or calls will occur. You can turn profiling on using the AllocationLoggingActive and CallLoggingActive properties.

The following example turns allocation logging on just before entering a loop, then turns logging back off when the loop completes:

private void btnCheckOut_Click(object sender, EventArgs e)
{
    CLRProfilerControl.AllocationLoggingActive = true;
    for (int i = 0; i < 20; i++)
    {
        m_cart.ProcessCart();
    }
    CLRProfilerControl.AllocationLoggingActive = false;
}

Inserting comments into the profiling log

The ability to insert comments into the profiling log is a cool feature that enables you to place your own visual markers into some of the profiler's graphical views.

Here's an example that inserts comments denoting the beginning and end of the execution of a loop:

private void btnCheckOut_Click(object sender, EventArgs e)
{
    CLRProfilerControl.LogWriteLine("Before ProcessCart loop");
    for (int i = 0; i < 20; i++)
    {
        m_cart.ProcessCart();
    }
    CLRProfilerControl.LogWriteLine("After ProcessCart loop");
}

After the application exits, your comments show up in several places in the CLRProfiler UI. First, the summary screen includes a button that brings up a window giving the timestamp of each comment. You can use this as a rudimentary timing mechanism in your application:

The comments also show up in various views throughout the UI For example, in the Timeline View, the comments show up as vertical green lines that indicate the time at which the comment was logged. You can see the text of a given comment by hovering the house over its green line. This allows you to see the state of the GC heap between different points of interest as your application ran.

Dumping the GC Heap

The final feature available via NetCFCLRProfilerControl is the ability to take a snapshot of the GC heap on demand. The DumpHeap method causes a heap view to be displayed immediately. You can also access all the heap views taken while the app ran from the Summary View after the application exits.

Thanks,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights.

Promoting all .Net Compact Framework Applications on a device using device.config

2008-03-12T03:01:01Z

By default, a managed device application is run with the version of the .Net Compact Framework it was built with. Only if that version is not present on the device will we "promote" the application to run with the latest installed version. This policy is considered safe and conservative because it isolates applications from unintended compatibility changes that may be introduced in new version of the Framework, at least in the most common scenario.

However, there are times when you may want to run an application with a version of NetCF other than the one it was built with. For example you may find that a newer version of NetCF performs better and is more stable than a previous version so you'd like to take advantage of those improvements without having to rebuild your application. Several previous blog posts (see http://blogs.msdn.com/davidklinems/archive/2005/11/09/491113.aspx) have shown how to do this using an application configuration file. This approach is manageable as long as you only have a few applications to promote. But if you'd like to promote a large number of applications it can quickly become a hassle to author all these configuration files and manage their deployment and updates.

Fortunately, version 3.5 of the Compact Framework lets you promote all applications on a device using a single device-wide configuration file. The syntax of this configuration file is the same as that used in the application configuration file: you specify the version of NetCF you'd like to run all applications with using the supportedRuntime element. The device-wide configuration file must be called device.config and must be located in the \windows directory.

Here's the contents of a sample device.config that will cause all applications to be run with .Net Compact Framework version 3.5:

There's one more case to consider. Suppose you've used device.config to promote all your applications only to discover that one of your applications doesn't work properly. In this scenario you'd want to "roll back" just that one application to the version it was built with. You'd do this by authoring an application configuration file for that application. Any supportedRuntime settings present in application configuration files will override the settings specified in device.config thus giving you a quick way to opt-out of your global policy.

Thanks,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights.

The CLRProfiler for the .Net Compact Framework, Part IV: The Call Tree View

2008-02-12T21:10:23Z

This series of posts provides an introduction to using the CLRProfiler for the .Net Compact Framework. In previous posts (part 1, part 2, and part 3) we've looked at various profiler features as we try to solve a performance problem with a sample application. So far, we've learned that our performance problem is due to excessive boxing of value types. We've also learned what type of objects we are boxing and what methods in the sample are causing the boxing to occur.

In some cases, knowing which method is causing the problem is all you need, especially if the method is small and straightforward. However, there are cases where even more detail is needed. In this post I'll use the Call Tree View to determine the exact line of source code that is causing my performance issue!

After you've finished profiling, you can bring up the Call Tree View from the summary page:

The Call Tree View

The Call Tree View shows every allocation made as a result of every method call. For all but the simplest programs the Call Tree View contains a huge amount of data. Fortunately, navigating through all this data isn't very hard once you get used to a few tricks.

Method calls are shown in black and object allocations are shown in green. For example, the following picture shows the initial state of the Call Tree View. You can see that only one method (the program's Main method) has been called at this outermost level. You can also see that several allocations have already been made. These initial allocations are made by the CLR just before it starts executing a program. As you can see, these initial allocations represent several of the common exception types as well as the initial Application Domain.

Along with the individual method calls and object allocations, the Call Tree View also shows summary statistics such as the number of method calls that result from a given call, the number of bytes and objects allocated by a given call and so on. You can sort the data either by number of calls or by the amount of memory allocated.

As you start to expand the tree, you'll notice that one child node at each level is highlighted in bold. The highlighted node represents the method that has allocated the most memory (or generated the most calls, depending on your sort preference). This is the key to drilling into the data quickly - just follow the highlighted nodes until you find the method you are looking for.

Back to the Example

In previous posts we've established that the objects we are boxing are of type Box.Block and that the boxing is occurring in calls to RotateGameBlocks and InitializeGameBlocks. By drilling through the data in the Call Tree View we can find out exactly which line of code in RotateGameBlocks (or InitializeGameBlocks) is causing the boxing to occur.

The following picture shows the result of following the highlighted notes until I have found the RotateGameBlocks method. Some of the methods in the tree look familiar, like Main, and Application.Run, but many methods don't. Method names that you don't recognize are likely part of NetCF's internal implementation. For example, methods that contain "AGL" in their name are part of NetCF's implementation of Windows Forms.

When looking at the child nodes of RotateGameBlocks, we see a green line representing the allocations of Box.Block. The Objects column tells us that a call to RotateGameBlocks causes 14,400 instances of Box.Block to be allocated. Furthermore, we can see that RotateGameBlocks calls ArrayList.set_Item 14,400 times. The fact that the number of calls to ArrayList.set_Item and the number of instances of Box.Block that are allocated are the same indicates that the calling ArrayList.set_Item is the line in my sample that is causing our boxing to occur.

In my next post I'll describe a set of managed APIs you can use to control profiling programmatically.

Thanks,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights.

PowerToys for the .Net Compact Framework version 3.5 now released!

2007-12-11T02:21:57Z

We've just released the final 3.5 version of the PowerToys for NetCF to the web.

You'll need to install the 3.5 NetCF runtime on your device to use these tools. For now, you can get the correct runtime by installing Visual Studio 2008. In a few weeks we hope to post a standalone install for NetCF 3.5.

The PowerToys release includes all the tools you need to diagnose issues in your .Net device applications, including the Remote Performance Monitor, CLR Profiler, the Logging Utility and more...

Thanks,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights

The CLRProfler for the .Net Compact Framework, Part III: The Timeline View

2007-11-28T22:31:37Z

In the first two parts of this series (part 1 and part 2) I described how to get started using the CLRProfiler and how to use the histograms and the "Show Who Allocated" view to see what types of objects you're allocating and to determine where in your application the allocations are occurring.

In this post I'll talk about my favorite view: the Timeline view. I like the Timeline view because it lets you look at the contents of the GC heap over time. Using this view you can see not only where in your application you are allocating objects, but when. You can also see which objects are collected each time the GC ran. By including a time dimension, the Timeline view combined with "Show Who Allocated" can give you a complete picture of how the use of managed memory varies as your application runs.

After you've completed profiling, you can launch the Timeline view from the Summary page:

The Timeline View

The Timeline view has two panes. The right hand pane is the legend and the left hand pane is the contents of the heap. As with the histograms we looked at in part 2, the colors in the legend describe the types of objects present in the heap. Free space in the heap is always white:

The horizontal axis in this view represents time. If you look at the axis you can see when each GC occurred as indicated by the vertical blue markers. By correlating each GC with the contents of the heap displayed above you can get a sense for which objects were freed in each collection.

Note: You may notice that each GC is labeled with a generation. The Compact Framework's garbage collector doesn't have the notion of generations so these markers are an artifact of the code we ported from the full .Net Framework. You can ignore them.

The vertical axis in the timeline view shows object addresses in the heap. Given that the segments of the GC heap are not necessary contiguous you may seen gaps in these numbers. If you'd like more information about the Compact Framework GC check out the following posts:

Back to the Example

Throughout this series of posts I've been using an example of a game I wrote that paints way too slowly. In part 2 we saw that the vast majority of objects I'm creating are of type Box.Block. Looking that the Timeline view confirms this as can be seen by looking at the legend and all the red in the view of the heap.

The time-based nature of this view allows us to get a few steps deeper into our analysis. First, we can see the points in time at which allocations of our Blocks occurred. In this case, that data isn't very interesting because the view tells me that I was almost continually allocating Blocks.

What's more important in my case is to discover which method in my application was doing the allocations at which point in time. It may be that early on in my application my allocations were coming from MethodA while later on they were coming from MethodB. This time-based analysis can be particularly useful if you'd like to see all that's happening when your application first starts, for example.

To see where your allocations are coming from at a given point in time just select that time in the view. This causes a vertical line to appear at that point in the graph. Then right-click and select "Show Who Allocated" from the context menu:

Doing so brings up the same Allocation Graph we looked at in part 2, except that the graph shows only the allocations done at that particular point in time. In part 2 we saw that my allocations were coming from two methods: InitializeGameBlocks and RotateGameBlocks. By using the timeline view I can see which of those methods was called at which times during my application.

Next time I'll describe the Call Tree view that will show even more detail by highlighting the exact lines within my methods that are causing the spurious allocations to occur.

Thanks,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights

Calling the EnumServices Win32 API from your .Net Compact Framework Application

2007-11-21T04:07:51Z

I was helping a customer use PInvoke to call EnumServices today and got stuck a few times so I thought it may be helpful to post the solution in case anyone else runs into this someday.

EnumServices returns a buffer containing a number of structures of type ServiceEnumInfo that describe basic information about the services on a device.

Each ServiceEnumInfo structure contains an embedded character array that represents the service's prefix and a pointer to a string that represents the name of the dll that implements the service. The dll names corresponding to the structures are laid out in memory just after the structures themselves. So the contents of the buffer you get back from calling EnumServices looks like this (this example is from a device with 3 services):

Here's some sample code that calls EnumServices and loops the buffer pulling out both the prefix name and the dll name for each service:

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Text;
using System.Windows.Forms;
using System.Runtime.InteropServices;
using System.Diagnostics;

namespace EnumServices
{
    public partial class Form1 : Form
    {
        // Managed definition of the native ServiceEnumInfo structure. Here's the corresponding native definition:
        //
        // typedef struct_ServiceEnumInfo {
        //    WCHAR szPrefix[6];
        //    WCHAR szDllName;
        //    HANDLE hServiceHandle;
        //    DWORD dwServiceState;
        // } ServiceEnumInfo;
        //
        struct ServiceEnumInfo
        {
            [MarshalAs(UnmanagedType.ByValTStr, SizeConst=6)] public String prefixName;
            public IntPtr pDllName; // this value is a pointer to the dll name - not the dll name itself.
            public IntPtr hServiceHandle;
            public int dwServiceState;
        }

[DllImport("coredll.dll")]
private static extern int EnumServices(IntPtr pBuffer, ref int numEntries, ref int cbBuf);

        public Form1()
        {
            InitializeComponent();
        }

        private void btnServices_Click(object sender, EventArgs e)
        {
            int numEntries = 0;
            int cbSize = 0;
            int structSize = Marshal.SizeOf(typeof(ServiceEnumInfo));

// call once to get required buffer size
int result = EnumServices(IntPtr.Zero, ref numEntries, ref cbSize);

// alloc a buffer of the correct size
IntPtr pBuffer = Marshal.AllocHGlobal(cbSize);

// call again to get the real stuff
result = EnumServices(pBuffer, ref numEntries, ref cbSize);

            // loop through the structure pulling out the prefix and the dll name
            for (int i = 0; i < numEntries; i++)
            {
                // move a pointer along to point to the "current" structure each time through the loop
                IntPtr pStruct = new IntPtr(pBuffer.ToInt32()+ (i * structSize));

// "translate" the pointer into an actual structure
ServiceEnumInfo sei = (ServiceEnumInfo)Marshal.PtrToStructure(pStruct,

typeof(ServiceEnumInfo));

string prefix = sei.prefixName;
string dllName = Marshal.PtrToStringUni(sei.pDllName);

                // use the prefix and dllName as needed....
                Debug.WriteLine(prefix);
                Debug.WriteLine(dllName);
            }

// remember to free the buffer that we allocated
Marshal.FreeHGlobal(pBuffer);

}
}
}

Thanks,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights.

Great .Net Compact Framework Content at Oredev

2007-11-10T03:47:00Z

Next week is the annual Øredev conference in Malmö, Sweden. We're very fortunate to have 3 detailed .Net Compact Framework sessions there. Doug Boling will be doing sessions on performance and on how to enable GPS in your applications and I'll be doing a session on how to use the .Net Compact Framework diagnostic tools to solve tough issues like performance problems or memory leaks.

I'll also be presenting a session on Silverlight 1.1.

If you're in that part of the world next week it's a great chance to get access to some .Net Compact Framework content. Doug Boling knows a ton about WindowsCE and is a great resource.

Thanks,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights.

What's new in the Remote Performance Monitor for .Net Compact Framework 3.5

2007-10-27T03:40:18Z

The Remote Performance Monitor (RPM) first shipped in version 2 service pack 1 of the .Net Compact Framework. Since that time, numerous customers have come to depend on RPM to help them diagnose performance problems and find memory leaks in their applications. With each release we add new features and fix bugs based on what customers tell us is important, and version 3.5 is no exception. This post summarizes what's new in v3.5.

Emulator support

One of the most glaring omissions from RPM in previous releases was the lack of support for emulators. Thankfully, in version 3.5 you can use the WindowsMobile and WindowsCE emulators with RPM just as you would a "real" device.

Auto-deployment of device-side components

The 3.5 version of RPM no longer requires any manual setup steps. Previously, you had to manually copy netcfrtl.dll and the VSD transports to your device before using RPM. Now, RPM does everything for you: if anything RPM depends on is missing from your device, those dependencies will be copied before you launch an application.

New User Interface

RPM's user interface has been streamlined to make the process of collecting and browsing performance statistics and analyzing GC heap dumps easier. We've removed the use of MDI to manage windows, simplified the menu options and added a dialog used to launch applications. Here's a screenshot of a section of the new UI. None of us are UI gurus here on the .Net Compact Framework team, but we hope our new approach is an improvement.

We also keep track of the applications you've previously launched so you don't have to retype them every time. I always hated that....

Improved Device Management

RPM now includes a new component that lets you manage the devices you'd like to connect to. The following device management form can be accessed from the Browse button on the launch dialog (see the previous picture):

This dialog makes it much easier for you to work with your devices. As with previous releases, RPM knows about Active Sync devices automatically. You can now add and save devices connected over tcp/ip so you don't have to retype the addresses each time you'd like to use the device. You can also set various device properties and designate one of your devices as the default device. Tagging a device as the default causes it to be automatically selected when the launch dialog is displayed.

Remote Installation of .Net CF

Installing .Net CF into RAM typically involves either rerunning the SDK setup program or browsing through your file system searching for the correct cab to install, manually copying it to the device and launching the cab installer. With RPM 3.5 you can install Net CF onto your device using the device management dialog as shown below:

RPM will detect the version of the OS on the device, along with the processor type, and deploy the correct cab file.

Deployment of your application from RPM

If the application you'd like to launch isn't present on your device, you can now use RPM to deploy it. On the launch dialog you'll see two fields: "Deploy application from" and "Application":

As you'd expect, "Deploy application from" is the location on your desktop computer where the .Net Compact Framework application executable you'd like to deploy resides. The "Application" field is the location on the device you'd like your application deployed to. There is one subtlety in the use of these fields to remember: in the "Deploy application from" field you specify the directory your application resides in, not including the name of the executable itself. In the "Application" field you specify the fully qualified name of the destination on device, including executable name.

For example, say I'd like RPM to deploy the file stored on my desktop computer at c:\temp\neilyoung\archives\riverboat.exe to the \program files\sjp directory on my device. The values of the two fields would be:

This feature can only be used to deploy applications consisting of a single file. If your application contains one executable file and several dlls you'll still have to deploy your application manually.

We hope these improvements make RPM an even better tool. As always, please keep the feedback coming.

Thanks,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights.

The CLRProfiler for the .Net Compact Framework, Part II: Histograms and "Show who Allocated"

2007-10-18T20:22:31Z

Yesterday I started a series of posts on how the use the CLRProfiler for the .Net Compact Framework. The first post contained the basic information you need to get started. I described how to install the profiler, launch an application on the device, and collect profiling data.

In order to direct the discussion, I've written a sample application that exhibits a performance problem that is surprisingly easy to fall into. Throughout these posts I'll show you how to use the profiler to diagnose the problem. To refresh your memory, the sample application is a basic game and the performance problem is that the main windows paints way too slowly.

After I stopped profiling the game in the first post, following summary page was displayed.

In this post I'll use some of the histograms to begin diagnosing our performance problem.

Histograms

The first thing that stands out at me when looking at the summary form is the amount of managed data I'm creating. While profiling the painting portion of my application I generated over 6MB of managed objects. That's clearly way too much for a relatively simple operation like painting my main window. My first step in determining what's going on is to get some basic statistics about the objects my application is using. For example, I'm interested in which objects I'm creating, how many of them there are and how long they live. This data can be obtained by looking at some of the histograms the profiler offers.

I can choose to view a histogram for all objects created as my application ran or only for those objects that were in the GC heap when my application exited. In my scenario I need to look at all objects. If I were to only look at the objects alive at the end of the run I may miss some important trend that occurred earlier on.

Clicking the "Histogram" button next to the "Allocated Bytes" value displays the following graph:

The histogram form has two panes. The pane on the right describes how many instances of each type of object were created and the total size of those instances. The pane on the left graphs type instances by size. The color coding next to the types in the right pane matches the bars in the left pane which show the relative amounts of objects created.

A quick glance at this form helps narrow my suspicions about what's causing my performance issue. As you can see, about 97% of the objects I created were of type Box.Block as indicated by the red box on the right hand pane and the red bar in the left hand pane. I can also see that each instance of Box.Block is relatively small at an average size of 136 bytes (see the right hand pane).

Who Allocated all those Objects?

Now that I know the majority of my objects are instances of Box.Block, I'd like to see where in my application those instances are getting created.

To determine the source of my allocations I can right-click on the bar that represents Box.Block in the histogram and select "Show Who Allocated" (the bar turns black when selected):

Doing so brings up a window referred to as an Allocation Graph:

The Allocation Graph traces the flow of every call that allocated an instance of Box.Block. I typically interpret this graph starting with the rightmost node. This node represents all instances of Box.Block in the system. Stepping back one level to the left we see two nodes representing methods that created instances of Box.Block: Form1.RotateGameBlocks and Form1.InitializeGameBlocks. The data in these nodes tell us that 75% of the Blocks were created in RotateGameBlocks and 25% were created in InitializeGameBlocks. Notice that the width of the lines connecting the nodes represents the percentage of instances that call created.

Now that I know where my objects are coming from I can dig into my code to see what's going on.

In some scenarios, the information we've learned so far may be all that we need to fix the problem. However, there are a few more pieces of data that may be required in some cases. For example, it may be useful to know the times at which Blocks were created and destroyed. Also, if RotateGameBlocks and InitializeGameBlocks are long, complicated methods, we may need to know the exact calls within those methods that caused the allocations. I'll describe how to get this information in future posts.

Thanks,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights

The CLRProfiler for the .Net Compact Framework, Part 1: Getting Started

2007-10-18T04:28:00Z

Version 3.5 of the .Net Compact Framework contains a new memory profiler called the CLRProfiler. The CLRProfiler is a great tool for looking into the details of how your application is allocating and using managed objects. For example, the profiler allows you to look at the contents of the GC heap at any point in time, provides a historical record of what's going on in the heap, let's you see which calls in your application are allocating which objects and so on. This level of detail is often needed to diagnose memory-related issues in your device applications. The Compact Framework's version of the CLRProfiler is an adaptation of the profiler that has been available for the full .Net Framework for some time.

In this series of posts I'll walk you through the primary features of the CLRProfiler for the .Net Compact Framework. The profiler contains numerous ways to analyze data about the GC heap so instead of briefly touching on all of them, I'll go into depth on the views I've found most useful.

We'll be looking at the profiler by way of an example, as I've found that learning a new tool is often easier if you have a specific problem to solve rather than just looking at the tool's features without any context. If you'd like more information on a view that I don't cover here you can read the document that ships with the full Framework's version of the profiler.

Our Sample

The sample I'll use to describe the profiler is the beginnings of a game I started to write using the Compact Framework. All the game currently does is allow you to start a new game and to rotate a set of blocks on the screen. Here's a simple view of the application:

The performance problem I'm having with this game involves my drawing logic. The blocks on the screen draw very slowing. On my Dell Axim I can literally see each column of blocks paint individually. Throughout these posts I'll use the CLRProfiler to figure out what I can do to make my game paint more quickly.

Launching an Application with the CLRProfiler

The CLRProfiler ships in the .Net Compact Framework Power Toys package. After installing the Power Toys you can find the CLRProfiler executable (NetCFClrProfiler.exe) in the bin directory of the .Net SDK. On my machine that directory is c:\Program Files\Microsoft.Net\SDK\CompactFramework\v3.5\bin. The Power Toys setup program also adds a menu item for the profiler to the Windows Start menu.

The main window of the profiler is strikingly simple:

Through this main window you can start and stop applications, take snapshots of the GC heap, and control various profiling options. These options include control over whether profiling is currently active and whether allocations, calls or both are logged.

Clicking the "Start Application…" button displays the following form:

On this form you enter the name of the device you'd like to connect to, the fully qualified path to the device executable you'd like to profile, and any command line parameters to be passed to the application. The CLRProfiler supports profiling applications on devices connected either over ActiveSync or TCP/IP. You can also profile applications running on emulators. The profiler supports the same types of devices that the .Net Compact Framework Performance Monitor does(in fact, the device selection and connectivity mechanisms are shared between the two tools).

In my example I have a device connected over ActiveSync and the name of my application is box.exe. Selecting "Connect" from the launch dialog starts the application on device and begins profiling. At this point just run your application as you normally would. The CLRProfiler causes your application to run much slower than normal. In order to speed up the debugging process you can turn profiling on and off for different sections of your application using the "Profiling active" checkbox on the profiler's main form.

When you're done profiling your application, you can stop it either by selecting the "Kill Application" button on the main form of the CLRProfiler or by just closing the application directly on the device.

The Summary Form

After your application exits the CLRProfiler displays the following summary form:

The summary form provides some general statistics about the use of managed memory as your application ran:

Heap Statistics. The statistics displayed in this group box describe the total size of the objects in the managed heap. The "Allocated Bytes" value counts the total size of allocations made as the application was running. When the Compact Framework garbage collector detects significant fragmentation in the heap, it will compact it. The "Relocated Bytes" value shows how many bytes the garbage collector moved around during compaction. "Final Heap Bytes" shows the size of the managed heap when the application exited and "Objects Finalized" is what you'd expect: the number of objects that had finalizers to run. Note that the Heap Statistics group also provides a count of critical finalizers run. The .Net Compact Framework doesn't have the notion of critical finalization so this value will always be 0.

There are several buttons in this box that launch viewers that allow you to analyze data in various ways. You can view data in histograms based on object size, age, address and so on. I'll describe most of these views in detail in subsequent posts.

Garbage Collection Statistics. This group box tells you how many garbage collections occurred while your application was being profiled and how many of those collections were induced by calls to GC.Collect. The "Timeline" button enables you to see how the contents of the GC heap changes over time. I think this is one of the coolest views. I'll describe it in great detail in a subsequent post.

GC Handle Statistics. The GC Handles group box shows you how many handles were created and destroyed while your application ran. Keep in mind that the Compact Framework CLR creates GCHandles under the covers as it executes your application so all of the handles you see here aren't likely to have been created explicitly by you.

Profiling Statistics. The CLRProfiler enables you to take snapshots of the GC heap as your application is running. I didn't do this while profiling my application, but if I would have I would be able to choose a snapshot from the dropdown and view it.

Getting back to our example, there are several pieces of data on this form that concern me, or are unexpected. The key to analyzing performance data such as this is not just looking at the raw values, but in interpreting the values in the context of what your application was doing as it was being profiled. For example, I profiled the time my application spent painting, yet I see that 5 garbage collections occurred and I created over 6 MB worth of objects. This data leads me to believe that I'm making the garbage collector work harder than it should for my scenario. Clearly there shouldn't be so much activity going on in the managed heap while I'm painting boxes on the screen!

In my next post, we'll use some of the histogram views to see how many objects my application is creating and of what type they are.

Thanks,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights.

Write your own GC Heap Viewer for the .Net Compact Framework

2007-09-21T19:52:00Z

The last few versions of the Remote Performance Monitor enable you to view snapshots of the GC heap on demand. The view of the heap presented by RPM is oriented around finding managed memory leaks. Specifically, the data is organized so that it's easy to see which GC root is responsible for keeping a given object instance alive.

We hope the views are well organized and easy to understand, but if you find they don't suite your needs for any reason you can write your own tool to analyze and display the heap data. For example, maybe you want a fancier graphical representation, or you want a view tailored to a scenario other than finding memory leaks.

You can't currently plug your new UI directly into RPM but you can save the heap snapshots to a file and create your own tool to view the data.

In this post I'll briefly describe how to save the heap data then I'll provide the details of the file format.

Saving a GC Log

Saving a snapshot of the GC heap to a file is easy: Given an open view, just select the "Save" option from the "File" menu:

You'll also be prompted to save any unsaved views when you close them.

The GC Log File Format

GC heap dump files are text files that include information for every object present in the GC heap at the time the heap dump was generated using RPM.

Each line in the text file contains one record. There are 5 different types of records:

AppDomain record. Identifies the application domain for which a dump file applies.
Type record. Describes a type present in the heap.
Object record. Describes an instance of a type present in the heap.
Root record. Describes a root instance.
End AppDomain record. Marks the end of the dump file.

Each record contains a number of elements separated by spaces. The first element in each line indicates the record type. Here's a portion of a dump file:

Note: This example is intended to show format only - it's likely not semantically accurate.

a 2 bubblecs.exe

t 2 System.RuntimeType

o 7c79f 2 64

t 1 System.NullReferenceException

o 7cb42 2 64

o 1ce004 1 18

t 3 System.OutOfMemoryException

o 1ce056 1d 118 1ce11f 1ce14c 1ce079 1ce113 1ce116 1ce267 1ce26a 1ce26d

r 22c81b 5 0

o 22c81e 1b 24

r 22c81e 5 0

o 22c823 1b 28

r 22c823 5 0

c bubblecs.exe

AppDomain Record

An AppDomain record initiates a section of the file pertaining to a particular application domain. Within that section is all the type, object and root information for the managed objects in that application domain. All AppDomain records are closed with a corresponding End AppDomain record.

The AppDomain record has 4 elements:

The letter "a". Identifies the record as an application record.
Version number. A number used to track the file format of the dump file itself. In the current releases this number will always be "2".
AppDomain Name. The name of the application domain from which this dump file was generated. This is typically the name of the exe that was run in the app domain.
Timestamp. The time at which the heap dump was take. RPM obtains this number by calling the WinCE API GetTickCount.

Example:

a 2 NHLSchedule.exe 444d20df

Type Record

Type records identify types in the GC heap. Each type record contains 3 elements:

The letter "t". Identifies the record as a type record.
Type ID. A unique numerical identifier for this type. This identifier is used to tie object instances to their types (see description of Object Records below).
Type name. The fully qualified name of the type.

Example:

t a1 Western.Pacific.SanJoseSharks

Note:

Type identifiers are not guaranteed to be unique across dump files. For example, the identifier for the "SanJoseSharks" type in the example above isn't guaranteed to be a1 in a different dump taken from the same application at a different point in time.

Object Record

Object records describe specific instances of types in the GC heap. Object records have a variable number of elements. The first 4 elements are required:

The letter "o". Identifies the record as an object record.
Object ID. A unique numerical identifier for this object.
Type ID. The identifier for the type of this object.
Object size. The size in bytes for this instance.

In addition to these required elements, object records will have a variable number of additional elements describing the instances that the object references.

Referenced Object IDs. The identifiers for all objects that this object references.

An example with referenced object ids:

o 1c15d2 3 1c 1c15db 1c15d8 1c15d5

Notes:

Object identifiers are not guaranteed to be unique across dump files. This point is important for tools writers: It is not possible to develop tools that identify trends across dump files because a given object instance in one dump isn't guaranteed to have the same identifier in a later dump of the same heap.
The type record corresponding to a particular object record is not guaranteed to precede the object record in the dump file.

Root Record

Root records identify GC roots. Each root record contains 4 required elements:

The letter "r". Identifies the record as a root record.
Object ID. The identifier of the object that is this root.
Root kind. The reason this object instance is a root. One of the following values:

1 The object instance is a local variable.

2 The object instance is on the finalizer queue waiting for its finalizer to be run. After the finalizer is run the object will be collected during the next GC.

3 A GC handle exists which refers to the object instance (i.e System.Runtime.InteropServices.GCHandle)

4 The object instance is a static variable.

5 The object instance is a root specific to the Compact Framework's GC implementation. Examples include interned strings or class descriptions.

0 The object instance is rooted internally by the Compact Framework. Examples include object instances for application domains, assemblies, exceptions and so on.

Root flags. A set of flags providing more information about the root:

0x1. The root is pinned. There are several cases in which a root may be pinned. For example, the root may refer to managed objects that have been passed out to native code via PInvoke or COM interop, the root may be referenced by a GCHandle created with a GCHandleType of Pinned or a pointer in unsafe code points to the root.

The .Net Compact Framework CLR may also pin objects on your behalf as it runs your application. The CLR will pin an object if it places a reference to that object in a register or refers to that object from the stack. An object will also be pinned if a local variable or method argument points to a field within the object. The CLR doesn't pin objects very often, but if one of the more obvious reasons don't explain why an object is pinned, it may be because of one of these "hidden" cases.

0x2. A GCHandle whose GCHandleType is Weak refers to the object.

0x4. The object is pointed to by a pointer in unsafe code or has a local variable or method argument points to a field within the object.

If the root kind is static, the root record will have an additional element:

Root container. The identifier of the type the static variable is contained in.

Here's a root record with the additional root container element:

r b2753 4 0 13c

End AppDomain Record

End AppDomain records close the section initiated by a corresponding AppDomain record. End AppDomain records have three elements:

The letter "c". Identifies the record as an end record
Application Name. The name of the application from which this dump file was generated. This name matches the name in the application record at the beginning of the file.
Timestamp. The timestamp from the corresponding AppDomain record.

Thanks,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights.

.Net Compact Framework: “Hey, what happened to the diagnostic tools?”

2007-09-13T03:33:00Z

If you’ve installed the Beta2 version of Orcas you may have noticed that the NetCF diagnostic tools (RPM, CLRProfler, ...) are missing. Don’t worry, these haven’t been cut from Orcas, they will just be distributed via the web in a separate “power toys” pack. A CTP of these tools is now available at: http://www.microsoft.com/downloads/details.aspx?familyid=C8174C14-A27D-4148-BF01-86C2E0953EAB&displaylang=en . This CTP works with Orcas Beta2.

We intend to distribute the final Power Toys release at the same time that Orcas ships.

Thanks,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights.

.Net Compact Framework Performance Webcast

2007-08-14T20:53:00Z

Last week I did an MSDN webcast on writing high performing device applications using the .Net Compact Framework. The webcast contains both in depth information on the internal workings of the Compact Framework and a set of tips and tricks for optimizing performance in everything from user interface to collections to networking. Here's the URL: http://msevents.microsoft.com/CUI/EventDetail.aspx?EventID=1032345907&Culture=en-US .

Also, I had promised those viewing the webcast live that I would post the "generics" slide without the performance numbers overlaying the code. Here it is:

Thanks,

Steven

Disclaimer(s):
This posting is provided "AS IS" with no warranties, and confers no rights.