Steven Pratschner's .Net CF WebLog

Another book update for “Customizing the Common Language Runtime”

2011-03-01T22:42:47Z

A reader recently pointed out another error in my hosting book. The error involves an API I used in the book that was cut before the final release went out.

On pages 253-256 I describe a hosting API named IHostControl::GetDomainNeutralAssemblies. The idea at the time was that this API could be used by a host to tell the CLR exactly which assemblies to load domain neutral. Unfortunately I published the book 6 months or so before the final product release (this was .Net Framework 2.0) and by the time the product shipped the CLR team had decided to remove the API.

As such, the options you have to tell the CLR which assemblies to load domain neutral aren’t nearly as granular. These options are represented by the enumeration called LoaderOptimization. This enumeration is described on page 252 of the book.

My apologies for this error. Bitten again by releasing the book before the product shipped…

Thanks,

Steven

The .Net Compact Framework Configuration Tool

2008-05-14T00:24:51Z

The .Net Compact Framework 3.5 Power Toys include a new utility called the NetCF Configuration Tool.

The Configuration Tool is a diagnostic tool. You won't typically use it in the course of everyday development, but it will come in handy for tasks like diagnosing failures related to device configuration and authoring configuration files.

While the tool isn't targeted at your application's end users, they will be able to use it with your help should you need them to provide you with information needed to debug a problem remotely. For this reason, the Configuration Tool runs directly on the mobile device instead of on a desktop machine connected to a device.

After describing how to install the tool, I'll discuss its four main functional areas:

Displaying which versions of NetCF are installed
Displaying the contents of the Global Assembly Cache
Authoring device.config files
Authoring application configuration files

Installing the Configuration Tool

The Configuration Tool isn't automatically installed on the device when you install the Power Toys. Instead, you must copy it to your device manually.

Fortunately there's only one file to copy: NetCFcfg.exe. This file is OS and processor-specific so you'll find it under the appropriate WindowsCE directory in the .Net Compact Framework SDK. I'm running a WindowsMobile 5.0 device, so the Configuration Tool executable for my device is in: C:\Program Files\Microsoft.NET\SDK\CompactFramework\v3.5\WindowsCE\wce500\armv4i on my desktop machine.

It doesn't matter which directory you put the file in on your device. I usually put it in the \windows directory but you can deploy the file to any location you please.

The About tab

The "About tab" describes which versions of NetCF are installed on a device. This is the tab that is displayed when you first launch the Configuration Tool:

It's not unusual to have more than one version of NetCF installed on a device. Most all Windows Mobile and WindowsCE devices come with a version of NetCF in ROM. Another version is often installed in RAM to support an application that requires a newer version of NetCF than the version that came with the device.

When multiple versions of NetCF are installed, it's not always obvious which version is being used to run your application. I've seen several cases where confusion results because an application is running with a different version than expected (see "promoting an application" for a general description of the rules used to determine which version of NetCF will be used to run an application). For example, remote tools like the CLRProfiler and Remote Performance Monitor launch an application on device in order to profile it or gather other performance statistics. If your application is launched with a version of NetCF other than the one the tool is expecting, the application may start but the tool won't be able to gather the diagnostic information it is looking for.

Knowing which versions of NetCF are installed on the device, which version was used to build your application, and the rules used to determine which version will be used to run your application can help you determine whether any unexpected behavior you are seeing results from the "wrong version of NetCF" problem.

The GAC tab

The GAC tab lists the assemblies installed in the Global Assembly Cache:

It helps to know the contents of the GAC when diagnosing assembly load failures. It may be you think an assembly is in the GAC when it isn't, or NetCF may load an assembly from the GAC when you don't expect it to. If you encounter a failure to load an assembly, use Loader Logging to find out why, then check the contents of the GAC using the Configuration Tool if the failure looks related to the GAC.

The Device Policy tab

In a previous post I described how to use device.config to cause all applications on a device to run with a given version of NetCF. The Configuration Tool enables you to edit device.config using a GUI rather than having to modify the XML by hand.

The Device Policy tab let's you choose a version of NetCF to run all "unconfigured" applications on the device. By "unconfigured" I mean all applications that do not have an application configuration file that contains a supportedRuntime element.

In addition to the installed versions of NetCF, you can also select the value "Default". Doing so will remove all supportedRuntime elements from device.config causing the device to revert to its default behavior for selecting a runtime.

Keep in mind that not all values you select on this tab are "valid" for all applications. For example, if I chose 1.0.4292 in the example above, no applications built with later versions of NetCF would run (unless they had configurations files of their own that overrode the device-wide setting).

The Application Policy tab

In addition to specifying device-wide policy using the Device Policy tab, you can also specify which version of NetCF should be used to run a specific application by using the Application Policy tab.

The top combo box on the Application Policy tab is pre-populated with all NetCF applications that are installed on the device. After selecting an application, use the lower combo box to select a version of NetCF to run the application. Under the covers this dialog is adding (or removing) supportedRuntime elements from your application's configuration file.

Any value chosen using the Application Policy tab overrides values chosen using the Device Policy tab.

Thanks,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights.

The CLR Profiler for the .Net Compact Framework Series Index

2008-05-09T00:04:17Z

I've completed all that I had planned to write (at least for now) about how to use the CLRProfiler with NetCF. Here's a brief explanation and a link to each post the series:

Part I, Getting Started. Describes how to install the profiler, launch an application, and begin collecting profiling data.
Part II, Histograms and "Show Who Allocated". The profiler displays various histograms you can use to determine the types and number of objects your application is allocating. You can also determine what methods in your application cause which types to get allocated.
Part III, The Timeline View. The timeline view shows the state of the GC heap over the lifetime of your application.
Part IV, The Call Tree View. Detailed information about every method call and every object allocation is shown in the Call Tree View.
Part V, Controlling the Profiler Programmatically. The CLRProfiler generates a large amount of data and significantly slows down your application. You can programmatically control when the profiler runs using an API that you call from your application. This API also allows you to insert comments into the profile data and to take a snapshot of the GC on demand.

Thanks,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights.

The CLRProfiler for the .Net Compact Framework, Part V: Controlling the profiler programmatically

2008-04-23T03:58:46Z

If you've used the CLRProfiler for NetCF you've probably noticed that your application runs much slower when being profiled. You likely have also seen the huge amount of data the profiler generates, even for relatively simple applications. The time it takes the profiler to run coupled with the volume of data it creates can sometimes make the profiler impractical to use. For example, I've had customers tell me that it can take several minutes to start an application while profiling. If it takes you several more minutes just to navigate to the portion of the application you'd like to profile, it can quickly become extremely frustrating.

The CLRProfiler ships with an additional assembly you can reference and call from your application to control the profiler while your application is running. This assembly, called NetCFCLRProfilerControl contains APIs you can use to start and stop profiling, insert comments into the profiling log, dump the contents of the GC heap on demand, and so on. The ability to control when the profiler collects data enables you to profile only the sections of your application you want to, rather than having the profiler collect data all the time. In this way, your application will start faster and there will be less data to sift through later.

In this post I'll describe how to use the APIs provided by the NetCFCLRProfilerControl.

Using the NetCFCLRProfilerControl Assembly

The following steps are required to use NetCFCLRProfilerControl from your application:

Reference NetCFCLRProfilerControl in your Visual Studio project.
Add code to your application to call the APIs.
Deploy NetCFCLRProfilerControl along with your application.
Start your application using the CLRProfiler.

Referencing NetCFCLRProfilerControl

NetCFCLRProfilerControl is installed on the desktop machine in same directory as the NetCFCLRProfiler executable. On my machine the files are in C:\Program Files\Microsoft.NET\SDK\CompactFramework\v3.5\bin.

You'll need to add a reference from your application to NetCFCLRProfilerControl to use its APIs. If you're using Visual Studio you can add this reference using the standard "Add Reference..." dialog.

Public APIs

NetCFCLRProfilerControl contains one public class called CLRProfilerControl which contains the following public static members:

Member	Notes
bool ProcessIsUnderProfiler { get; }	Returns true if your application was launched using the CLRProfiler. Note that this is independent of whether the profiler is actually logging allocations or calls at the time.
bool AllocationLoggingActive { set; get; }	A boolean property that controls whether the profiler logs managed allocations.
bool CallLoggingActive { set; get; }	A boolean property that controls whether the profiler logs method calls.
void LogWriteLine(string comment) void LogWriteLine(string format, params object[] args)	Enables you to insert comments into the profiling log. These comments show up in the various profiler views to help you narrow down exactly where in your application certain allocations or calls were made. See the example later in this post.
void DumpHeap()	Causes the profiler to display a graphical view of the GC heap. Individual views will also show up on the profiler's summary form when the application exits.

Deploying your application

NetCFCLRProfilerControl is not installed on the device automatically. You'll need to deploy it along with your application. It's probably easiest to deploy NetCFCLRProfilerControl to the same directory as your application but you can also install it in the GAC if you prefer.

Launching your application

The APIs provided by NetCFCLRProfilerControl have no effect if your application isn't started using the CLRProfiler. However, assuming you are going to use the APIs to control when the profiler logs allocations and/or calls, be sure to clear the relevant checkboxes on the profiler's main form before launching your application (otherwise you'll be logging all the stuff you wanted to prevent in the first place!).

After clearing the checkboxes, start your application using the profiler as you always have. The actions of the profiler will now be controlled by your application instead of the CLRProfiler's user interface.

Turning profiling on and off

Assuming you've started your application with the "Profile" checkboxes on the profiler's main form cleared, no logging of allocations or calls will occur. You can turn profiling on using the AllocationLoggingActive and CallLoggingActive properties.

The following example turns allocation logging on just before entering a loop, then turns logging back off when the loop completes:

private void btnCheckOut_Click(object sender, EventArgs e)
{
    CLRProfilerControl.AllocationLoggingActive = true;
    for (int i = 0; i < 20; i++)
    {
        m_cart.ProcessCart();
    }
    CLRProfilerControl.AllocationLoggingActive = false;
}

Inserting comments into the profiling log

The ability to insert comments into the profiling log is a cool feature that enables you to place your own visual markers into some of the profiler's graphical views.

Here's an example that inserts comments denoting the beginning and end of the execution of a loop:

private void btnCheckOut_Click(object sender, EventArgs e)
{
    CLRProfilerControl.LogWriteLine("Before ProcessCart loop");
    for (int i = 0; i < 20; i++)
    {
        m_cart.ProcessCart();
    }
    CLRProfilerControl.LogWriteLine("After ProcessCart loop");
}

After the application exits, your comments show up in several places in the CLRProfiler UI. First, the summary screen includes a button that brings up a window giving the timestamp of each comment. You can use this as a rudimentary timing mechanism in your application:

The comments also show up in various views throughout the UI For example, in the Timeline View, the comments show up as vertical green lines that indicate the time at which the comment was logged. You can see the text of a given comment by hovering the house over its green line. This allows you to see the state of the GC heap between different points of interest as your application ran.

Dumping the GC Heap

The final feature available via NetCFCLRProfilerControl is the ability to take a snapshot of the GC heap on demand. The DumpHeap method causes a heap view to be displayed immediately. You can also access all the heap views taken while the app ran from the Summary View after the application exits.

Thanks,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights.

Promoting all .Net Compact Framework Applications on a device using device.config

2008-03-12T03:01:01Z

By default, a managed device application is run with the version of the .Net Compact Framework it was built with. Only if that version is not present on the device will we "promote" the application to run with the latest installed version. This policy is considered safe and conservative because it isolates applications from unintended compatibility changes that may be introduced in new version of the Framework, at least in the most common scenario.

However, there are times when you may want to run an application with a version of NetCF other than the one it was built with. For example you may find that a newer version of NetCF performs better and is more stable than a previous version so you'd like to take advantage of those improvements without having to rebuild your application. Several previous blog posts (see http://blogs.msdn.com/davidklinems/archive/2005/11/09/491113.aspx) have shown how to do this using an application configuration file. This approach is manageable as long as you only have a few applications to promote. But if you'd like to promote a large number of applications it can quickly become a hassle to author all these configuration files and manage their deployment and updates.

Fortunately, version 3.5 of the Compact Framework lets you promote all applications on a device using a single device-wide configuration file. The syntax of this configuration file is the same as that used in the application configuration file: you specify the version of NetCF you'd like to run all applications with using the supportedRuntime element. The device-wide configuration file must be called device.config and must be located in the \windows directory.

Here's the contents of a sample device.config that will cause all applications to be run with .Net Compact Framework version 3.5:

There's one more case to consider. Suppose you've used device.config to promote all your applications only to discover that one of your applications doesn't work properly. In this scenario you'd want to "roll back" just that one application to the version it was built with. You'd do this by authoring an application configuration file for that application. Any supportedRuntime settings present in application configuration files will override the settings specified in device.config thus giving you a quick way to opt-out of your global policy.

Thanks,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights.

The CLRProfiler for the .Net Compact Framework, Part IV: The Call Tree View

2008-02-12T21:10:23Z

This series of posts provides an introduction to using the CLRProfiler for the .Net Compact Framework. In previous posts (part 1, part 2, and part 3) we've looked at various profiler features as we try to solve a performance problem with a sample application. So far, we've learned that our performance problem is due to excessive boxing of value types. We've also learned what type of objects we are boxing and what methods in the sample are causing the boxing to occur.

In some cases, knowing which method is causing the problem is all you need, especially if the method is small and straightforward. However, there are cases where even more detail is needed. In this post I'll use the Call Tree View to determine the exact line of source code that is causing my performance issue!

After you've finished profiling, you can bring up the Call Tree View from the summary page:

The Call Tree View

The Call Tree View shows every allocation made as a result of every method call. For all but the simplest programs the Call Tree View contains a huge amount of data. Fortunately, navigating through all this data isn't very hard once you get used to a few tricks.

Method calls are shown in black and object allocations are shown in green. For example, the following picture shows the initial state of the Call Tree View. You can see that only one method (the program's Main method) has been called at this outermost level. You can also see that several allocations have already been made. These initial allocations are made by the CLR just before it starts executing a program. As you can see, these initial allocations represent several of the common exception types as well as the initial Application Domain.

Along with the individual method calls and object allocations, the Call Tree View also shows summary statistics such as the number of method calls that result from a given call, the number of bytes and objects allocated by a given call and so on. You can sort the data either by number of calls or by the amount of memory allocated.

As you start to expand the tree, you'll notice that one child node at each level is highlighted in bold. The highlighted node represents the method that has allocated the most memory (or generated the most calls, depending on your sort preference). This is the key to drilling into the data quickly - just follow the highlighted nodes until you find the method you are looking for.

Back to the Example

In previous posts we've established that the objects we are boxing are of type Box.Block and that the boxing is occurring in calls to RotateGameBlocks and InitializeGameBlocks. By drilling through the data in the Call Tree View we can find out exactly which line of code in RotateGameBlocks (or InitializeGameBlocks) is causing the boxing to occur.

The following picture shows the result of following the highlighted notes until I have found the RotateGameBlocks method. Some of the methods in the tree look familiar, like Main, and Application.Run, but many methods don't. Method names that you don't recognize are likely part of NetCF's internal implementation. For example, methods that contain "AGL" in their name are part of NetCF's implementation of Windows Forms.

When looking at the child nodes of RotateGameBlocks, we see a green line representing the allocations of Box.Block. The Objects column tells us that a call to RotateGameBlocks causes 14,400 instances of Box.Block to be allocated. Furthermore, we can see that RotateGameBlocks calls ArrayList.set_Item 14,400 times. The fact that the number of calls to ArrayList.set_Item and the number of instances of Box.Block that are allocated are the same indicates that the calling ArrayList.set_Item is the line in my sample that is causing our boxing to occur.

In my next post I'll describe a set of managed APIs you can use to control profiling programmatically.

Thanks,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights.

PowerToys for the .Net Compact Framework version 3.5 now released!

2007-12-11T02:21:57Z

We've just released the final 3.5 version of the PowerToys for NetCF to the web.

You'll need to install the 3.5 NetCF runtime on your device to use these tools. For now, you can get the correct runtime by installing Visual Studio 2008. In a few weeks we hope to post a standalone install for NetCF 3.5.

The PowerToys release includes all the tools you need to diagnose issues in your .Net device applications, including the Remote Performance Monitor, CLR Profiler, the Logging Utility and more...

Thanks,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights

The CLRProfler for the .Net Compact Framework, Part III: The Timeline View

2007-11-28T22:31:37Z

In the first two parts of this series (part 1 and part 2) I described how to get started using the CLRProfiler and how to use the histograms and the "Show Who Allocated" view to see what types of objects you're allocating and to determine where in your application the allocations are occurring.

In this post I'll talk about my favorite view: the Timeline view. I like the Timeline view because it lets you look at the contents of the GC heap over time. Using this view you can see not only where in your application you are allocating objects, but when. You can also see which objects are collected each time the GC ran. By including a time dimension, the Timeline view combined with "Show Who Allocated" can give you a complete picture of how the use of managed memory varies as your application runs.

After you've completed profiling, you can launch the Timeline view from the Summary page:

The Timeline View

The Timeline view has two panes. The right hand pane is the legend and the left hand pane is the contents of the heap. As with the histograms we looked at in part 2, the colors in the legend describe the types of objects present in the heap. Free space in the heap is always white:

The horizontal axis in this view represents time. If you look at the axis you can see when each GC occurred as indicated by the vertical blue markers. By correlating each GC with the contents of the heap displayed above you can get a sense for which objects were freed in each collection.

Note: You may notice that each GC is labeled with a generation. The Compact Framework's garbage collector doesn't have the notion of generations so these markers are an artifact of the code we ported from the full .Net Framework. You can ignore them.

The vertical axis in the timeline view shows object addresses in the heap. Given that the segments of the GC heap are not necessary contiguous you may seen gaps in these numbers. If you'd like more information about the Compact Framework GC check out the following posts:

Back to the Example

Throughout this series of posts I've been using an example of a game I wrote that paints way too slowly. In part 2 we saw that the vast majority of objects I'm creating are of type Box.Block. Looking that the Timeline view confirms this as can be seen by looking at the legend and all the red in the view of the heap.

The time-based nature of this view allows us to get a few steps deeper into our analysis. First, we can see the points in time at which allocations of our Blocks occurred. In this case, that data isn't very interesting because the view tells me that I was almost continually allocating Blocks.

What's more important in my case is to discover which method in my application was doing the allocations at which point in time. It may be that early on in my application my allocations were coming from MethodA while later on they were coming from MethodB. This time-based analysis can be particularly useful if you'd like to see all that's happening when your application first starts, for example.

To see where your allocations are coming from at a given point in time just select that time in the view. This causes a vertical line to appear at that point in the graph. Then right-click and select "Show Who Allocated" from the context menu:

Doing so brings up the same Allocation Graph we looked at in part 2, except that the graph shows only the allocations done at that particular point in time. In part 2 we saw that my allocations were coming from two methods: InitializeGameBlocks and RotateGameBlocks. By using the timeline view I can see which of those methods was called at which times during my application.

Next time I'll describe the Call Tree view that will show even more detail by highlighting the exact lines within my methods that are causing the spurious allocations to occur.

Thanks,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights

Calling the EnumServices Win32 API from your .Net Compact Framework Application

2007-11-21T04:07:51Z

I was helping a customer use PInvoke to call EnumServices today and got stuck a few times so I thought it may be helpful to post the solution in case anyone else runs into this someday.

EnumServices returns a buffer containing a number of structures of type ServiceEnumInfo that describe basic information about the services on a device.

Each ServiceEnumInfo structure contains an embedded character array that represents the service's prefix and a pointer to a string that represents the name of the dll that implements the service. The dll names corresponding to the structures are laid out in memory just after the structures themselves. So the contents of the buffer you get back from calling EnumServices looks like this (this example is from a device with 3 services):

Here's some sample code that calls EnumServices and loops the buffer pulling out both the prefix name and the dll name for each service:

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Text;
using System.Windows.Forms;
using System.Runtime.InteropServices;
using System.Diagnostics;

namespace EnumServices
{
    public partial class Form1 : Form
    {
        // Managed definition of the native ServiceEnumInfo structure. Here's the corresponding native definition:
        //
        // typedef struct_ServiceEnumInfo {
        //    WCHAR szPrefix[6];
        //    WCHAR szDllName;
        //    HANDLE hServiceHandle;
        //    DWORD dwServiceState;
        // } ServiceEnumInfo;
        //
        struct ServiceEnumInfo
        {
            [MarshalAs(UnmanagedType.ByValTStr, SizeConst=6)] public String prefixName;
            public IntPtr pDllName; // this value is a pointer to the dll name - not the dll name itself.
            public IntPtr hServiceHandle;
            public int dwServiceState;
        }

[DllImport("coredll.dll")]
private static extern int EnumServices(IntPtr pBuffer, ref int numEntries, ref int cbBuf);

        public Form1()
        {
            InitializeComponent();
        }

        private void btnServices_Click(object sender, EventArgs e)
        {
            int numEntries = 0;
            int cbSize = 0;
            int structSize = Marshal.SizeOf(typeof(ServiceEnumInfo));

// call once to get required buffer size
int result = EnumServices(IntPtr.Zero, ref numEntries, ref cbSize);

// alloc a buffer of the correct size
IntPtr pBuffer = Marshal.AllocHGlobal(cbSize);

// call again to get the real stuff
result = EnumServices(pBuffer, ref numEntries, ref cbSize);

            // loop through the structure pulling out the prefix and the dll name
            for (int i = 0; i < numEntries; i++)
            {
                // move a pointer along to point to the "current" structure each time through the loop
                IntPtr pStruct = new IntPtr(pBuffer.ToInt32()+ (i * structSize));

// "translate" the pointer into an actual structure
ServiceEnumInfo sei = (ServiceEnumInfo)Marshal.PtrToStructure(pStruct,

typeof(ServiceEnumInfo));

string prefix = sei.prefixName;
string dllName = Marshal.PtrToStringUni(sei.pDllName);

                // use the prefix and dllName as needed....
                Debug.WriteLine(prefix);
                Debug.WriteLine(dllName);
            }

// remember to free the buffer that we allocated
Marshal.FreeHGlobal(pBuffer);

}
}
}

Thanks,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights.

Great .Net Compact Framework Content at Oredev

2007-11-10T03:47:00Z

Next week is the annual Øredev conference in Malmö, Sweden. We're very fortunate to have 3 detailed .Net Compact Framework sessions there. Doug Boling will be doing sessions on performance and on how to enable GPS in your applications and I'll be doing a session on how to use the .Net Compact Framework diagnostic tools to solve tough issues like performance problems or memory leaks.

I'll also be presenting a session on Silverlight 1.1.

If you're in that part of the world next week it's a great chance to get access to some .Net Compact Framework content. Doug Boling knows a ton about WindowsCE and is a great resource.

Thanks,

Steven

This posting is provided "AS IS" with no warranties, and confers no rights.