jaredpar's WebLog : PInvoke

Properly Incrementing an IntPtr

Jared Parsons — Tue, 11 Nov 2008 16:00:00 GMT

Just as native pointer types are moved around with pointer arithmetic in native code, it can also be useful to move IntPtr types around in managed code. Say for instance there is an IntPtr available which points to a native array of Dog instances. To access the values of that array individually requires pointer arithmetic of a fashion.

For the most part this is a straight forward operation if the underlying native memory is understood. Say there is an array of Dog instances with length 10 and the Dog structure has a size of 8. The total amount of memory will be 80 bytes and with a valid dog instance being available at every 8 bytes. So if the start address is 1000 then 1000,1008,1016 and so on will point to a valid instance.

The native size of any data structure can be calculated via Marshal.SizeOf(tyepof(Dog)) [1]. With a pointer to the start of the array, the Nth Dog instance can be accessed with a pointer of address = (N*sizeof(Dog))+startAddress

The address of a pointer can be accessed by 1 of 2 functions

.ToIn32()
.ToInt64()

Unless you are writing an application that will every only run on a 32 bit system, don’t use method #1 (even then still don’t). Native pointer addresses vary in size depending on version of the OS a program is running on. 64 bit systems have a much larger address size (long vs int). Consequently calling .ToInt32 on a 64bit system will truncate the actual address to a valid that is no longer valid. This will eventually lead to a random error PInvoke’ing a function that is difficult to track down.

Instead use .ToInt64(). This method is safe on both 32 and 64 bit systems. Additionally constructing an IntPtr instance with either value is safe. The class knows what version of windows it’s executing on and will adjust the size in a safe way [2].

In many of my projects I define a class similar to the following to take care of this automatically.

public static class IntPtrExtensions
{
    public static IntPtr Increment(this IntPtr ptr, int cbSize)
    {
        return new IntPtr(ptr.ToInt64() + cbSize);
    }

    public static IntPtr Increment<T>(this IntPtr ptr)
    {
        return ptr.Increment(Marshal.SizeOf(typeof(T)));
    }

    public static T ElementAt<T>(this IntPtr ptr, int index)
    {
        var offset = Marshal.SizeOf(typeof(T))*index;
        var offsetPtr = ptr.Increment(offset);
        return (T)Marshal.PtrToStructure(offsetPtr, typeof(T));
    }
}

[1] It’s highly advisable to not calculate this value yourself.

[2] See the post “Is IntPtr(long) truncating?” for more details

Dereference a double IntPtr

Jared Parsons — Wed, 05 Nov 2008 16:00:10 GMT

A common PInvoke question is how to deal with a double pointer. More specifically, how can one dereference an IntPtr to another pointer without using unsafe code?

Dereferencing a double pointer is done the same way a dereference to any other structure is done: Marshal.PtrToStructure. PtrToStructure is used to transform a native pointer, in the form of an IntPtr, into a managed version of the native data structure the native pointer points to.

In the case of a double pointer, the native data structure the pointer points to is just another native pointer. The managed equivalent is the IntPtr (or UIntPtr) class.

For Example, say we had the following native data signature

void GetDoublePointer(int** ppData)

This function returns a pointer to a pointer that points to an int. (Pointer->Pointer->int). We can then use the following C# code to access the final int value.

[DllImport("PInvokeSample.dll")]
static extern void GetDoublePointer(IntPtr doublePtr);

static void Main(string[] args)
{
    var ptr = Marshal.AllocHGlobal(Marshal.SizeOf(typeof(IntPtr)));
    try
    {
        GetDoublePointer(ptr);
        var deref1 = (IntPtr)Marshal.PtrToStructure(ptr, typeof(IntPtr));
        var deref2 = (int)Marshal.PtrToStructure(deref1, typeof(int));
        Console.WriteLine(deref2);
    }
    finally
    {
        Marshal.FreeHGlobal(ptr);
    }
}

Is IntPtr(long) truncating?

Jared Parsons — Tue, 28 Oct 2008 15:00:00 GMT

The short answer is: No, not when it matters

A colleague and I were discussing a particular scenario around IntPtr,PInvoke and 64 bit correctness. Eventually our discussion lead us to the IntPtr constructor which takes a long. To my surprise the code for the constructor is the following.

public unsafe IntPtr(long value) { this.m_value = (void*) ((int) value); }

The problem is long value is arbitrarily truncated to an int. This has the effect of essentially losing any address over the 4 GB range (in other words, no 64 bit addresses). This much to big of a hole to actually be the real behavior so I decided to see if it was a bug in the disassembler. I was using .Net Reflector so I switched to IL mode.

    L_0000: ldarg.0 
    L_0001: ldarg.1 
    L_0002: conv.ovf.i4 
    L_0003: conv.i 
    L_0004: stfld void* System.IntPtr::m_value
    L_0009: ret

This confirmed it is indeed truncating the value (and doing an overflow check to boot). But wait, mscorlib.dll is a processor specific DLL so perhaps this is just a 32 bit OS thing. I switched over to a 64 bit machine, fired up Reflctor and found to my dismay that it had the exact same code.

After a few minutes I thought to open up task manager and to my surprise reflector was running in a WoW64 bit process. This meant it was still loading up the 32 bit version of mscorlib.dll. Next I fired up ildasm, loaded up a 64 bit mscorlib and confirmed that the code will not truncate on 64 bit machines.

  IL_0000:  ldarg.0
  IL_0001:  ldarg.1
  IL_0002:  conv.u
  IL_0003:  stfld      void* System.IntPtr::m_value
  IL_0008:  ret

The conv.u code is a conversion to unsigned native platform int. On a 64 bit machine this will be an unsigned 8 byte number(see OpCodes.Conv_U for more details).

So what does this mean for the developer. Essentially IntPtr(long) will do the right thing independently of the platform a developer is using. On a 32 bit platform it will (correctly) throw exceptions if a non-4GB address is passed in. In 64 bit land it will essentially do nothing and rely on the programmer to give correct addresses.

PInvoke and bool (or should I say BOOL)?

Jared Parsons — Tue, 14 Oct 2008 15:00:00 GMT

Boolean value types are a constant source of problems for people attempting to generate PInvoke signatures. It's yet another case of managed and native types differing on size.

There are two common boolean types in native code: bool and BOOL. As with most PInvoke issue the main issue is to understand the size

bool: 1 byte

bool SomeOperation();

BOOL: 4 bytes

BOOL SomeOtherOperation();

Fortunately there is only one managed boolean type. Unfortunately though, it has 2 sizes that must be considered. In managed code a bool is a single byte. Yet the default marshalling for a bool will expand it to 4 bytes. So even though bool is 1 byte in managed code, it is by default compatible with the 4 byte native version BOOL.

Translating a PInvoke signature for a native 1 byte bool is simple enough though. The UnmanagedType.I1 value allows for this scenario. It tells the CLR that while the managed code has a boolean value the corresponding native item is the 1 byte version of bool. The CLR will then make the necessary size adjustments. When converting native -> managed it will consider a non-zero value true and 0 to be false.

Native:

bool SomeOperation();

Managed:

[return: MarshalAs(UnmanagedType.I1)]
public static extern bool SomeOperation();

BOOL's can be usefully marshalled in two different ways. The first is simply as a int (signed or unsigned). This works because int matches the size requirement of a BOOL and the sign is not really important as true or false is simply 0 or not 0.

public static extern int SomeOtherOperation();

This is certainly sub-optimal since we're now returning a non-boolean type for what is intended to be a boolean operation but some people choose to Marshal it this way.

Marshaling a native BOOL to a managed bool works without any annotations. However it is preferred that you include the annotations to make it much clearer what the native data type is. The UnmanagedType enumeration contains a Bool value for just this purpose. This can be applied to any managed bool return or parameter.

[return: MarshalAs(UnmanagedType.Bool)]
public static extern bool SomeOtherOperation();

And presto, we have maintained the boolean semantics of the operation.

Hint: If you are using the PInvoke Interop Assistant, it will automatically do these conversions for you.

Edit

Fixed a mixup on the default native size of bool. Thanks to Jachym Kouba for pointing out the mistake

When is a LONG not a long?

Jared Parsons — Thu, 02 Oct 2008 15:00:00 GMT

Answer: When PInvoke is involved.

I ran across a common error today on stackoverflow regarding P/Invoke that is worth blogging about. The question regarded the translation of a native API with a parameter of type LONG. The user mistakenly used the .Net long type as the parameter. The error is that a C++ LONG is not the same as a .Net long.

When talking about types and PInvoke it's easier to discuss byte size and signed-ness than type names. Otherwise confusion around long and short crop up. Really there are four integer byte sizes each of which can be signed or unsigned: 1,2,4 and 8

The problem the user encountered is the C++ long is 4 byte signed and .Net long is 8 byte signed. PInvoke requires the parameters to have the same size. Below is a quick table of the various types in C++ and .Net.

1 byte

C++ - char, __int8, BYTE, BOOLEAN
.Net - byte

2 byte

C++ - wchar, __int16, short, WORD
.Net - char, short

4 byte

C++ - int, LONG, long, __int32, DWORD
.Net - int

8 byte

C++ - __int64, LONGLONG, DWORDLONG, LARGE_INTEGER
.Net - long

Based on this table when translating a C++ LONG, you should use a .Net int.

Edit1: Moved C++ short to 2 byte, added several other C++ types. (thanks Raymond)

PInvoke Interop Assistant on Channel 9

Jared Parsons — Thu, 11 Sep 2008 15:00:56 GMT

Beth Massi dropped by my office a couple of months ago and we did a channel 9 video on the PInvoke Interop Assistant. Mainly an overview of the product and a bit of a tutorial. Check out the video here.

http://channel9.msdn.com/posts/funkyonex/The-P-Invoke-Interop-Assistant/

PInvoke Interop Assistant now on CodePlex

Jared Parsons — Tue, 17 Jun 2008 15:00:53 GMT

I'm happy to announce that the PInvoke Interop Assistant tool is now available on CodePlex. This includes the binaries, source code and the dev authored unit tests.

It is hosted on the more general CLR Interop Tools page.

http://www.codeplex.com/clrinterop

I will be actively maintaining this tool in the future and (hopefully) adding more features. There were several features we cut just before release due to QA costing that I would like to add back (including VS integration, wrapper functions).

Please post any suggestions you have for the tool on the site and we will definitely consider them.

Making PInvoke Easy

Jared Parsons — Fri, 14 Mar 2008 19:58:39 GMT

I very excited to announce we recently released a tool I've been working on to MSDN that will greatly help with using PInvoke in managed code. The tool is called the "PInvoke Interop Assistant" and is included as part of a MSDN article on marshalling data for PInvoke and Reverse PInvoke scenarios.

Here is a link to the article and tool

Article: http://msdn2.microsoft.com/en-us/magazine/cc164193.aspx
Tool: CLRInsideOut2008_01.exe

The motivation behind this tool is writing PInvoke is a hard and often tedious task. There are many rules you must obey and many exceptions that must be taken into account. Anything beyond simple data structures gets very involved and subtle semantics of C can greatly change the needed signature. Incorrect translations often result in obscure exceptions or crashes.

In short, it's not any fun.

The tool works in several different ways to make PInvoke generation an easier process. The goal is to make generating managed code for structs, unions, enums, constants, functions, typedefs , etc ... as easy as possible. The resulting code can be generated in both VB and C#.

The GUI version of the tool operates in 3 modes.

SigImp Search: Search for a commonly used function and translate it into managed code.
SigImp Translate Snippet: Directly translate C code into managed PInvoke signatures.
SigExp: Convert managed binaries into C++ Reverse PInvoke scenarios

The first two are the parts I worked on and represent the PInvoke scenarios. The third part was written by Ladi Prosek and will be covered in a different article. We chose the names SigImp and SigExp to mirror the tblimp/tlbexp tool base since they have similar functions.

Directly translating C code into PInvoke Signatures

Most adventures in PInvoke start with a developer having a small set of C code they would like to use from a managed binary. Typically it's one or two functions with several supporting C structs. Before, all of this would be hand translated into managed code from scratch. With this tool all you must do is paste the code into the tool and it will generate the interop signature for you.

For instance assume you wanted to translate the following C code into VB.

struct S1
{
  int a;
  char[10] b;
};

float CalculateData(S1* p);

Start up the tool and switch to the "SigImp Translate Snippet" tab. Then paste the code in and then hit the Generate button.

You can also set click the "Auto Generate" box and watch the code update as you type.

This translation is not limited to built-in C types. It will also resolve most commonly used windows types such as HANDLE, DWORD all the way up to complex structs such as WIN32_FIND_DATA

Searching for a commonly used function

Often developers want to use C functions familiar to them in managed code. This can be a tedious task as well because if the signature is not already available you are back to coding from scratch. Even adding a constant value can be tricky if you don't know which header file to look in.

The tool also provides a database of many commonly used functions, structs, constants, etc ... It is essentially anything that is included from windows.h. Switch to the SigImp search tab, type the name of what you are looking for and hit generate. For example if I want to see the value for WM_PAINT just type it in.

In addition this part of the tool will also do dependency calculation. For instance if choose a method which has a parameter that is a C structure it will automatically generate the structure with the function. For instance if you choose the function FindFirstFile it will determine that the function depends on the WIN32_FIND_DATA structure. Furthermore it will notice that WIN32_FIND_DATA depends on FILETIME and generate both in addition to the method.

<System.Runtime.InteropServices.StructLayoutAttribute( _
    System.Runtime.InteropServices.LayoutKind.Sequential, _
    CharSet:=System.Runtime.InteropServices.CharSet.[Unicode])> _
Public Structure WIN32_FIND_DATAW
    '''DWORD->unsigned int
    Public dwFileAttributes As UInteger
    '''FILETIME->_FILETIME
    Public ftCreationTime As FILETIME
    '''FILETIME->_FILETIME
    Public ftLastAccessTime As FILETIME
    '''FILETIME->_FILETIME
    Public ftLastWriteTime As FILETIME
    '''DWORD->unsigned int
    Public nFileSizeHigh As UInteger
    '''DWORD->unsigned int
    Public nFileSizeLow As UInteger
    '''DWORD->unsigned int
    Public dwReserved0 As UInteger
    '''DWORD->unsigned int
    Public dwReserved1 As UInteger
    '''WCHAR[260]
    <System.Runtime.InteropServices.MarshalAsAttribute( _
        System.Runtime.InteropServices.UnmanagedType.ByValTStr, SizeConst:=260)> _
    Public cFileName As String
    '''WCHAR[14]
    <System.Runtime.InteropServices.MarshalAsAttribute( _
        System.Runtime.InteropServices.UnmanagedType.ByValTStr, SizeConst:=14)> _
    Public cAlternateFileName As String
End Structure

<System.Runtime.InteropServices.StructLayoutAttribute( _
    System.Runtime.InteropServices.LayoutKind.Sequential)> _
Public Structure FILETIME
    '''DWORD->unsigned int
    Public dwLowDateTime As UInteger
    '''DWORD->unsigned int
    Public dwHighDateTime As UInteger
End Structure

Partial Public Class NativeMethods
    '''Return Type: HANDLE->void*
    '''lpFileName: LPCWSTR->WCHAR*
    '''lpFindFileData: LPWIN32_FIND_DATAW->_WIN32_FIND_DATAW*
    <System.Runtime.InteropServices.DllImportAttribute("kernel32.dll", EntryPoint:="FindFirstFileW")> _
    Public Shared Function FindFirstFileW( _
        <System.Runtime.InteropServices.InAttribute(), _
            System.Runtime.InteropServices.MarshalAsAttribute(System.Runtime.InteropServices.UnmanagedType.LPWStr)> _
            ByVal lpFileName As String, _
        <System.Runtime.InteropServices.OutAttribute()> _
        ByRef lpFindFileData As WIN32_FIND_DATAW) As System.IntPtr
    End Function
End Class

Translating Large Code bases

The snippet translator works well for small snippets of code. If you are trying to translate a much larger code base, say several interdependent header files the small snippet dialog won't work well. To work with larger code bases you should use the command line version of the tool; sigimp.exe. It is designed to process several header files and produce a mass output.

Wrapping Up

This tool started out as a pet project of mine some time ago. I'm extremely excited that customers are now going to be able to take advantage of it and I greatly look forward to any feedback you have. I will post a couple more articles in the future detailing how this tool works under the hood.

Follow up: PInvoke with 32 and 64 bit

Jared Parsons — Fri, 14 Apr 2006 00:44:00 GMT

Recently after I made my original post titled “PInvoke with 32 and 64 bit”, a developer emailed me. They pointed out that there is an internal structure that dynamically resizes based on the architecture; IntPtr. IntPtr will be size of the native chip pointer value. So for values like size_t that map to the pointer size, you can use the Address field of IntPtr.

This approach has some small down sides.

The type you are marshaling must have the same size as IntPtr for all of the architecture's you run on
If you expose an interop API that uses IntPtr in this manner, you have to find a good way to inform your users that they really need to pass the value in the address of the IntPtr

A way to get around #2 is to define your own type (like I did with SizeT) and just wrap an IntPtr. If you have no other fields in your type it will Marshall like an IntPtr. You can provide friendly properties that assign values into the IntPtr's Address property.

If you have a type who's size does not change in the same way as IntPtr then you'll probably have to go the custom marshal route

Marshaling Nested Data Structures Part 4

Jared Parsons — Mon, 18 Jul 2005 16:51:00 GMT

This is part 4 of a series. You can find part one here. Please refer to that article for all of the Native definitions of the structures that I use here.

In the previous article we were left with a solution where using the code was very clean but the actualy implementation had extra allocation and perf overhead. We'll conquer both of those in this installment by implementing a custom marshaler for our Course object. This is accomplished by implementing ICustomMarshaler and call the class CourseMarshaler. (ICustomMarshaler docs)

Custom marshaling is exactly like it sounds. The marshaler itself does practically nothing. Your code must transform the managed data structure into a native format and vice versa. It can be tedious at times but this method can be used to marshal even the most complex of structures. There are 6 methods you must implement. Lets go over each of them.

CleanupData() - Use this method to dispose of anything on your managed object when it's no longer needed. Typically there is nothing to do here and this method remains blank as it does in our case.

CleanupNativeData() - Use this method to free up any data associated with the Native pointer after the runtime is finished using it. In our case, we have to allocate memory to Marshal the data structure into, so we need to free up this pointer.

public void CleanUpNativeData(IntPtr pNativeData)
{
    Marshal.FreeCoTaskMem(pNativeData);
}

GetNativeDataSize() - Returns the size of the unmanaged data structure. As previously discussed, this size is 268.

MarshalManagedToNative() - Takes in an object and returns a pointer to the memory that contains the Native structure in memory. To complete this we need to allocate a block of memory, and then marshal each of the fields in order into that memory block and return the pointer to the front of the block. The pointer we allocate will be freed later when the runtime passes it into CleanupNativeData(). Some error checking was removed for the sake of brevity.

public IntPtr MarshalManagedToNative(object managedObj)
{
    Course course = (Course)managedObj;

    IntPtr ptr = Marshal.AllocCoTaskMem(this.GetNativeDataSize());
    if (IntPtr.Zero == ptr)
    {
        throw new Exception("Could not allocate memory");
    }

    // Write the Int values in order into memory
    Marshal.WriteInt32(ptr, 0, course.Id);
    Marshal.WriteInt32(ptr, Marshal.SizeOf(typeof(Int32)), course.Count);
            
    // Now we need to Marshal each of the Student elements into the "array".  This 
    // starts immediately after the Ints
    IntPtr cur = new IntPtr(ptr.ToInt32() + (2 * Marshal.SizeOf(typeof(Int32))));
    for (int i = 0; i < course.Count; i++)
    {
         Student student = course.Students[i];
         Marshal.StructureToPtr(student, cur, false);
         cur = new IntPtr(cur.ToInt32() + Marshal.SizeOf(typeof(Student)));
    }
    return ptr;
}

MarshalManagedToNative() - Marshal the Native Struct into a managed version. This is almost identical to the sample that we did in part 3. Code reposted below.

public object MarshalNativeToManaged(IntPtr ptr)
{
    int courseId = Marshal.ReadInt32(ptr);
    int count = Marshal.ReadInt32(ptr, Marshal.SizeOf(typeof(Int32)));

    // Set the int values
    Course course = new Course();
    course.Id = courseId;
    course.Count = count;

    // Now read out the Student structures
    ptr = new IntPtr(ptr.ToInt32() + (2 * Marshal.SizeOf(typeof(Int32))));
    for (int i = 0; i < count; i++)
    {
        Student student = (Student)Marshal.PtrToStructure(ptr, typeof(Student));
        course.Students[i] = student;
        ptr = new IntPtr(ptr.ToInt32() + Marshal.SizeOf(typeof(Student)));
    }

    return course;
}

GetInstance() - This method is not a part of the ICustomMarshal interface but it's a static method that must be implemented by any custom marashaler. The runtime uses this to create an instance of the object.

Now we have a complete implementation of ICustomMarshal. Really the only new method that we had to implement was MarshalManagedToNative() and that's just the opposite of what we did in part 3. There are a couple of tidbits left that we have to alter.

The first is that we must convert the managed Course from a struct to a class. This is very important. A custom marshaler can only be applied to reference types. Changing Course to a class has a couple of other implications as well. Structs in C# are stored in the stack and reference types are stored on the heap. This has implications to Marshalling as well. When you Marshal a struct (or any ValueType), the runtime is expecting to find a stack based value (or better, a non pointer value) on the Native end. Now we are Marshalling a reference type so the runtime will expect to find a pointer value on the other end.

Also we can do away with the StructLayout attribute on the Course type. We are hand Marshaling this now so we don't need to provide any hints to the runtime. Now we are left with just a vanilla class.

public class Course
{
    public int Id;
    public int Count;
    public List<Student> Students = new List<Student>(5);
}

Now we just need to inform the runtime about how to link our custom marshaler (called CourseMarshaler in my code) to the Course class. Every place that we declare a P/Invoke method we need to add custom Marshalling data. This is done by adding the MarshalAs attribute. Lets use the GetCourseInfo() method for an example. Here is our updated definition.

[DllImport("Enrollment.dll", CharSet = CharSet.Unicode)]
[return: MarshalAs(UnmanagedType.CustomMarshaler, MarshalTypeRef=typeof(CourseMarshaler))]
public static extern Course GetCourseInfo(int id);

We've made two important changes here as well. The first is we changed the return type of the method from an IntPtr to a Course. Remember that this is a class now so the runtime is expecting a pointer value. We take advantage of that here. Also we've added the MarshalAs attribute to the return type to tell the runtime to use the Custom marshaler that we created.

This makes the use code even cleaner since we aren't dealing with an IntPtr return type anymore.

static void Main(string[] args)
{
    Course course = Enrollment.GetCourseInfo(42);
    Student first = course.Students[0];
}

That essentially concludes this series on Marshalling Nested Data Structures. I may add an additional chapter on common tips for debugging common marshalling problems if I have some time. Hope you enjoyed this.

This posting is provided "AS IS" with no warranties, and confers no rights.
Use of included script samples are subject to the terms specified at http://www.microsoft.com/info/cpyright.htm.

Marshaling Nested Data Structures Part 3

Jared Parsons — Fri, 15 Jul 2005 02:58:00 GMT

This is part 3 of a series. You can find part one here. Please refer to that article for all of the Native definitions of the structures that I use here.

In part 2 of this series I demonstrated how you can Marshal nested data structures by flattening an array of structures into it's individual elements. Using this is very awkward and is only practical when the size of your array is small. This installment will get around these limitations with a bit of Marshalling help. We'll get back indexing and remove the awkward flattening.

Once again, don't forget that the only 2 things that matter when Marshalling data

Byte Size
Byte Layout

When an array of structures are declared as a member of a struct, in the same manner as the Course structure, the data is inlined after the first two ints. We can get this data to Marshall properly by inserting a structure with the same size as the array of Students. The Native Student Structure is 52 bytes so we need a 260 byte structure. The following will do.

[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Unicode, Size = 260 )]
    public struct ArrayBlob
    {
    }

[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Unicode)]
    public struct Course
    {
        public int Id;
        public int Count;
        public ArrayBlob Blob;

        /* ... Methods ... */
    }

This structure meets all of our requirments. Course is now properly formatted and can be Marshalled back and forth successfully. However our student array is stuck in a chunk of data that has no access methods. This blob is just an array of students in memory. Getting the Students out in C would be a snap. We could just grab the address, cast it the the appropriate pointer and run away with the Student elements.

Student *elements = (Student*)(&(myCourse.Blob);

Unfortunately I'm giving these examples in Safe code so this is not allowed. You can get the same effect with C# although it's a bit slower and forces a memory allocation. We implement this as in indexer into the Course struct.

public Student this[int index]
        {
            get
            {
                if ( index >= Count )
                {
                    throw new ArgumentOutOfRangeException();
                }

                IntPtr ptr = Marshal.AllocHGlobal(Marshal.SizeOf(typeof(ArrayBlob)));
                try
                {
                    Marshal.StructureToPtr(this.Blob, ptr, false);
                    int offset = index * Marshal.SizeOf(typeof(Student));
                    IntPtr structPtr = new IntPtr(ptr.ToInt32() + offset);
                    return (Student)Marshal.PtrToStructure(structPtr, typeof(Student));
                }
                finally
                {
                    Marshal.FreeHGlobal(ptr);
                }
            }
        }

Some error checking was removed for brevity sake. Granted this is still a suboptimal way of accessing the members of the structure. Every index requires an allocation. But if you only interop for brief times and in non perf sensitive areas this code will do just fine. Here's an example.

public static class Enrollment
    {
        [DllImport("Enrollment.dll", CharSet = CharSet.Unicode)]
        public static extern IntPtr CreateStudent(
            [MarshalAs(UnmanagedType.LPWStr)] String firstName,
            [MarshalAs(UnmanagedType.LPWStr)] String lastName,
            int bDay,
            int bMon,
            int bYear);

        [DllImport("Enrollment.dll", CharSet = CharSet.Unicode)]
        public static extern void UpdateStudentInfo(
            [MarshalAs(UnmanagedType.LPWStr)] String firstName,
            [MarshalAs(UnmanagedType.LPWStr)] String lastName,
            int bDay,
            int bMon,
            int bYear,
            ref Student student);

        [DllImport("Enrollment.dll", CharSet = CharSet.Unicode)]
        public static extern IntPtr GetCourseInfo(int id);
    }


    class MarshalFun
    {
        static void Main(string[] args)
        {
            IntPtr ptr = Enrollment.GetCourseInfo(42);
            Course course = (Course)Marshal.PtrToStructure(ptr, typeof(Course));

            Student first = course[0];
            Student second = course[1];
        }
    }

This code assumes that at least 2 students were returned from GetCourseInfo(). This code is much cleaner than the previous example. It will also work with very large sized arrays. However it does have undue perf and memory overhead. Next time we'll look at a way to have the clean code and remove all of the perf problems present in the current code.

This posting is provided "AS IS" with no warranties, and confers no rights.
Use of included script samples are subject to the terms specified at http://www.microsoft.com/info/cpyright.htm.

Marshaling Nested Data Structures Part 2

Jared Parsons — Tue, 12 Jul 2005 18:00:00 GMT

This is part 2 of a series. You can find part one here. Please refer to that article for all of the Native definitions of the structures that I use here.

The most important thing to remember when Marshalling data is that the .Net Runtime really only cares about byte layout and how many bytes you are trying to move. If you mangle your managed structure to match the byte size and layout of your unmanaged structure, the runtime will Marshal it sucessfully almost all of the time. Note by successfully I mean that the method will complete without causing the dreaded AccessViolationException but the data might not be quite what you were expecting.

For example. Here is the typical managed definition for the Student struct definition.

    [StructLayout(LayoutKind.Sequential, CharSet = CharSet.Unicode)]
    public struct Student
    {
        [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 10)]
        public string FirstName;

        [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 10)]
        public string LastName;

        public int BirthDay;
        public int BirthMonth;
        public int BirthYear;
    }

Notice that the size of this structure is 52 bytes.

FirstName 2 (size of char) * 10 (size of array)
LastName 2 (size of char) * 10 (size of array)
12 bytes (4 bytes for each of the int's)

This is the same size as the native definiton of Student. In memory these bytes will all appear in order with no spaces between them (FirstName and LastName are set as ByValTStr's so they will be inlined). So LastName is 20 bytes offset of a student structure, BirthDay is 40 bytes offset and so on.

You don't always have to be so perfect though. In this case it's only important because we want to accesss the fields of the managed structure. If this was not important to us, we could have just as easily declared the structure as so.

    [StructLayout(LayoutKind.Sequential, Size = 52, CharSet = CharSet.Unicode)]
    public struct Student
    { 

    }

Note that i didn't leave anything out in the structure above. It has the correct CharSet and Size so this could be successfully used wherever we need the Student structure. It just doesn't help us very much in Managed code :)

Now onto the Course. A lot of developers get tripped up when trying to define structs like Course in managed code. Don't forget, all we need to do is get the bytes correct and the Marshaller will work out the rest.

This structure is very small. There are only 5 inlined elements in the student array. This means that we could easily define our structure as so.

    [StructLayout(LayoutKind.Sequential, CharSet = CharSet.Unicode)]
    public struct Course
    {
        public int Id;
        public int Count;
        Student Student0;
        Student Student1;
        Student Student2;
        Student Student3;
        Student Student4;
    }

I realize there are several downsides to defining our structure like this. For one, it doesn't look very clean. Secondly we've lost the ability to easily index the array of students. Also this works easily because there are only 5 elements in the array. If there were much more than that this wouldn't be pheasible at all. However this structure is layed out exactly like the one defined in Native code so this will Marshal correctly.

The next part to this series will try to make things a bit more pleasing to the eye. We'll get back indexing at the cost of a bit of manual Marshalling.

This posting is provided "AS IS" with no warranties, and confers no rights.
Use of included script samples are subject to the terms specified at http://www.microsoft.com/info/cpyright.htm.

Marshaling Nested Data Structures Part 1

Jared Parsons — Mon, 11 Jul 2005 19:47:00 GMT

A frequent question that pops up on newsgroups such as microsoft.dotnet.framework.interop is how to Marshal nested structures (and arrays of nested structures) via P/Invoke. The documentation on the subject that I have found usually only covers structures that defined in the UnmanagedType enumeration and just touch on the idea of structures containing nested arrays of custom structures. Part of the reason is that the explinations for these problems is quite involed and scenario specific. After answering it a few times I decided to make a multi part blog post on the subject. This will consist of at least 4 entries (might add some more if something comes up).

Index

The first installment is just going to be laying out our Native data structures that will be used throughout this series. I'm going to use a generic(and quite contrived) university classroom examle. Here are the C struct definitions and a couple of functions that I will be using.

struct Student 
{ 
  WCHAR FirstName[10]; 
  WCHAR LastName[10]; 
  int BirthDay; 
  int BirthMonth; 
  int BirthYear; 
}; 

struct Course{ 
  int Id; 
  int Count; 
  Student Students[5]; 
}; 

	// Create a student with the specified information.  Returned structure should be freed with 
	// GlobalFree
__declspec(dllexport) Student* CreateStudent(
    LPCWSTR firstName, 
    LPCWSTR lastName, 
    int bDay, 
    int bMon, 
    int bYear)

	// Update an existing student with the specified information
__declspec(dllexport) void UpdateStudentInfo(
    LPCWSTR firstName, 
    LPCWSTR lastName, 
    int bDay, 
    int bMon, 
    int bYear, 
    Student *student)

	// Get information for a particurlar course.  Returned structure should be freed with 
	// GlobalFree
__declspec(dllexport) Course *GetCourseInfo(int courseId)

The environment I am coding and testing my fixes in is Microsoft Visual Studio 2005 Beta 2. All of these examples should work in all versions of Visual Studio though. If you find an inconsistencies please let me know. All native examples will also be in Unicode.

This posting is provided "AS IS" with no warranties, and confers no rights.
Use of included script samples are subject to the terms specified at http://www.microsoft.com/info/cpyright.htm.

Interop and Unions

Jared Parsons — Fri, 13 May 2005 18:31:00 GMT

A frequent question I see pop up on news groups is how to create Unions in managed code for interop with P/Invoke. .NET does not directly support unions but you can use some Marshal magic to get them working. Take the following C union.

union Foo

{

int a;

float b;

}

Really what this translates to in the native world is a data structure that is 4 bytes long (the key to Marshalling is never forgetting that you are just dealing with bytes). To simulate this in .NET you could probably get away with creating a structure that is 4 bytes long and passing that in. You'd loose the easy type switch that you get with unions but it would most likely work. To get the nice type switch you do the following.

[StructLayout(LayoutKind.Explicit)]

public struct ManagedFoo

{

[FieldOffset(0)] int a;

[FieldOffset(0)] float b;

}

The attributes here will cause the fields to both be aligned at the very begining of the structure. This gives you the benefits of a Union in C# and the compatibility to Marshall with P/Invoke.

There is one catch with this technique. You cannot mix value and reference types in a Managed union. If you do you will get a nasty type load exception at runtime. The best work around in that case is to use an IntPtr in place of the reference types and do some additional Marshalling on that field.