SpankyJ : 64bit CLR Implementation Goo

BigArray, getting around the 2GB array size limit

joshwil — Thu, 11 Aug 2005 04:14:00 GMT

I’ve received a number of queries as to why the 64-bit version of the 2.0 .Net runtime still has array maximum sizes limited to 2GB. Given that it seems to be a hot topic of late I figured a little background and a discussion of the options to get around this limitation was in order.

First some background; in the 2.0 version of the .Net runtime (CLR) we made a conscious design decision to keep the maximum object size allowed in the GC Heap at 2GB, even on the 64-bit version of the runtime. This is the same as the current 1.1 implementation of the 32-bit CLR, however you would be hard pressed to actually manage to allocate a 2GB object on the 32-bit CLR because the virtual address space is simply too fragmented to realistically find a 2GB hole. Generally people aren’t particularly concerned with creating types that would be >2GB when instantiated (or anywhere close), however since arrays are just a special kind of managed type which are created within the managed heap they also suffer from this limitation.

<Sidenote> managed arrays: arrays are a first class type in the CLR world and they are laid out in one contiguous block of memory in the managed garbage collected heap. In the CLR 1.1 they can be thought of as the only generic type (in 2.0 we’re introducing a much more universal concept of generics) in that you can have an array that is of the type of any managed type that you like (primitive types, value types, reference types). It is interesting to think about what that means in context of the 2GB object instance size limit imposed on objects in the managed heap. With value types (bool, char, int, long, struct X {}, etc…) the actual data of the instance for each element in the array will be laid out contiguous with the next element in memory, since the 2GB limit discussed earlier applies to the total array size, and the array size is a factor of the type size the maximum number of elements you can store in an array of type X will vary proportionally to the size of X.

Differing from this are arrays of reference types (e.g. objects, strings, class Y {}, etc…), for these arrays the actual array will be that of a bunch of references, initially null. To initialize the array your code will need to go through one element at a time and create or assign an appropriate instance of the type to that array element. The 2GB size limit for arrays applies to this array of references, not the instances of the objects themselves. On a 32-bit machine if you create an array of type object (object[]) and one instance of type object per element in the array then your available virtual address space will end up limiting the size of your array as you will never be able to fit enough objects in memory to be able to fill up a 2GB object array with unique object references.</Sidenote>

The developer visible side of this is that array indexes are signed integers (with a byte[] you can use the full positive space of the signed integer as an index (assuming the array is 0 based), with other types you use some subset of that space until the total array size is 2GB). While some of the BCL APIs that deal with arrays have alternate signatures that take longs this isn’t yet ubiquitous in the framework (i.e. the IList interface (which the BCL’s Array class implements) uses int indexes).

It is debatable whether or not we should have included a “Big Array” implementation in the 2.0 64-bit runtime, and I’m sure that debate will rage for some years to come. However, as 2.0 is getting ready to ship and there currently isn’t any support for this we are going to have to live without it until at least the next version.

So, what is there to do in .Net 2.0 if you have an application which requires arrays that are very large?

Well, first switch to 64-bit! As mentioned, it is next to impossible to allocate a full 2GB array on 32-bit because of the way that the virtual address space is broken up by modules and other various allocations. Simply switching to 64-bit will buy you the ability to allocate those full 2GB blocks (well, close anyway, the total object size is limited to 2GB, but there is some CLR book-keeping goo in there that takes a few bytes).

What if that still isn’t enough? You have a couple of choices:

A) Rethink your application’s design? Do you really need a single gigantor array to store your data? Keep in mind that if you’re allocating 8GB of data in a single array and then accessing it in a sparse and random manner you’re going to be in for a world of paging pain unless you have a ton of physical memory. It is very possible that there is another data organization scheme you can use where you can group data into frequently used groups of some sort or another and manage to keep under the 2GB limit for an individual group. If you choose correctly you can vastly improve your applications performance due to lower paging and better cache access characteristics that come from keeping things that are used together close to one another.

B) Use native allocations. You can always P/Invoke to NT’s native heap and allocate memory which you can then use unsafe code to access. This isn’t going to work if you want to have an array full of object references, but if you just need a huge byte[] to store an image this might work out fine, even great. The added cost of the P/Invoke is low because the NT APIs have simple signatures that don’t require marshaling and the code executed when allocating an 8GB block is probably mostly zeroing the memory anyway. If you choose this option you will have to write a small memory management class of some kind and be comfortable using unsafe code. I know that Paint.Net (http://blogs.msdn.com/joshwil/archive/2005/04/07/406218.aspx) uses this very method for allocating the memory in which they store the image (and it’s various layers) which you’re editing. This is a good solution for the case where you really need a single unbroken allocation of some large size. While it isn’t a very general purpose solution it works out great for the Paint.Net guys.

C) Write your own BigArray class.

I’d stress that option C is my least favorite of the above three, but I will acknowledge that there are probably cases where it is the right thing to do. Given that, I have gone and written one myself. This is a very bare bones implementation, just the array allocation and accessors are implemented, I will leave implementing any extended functionality (like the functionality provided by the static members of the Array class, Sort, Copy, etc… or writing big collections on top of it) as an exercise for the reader.

// Goal: create an array that allows for a number of elements > Int.MaxValue
class BigArray<T>
{
    // These need to be const so that the getter/setter get inlined by the JIT into
    // calling methods just like with a real array to have any chance of meeting our
    // performance goals.
    //
    // BLOCK_SIZE must be a power of 2, and we want it to be big enough that we allocate
    // blocks in the large object heap so that they don't move.
    internal const int BLOCK_SIZE = 524288;
    internal const int BLOCK_SIZE_LOG2 = 19;

    // Don't use a multi-dimensional array here because then we can't right size the last
    // block and we have to do range checking on our own and since there will then be
    // exception throwing in our code there is a good chance that the JIT won't inline.
    T[][] _elements;
    ulong _length;

    // maximum BigArray size = BLOCK_SIZE * Int.MaxValue
    public BigArray(ulong size)
    {
            int numBlocks = (int)(size / BLOCK_SIZE);
            if ((numBlocks * BLOCK_SIZE) < size)
            {
                numBlocks += 1;
            }

            _length = size;
            _elements = new T[numBlocks][];
            for (int i=0; i<(numBlocks-1); i++)
            {
                _elements[i] = new T[BLOCK_SIZE];
            }
            // by making sure to make the last block right sized then we get the range checks
            // for free with the normal array range checks and don't have to add our own
            _elements[numBlocks-1] = new T[NumElementsInLastBlock];
    }

    public ulong Length
    {
        get
        {
            return _length;
        }
    }

    public T this[ulong elementNumber]
    {
        // these must be _very_ simple in order to ensure that they get inlined into
        // their caller
        get
        {
            int blockNum = (int)(elementNumber >> BLOCK_SIZE_LOG2);
            int elementNumberInBlock = (int)(elementNumber & (BLOCK_SIZE – 1));
            return _elements[blockNum][elementNumberInBlock];
        }
        set
        {
            int blockNum = (int)(elementNumber >> BLOCK_SIZE_LOG2);
            int elementNumberInBlock = (int)(elementNumber & (BLOCK_SIZE – 1));
            _elements[blockNum][elementNumberInBlock] = value;
        }
    }
}

The beauty of this implementation is that the JIT already understands single dimensional array accesses intrinsically, including range checking code. In practice this class ends up being almost as fast as real array access for small arrays (< BLOCK_SIZE) and not too much slower once you get to reasonably big arrays. It doesn’t waste much space compared to a normal array because the last block is right sized and the performance is good because the getter and setter for array elements are simple enough that they should get inlined into the calling method, this becomes very important for getting anywhere close to normal array access speeds.

Here is an example of big array usage:

public static void Main()
{
    long size = 0x1FFFFFFFF;
    BigArray<int> baInt = new BigArray<int>(size);
    long len = baInt.Length;
    for (long i=0; i<len; i++)
    {
        baInt[i] = i;
    }
    Console.WriteLine("baInt[len/2]=" + baInt[len/2]);
}

You could imagine also exposing the fact that this BigArray<T> implementation has blocks through a couple of properties and a indexer of this[int block, int element] which would allow people to intelligently write code to do block based access on the array (e.g. merge sorts that are block intelligent). This can be important for performance as we know that elements within a single block are contiguous in memory, however we cannot make that guarantee about elements in neighboring blocks.

It is worth noting that given the allocation scheme of the BigArray<T> constructor we may very well have multiple garbage collections while it runs, because of this you don’t really want to be using large instances of this class in a throw away manner. My advice would be to use this carefully and sparingly, instead favoring architectures which don’t require such large single arrays.

What is the difference in a P/Invoke signature between “byref byte” and “byte[]”?

joshwil — Thu, 11 Aug 2005 04:11:00 GMT

Lately we’ve seen a spate of issues coming up on 64-bit platforms within the Developer Division around usages of P/Invoke signatures which declare a parameter as type “byref byte” where the developer really means “byte[]” (the corresponding native parameter type being something like LPBYTE). Usually when something works on 32-bit and doesn’t work on 64-bit we quickly get a phone call or email indicating that this must be a CLR problem, and this case was no different.

I received an email which pointed me to the following P/Invoke signature:
[DllImport(“kernel32.dll”)]
public static extern int ReadProcessMemory(
                             IntPtr hProcess,
                             IntPtr lpBaseAddress,
                             ref byte lpBuffer,
                             IntPtr nSize,
                             IntPtr lpNumberOfBytesWritten
);

Looking at MSDN we can see that the C prototype for this function is:
BOOL ReadProcessMemory(
         HANDLE hProcess,
         LPCVOID lpBaseAddress,
         LPVOID lpBuffer,
         SIZE_T nSize,
         SIZE_T* lpNumberOfBytesRead
);

There are a number of problems with the P/Invoke declaration (it’s return type should be BOOL for instance and the nSize parameter should probably be a UIntPtr instead of IntPtr), those aside, the real problem is that the lpBuffer parameter shouldn’t be defined as a byref byte. The intended usage was:

byte[] b = new byte[100];
ReadProcessMemory(…, …, ref b[0], …, …);

The expectation being that this would result in a pointer to the beginning of the byte array being delivered to the native code to play with. However that wasn’t happening and ReadProcessMemory was returning a failure (something that was actually very convenient in tracking down this bug). In the end though, this isn’t a CLR problem, it is a usage problem with the P/Invoke signature declaration. If that’s the case then you might ask: why did it work on 32-bit in the first place? Well, because of an “optimization” (I put it in quotes for a reason) in the x86 P/Invoke code “byref byte” means that we just happen to pin the reference to the byte which is passed through the P/Invoke layer and we pass that pinned original reference on to the native code.

This means that if you pass in a reference to the first byte (or any byte for that matter) of an array of bytes then we will pass a pointer to that and the native code can party on the rest of the array just as if we passed an interior pointer into the object (well, we did). It is very possible that this makes a lot of sense to those C++ programmers out there who have become very accustomed to a reference and a pointer being the same thing, and being able to do fancy pointer math on references just by casting them to pointers.

It turns out that in the 64-bit implementation of P/Invoke (which under the covers is radically different than the 32-bit implementation) we decided to more accurately represent a “byref byte” as a reference to a single byte, in fact, we allocate the byte on the interop layer’s stack and pass along a reference to that to the native code. On the way back to managed code we copy that byte back into the GCHeap wherever the managed object identified by the incoming object reference is currently living (in this case some byte in a byte array). This decision was also made as an “optimization” to avoid some of the frequent pinning that the CLR does during interop (as pinning can be rather hard for the GC to deal with, especially for very small objects and generally the less of it the better).

We do this for small native types that we can move around with an instruction or two, however for larger types (like an actual byte array, specified as a “byte[]” in a P/Invoke signature (or a “byref byte” identified by an array attribute of sorts) we still do go ahead pin the reference in the GC Heap and pass along the pinned reference to native code to party. This is what the developer of the above code intended to happen.

The correct P/Invoke signature would be (conveniently this can be found on http://www.pinvoke.net):
[DllImport(“kernel32.dll”)]
public static extern bool ReadProcessMemory(
                              IntPtr hProcess,
                              IntPtr lpBaseAddress,
                              [Out] byte[] lpBuffer,
                              UIntPtr nSize,
                              IntPtr lpNumberOfBytesWritten
);

Given this fixed signature we will pin the byte[] reference and pass along a pointer to the unmanaged code and everything will work as expected. Fortunately in this case for the group that wrote this code ReadProcessMemory was able to return a failure when it received what it deemed to be a bad pointer for lpBuffer, in most cases you will probably just end up seeing spectacular failures when the native code that you’re P/Invoke-ing out to starts overwriting your application’s stack. So it is very important to remember to get your P/Invoke signatures right!!! It will save you some serious debugging later.

How the OS Loader will force .Net v1.0/1.1 executables to run under WOW64 on a 64-Bit Machine

joshwil — Fri, 15 Oct 2004 20:15:00 GMT

[10/15, 2:04pm, fixed a couple typos; 10/15, 4:51pm, clarified a point]

Before you read this entry, you might want to read these two entries:

- http://blogs.msdn.com/joshwil/archive/2004/03/13/89163.aspx

- http://blogs.msdn.com/joshwil/archive/2004/03/11/88280.aspx

In case you skipped the link and kept on reading I’ll summarize the first post linked to (I however believe that they are really worth reading):

- “Bitness” is what we call an assembly’s ability to tell the OS and CLR what type of machine the assembly is safe to run on (32-bit vs. 64-bit, and on 64-bit: X64, IA64 or both).

- .Net 1.0 and 1.1 assemblies didn’t know anything about bitness

- There are things that you can do in a managed assembly which force you to need to bind it to one platform; these include: p/invoke, unsafe code, managed c++, etc…

The point is, in v1.0 and v1.1 we let you create assemblies that might very well have problems running on a 64bit platform, because of that we have decided that with .Net 2.0 (which is the first version of .Net to support 64-bit platforms natively) there will be an enforced loader policy such that 1.0 and 1.1 apps will be loaded by the 32-bit runtime on the machine.

We have gone back and forth on the right way to do this and have settled on the following:

- .Net 2.0 compilers will produce PE images with the IMAGE_COR20_HEADER.MinorRuntimeVersion set to 5, this matters in the case of MSIL (/platform:anycpu) images where the loader has the opportunity to choose whether the image should be loaded as a 32-bit or 64-bit image.

- MinorRuntimeVersion <5 will be used by the OS Loader to determine if the assembly is a .Net 2.0+ assembly or an older .Net assembly

- This will cause only .Net 2.0 executables to be loaded by the 64-bit runtime.

For reference, here is the definition of the IMAGE_COR20_HEADER:

// CLR 2.0 header structure.

typedef struct IMAGE_COR20_HEADER

{

// Header versioning

ULONG cb;

USHORT MajorRuntimeVersion;

USHORT MinorRuntimeVersion;

// Symbol table and startup information

IMAGE_DATA_DIRECTORY MetaData;

ULONG Flags;

ULONG EntryPointToken;

// Binding information

IMAGE_DATA_DIRECTORY Resources;

IMAGE_DATA_DIRECTORY StrongNameSignature;

// Regular fixup and binding information

IMAGE_DATA_DIRECTORY CodeManagerTable;

IMAGE_DATA_DIRECTORY VTableFixups;

IMAGE_DATA_DIRECTORY ExportAddressTableJumps;

// Precompiled image info (internal use only - set to zero)

IMAGE_DATA_DIRECTORY ManagedNativeHeader;

} IMAGE_COR20_HEADER;

It turns out that the 2.0 in the header structure name is kind of a misnomer and results from some pre-1.0 definitions of this structure. All released versions of the .Net runtime have had images that contain a IMAGE_COR20_HEADER structure. Additionally, all released versions of the .Net runtime have specified the MajorRuntimeVersion=2 and MinorRuntimeVersion=0. Kind of weird you might say for a v1.0/1.1 product? Yeah… That’s history for you…

What does this mean?

Well, it means that if you have a v1.0/1.1 assembly and run it on a 64-bit box with .Net 2.0 installed it will run under the 32-bit runtime in the WOW64. Whether it runs under a 1.1 32-bit runtime or the 2.0 32-bit runtime will be determined by the CLR’s loader policy, just as it would on a native 32-bit box. If you’re using v2.0 assemblies on a 64-bit machine and you have compiled them without the /platform switch or with /platform:anycpu (the default) then your image will load in the native runtime on whatever box you put it on. And of course the /platform:x86, /platform:x64, /platform:itanium switches work as they imply.

NOTE: /platform:anycpu doesn’t keep you from shooting yourself in the foot with a bad P/Invoke signature, unsafe code, etc… See this (http://blogs.msdn.com/joshwil/archive/2004/03/16/90612.aspx) blog entry for an example.

If you do have a managed v1.0/1.1 app which you believe to be 64-bit safe and whidbey compatible there is an easy way to make it run in 64-bit mode… You just have to whack the MinorRuntimeVersion to 5 in the IMAGE_COR20_HEADER for the image. Along these lines there will be a tool in the .Net 2.0 SDK called corflags.exe which will allow you to modify an image this way. A v1.0/1.1 image which has had its MinorRuntimeVersion whacked will still be compatible with the v1.0/1.1 runtimes (as per the version it was compiled against).

WARNING: If you don’t know what’s going on inside of some v1.0/1.1 image (if you didn’t write it say) you should be _VERY_ careful bumping it up to 64-bit as it may break in unexpected ways.

If you are a compiler writer and you want your application to run in 64-bit native mode under .Net 2.0 then you will need to produce images with this updated IMAGE_COR20_HEADER.

NOTE: .Net 2.0 assemblies that were compiled against the Beta 1 version of the framework (and the community drops up to now [10/15/04]) will act like v1.0/1.1 assemblies on newer builds of 64-bit OSes and load under the WOW64 in 32-bit mode.

What about v1.0/1.1 dlls?

If you make a .Net 2.0 executable and link against a v1.0/1.1 dll you will be able to load it into your process as if it was a .Net 2.0 MSIL assembly. If that v1.0/1.1 dll has code that isn’t safe to run in 64-bit mode it may crash.

Appendix: Why I think this is a good solution:

There are a number of ways in which we could have solved this problem. An easy one would have been to use the version string contained within the metadata for managed images. That string looks something like “v1.1.4322”, it represents the version of the framework which the assembly was compiled against, and.Net 2.0 assemblies will look something like “v2.0.X” (X still to be determined). You can argue that this makes sense, after all, Microsoft’s .Net 2.0 compilers (csc.exe, vbc.exe, cl.exe, etc…) will produce images that will be marked correctly. They need to since they support new features of the .Net 2.0 runtime (generics comes to mind) and bind against 2.0 frameworks libraries.

This however forces a tight binding between the runtime version and the compiler version, and doesn’t recognize the fact that there are plenty of assemblies compiled against the v1.0/1.1 frameworks that will run fine in 64-bit mode under v2.0. But even given that, its downfall is that it is a very Microsoft centric view, it assumes that the compilers will be updated at the same time as the runtime (which currently our compilers are). That tight binding wouldn’t recognize that there are compilers out there that only produce code which is safe to run in either 32-bit or 64-bit mode. Those compilers currently produce an image that works with the v1.0 or v1.1 runtime, and the compiler vendors may very well want to produce an image that also runs in 64-bit mode under the v2.0 runtime.

That’s the distinction, while Microsoft is producing compilers in this version of .Net that won’t create images which will run under any runtime prior to v2.0, other compiler vendors might very well want to produce images that run under prior versions (for deployments that maybe don’t have v2.0 installed yet) and yet still run in 64-bit native mode under v2.0 on a 64-bit machine.

This does put a great responsibility upon compiler vendors who chose to produce images that are marked this way, those compilers should only produce code which is safe to run in both 32-bit and 64-bit mode. Alternatively, they can do as our compilers do and let you shoot yourself in the foot if you want to by producing MSIL images with bad P/Invoke signatures and unsafe code… That puts the onus on the developer.

ArrayList’s vs. generic List for primitive types and 64-bits

joshwil — Tue, 13 Apr 2004 21:41:00 GMT

The dynamically growing ArrayList structure is an interesting one, the other day I was looking at memory usage of some performance scenarios on 64-bit machines vs. 32-bit machines and noticed that ArrayLists are double the size on 64-bit machines. After looking at the screen for a second wondering why that was it hit me… Under the covers an ArrayList is just a big Object[], and what’s an Object[]? It’s an array of references to Objects, references which are bigger on 64-bit platforms because they have to be able to reference more memory. Unfortunately that’s just a fact of life with bigger pointers.

But that realization did make me wonder what’s the difference in memory usage characteristics between using an ArrayList and using our new (v2.0) generic List<T> when you’re filling the list with primitive (non-reference) data types. It turns out to be interesting (at least to me) though predictable, so I thought I’d show it here…

As a spoiler, if you think that using the generic List<T> is better, you’re right! But you might not have guessed how right you are!!

Here’s my test code (I use a compile time define to compile with the generic v. non-generic list), I decided to just play with differences between lists containing ints here because they have an interesting property in that the size of a primitive int is the same as an Object reference on a 32bit machine. More about that at the end.

// SIZE is big enough that the list size dominates my memory usage
const int SIZE = 1000000;

public static void Main()
{
#if ARRAYLIST
     ArrayList list = new ArrayList();
#else
     List<int> list = new List<int>();
#endif
     for (int i=0; i<SIZE; i++)
     {
         list.Add(i);
     }
}

First off let’s think about what happens in these two scenarios:

1) ArrayList, under the covers this is an Object[], so to put an int value into the ArrayList we’ll have to box it (in the process creating a heap allocated Int32) and then put the reference to the boxed value into the array.
2) List<int>, with the generic implementation under the covers we’re using an int[], no boxing needed and instead we just shove the actual value being added into the array (thereby avoiding creating the heap allocated Int32).

Given that, we would expect the ArrayList implementation to be more of a memory hog, and it is

Results from a 32bit machine:

Here we can see that 19MB is allocated by the 32-bit ArrayList test, 11MB of which is simply heap allocated Int32 objects which result from the boxing of the int values. We also have 8MB worth of Object[] that have been allocated, notice that the amount of allocated Object[] is larger than the actual amount of space needed because I didn’t set the ArrayList capacity to something near what I knew we would actually. Instead I relied on the ArrayList auto-resizing which uses a 2x growth factor thus possibly ending up with up to ½ of the underlying array being unused, in this example that doesn’t effect the point that I’m trying to make so I ignored it.

On the other hand, using the List<int> version of the test we under the hood allocate a large Int32[] instead Object[], and we don’t need to heap allocate any Int32 objects. These differences result in memory usage of only 8MB for the Int32[]. Conveniently for my example on a 32-bit machine the size of an Object reference is the same as a 32-bit int so these arrays are the same size.

What do we see on a 64bit machine?

In this case the ArrayList test uses approximately two times more memory than the 32-bit version, this is unfortunately a fact of life in the 64-bit world… Given this data we can see that the reason is two fold:

1) References are bigger, and therefore our Object[] which underlies the ArrayList is double the size of it’s 32-bit version (16MB instead of 8MB)
2) Objects take more space in memory in a 64bit process. Every managed object has a couple pieces of CLR goo attached to it, a MethodTable* and a sync block, and these each end up being 8 bytes instead of 4 bytes like they are on 32-bit. In a small object like Int32 this combined with the fact that we keep the total size pointer size aligned results in an object that is two times as big on 64-bit. This results in 23MB of Int32 objects in a 64-bit process vs. 11MB in a 32-bit process. (see appendix below)

However, we can see that if we had instead used the generic List<int> we would have only allocated the same 8MB that we did in a 32-bit process. It is of note that since object references are 8 bytes on a 64-bit machine but an int is still 4 bytes we end up with the size of the underlying array being 50% smaller in the List<int> test case than the ArrayList case whereas on 32-bit they were the same. With smaller primitive data types (short, char, bool) the difference is of course more significant and is visible on both platforms since at that point the types are smaller than a object reference on 32-bit platforms as well.

Given that, what’s the difference in memory usage on the two platforms for the two solutions?

              ArrayList    List<int>    Difference (%)
32-bit   19MB         8MB          237%
64-bit   39MB         8.1MB        481%

We can see that though it is a definitely a better idea to use the generic List<int> on both platforms it becomes especially important on a 64-bit machine…

Then again, how many people are using ArrayList’s for big lists of ints and other primitive types? That’s a question that I don’t know the answer to…

The analysis (from a memory perspective) would be a lot different if we were talking reference types instead of ints. In that case we don’t have the option to avoid creating object instances; and the size of the reference to the reference type is the same size as that of a reference to an object. There are still plenty of reasons to use the generic List<T> in that case… But memory usage would no longer factor in.

Appendix: using SOS to look at Int32 objects in memory

64bit, Int32:
0:003> !do 000007ffe724d768
Name: System.Int32
MethodTable: 000000005cbf4408
EEClass: 000000005cc66380
Size: 24(0x18) bytes
(C:\WINDOWS\Microsoft.NET\Framework64\v2.0.amd64chk\assembly\GAC_64\mscorlib\2.0.3600.0__b77a5c561934e089\mscorlib.dll)
Fields:
MT Field Offset Type Attr Value Name
000000005cbf4408 40003cf 8 System.Int32 instance 3 m_value

32bit, Int32:
0:003> !do 00a56adc
Name: System.Int32
MethodTable: 78c12528
EEClass: 78c4ae44
Size: 12(0xc) bytes
(C:\WINDOWS\Microsoft.NET\Framework\v2.0.x86ret\assembly\GAC_32\mscorlib\2.0.3600.0__b77a5c561934e089\mscorlib.dll)
Fields:
MT Field Offset Type Attr Value Name
78c12528 40003bc 4 System.Int32 instance 437 m_value

As promised, what is the WOW64 and what does it mean to managed apps that you run on 64bit machines?

joshwil — Fri, 12 Mar 2004 00:11:00 GMT

So, what is the WOW64? If you already have a firm grasp please feel free to skip these first few paragraphs to the section titled “Break in here for the .Net perspective”.

When I went and Google’d a bit to see if anyone had a convincing answer out there what I seemed to find instead was confusion. Note this article titled “WOW64 for AMD Released to the Public”. With a firmer understanding of what the WOW64 is the fact that this headline is misleading would not have escaped that journalist. So lets start with the name:

WOW64 = Windows On Windows64

It might be more appropriately written “Windows32 On Windows64” but then the acronym isn’t nearly as cool. Basically it is a layer of code that allows for 32bit processes to run just like they were running on a 32bit system under a 32bit OS (e.g. your normal managed app running on the CLR on WindowsXP on a Pentium 4) even though in reality they are running on a 64bit OS. In fact you can find a good indicator of this by looking in your WINDOWS directory where now you will not only find the misnamed SYSTEM32 directory (which holds the 64bit dlls for the 64bit OS) but also the SYSWOW64 directory which holds 32bit versions of dlls for the virtual 32bit OS.

<historical note> I believe that there was a WOW32 effort as well during the move from Win16 to Win32. That move was accompanied by a significant change in programming model as the Win32 API was introduced (The Win16 stuff still lives in WINDOWS\SYSTEM). Generally the Win64 platform uses the Win32 API with platform specific data types (pointers and such) just expanded to fit the new hardware. This is why Windows decided to stick with the SYSTEM32 directory as the primary home for Win64 dlls even though the name can be misleading. This entry and discussion on Raymond’s blog is an interesting read that talks about the joys of backwards compatibility and Win16. </historical note>

You can imagine (and not be too far off) that once you’ve been kicked off by the loader as a 32bit process WINDOWS\SysWow64 gets aliased (for your purposes) to WINDOWS\System32. This has side effects, like the fact that one of my favorite little text editors (EmEditor) which is 32bit can’t find files in the real WINDOWS\System32 directory when running in the WOW64 on my 64bit boxes (it took me a while to figure out why I couldn’t find a couple of scripts that I knew for sure I had dropped in there). There are other parts of the system which are also split into two parts, most notably the registry, unfortunately I don’t know nearly enough about the technical details of that split to discuss it intelligently here.

Under the hood Windows is doing a lot of work to make sure things continue to look just like you’re on a 32bit box and this generally pays off in app compatibility. As would be expected however there is a performance hit that you take for running in this mode, it is an especially heavy hit on IA64 where in the current implementation the x86 instruction set is emulated with software. It is worth noting however that it does work pretty darn well, and the x86 CLR that we’ll be shipping to Win64 users to run in the WOW64 is the same build as that which we ship to x86 users (we just have to mess with the installer a bit). Other large apps can run in the WOW64 and be quite snappy on 64bit Extended hardware platforms (read:AMD64, IA32e) which can natively run the 32bit x86 instructions in a special more of the processor. I've played with VS for instance running on an AMD64 machine like this and it's great.

It is important to understand that once you’re loaded up in the WOW64 you’re a 32bit process and there’s no turning back. You have to use 32bit dlls all around. On the other hand, once you start up 64bit you’re 64bit, no 32bit dlls allowed. So, unfortunately this is not one of those things you can slowly transition. It is possible to kick off another process of a different bitness from your own and communicate with it through RPC or out of proc COM, and this can serve as a transition for some. But hopefully over the next couple months I’ll help to convince you that the move to running native as a 64bit process is going to be worth it.

>> Break in here for the .NET perspective <<

So, what does this mean for .Net and the CLR? The Whidbey CLR will be available in 32bit and 64bit versions with the 64bit version supporting x64 (AMD64/IA32e) and IA64 (Itanium). And the current plan is that when you install the 64bit CLR on a machine we’ll slap down the 32bit version of Whidbey at the same time. Yes, you read that right – your 64bit machine will end up with two copies of the runtime. The 32bit version will be installed in \WINDOWS\Microsoft.Net\Framework just as it would on a native 32bit machine whereas the 64bit version ends up in \WINDOWS\Microsoft.Net\Framework64, no rocket science here, though the GAC issues can be interesting (I’ll talk about these for sure in a later blog entry). Why do we need two frameworks on the machine you might ask? Isn’t one of the cool things about compiling to IL that the JIT takes care of the hardware specific stuff and your code just automatically takes advantage of the new platform it is there?

Well… There are many cases where you just want your code to “float up” to the new platform and run as native 64bit code. In fact, if you’re writing a fully managed app today chances are good that this is what you desire. But, there are still a lot of apps that use PInvoke, COM Interop to inproc servers (including using VB libraries pre-.Net) or APIs that won’t initially be available in 64bit which could break horribly. Remember that once you’re process is started as 64bit that’s it. Trying to load and execute code from a 32bit dll at that point is undefined and your process will certainly die a quick and painful death. While it is true that most of the dlls that come with Windows will be available on both 32bit and 64bit platforms many custom unmanaged code out there hasn’t yet made the jump. The plus side to not going to a new Win64 API is that if you define your PInvoke signatures correctly, most of the Win32 API is still there in a 64bit version and you’ll just pick up the new dlls and run native! In cases where you’re using some unmanaged code that you have no control over and isn’t available in a 64bit version it behooves you to specify that your app requires that it be run in a 32bit process.

How do you specify that? Well, fundamentally it is specified in the CLR headers of your PE image (the format of your dll or exe on disk and in memory). Realistically it is set at compile time (e.g. with C# use the /platform switch). Your exes and dlls can be marked one of four things: MSIL, x86, x64, IA64 (names vary depending on who you talk to). Let’s start out with the obvious ones:

x86: on a 32bit platform this will just run, no hassles. On a 64bit platform this will get started up as a 32bit process in the WOW64 running under the 32bit CLR. You’ll load 32bit .Net FX images out of the GAC (like MSCorLib, System.Web, System.Windows.Forms, etc..). Your code will be JITted by the x86 JITter and will run (in emulation on IA64) as x86 code.

x64: on a 64bit x64 platform this will run as a 64bit process. On a IA64 or x86 box you’ll get a BadImageFormatException. When you’re running as a 64bit process your image will be JITted 64bit native, we’ll pull 64bit images out of the GAC (be it by loading a 64bit native image that has been previously ngen’d or grabbing IL out of the GAC and JITting it at runtime to be 64bit).

IA64: like x64, but swap x64 for IA64.

Then there is MSIL (or “anycpu” as I think the C# compiler switch goes, some people will also refer to this as agnostic or neutral), this indicates that the code here really isn’t processor specific, on a 32bit platform it will get loaded up as a 32bit process and on a 64bit platform it will float up to a 64bit process and run under the 64bit CLR. This is rather important to know as this is currently the default for applications that are compiled using Whidbey compilers (disclaimer: this is still being hotly debated as to whether or not it is the right default, “currently” in this case means if you look at the build that I’m working with on my box on 3/11/04, what we ship may/may not reflect that).

<1.0&1.1 Note> As for 1.0/1.1 apps that are deployed onto a 64bit box, right now we’re probably going with a model that assumes that those assemblies (which didn’t know about bitness) will run in a 32bit process under the WOW64 as if they had been compiled with /platform:x86. The thought process leading us this direction is that we’d prefer to be conservative and by running 1.0/1.1 apps in the WOW64 we ensure that they don’t run into bitness issues that their developers might not have anticipated as it wasn’t part of the .NET world when they were being developed.

Under that model, if you do want your application to run in a 64bit process you’ll need to explicitly recompile it with one of the Whidbey compilers which know about bitness. Then (currently, see disclaimer above) if you don’t specify /platform:x86 your app will run 64bit when/if it is deployed to an x64 or IA64 box. </1.0&1.1 Note>

<GEEK> There is actually some interesting stuff that goes on under the covers here as well, that is if you’re interested in PE images and loader magic, x64 and IA64 images generated by the compiler with /platform:x64 or :IA64 will be PE32+ images (the 64bit extension to PE32) whereas x86 and MSIL are PE32 images (other wise they wouldn’t work on 32bit OS’s). When the OS loader comes across a managed image the first thing it does is hand it to some CLR code we call the shim (mscoree.dll) which interrogates the image, potentially makes some fixups and gives it back to the OS loader to let the OS then kick it off and trigger the actual runtime to start up. But, to get the OS loader to load your app as a 64bit process you have to give it a 64bit image, so in the shim on 64bit machines we will actually modify MSIL images in memory to turn them into PE32+ images before handing them back to the OS loader. This then in turn causes the loader to start up the right (64bit) runtime. </GEEK>

>> End .NET perspective <<

Recap: So, what have I talked about?

- WOW64 isn’t the OS per se, but a subset of the Win64 OS which enables a 32bit application to run inside of a 32bit process on a 64bit OS while using 32bit system dlls and such.

- Whidbey CLR will include both 32bit and 64bit versions, both of which will be installed on 64bit machines. This allows both 32bit and 64bit managed applications to run in bitness correct native process depending on how the assemblies are tagged at compile time.

- Once a process is started up as either 32bit or 64bit all of the dlls/assemblies that are loaded into that process have to be compatible with that bitness. There is a significant complexity for instance in the GAC to make this possible for .NET FX images (I promise to talk more about that in a later blog entry).

- If you have a 32bit managed app which has dependencies on 32bit unmanaged code then you’ll need to either find a 64bit version of the unmanaged code or tag your managed app as x86 at compile time to make sure that you don’t float up to a native 64bit process. This will result in you having to live with your process running under the WOW64.