SpankyJ : Writing code targeted at 64bit.

BigArray, getting around the 2GB array size limit

joshwil — Thu, 11 Aug 2005 04:14:00 GMT

I’ve received a number of queries as to why the 64-bit version of the 2.0 .Net runtime still has array maximum sizes limited to 2GB. Given that it seems to be a hot topic of late I figured a little background and a discussion of the options to get around this limitation was in order.

First some background; in the 2.0 version of the .Net runtime (CLR) we made a conscious design decision to keep the maximum object size allowed in the GC Heap at 2GB, even on the 64-bit version of the runtime. This is the same as the current 1.1 implementation of the 32-bit CLR, however you would be hard pressed to actually manage to allocate a 2GB object on the 32-bit CLR because the virtual address space is simply too fragmented to realistically find a 2GB hole. Generally people aren’t particularly concerned with creating types that would be >2GB when instantiated (or anywhere close), however since arrays are just a special kind of managed type which are created within the managed heap they also suffer from this limitation.

<Sidenote> managed arrays: arrays are a first class type in the CLR world and they are laid out in one contiguous block of memory in the managed garbage collected heap. In the CLR 1.1 they can be thought of as the only generic type (in 2.0 we’re introducing a much more universal concept of generics) in that you can have an array that is of the type of any managed type that you like (primitive types, value types, reference types). It is interesting to think about what that means in context of the 2GB object instance size limit imposed on objects in the managed heap. With value types (bool, char, int, long, struct X {}, etc…) the actual data of the instance for each element in the array will be laid out contiguous with the next element in memory, since the 2GB limit discussed earlier applies to the total array size, and the array size is a factor of the type size the maximum number of elements you can store in an array of type X will vary proportionally to the size of X.

Differing from this are arrays of reference types (e.g. objects, strings, class Y {}, etc…), for these arrays the actual array will be that of a bunch of references, initially null. To initialize the array your code will need to go through one element at a time and create or assign an appropriate instance of the type to that array element. The 2GB size limit for arrays applies to this array of references, not the instances of the objects themselves. On a 32-bit machine if you create an array of type object (object[]) and one instance of type object per element in the array then your available virtual address space will end up limiting the size of your array as you will never be able to fit enough objects in memory to be able to fill up a 2GB object array with unique object references.</Sidenote>

The developer visible side of this is that array indexes are signed integers (with a byte[] you can use the full positive space of the signed integer as an index (assuming the array is 0 based), with other types you use some subset of that space until the total array size is 2GB). While some of the BCL APIs that deal with arrays have alternate signatures that take longs this isn’t yet ubiquitous in the framework (i.e. the IList interface (which the BCL’s Array class implements) uses int indexes).

It is debatable whether or not we should have included a “Big Array” implementation in the 2.0 64-bit runtime, and I’m sure that debate will rage for some years to come. However, as 2.0 is getting ready to ship and there currently isn’t any support for this we are going to have to live without it until at least the next version.

So, what is there to do in .Net 2.0 if you have an application which requires arrays that are very large?

Well, first switch to 64-bit! As mentioned, it is next to impossible to allocate a full 2GB array on 32-bit because of the way that the virtual address space is broken up by modules and other various allocations. Simply switching to 64-bit will buy you the ability to allocate those full 2GB blocks (well, close anyway, the total object size is limited to 2GB, but there is some CLR book-keeping goo in there that takes a few bytes).

What if that still isn’t enough? You have a couple of choices:

A) Rethink your application’s design? Do you really need a single gigantor array to store your data? Keep in mind that if you’re allocating 8GB of data in a single array and then accessing it in a sparse and random manner you’re going to be in for a world of paging pain unless you have a ton of physical memory. It is very possible that there is another data organization scheme you can use where you can group data into frequently used groups of some sort or another and manage to keep under the 2GB limit for an individual group. If you choose correctly you can vastly improve your applications performance due to lower paging and better cache access characteristics that come from keeping things that are used together close to one another.

B) Use native allocations. You can always P/Invoke to NT’s native heap and allocate memory which you can then use unsafe code to access. This isn’t going to work if you want to have an array full of object references, but if you just need a huge byte[] to store an image this might work out fine, even great. The added cost of the P/Invoke is low because the NT APIs have simple signatures that don’t require marshaling and the code executed when allocating an 8GB block is probably mostly zeroing the memory anyway. If you choose this option you will have to write a small memory management class of some kind and be comfortable using unsafe code. I know that Paint.Net (http://blogs.msdn.com/joshwil/archive/2005/04/07/406218.aspx) uses this very method for allocating the memory in which they store the image (and it’s various layers) which you’re editing. This is a good solution for the case where you really need a single unbroken allocation of some large size. While it isn’t a very general purpose solution it works out great for the Paint.Net guys.

C) Write your own BigArray class.

I’d stress that option C is my least favorite of the above three, but I will acknowledge that there are probably cases where it is the right thing to do. Given that, I have gone and written one myself. This is a very bare bones implementation, just the array allocation and accessors are implemented, I will leave implementing any extended functionality (like the functionality provided by the static members of the Array class, Sort, Copy, etc… or writing big collections on top of it) as an exercise for the reader.

// Goal: create an array that allows for a number of elements > Int.MaxValue
class BigArray<T>
{
    // These need to be const so that the getter/setter get inlined by the JIT into
    // calling methods just like with a real array to have any chance of meeting our
    // performance goals.
    //
    // BLOCK_SIZE must be a power of 2, and we want it to be big enough that we allocate
    // blocks in the large object heap so that they don't move.
    internal const int BLOCK_SIZE = 524288;
    internal const int BLOCK_SIZE_LOG2 = 19;

    // Don't use a multi-dimensional array here because then we can't right size the last
    // block and we have to do range checking on our own and since there will then be
    // exception throwing in our code there is a good chance that the JIT won't inline.
    T[][] _elements;
    ulong _length;

    // maximum BigArray size = BLOCK_SIZE * Int.MaxValue
    public BigArray(ulong size)
    {
            int numBlocks = (int)(size / BLOCK_SIZE);
            if ((numBlocks * BLOCK_SIZE) < size)
            {
                numBlocks += 1;
            }

            _length = size;
            _elements = new T[numBlocks][];
            for (int i=0; i<(numBlocks-1); i++)
            {
                _elements[i] = new T[BLOCK_SIZE];
            }
            // by making sure to make the last block right sized then we get the range checks
            // for free with the normal array range checks and don't have to add our own
            _elements[numBlocks-1] = new T[NumElementsInLastBlock];
    }

    public ulong Length
    {
        get
        {
            return _length;
        }
    }

    public T this[ulong elementNumber]
    {
        // these must be _very_ simple in order to ensure that they get inlined into
        // their caller
        get
        {
            int blockNum = (int)(elementNumber >> BLOCK_SIZE_LOG2);
            int elementNumberInBlock = (int)(elementNumber & (BLOCK_SIZE – 1));
            return _elements[blockNum][elementNumberInBlock];
        }
        set
        {
            int blockNum = (int)(elementNumber >> BLOCK_SIZE_LOG2);
            int elementNumberInBlock = (int)(elementNumber & (BLOCK_SIZE – 1));
            _elements[blockNum][elementNumberInBlock] = value;
        }
    }
}

The beauty of this implementation is that the JIT already understands single dimensional array accesses intrinsically, including range checking code. In practice this class ends up being almost as fast as real array access for small arrays (< BLOCK_SIZE) and not too much slower once you get to reasonably big arrays. It doesn’t waste much space compared to a normal array because the last block is right sized and the performance is good because the getter and setter for array elements are simple enough that they should get inlined into the calling method, this becomes very important for getting anywhere close to normal array access speeds.

Here is an example of big array usage:

public static void Main()
{
    long size = 0x1FFFFFFFF;
    BigArray<int> baInt = new BigArray<int>(size);
    long len = baInt.Length;
    for (long i=0; i<len; i++)
    {
        baInt[i] = i;
    }
    Console.WriteLine("baInt[len/2]=" + baInt[len/2]);
}

You could imagine also exposing the fact that this BigArray<T> implementation has blocks through a couple of properties and a indexer of this[int block, int element] which would allow people to intelligently write code to do block based access on the array (e.g. merge sorts that are block intelligent). This can be important for performance as we know that elements within a single block are contiguous in memory, however we cannot make that guarantee about elements in neighboring blocks.

It is worth noting that given the allocation scheme of the BigArray<T> constructor we may very well have multiple garbage collections while it runs, because of this you don’t really want to be using large instances of this class in a throw away manner. My advice would be to use this carefully and sparingly, instead favoring architectures which don’t require such large single arrays.

What is the difference in a P/Invoke signature between “byref byte” and “byte[]”?

joshwil — Thu, 11 Aug 2005 04:11:00 GMT

Lately we’ve seen a spate of issues coming up on 64-bit platforms within the Developer Division around usages of P/Invoke signatures which declare a parameter as type “byref byte” where the developer really means “byte[]” (the corresponding native parameter type being something like LPBYTE). Usually when something works on 32-bit and doesn’t work on 64-bit we quickly get a phone call or email indicating that this must be a CLR problem, and this case was no different.

I received an email which pointed me to the following P/Invoke signature:
[DllImport(“kernel32.dll”)]
public static extern int ReadProcessMemory(
                             IntPtr hProcess,
                             IntPtr lpBaseAddress,
                             ref byte lpBuffer,
                             IntPtr nSize,
                             IntPtr lpNumberOfBytesWritten
);

Looking at MSDN we can see that the C prototype for this function is:
BOOL ReadProcessMemory(
         HANDLE hProcess,
         LPCVOID lpBaseAddress,
         LPVOID lpBuffer,
         SIZE_T nSize,
         SIZE_T* lpNumberOfBytesRead
);

There are a number of problems with the P/Invoke declaration (it’s return type should be BOOL for instance and the nSize parameter should probably be a UIntPtr instead of IntPtr), those aside, the real problem is that the lpBuffer parameter shouldn’t be defined as a byref byte. The intended usage was:

byte[] b = new byte[100];
ReadProcessMemory(…, …, ref b[0], …, …);

The expectation being that this would result in a pointer to the beginning of the byte array being delivered to the native code to play with. However that wasn’t happening and ReadProcessMemory was returning a failure (something that was actually very convenient in tracking down this bug). In the end though, this isn’t a CLR problem, it is a usage problem with the P/Invoke signature declaration. If that’s the case then you might ask: why did it work on 32-bit in the first place? Well, because of an “optimization” (I put it in quotes for a reason) in the x86 P/Invoke code “byref byte” means that we just happen to pin the reference to the byte which is passed through the P/Invoke layer and we pass that pinned original reference on to the native code.

This means that if you pass in a reference to the first byte (or any byte for that matter) of an array of bytes then we will pass a pointer to that and the native code can party on the rest of the array just as if we passed an interior pointer into the object (well, we did). It is very possible that this makes a lot of sense to those C++ programmers out there who have become very accustomed to a reference and a pointer being the same thing, and being able to do fancy pointer math on references just by casting them to pointers.

It turns out that in the 64-bit implementation of P/Invoke (which under the covers is radically different than the 32-bit implementation) we decided to more accurately represent a “byref byte” as a reference to a single byte, in fact, we allocate the byte on the interop layer’s stack and pass along a reference to that to the native code. On the way back to managed code we copy that byte back into the GCHeap wherever the managed object identified by the incoming object reference is currently living (in this case some byte in a byte array). This decision was also made as an “optimization” to avoid some of the frequent pinning that the CLR does during interop (as pinning can be rather hard for the GC to deal with, especially for very small objects and generally the less of it the better).

We do this for small native types that we can move around with an instruction or two, however for larger types (like an actual byte array, specified as a “byte[]” in a P/Invoke signature (or a “byref byte” identified by an array attribute of sorts) we still do go ahead pin the reference in the GC Heap and pass along the pinned reference to native code to party. This is what the developer of the above code intended to happen.

The correct P/Invoke signature would be (conveniently this can be found on http://www.pinvoke.net):
[DllImport(“kernel32.dll”)]
public static extern bool ReadProcessMemory(
                              IntPtr hProcess,
                              IntPtr lpBaseAddress,
                              [Out] byte[] lpBuffer,
                              UIntPtr nSize,
                              IntPtr lpNumberOfBytesWritten
);

Given this fixed signature we will pin the byte[] reference and pass along a pointer to the unmanaged code and everything will work as expected. Fortunately in this case for the group that wrote this code ReadProcessMemory was able to return a failure when it received what it deemed to be a bad pointer for lpBuffer, in most cases you will probably just end up seeing spectacular failures when the native code that you’re P/Invoke-ing out to starts overwriting your application’s stack. So it is very important to remember to get your P/Invoke signatures right!!! It will save you some serious debugging later.

Bit specific code in agnostic assemblies???

joshwil — Thu, 11 Aug 2005 04:09:00 GMT

In previous blog entries I’ve spent some time talking about how to mark assemblies as bit specific and how the loader deals with those markings.

What however is the preferred mode of an application? I will posit that it is to be compiled agnostic and to run equally well on both 32-bit and 64-bit platforms. It makes a lot of things easier: development, build, testing, deployment, servicing…

Caveat: The following discussion deals only with fully IL assemblies. If you generate managed C++ code you may end up with some native code in your image at which point it has to be tied to one specific platform.

If you have a reason to tie yourself to only one platform (e.g. x86 because you only have an x86 version of some native DLL that you need to P/Invoke to) then your decision is easy and it’s been made for you. Just flip the /platform:x86 switch on your compiler and go. However, if you have some code that works on both 32-bit and 64-bit platforms with just some subtle difference then you have a couple of options to think about for implementing the differences:

1) Use compile time defines (#if/#else in C#) to separate your 64-bit code from 32-bit code. Use the /platform:X switch of your compiler to generate different assemblies for 32-bit and (both) 64-bit platforms.
2) Use runtime if/else blocks to separate 32-bit and 64-bit code.

Both of these end up having their place. In most cases I’ve seen, people have only a small amount of code which needs to be bit specific, and in those cases dealing with the rest of the hassles around building and deploying multiple assemblies aren’t really worth it…

But, what about the runtime cost of the check? What if your bit specific code is on the hot path? Won’t that hurt?

Actually, that’s the cool part, if you do it right it won’t[1]. So, what are your options for determining bitness of a process at runtime?

A) if (Marshal.SizeOf(IntPtr.GetType()) == 8) {/*64*/} else {/*32*/ }
B) if (IntPtr.Size == 8) {/*64*/} else {/*32*/}
C) readonly static bool is64Bit = (IntPtr.Size==8);
if (is64Bit) {/*64*/} else {/*32*/}

Of those options there are 2 right ways and a wrong way. Unfortunately, some of the early information coming out of Microsoft indicated that you should use Marshal.SizeOf, which is definitely the wrong way to do this. That check involves a call to the marshaling code in mscorlib.dll and since the JIT (or ngen) compiler doesn’t know at JIT (or ngen) time what the result will be the unused half of the code can’t be optimized away as dead code.

The easiest way to do this is B, since IntPtr.Size is a constant which is hard-coded into mscorlib.dll when we build the runtime, the JIT (or ngen) can check the loaded mscorlib.dll (which will vary depending on bitness) and optimize away the check and the unused half of the code.

Option C works also, but it has a potentially subtle bug to it. If you don’t mark the static variable definition as readonly then the JIT (and ngen) won’t be able to optimize away the check and unused code. This is because it has to assume that the value can change at runtime. This is very important to remember because without this keyword this solution will become almost as bad as A.

Recommendation: for simple cases, use “if (IntPtr.Size==8)” to determine 64-bitness. For more complex cases consider using a static boolean, but remember to mark it as readonly.

Unfortunately the if/else solution won’t work well for cases where you need different structure definitions on 64-bit and 32-bit platforms for P/Invoke-ing to native routines. If you have a very small number of usages you might consider having two separate structure definitions and P/Invoke declarations, and using if/else to determine which one you use (maybe hiding the bitness stuff behind a wrapper). However, if it is a frequently used structure then it probably makes more sense to just use platform specific assemblies and compile time defines to determine structure layout as then changes only need to be made at the structure definition site.

If you’d like to see some of this stuff in action I’ve posted the source to a test that you can run and then inspect in the debugger to see what the JIT does (http://homepage.mac.com/willij3/blog/testing_bitness.cs). I’m sure it can be done more easily in VS, but I’ve been using WinDbg with SOS’s !name2ee and !u commands to disassemble the resulting code.

[1] Well, there is a small cost involved in the JIT having to parse the extra IL code for both platforms before it can evaluate the const condition and throw away half. However this cost is minimal and for frequently executed code is trivial. For ngen’d code the cost at runtime is non-existant.

Isn’t my code going to be faster on 64-bit???

joshwil — Fri, 29 Apr 2005 07:01:00 GMT

[updated 10:50am 5/2/05: It turns out that I copied and pasted an error in my code from the newsgroup posting I was answering. However a kind reader was able to spot it and I've fixed it, I'm getting new data and will updated graphs later today, however the points of the article remain valid]

[updated 8:04am 5/3/05: Added new graphs for data from fixed code. As expected, the results are the same, the peaks just moved to the left]

Subtitle: Micro-optimizations for 64-bit platforms

DISCLAIMER: As usual with performance discussions, your mileage will vary, and it is highly recommended that you test your specific scenario and not make any vast over-generalizations about the data contained here.

The other day on the new MSDN forums a question came up of what the performance difference of a piece of code would be when it was run on 64-bit vs. 32-bit. In this case the poster specifically talked about the question of what the performance difference between managed code running natively on an X64 64-bit CLR and the corresponding managed code running natively on a 32-bit X86 CLR for a simple copy loop which moves data from one byte array to another. I decided it would be interesting to do an analysis of this and so here we are.

I wrote up a little unsafe C# code which approximates what I believe the poster to be talking about, it goes something like this:

class ByteCopyTest
{
byte[] b1;
byte[] b2;

int iters;

    public ByteCopyTest (int size, int iters)
    {
        b1 = new byte[size];
        b2 = new byte[size];

        this.iters = iters;
    }

    unsafe public ulong DoIntCopySingle()
    {
        Timer t = new Timer();

        t.Start();
        int intCount = b1.Length / 4;
        fixed (byte* pbSrc = &b1[0], pbDst = &b2[0])
        {
            for (int j=0; j<iters; j++)
            {
                int* piSrc = (int*)pbSrc;
                int* piDst = (int*)pbDst;

                for (int i=0; i<intCount; ++i)
                {
                    *piDst++ = *piSrc++;
                }
            }
        }
        t.End();

        return t.GetTicks();
    }
}

Here we can see a simple piece of code that facilitates coping from one byte array into another. It is then easy enough to run this test under both the 64-bit and 32-bit CLR to compare performance. In this case I varied the byte array in size from 256 Bytes up to 256MB and ran a varying number of iterations so that each time measured is for copying the same amount of total data (about 5.1GB).

Something to note about these tests, is that they aren’t actually testing the internal working of the CLR so much as they test the code-generation capabilities of the JIT32 and JIT64 and the memory/cache of the machine that the test is run on.

Here, when using a copy loop that copies a single int (4-bytes) at a time from one array to another we can see that the JIT32 seems to generate better code and in many cases the 32-bit version wins. We can see that in both cases the time taken goes drastically up when we go from 1MB to 2MB and then levels off somewhat. This is where the processors on die cache stops being able to keep up as well and our program’s run time ends up being ruled by memory access, we will see later that the particular implementation of the copy loop at this point ceases to matter much.

While that is interesting, it might be even more interesting to compare a copy loop that uses a long (8-byte) instead of an int given that registers are 8-bytes wide on the X64 platform that means we can fit the whole long into a single register in the inner copy loop.

Here we can see that the long based copy loops definitely out perform the int based copy loops, and they do so consistently on both platforms… That this is faster on 32-bit is interesting, it turns out that the loop overhead is so great that breaking the long down into two 4-byte pieces to copy it inside of the loop is a win, effectively we’ve just made the jit unroll our loop one level for us. In this case it turns out to be a win.

We can see however that even with 32-bit beating it’s int based implementation the 64-bit version has a considerably smaller inner loop with fewer memory accesses which shows in the data above where we consistently see the 64-bit long based copy loop wining.

We still see a plateau starting at copies of 2MB, here the latency of memory access takes over and the processor can’t keep up with the code. At this point the processor will be spending many cycles spinning waiting for data and few extra instructions aren’t going to hurt as badly.

The positive results of using the long copy loop on 32-bit invites us to try a copy loop which copies two ints or longs at a time instead of one to try and better utilize that processor. An implementation of this would look like:

    unsafe public ulong DoLongCopyDouble()
    {
        Timer t = new Timer();
        t.Start();
        int longCount = b1.Length / 16;
        fixed (byte* pbSrc = &b1[0], pbDst = &b2[0])
        {
            for (int j=0; j<iters; j++)
            {
                long* plSrc = (long*)pbSrc;
                long* plDst = (long*)pbDst;

                for (int i=0; i<longCount; ++i)
                {
                    plDst[0] = plSrc[0];
                    plDst[1] = plSrc[1];
                    plDst += 2;
                    plSrc += 2;
                }
            }
        }
        t.End();

return t.GetTicks();
}

We will call this a “double” copy loop (and our former code a “single” copy loop). Let’s look and see how the double copy loops do on 64-bit:

Here we can see that the double long copy loop wins over the others, and, interestingly the double int and single long loops are very close. This would be expected as they are coping the same amount of data per iteration through the inner loop, however, the double int implementation uses more instructions to do it and does look to be a bit slower through most of the graph.

When we put everything together into a single graph we can see that the best of the implementations (double long on 64-bit) beats the worst of the implementations (single int on 64-bit) by around 50% which is significant. Most of the implementations fall somewhere in the middle however and vary minimally from implementation to implementation.

We can see that unrolling the loop only works so far before we see diminishing returns in that on the 32-bit platform the double long implementation isn’t that much faster than the double int implementation even though it is moving twice as much data per iteration of the inner loop. This code is getting to the point where loop overhead is lost in the noise of memory access.

What is the moral of the story? This code can be faster on 64-bit for certain scenarios, but if you’re writing it you might have to think about it (once again good engineering triumphs over good hardware). For instance, you might have written the single int copy loop for some super optimized routine in your code when thinking about a 32-bit target, if that is the case then that piece of code may run marginally slower on 64-bit (or not, see other graphs below), and if it’s really important you might consider revising it to be long based for a 64-bit win. In the end we’ve seen that making it long based actually results in a win for both 32-bit and 64-bit platforms. This supports an assertion that you will commonly hear me broadcasting to anyone who will listen, “Good Engineering == Better Performance”. It’s true regardless of platform.

While examining this copy loop is a fun game to play, chances are that most of your code isn’t this low level. Chances are also good that most of your code is already fast enough on both platforms. As Rico is apt to say, premature optimization is the root of all evil. I highly recommend that you profile, a lot. And then make educated decisions about the parts of your program which it would make sense to specifically do some work to try and optimize for 64-bit. The likelihood is high that places where you can find something very low level that is 64-bit specific are few and far between. Often the hot spots that you find will be places where optimization just plain makes sense regardless of the target hardware platform. Then it’s just a task to think about that general optimization and hopefully keep 64-bit in mind.

Well, we’ve managed to make it to the end of this post without me directly answering the question posed in the title… In case you’ve forgotten, it is “Isn’t my code going to be faster on 64-bit???”

Maybe.

I know, a pointy haired answer at best… The fact of the matter is that there are a lot of cases where 64-bit processors will provide a significant boost to current applications which can take advantage of the additional memory and registers. However, there are some applications which just by their nature will run at a comparable speed to their 32-bit siblings. And some that will run somewhat slower. It is unfortunately impossible to provide a universal answer to the question for every application under the sun.

The big blocker to a universal speed boost from 64-bit processors is that they don’t fundamentally change one of the big limiting factors of many applications, I/O, both to memory and to the disk or network. Given that most of the time processors in modern machines are spinning, waiting for something to do, the difference of a few instructions in a tight loop when you’re waiting on memory can be so small as to not matter.

Which brings us to an interesting point, as can be clearly seen in the graphs above, running out of cache can be a significant problem on modern processors… This unfortunately is the current challenge for 64-bit computing, a challenge which is somewhat increased by managed runtimes which have a tendency to exacerbate coding patterns which are very reference heavy. References (pointers for you old-school c++ types like me) grow on 64-bit, in fact they double in size from 4 bytes (32-bits) to 8 bytes (64-bits). Depending on application architecture this can have a big effect on cache utilization and correspondingly performance.

So, maybe.

I’ll leave you with this sentiment: “Good Engineering == Good 64-bit Performance!”

The code for this example can be found here.

Paint.Net is pretty cool, 20 minutes to get it running on 64-bit is even cooler!

joshwil — Thu, 07 Apr 2005 20:35:00 GMT

Last night I played with Paint.Net for a while and got it running on the native 64-bit CLR with whidbey... Pretty cool stuff!!!

Actually, a while might be overstating things... It took longer to install a copy of VS 2005 on my clean AMD64 machine than it did to get Paint.Net running natively, in fact there were no code changes needed to get it to run on .Net 2.0 (just had to change a couple project settings) and only 3 small changes to the code to get it to run on 64-bit (the biggest being removing a dependency on some assembly code used to figure out whether or not a machine has HyperThreading). All of this took < 20 minutes.

Check out a blog about Paint.Net here: http://blogs.msdn.com/rickbrew/

Here's some screen shots to whet your appitite

Paint.Net running on 64-bit CLR

Copy and Paste between 32-bit and 64-bit versions of Paint.Net

A 2GB image in Paint.Net, note the 4GB memory usage (it actually has a 6GB VM footprint at this point, yes... this machine does have a lot of RAM).

Note: this is all running on a "very close to done" internal Beta2 .Net Framework.

Good blog entries on .Net GC

joshwil — Thu, 28 Oct 2004 17:15:00 GMT

I just ran across these, they are a couple of really good reads on the .Net GC...

http://blogs.msdn.com/maoni/archive/2004/06/15/156626.aspx

http://blogs.msdn.com/maoni/archive/2004/09/25/234273.aspx

Good reading about 64-bit .Net and the GAC... Via, Junfeng.

joshwil — Wed, 27 Oct 2004 16:00:00 GMT

Here are a couple of really worthwhile entries written by Junfeng on the GAC, why you might compile platform specifc and 64-bit... I don't know if I need to bother with my intended GAC entry anymore...

http://blogs.msdn.com/junfeng/archive/2004/08/11/212555.aspx

http://blogs.msdn.com/junfeng/archive/2004/09/12/228635.aspx

How the OS Loader will force .Net v1.0/1.1 executables to run under WOW64 on a 64-Bit Machine

joshwil — Fri, 15 Oct 2004 20:15:00 GMT

[10/15, 2:04pm, fixed a couple typos; 10/15, 4:51pm, clarified a point]

Before you read this entry, you might want to read these two entries:

- http://blogs.msdn.com/joshwil/archive/2004/03/13/89163.aspx

- http://blogs.msdn.com/joshwil/archive/2004/03/11/88280.aspx

In case you skipped the link and kept on reading I’ll summarize the first post linked to (I however believe that they are really worth reading):

- “Bitness” is what we call an assembly’s ability to tell the OS and CLR what type of machine the assembly is safe to run on (32-bit vs. 64-bit, and on 64-bit: X64, IA64 or both).

- .Net 1.0 and 1.1 assemblies didn’t know anything about bitness

- There are things that you can do in a managed assembly which force you to need to bind it to one platform; these include: p/invoke, unsafe code, managed c++, etc…

The point is, in v1.0 and v1.1 we let you create assemblies that might very well have problems running on a 64bit platform, because of that we have decided that with .Net 2.0 (which is the first version of .Net to support 64-bit platforms natively) there will be an enforced loader policy such that 1.0 and 1.1 apps will be loaded by the 32-bit runtime on the machine.

We have gone back and forth on the right way to do this and have settled on the following:

- .Net 2.0 compilers will produce PE images with the IMAGE_COR20_HEADER.MinorRuntimeVersion set to 5, this matters in the case of MSIL (/platform:anycpu) images where the loader has the opportunity to choose whether the image should be loaded as a 32-bit or 64-bit image.

- MinorRuntimeVersion <5 will be used by the OS Loader to determine if the assembly is a .Net 2.0+ assembly or an older .Net assembly

- This will cause only .Net 2.0 executables to be loaded by the 64-bit runtime.

For reference, here is the definition of the IMAGE_COR20_HEADER:

// CLR 2.0 header structure.

typedef struct IMAGE_COR20_HEADER

{

// Header versioning

ULONG cb;

USHORT MajorRuntimeVersion;

USHORT MinorRuntimeVersion;

// Symbol table and startup information

IMAGE_DATA_DIRECTORY MetaData;

ULONG Flags;

ULONG EntryPointToken;

// Binding information

IMAGE_DATA_DIRECTORY Resources;

IMAGE_DATA_DIRECTORY StrongNameSignature;

// Regular fixup and binding information

IMAGE_DATA_DIRECTORY CodeManagerTable;

IMAGE_DATA_DIRECTORY VTableFixups;

IMAGE_DATA_DIRECTORY ExportAddressTableJumps;

// Precompiled image info (internal use only - set to zero)

IMAGE_DATA_DIRECTORY ManagedNativeHeader;

} IMAGE_COR20_HEADER;

It turns out that the 2.0 in the header structure name is kind of a misnomer and results from some pre-1.0 definitions of this structure. All released versions of the .Net runtime have had images that contain a IMAGE_COR20_HEADER structure. Additionally, all released versions of the .Net runtime have specified the MajorRuntimeVersion=2 and MinorRuntimeVersion=0. Kind of weird you might say for a v1.0/1.1 product? Yeah… That’s history for you…

What does this mean?

Well, it means that if you have a v1.0/1.1 assembly and run it on a 64-bit box with .Net 2.0 installed it will run under the 32-bit runtime in the WOW64. Whether it runs under a 1.1 32-bit runtime or the 2.0 32-bit runtime will be determined by the CLR’s loader policy, just as it would on a native 32-bit box. If you’re using v2.0 assemblies on a 64-bit machine and you have compiled them without the /platform switch or with /platform:anycpu (the default) then your image will load in the native runtime on whatever box you put it on. And of course the /platform:x86, /platform:x64, /platform:itanium switches work as they imply.

NOTE: /platform:anycpu doesn’t keep you from shooting yourself in the foot with a bad P/Invoke signature, unsafe code, etc… See this (http://blogs.msdn.com/joshwil/archive/2004/03/16/90612.aspx) blog entry for an example.

If you do have a managed v1.0/1.1 app which you believe to be 64-bit safe and whidbey compatible there is an easy way to make it run in 64-bit mode… You just have to whack the MinorRuntimeVersion to 5 in the IMAGE_COR20_HEADER for the image. Along these lines there will be a tool in the .Net 2.0 SDK called corflags.exe which will allow you to modify an image this way. A v1.0/1.1 image which has had its MinorRuntimeVersion whacked will still be compatible with the v1.0/1.1 runtimes (as per the version it was compiled against).

WARNING: If you don’t know what’s going on inside of some v1.0/1.1 image (if you didn’t write it say) you should be _VERY_ careful bumping it up to 64-bit as it may break in unexpected ways.

If you are a compiler writer and you want your application to run in 64-bit native mode under .Net 2.0 then you will need to produce images with this updated IMAGE_COR20_HEADER.

NOTE: .Net 2.0 assemblies that were compiled against the Beta 1 version of the framework (and the community drops up to now [10/15/04]) will act like v1.0/1.1 assemblies on newer builds of 64-bit OSes and load under the WOW64 in 32-bit mode.

What about v1.0/1.1 dlls?

If you make a .Net 2.0 executable and link against a v1.0/1.1 dll you will be able to load it into your process as if it was a .Net 2.0 MSIL assembly. If that v1.0/1.1 dll has code that isn’t safe to run in 64-bit mode it may crash.

Appendix: Why I think this is a good solution:

There are a number of ways in which we could have solved this problem. An easy one would have been to use the version string contained within the metadata for managed images. That string looks something like “v1.1.4322”, it represents the version of the framework which the assembly was compiled against, and.Net 2.0 assemblies will look something like “v2.0.X” (X still to be determined). You can argue that this makes sense, after all, Microsoft’s .Net 2.0 compilers (csc.exe, vbc.exe, cl.exe, etc…) will produce images that will be marked correctly. They need to since they support new features of the .Net 2.0 runtime (generics comes to mind) and bind against 2.0 frameworks libraries.

This however forces a tight binding between the runtime version and the compiler version, and doesn’t recognize the fact that there are plenty of assemblies compiled against the v1.0/1.1 frameworks that will run fine in 64-bit mode under v2.0. But even given that, its downfall is that it is a very Microsoft centric view, it assumes that the compilers will be updated at the same time as the runtime (which currently our compilers are). That tight binding wouldn’t recognize that there are compilers out there that only produce code which is safe to run in either 32-bit or 64-bit mode. Those compilers currently produce an image that works with the v1.0 or v1.1 runtime, and the compiler vendors may very well want to produce an image that also runs in 64-bit mode under the v2.0 runtime.

That’s the distinction, while Microsoft is producing compilers in this version of .Net that won’t create images which will run under any runtime prior to v2.0, other compiler vendors might very well want to produce images that run under prior versions (for deployments that maybe don’t have v2.0 installed yet) and yet still run in 64-bit native mode under v2.0 on a 64-bit machine.

This does put a great responsibility upon compiler vendors who chose to produce images that are marked this way, those compilers should only produce code which is safe to run in both 32-bit and 64-bit mode. Alternatively, they can do as our compilers do and let you shoot yourself in the foot if you want to by producing MSIL images with bad P/Invoke signatures and unsafe code… That puts the onus on the developer.

ArrayList’s vs. generic List for primitive types and 64-bits

joshwil — Tue, 13 Apr 2004 21:41:00 GMT

The dynamically growing ArrayList structure is an interesting one, the other day I was looking at memory usage of some performance scenarios on 64-bit machines vs. 32-bit machines and noticed that ArrayLists are double the size on 64-bit machines. After looking at the screen for a second wondering why that was it hit me… Under the covers an ArrayList is just a big Object[], and what’s an Object[]? It’s an array of references to Objects, references which are bigger on 64-bit platforms because they have to be able to reference more memory. Unfortunately that’s just a fact of life with bigger pointers.

But that realization did make me wonder what’s the difference in memory usage characteristics between using an ArrayList and using our new (v2.0) generic List<T> when you’re filling the list with primitive (non-reference) data types. It turns out to be interesting (at least to me) though predictable, so I thought I’d show it here…

As a spoiler, if you think that using the generic List<T> is better, you’re right! But you might not have guessed how right you are!!

Here’s my test code (I use a compile time define to compile with the generic v. non-generic list), I decided to just play with differences between lists containing ints here because they have an interesting property in that the size of a primitive int is the same as an Object reference on a 32bit machine. More about that at the end.

// SIZE is big enough that the list size dominates my memory usage
const int SIZE = 1000000;

public static void Main()
{
#if ARRAYLIST
     ArrayList list = new ArrayList();
#else
     List<int> list = new List<int>();
#endif
     for (int i=0; i<SIZE; i++)
     {
         list.Add(i);
     }
}

First off let’s think about what happens in these two scenarios:

1) ArrayList, under the covers this is an Object[], so to put an int value into the ArrayList we’ll have to box it (in the process creating a heap allocated Int32) and then put the reference to the boxed value into the array.
2) List<int>, with the generic implementation under the covers we’re using an int[], no boxing needed and instead we just shove the actual value being added into the array (thereby avoiding creating the heap allocated Int32).

Given that, we would expect the ArrayList implementation to be more of a memory hog, and it is

Results from a 32bit machine:

Here we can see that 19MB is allocated by the 32-bit ArrayList test, 11MB of which is simply heap allocated Int32 objects which result from the boxing of the int values. We also have 8MB worth of Object[] that have been allocated, notice that the amount of allocated Object[] is larger than the actual amount of space needed because I didn’t set the ArrayList capacity to something near what I knew we would actually. Instead I relied on the ArrayList auto-resizing which uses a 2x growth factor thus possibly ending up with up to ½ of the underlying array being unused, in this example that doesn’t effect the point that I’m trying to make so I ignored it.

On the other hand, using the List<int> version of the test we under the hood allocate a large Int32[] instead Object[], and we don’t need to heap allocate any Int32 objects. These differences result in memory usage of only 8MB for the Int32[]. Conveniently for my example on a 32-bit machine the size of an Object reference is the same as a 32-bit int so these arrays are the same size.

What do we see on a 64bit machine?

In this case the ArrayList test uses approximately two times more memory than the 32-bit version, this is unfortunately a fact of life in the 64-bit world… Given this data we can see that the reason is two fold:

1) References are bigger, and therefore our Object[] which underlies the ArrayList is double the size of it’s 32-bit version (16MB instead of 8MB)
2) Objects take more space in memory in a 64bit process. Every managed object has a couple pieces of CLR goo attached to it, a MethodTable* and a sync block, and these each end up being 8 bytes instead of 4 bytes like they are on 32-bit. In a small object like Int32 this combined with the fact that we keep the total size pointer size aligned results in an object that is two times as big on 64-bit. This results in 23MB of Int32 objects in a 64-bit process vs. 11MB in a 32-bit process. (see appendix below)

However, we can see that if we had instead used the generic List<int> we would have only allocated the same 8MB that we did in a 32-bit process. It is of note that since object references are 8 bytes on a 64-bit machine but an int is still 4 bytes we end up with the size of the underlying array being 50% smaller in the List<int> test case than the ArrayList case whereas on 32-bit they were the same. With smaller primitive data types (short, char, bool) the difference is of course more significant and is visible on both platforms since at that point the types are smaller than a object reference on 32-bit platforms as well.

Given that, what’s the difference in memory usage on the two platforms for the two solutions?

              ArrayList    List<int>    Difference (%)
32-bit   19MB         8MB          237%
64-bit   39MB         8.1MB        481%

We can see that though it is a definitely a better idea to use the generic List<int> on both platforms it becomes especially important on a 64-bit machine…

Then again, how many people are using ArrayList’s for big lists of ints and other primitive types? That’s a question that I don’t know the answer to…

The analysis (from a memory perspective) would be a lot different if we were talking reference types instead of ints. In that case we don’t have the option to avoid creating object instances; and the size of the reference to the reference type is the same size as that of a reference to an object. There are still plenty of reasons to use the generic List<T> in that case… But memory usage would no longer factor in.

Appendix: using SOS to look at Int32 objects in memory

64bit, Int32:
0:003> !do 000007ffe724d768
Name: System.Int32
MethodTable: 000000005cbf4408
EEClass: 000000005cc66380
Size: 24(0x18) bytes
(C:\WINDOWS\Microsoft.NET\Framework64\v2.0.amd64chk\assembly\GAC_64\mscorlib\2.0.3600.0__b77a5c561934e089\mscorlib.dll)
Fields:
MT Field Offset Type Attr Value Name
000000005cbf4408 40003cf 8 System.Int32 instance 3 m_value

32bit, Int32:
0:003> !do 00a56adc
Name: System.Int32
MethodTable: 78c12528
EEClass: 78c4ae44
Size: 12(0xc) bytes
(C:\WINDOWS\Microsoft.NET\Framework\v2.0.x86ret\assembly\GAC_32\mscorlib\2.0.3600.0__b77a5c561934e089\mscorlib.dll)
Fields:
MT Field Offset Type Attr Value Name
78c12528 40003bc 4 System.Int32 instance 437 m_value

Unsafe code, Stacks and IA64...

joshwil — Mon, 12 Apr 2004 15:35:00 GMT

Sometimes the differences between platforms can show up in interesting ways. Last week I was looking at a bug that was filed about a difference in error mode between IA64 and x64/x86 platforms… I thought the investigation led me down an interesting path so I thought I’d share it with you.

What might you assume the thread stack layout to look like on x86, x64, IA64? Well, fundamentally they all look pretty similar, something like this (artistic license has been taken):

-- top of stack

0x9000 -- Frame A

0x8000 -- Frame B

0x7000 -- Frame C

0x6000 -- Frame D [call graph looks like: A()->B()->C()->D()]

0x5FF8 <return address>

0x5FF0 BYTE* ptr

0x5F00 BYTE[] a (stack allocated byte array, size F0)

0x5000 <space>

0x1000 Soft guard

0x0000 Hard guard

Of course that has been significantly simplified for the purposes of this discussion, and some of the addresses might be a little bogus as I just made them up. The interesting take-away’s however are:

1) The stack “grows” down. That is D() is called by C() and therefore D’s frame is at a lower address than C’s.

2) At the end of the stack is a “guard” region, this is to implement stack overflow exception handling. If you touch the soft guard the OS will raise a stack overflow exception and you will be given the stack space that is the guard region to deal with it.

3) After the soft guard is the hard guard. The hard guard is always unallocated memory which will cause an AV if you touch it, in fact, if you’re dealing with a stack overflow caused by touching the soft guard and you use up too much stack and touch the hard guard then you’ll get an AV which will take down the process.

4) There is no stack guard region at the top of the stack to protect you from stack underflow, up there is just some random memory, could be another thread stack, could be the managed heap, could be the end of memory…

If you read off the top of your stack the results are undefined, but you can safely assume that if you keep reading then at some point you’ll get an AV, or so it seems.

Who would read off the top of the stack you might ask? Probably no one, but yesterday I ran into a test case that was doing just that, it would create a stack based byte array and then pass a pointer to the first element of the array to our unsafe string constructor which takes a byte* and a length. Instead of giving it the actual length of the byte[] that was created on the stack, the test case would proceed to pass a length like Int32.MaxValue or some other such huge (and incorrect) thing.

What happens behind the scenes in the BCL at this point isn’t exactly rocket science, we create a string and proceed to read bytes out of the passed in byte[], it is very similar in concept to if you wrote the following c# code yourself:

public unsafe void EventuallyBlowUp()

{

SByte* p1 = stackalloc SByte[256];

SByte temp;

for (int i=0; i<Int32.MaxValue; i++)

{

temp = *(p1 + i);

}

That’s over simplified, really we make a string after doing some range checks and such and then memcpy the data from the byte* into the string (there’s a reason that this code is marked as unsafe). Note that while the stack grows down, our reading of the data from the SByte[] results in addresses that grow up. Therefore at some point if the offset gets big enough we read off the top of the stack and into random memory.

In this specific test case we were looking for the “expected” AV to happen and be converted into our new AccessViolationException (I think this is new in v2.0). But on IA64 it wasn’t, instead it was coming back as a StackOverflowException. Confusion ensued… For a while I was convinced we had something weird going on where in this random case we had two thread stacks next to each other and for some reason instead of getting the expected AV when we hit the hard guard for the next stack we were skipping into its soft guard and getting a stack overflow instead, the problem however didn’t turn out to be nearly so convoluted.

First a little background, the IA64 platform actually has two stacks for a thread, the “normal” stack and the “backing store”. I really should get around to writing up a piece on the IA64 calling convention and by association the interesting thing that is the backing store and rotating register stack… but for now it is enough to know that IA64 has this other thing called the backing store which is used for storing register values to memory from registers that have been allocated by a function for use as input, locals and output… And this backing store is laid out in memory such that it is next to and contiguous with the “normal stack”… And it grows up instead of down. The picture looks something like this:

0x1a000 Backing store hard guard

0x19000 Backing store soft guard

0x14000 <space>

0x13000 -- Frame D rotating register store

0x12000 -- Frame C rotating register store

0x11000 -- Frame B rotating register store

0x10000 -- Frame A rotating register store

-- “top” of “backing store” stack

-- top of “normal” stack

0x9000 -- Frame A

0x8000 -- Frame B

0x7000 -- Frame C

0x6000 -- Frame D [call graph looks like: A()->B()->C()->D()]

0x5FF8 <return address>

0x5FF0 BYTE* ptr

0x5F00 BYTE[] a (stack allocated byte array, size F0)

0x5000 <space>

0x1000 Soft guard

0x0000 Hard guard

When we have code like that which we saw above, and we run it on an IA64 box the result of running off the top of our “normal” stack (where the byte[] is allocated) is different. Instead of immediately running into random memory (and presumably AVing), we will consistently run into a known piece of memory that is the backing store stack. And, as we continue reading up that stack eventually we will run into the backing store soft guard region and cause the OS to issue a stack overflow exception which the CLR will convert to a managed StackOverflowException and return to the code in EventuallyBlowUp(). Maybe EventuallyBlowUp’s caller deals with the stack overflow, maybe not, of course the same can be said for the AV.

The moral of the story, it’s difficult to completely abstract away the underlying platform. In this case we had a discussion about whether or not to “fix the bug” in the string constructor such that it would always return an AV by checking whether or not the requested start offset and length when used with the given pointer (if it was stack allocated) would result in stack underflow. We decided for now to leave it like it is because it’s unsafe code and the current implementation makes the failure mode match that of a programmer writing similar unsafe code themselves.

Fixing the general unsafe code stack underflow case is of course far from trivial, and of debatable value.

64-bit calling convention and PInvoke bugs

joshwil — Tue, 16 Mar 2004 18:28:00 GMT

Ok, so Word for the Mac is failing me right now. I've tried twice to start this entry there and both times Word has gone kaput on me. Back to my trusty text editor… As for the inevitable "why Mac?" question... Well, I still haven't found a laptop I like as much as my Titanium Powerbook. What can I say, I'm a hardware snob.

I originally intended to write my next entry about the GAC and its usefulness on 64-bit machines (for both the 32-bit and 64-bit CLR(s) that live there). I think this an interesting topic, especially given this article on Chris Sells' site. Alas, in writing it I realized that I need to do a little research and talk to a couple people before I feel completely competent with my facts.

So, to whet your appetite while we wait, how about an entry about the managed x64 calling convention, and a fun PInvoke bug that shows up on 64bit platforms because of the hardware difference and this calling convention? I would highly recommend reading Raymond's treatment of x64 calling convention.

One of the nice things about x64 is that we have narrowed ourselves down to one standard calling convention unlike x86 (of course if you're writing assembly you can do whatever you want). Both native code generated by the VC++ compiler and JITted managed code follows this convention. And it goes something like this:

- Arguments 1-4 are passed in registers rcx,rdx,r8,r9 (or if floating point Xmm0-Xmm3)

- Spill space is allocated by the caller for the enregistered parameters

- Additional parameters are passed on the stack previous (stack grows down) to this spill space in SLOT sized (read: 8-byte) chunks (i.e. even if you have a 1-byte bool, if you pass it on the stack it will take 8 bytes).

- The call instruction pushes an 8-byte return address onto the stack; this value will immediately follow the spill space for rcx.

- The stack must always be aligned to 16 bytes by non-leaf functions (read: if you make a call then you have to align it in the prolog).

- Non floating point returns are through rax (exception "retbufarg" which is treated later).

- Floating point returns are in Xmm0.

Floating point note: enregistered floating point parameters are put into the floating point register corresponding to their correct position in the argument list (e.g. if parameter 3 is floating point then it will be in Xmm2 instead of r8, this is different from IA64 where the floating point registers are filled using a "next available" heuristic).

That's the basics. Here are a couple of rules that build on that:

- If there is a "this" parameter (i.e. instance methods) it is put at the front of the argument list as arg1 and other args are moved by 1 slot.

- If there is a "retbufarg" then it will be treated as arg1, moving other arguments by 1 slot (including the “this” parameter). (e.g. arg1=retbufarg, arg2=this, arg3=declaration arg1, arg4=declaration arg2, etc...)

Most people (at least those reading this blog right?) know what a "this" parameter is, but what's a "retbufarg" parameter? It is a "secret" reference to space that is caller-allocated to receive the return value. This "retbufarg" parameter is passed when we can't put the return value in the return register rax. On x64 this happens when:

- the return value > 64-bits (e.g. won't fit in rax), excepting Doubles which will be returned through Xmm0.

- the size of the return value is not a power of two. e.g. a 7-byte value class (struct) returned by value will be returned by reference in a retbufarg.

Ok, so that's all well and good, but why did I need to know that you might be asking? Well, b/c it can affect lots of things. Lets take a PInvoke example that one of the devs on the 64-bit CLR team ran into on Thursday:

// defintion that worked on 32-bit

[DllImport(ExternDll.User32, ExactSpelling=true)]

public static extern IntPtr MonitorFromPoint(int x, int y, int flags);

The actual Win32 API specified that MonitorFromPoint() takes a POINT structure and an int argument named flags. Someone decided that it would be nice to not have to define a POINT structure (which is just an 8-byte structure consisting of two ints, x and y) and instead wrote their PInvoke using the two ints shown above.

This works on x86 where those parameters are passed on the stack. In fact, because you get lucky with the calling convention they look to the Win32 API as if you had correctly declared the POINT structure and passed it instead.

But!! On x64 this breaks in a rather interesting way... Let's go back to the calling convention discussion above. Using this scheme, the parameters will be set up as such:

rcx <- x

rdx <- y

r8 <- flags

Now, these register slots on x64 are 8 bytes wide, which means our 8-byte POINT structure, when passed by value, should actually be passed in a single register. What was the MonitorFromPoint() Win32 API expecting?

rcx <- POINT { LONG x, LONG y }

rdx <- flags

NOTE: keep in mind that the LONG as specified by MSDN here is the c++ LONG which is still 32 bits on 64-bit platforms, not 64 bits like the C# long. It is the equivalent of the C# int.

[correction made here, x/y high/low were reversed]

MonitorFromPoint() expected that x was the low 4 bytes of rcx and y was the high 4 bytes. As can be imagined, this code failed horribly on x64 as such:

-Specifically, the call was in some code that tried to compensate for multiple monitors by putting a dialog on the monitor where your mouse is.

return new Screen(SafeNativeMethods.MonitorFromPoint(point.X, point.Y, MONITOR_DEFAULTTONEAREST));

- The calculation depends on the x and y that you pass it (remember that the monitor’s upper left hand corner actually starts at 2000, 2000 or something like that)

[correction made here, re:messing up x/y position within struct... wrote it too late at night]

- The calculation that we do ends up FUBAR because the x you give the method ends up being seen by the Win32 API as the whole POINT structure. Thus, it thinks that y==0, and the dialog ends up pretty much unusable up in the upper left hand corner of the screen (halfway off the screen) with its title bar inaccessible to grab it and move it.

So, the fix, if you haven’t already guessed, is to define a POINT structure containing 2 ints “x” and “y” which you then correctly define as the first parameter to MonitorFromPoint(), in this way ensuring that the usage of MonitorFromPoint() is correct.

public static extern IntPtr MonitorFromPoint(NativeMethods.POINT pt, int flags);

NOTE: this will fail in the same way on IA64, but since this is an entry about the x64 calling convention, I thought I'd stick to talking about x64.

PInvoke errors are insidious because you might take for granted that the method you're calling is declared correctly. You would be likely to spend hours having to convince yourself that your managed code is correct. Or even worse, spend hours looking at your unmanaged code (or the disassmbly of some unmanaged code in Win32 for instance), convinced it is broken. Usually if there are PInvokes involved, I would take a look at those first, hopefully some of the CDP (customer debug probes) that are going into CLR for V2.0 will help out a lot. I haven't really played with them at all, but Adam Nathan's blog would probably be a good place to start.

Additionally, Raymond discusses what can go wrong when you mismatch calling conventions. This is something you might think impossible on 64-bit as we only have the one... But, a PInvoke declaration can have calling convention assumptions built into it, as seen above... Yet another case of "old problem, new form"!

Behavior of 1.0/1.1 managed apps on 64bit machines

joshwil — Sat, 13 Mar 2004 21:55:00 GMT

As I alluded to in my previous post there are multiple ways that we can go in terms of supporting legacy 1.0/1.1 assemblies on a Win64 machine. The context of the 1.0/1.1 support story is made somewhat simpler by the current plan of having both a v2.0 32bit CLR and a v2.0 64bit CLR on the box but no 1.0/1.1 CLR bits.

I mentioned that the 1.0/1.1 compilers didn't know anything about “bitness”. Basically they spit out a PE image that said “Hey! I'm managed! Run me with the CLR!” (gross simplification), whereas the v2.0 compilers produce images that range from “Hey! I'm managed, and I can run everywhere!!” to “Hey! I'm managed and I only run on x86!” etc...

This brings us to the fundamental question of this post -- what to do with 1.0/1.1 assemblies?

Option 1: call them “Legacy” assemblies since they don't know about “bitness”. Require them to run in the WOW64 under the 32bit CLR as we can't say for sure that the developer who created them was thinking about 64bit compatibility when they were created (remember that many of these were created years before even a 64bit alpha of .NET was available at PDC last year). Additionally, make the loader get angry and spew something along the lines of “BAD_IMAGE_FORMAT” if you try to load a legacy assembly in a native 64bit managed process just as if you had tried to load a v2.0 assembly marked x86 only.

Option 2: treat them like the v2.0 notion of MSIL assemblies, allow them to be used from both 32bit and 64bit managed processes. By default if they are an exe kick off the 64bit CLR when someone tries to start them. This would cause them to run as a 64bit process even though their creators probably didn't have that potential in mind when the code was written and tested.

Cases can be made for both sides. Right now the more conservative approach is “Option 1” which is what we are leaning towards. But there are definitely some negatives to that, the primary one in my mind being that is makes the transition to 64bit harder for groups that have dependencies on a lot of managed code that they don't own but are willing to do the testing legwork to make sure they work in 64bit mode anyway. In effect it makes the 1.0/1.1 managed code assemblies much like 32bit native code components as dependencies for moving your app to 64bit because in that scenario we won't let you load 1.0/1.1 assemblies in your 64bit process.

One of the great things about managed code is that frequently there isn't much if any work to be done to move it to 64bit. But given “Option 1” above we would at least require the work of a recompile (though someone could imagine a tool that would be frightfully dangerous which would modify 1.0/1.1 headers to look like 2.0 headers to pretend to be a v2.0 compiled MSIL image... Please don't do this!!). If you don't own the managed code you're using that means waiting for whoever does to recompile and give you the properly tagd version before you can move your app to 64bit.

Mind you, that is probably better than the alternative. If we were to just load up 1.0/1.1 images in a 64bit process expecting that they should be the equivalent of v2.0’s MSIL (which is what the compilers are currently producing as a default) you could end up with all manner of random execution failure, usually related to calling into some native 32bit code or other... “Option 2” would allow those who are willing do the legwork in testing to do their due diligence, test their application in a 64bit environment thoroughly even though it might contain 1.0/1.1 components and be able to say with reasonable confidence that their customers wont have problems running 64bit native. The fact that I id “willing to do the legwork in testing” and “due diligence” etc.. should be setting off huge danger signals in your head. How many people are willing to toughly test some component they paid for, isn’t that part of what you paid for??

There are of course all manner of “in-between” scenarios, few of which are supportable or justifiable, so for the purposes of this debate lets stick to these two options.

The main reason I started writing this post however wasn't to make up your mind but to poll your thoughts…

Thoughts?

As promised, what is the WOW64 and what does it mean to managed apps that you run on 64bit machines?

joshwil — Fri, 12 Mar 2004 00:11:00 GMT

So, what is the WOW64? If you already have a firm grasp please feel free to skip these first few paragraphs to the section titled “Break in here for the .Net perspective”.

When I went and Google’d a bit to see if anyone had a convincing answer out there what I seemed to find instead was confusion. Note this article titled “WOW64 for AMD Released to the Public”. With a firmer understanding of what the WOW64 is the fact that this headline is misleading would not have escaped that journalist. So lets start with the name:

WOW64 = Windows On Windows64

It might be more appropriately written “Windows32 On Windows64” but then the acronym isn’t nearly as cool. Basically it is a layer of code that allows for 32bit processes to run just like they were running on a 32bit system under a 32bit OS (e.g. your normal managed app running on the CLR on WindowsXP on a Pentium 4) even though in reality they are running on a 64bit OS. In fact you can find a good indicator of this by looking in your WINDOWS directory where now you will not only find the misnamed SYSTEM32 directory (which holds the 64bit dlls for the 64bit OS) but also the SYSWOW64 directory which holds 32bit versions of dlls for the virtual 32bit OS.

<historical note> I believe that there was a WOW32 effort as well during the move from Win16 to Win32. That move was accompanied by a significant change in programming model as the Win32 API was introduced (The Win16 stuff still lives in WINDOWS\SYSTEM). Generally the Win64 platform uses the Win32 API with platform specific data types (pointers and such) just expanded to fit the new hardware. This is why Windows decided to stick with the SYSTEM32 directory as the primary home for Win64 dlls even though the name can be misleading. This entry and discussion on Raymond’s blog is an interesting read that talks about the joys of backwards compatibility and Win16. </historical note>

You can imagine (and not be too far off) that once you’ve been kicked off by the loader as a 32bit process WINDOWS\SysWow64 gets aliased (for your purposes) to WINDOWS\System32. This has side effects, like the fact that one of my favorite little text editors (EmEditor) which is 32bit can’t find files in the real WINDOWS\System32 directory when running in the WOW64 on my 64bit boxes (it took me a while to figure out why I couldn’t find a couple of scripts that I knew for sure I had dropped in there). There are other parts of the system which are also split into two parts, most notably the registry, unfortunately I don’t know nearly enough about the technical details of that split to discuss it intelligently here.

Under the hood Windows is doing a lot of work to make sure things continue to look just like you’re on a 32bit box and this generally pays off in app compatibility. As would be expected however there is a performance hit that you take for running in this mode, it is an especially heavy hit on IA64 where in the current implementation the x86 instruction set is emulated with software. It is worth noting however that it does work pretty darn well, and the x86 CLR that we’ll be shipping to Win64 users to run in the WOW64 is the same build as that which we ship to x86 users (we just have to mess with the installer a bit). Other large apps can run in the WOW64 and be quite snappy on 64bit Extended hardware platforms (read:AMD64, IA32e) which can natively run the 32bit x86 instructions in a special more of the processor. I've played with VS for instance running on an AMD64 machine like this and it's great.

It is important to understand that once you’re loaded up in the WOW64 you’re a 32bit process and there’s no turning back. You have to use 32bit dlls all around. On the other hand, once you start up 64bit you’re 64bit, no 32bit dlls allowed. So, unfortunately this is not one of those things you can slowly transition. It is possible to kick off another process of a different bitness from your own and communicate with it through RPC or out of proc COM, and this can serve as a transition for some. But hopefully over the next couple months I’ll help to convince you that the move to running native as a 64bit process is going to be worth it.

>> Break in here for the .NET perspective <<

So, what does this mean for .Net and the CLR? The Whidbey CLR will be available in 32bit and 64bit versions with the 64bit version supporting x64 (AMD64/IA32e) and IA64 (Itanium). And the current plan is that when you install the 64bit CLR on a machine we’ll slap down the 32bit version of Whidbey at the same time. Yes, you read that right – your 64bit machine will end up with two copies of the runtime. The 32bit version will be installed in \WINDOWS\Microsoft.Net\Framework just as it would on a native 32bit machine whereas the 64bit version ends up in \WINDOWS\Microsoft.Net\Framework64, no rocket science here, though the GAC issues can be interesting (I’ll talk about these for sure in a later blog entry). Why do we need two frameworks on the machine you might ask? Isn’t one of the cool things about compiling to IL that the JIT takes care of the hardware specific stuff and your code just automatically takes advantage of the new platform it is there?

Well… There are many cases where you just want your code to “float up” to the new platform and run as native 64bit code. In fact, if you’re writing a fully managed app today chances are good that this is what you desire. But, there are still a lot of apps that use PInvoke, COM Interop to inproc servers (including using VB libraries pre-.Net) or APIs that won’t initially be available in 64bit which could break horribly. Remember that once you’re process is started as 64bit that’s it. Trying to load and execute code from a 32bit dll at that point is undefined and your process will certainly die a quick and painful death. While it is true that most of the dlls that come with Windows will be available on both 32bit and 64bit platforms many custom unmanaged code out there hasn’t yet made the jump. The plus side to not going to a new Win64 API is that if you define your PInvoke signatures correctly, most of the Win32 API is still there in a 64bit version and you’ll just pick up the new dlls and run native! In cases where you’re using some unmanaged code that you have no control over and isn’t available in a 64bit version it behooves you to specify that your app requires that it be run in a 32bit process.

How do you specify that? Well, fundamentally it is specified in the CLR headers of your PE image (the format of your dll or exe on disk and in memory). Realistically it is set at compile time (e.g. with C# use the /platform switch). Your exes and dlls can be marked one of four things: MSIL, x86, x64, IA64 (names vary depending on who you talk to). Let’s start out with the obvious ones:

x86: on a 32bit platform this will just run, no hassles. On a 64bit platform this will get started up as a 32bit process in the WOW64 running under the 32bit CLR. You’ll load 32bit .Net FX images out of the GAC (like MSCorLib, System.Web, System.Windows.Forms, etc..). Your code will be JITted by the x86 JITter and will run (in emulation on IA64) as x86 code.

x64: on a 64bit x64 platform this will run as a 64bit process. On a IA64 or x86 box you’ll get a BadImageFormatException. When you’re running as a 64bit process your image will be JITted 64bit native, we’ll pull 64bit images out of the GAC (be it by loading a 64bit native image that has been previously ngen’d or grabbing IL out of the GAC and JITting it at runtime to be 64bit).

IA64: like x64, but swap x64 for IA64.

Then there is MSIL (or “anycpu” as I think the C# compiler switch goes, some people will also refer to this as agnostic or neutral), this indicates that the code here really isn’t processor specific, on a 32bit platform it will get loaded up as a 32bit process and on a 64bit platform it will float up to a 64bit process and run under the 64bit CLR. This is rather important to know as this is currently the default for applications that are compiled using Whidbey compilers (disclaimer: this is still being hotly debated as to whether or not it is the right default, “currently” in this case means if you look at the build that I’m working with on my box on 3/11/04, what we ship may/may not reflect that).

<1.0&1.1 Note> As for 1.0/1.1 apps that are deployed onto a 64bit box, right now we’re probably going with a model that assumes that those assemblies (which didn’t know about bitness) will run in a 32bit process under the WOW64 as if they had been compiled with /platform:x86. The thought process leading us this direction is that we’d prefer to be conservative and by running 1.0/1.1 apps in the WOW64 we ensure that they don’t run into bitness issues that their developers might not have anticipated as it wasn’t part of the .NET world when they were being developed.

Under that model, if you do want your application to run in a 64bit process you’ll need to explicitly recompile it with one of the Whidbey compilers which know about bitness. Then (currently, see disclaimer above) if you don’t specify /platform:x86 your app will run 64bit when/if it is deployed to an x64 or IA64 box. </1.0&1.1 Note>

<GEEK> There is actually some interesting stuff that goes on under the covers here as well, that is if you’re interested in PE images and loader magic, x64 and IA64 images generated by the compiler with /platform:x64 or :IA64 will be PE32+ images (the 64bit extension to PE32) whereas x86 and MSIL are PE32 images (other wise they wouldn’t work on 32bit OS’s). When the OS loader comes across a managed image the first thing it does is hand it to some CLR code we call the shim (mscoree.dll) which interrogates the image, potentially makes some fixups and gives it back to the OS loader to let the OS then kick it off and trigger the actual runtime to start up. But, to get the OS loader to load your app as a 64bit process you have to give it a 64bit image, so in the shim on 64bit machines we will actually modify MSIL images in memory to turn them into PE32+ images before handing them back to the OS loader. This then in turn causes the loader to start up the right (64bit) runtime. </GEEK>

>> End .NET perspective <<

Recap: So, what have I talked about?

- WOW64 isn’t the OS per se, but a subset of the Win64 OS which enables a 32bit application to run inside of a 32bit process on a 64bit OS while using 32bit system dlls and such.

- Whidbey CLR will include both 32bit and 64bit versions, both of which will be installed on 64bit machines. This allows both 32bit and 64bit managed applications to run in bitness correct native process depending on how the assemblies are tagged at compile time.

- Once a process is started up as either 32bit or 64bit all of the dlls/assemblies that are loaded into that process have to be compatible with that bitness. There is a significant complexity for instance in the GAC to make this possible for .NET FX images (I promise to talk more about that in a later blog entry).

- If you have a 32bit managed app which has dependencies on 32bit unmanaged code then you’ll need to either find a 64bit version of the unmanaged code or tag your managed app as x86 at compile time to make sure that you don’t float up to a native 64bit process. This will result in you having to live with your process running under the WOW64.