David Notario's WebLog

Gotchas with Reverse Pinvoke (unmanaged to managed code callbacks)

davidnotario — Fri, 13 Jan 2006 12:36:00 GMT

One of the first things I had to do when I started working in Outlook Web Access (will call it OWA from now on) was integrating an unmanaged component that another group at Microsoft wrote. After spending some time with it, I decided the best way to bring it in would be writing a very small unmanaged library that wrapped its behavior and exposed a very simple interface to OWA. This interface would consist of only 3 or 4 APIs, and we would use them via PInvoke.

In order to avoid keeping unnecessary state around in the unmanaged wrapper dll, one of the APIs would do callbacks to OWA in order to report incremental results. This is something I hadn't done before in managed code. This is how it looks like:

C++ side:

typedef void (__stdcall *PFN_MYCALLBACK)();
int __stdcall MyUnmanagedApi(PFN_ MYCALLBACK callback);

C# side

public delegate void MyCallback();
[DllImport("MYDLL.DLL")] public static extern void MyUnmanagedApi(MyCallback callback);

public static void Main()
{

MyUnmanagedApi(
    delegate()
    {
      Console.WriteLine("Called back by unmanaged side");
    }
   );
}

The CLR will do all the magic to marshal our anonymous delegate to an unmanaged pointer that can be passed out to the C++ side. However, when we look a bit closely at things we may find some interesting problems:

What happens if my callback throws, what will my unmanaged code see? For example:

delegate()
{
Console.WriteLine("Called back by unmanaged side");
throw new ApplicationException("Let's see what happens");
}

If you try this out you will see that your C++ code will just see an SEH exception fly through... oops, my C++ code wasn't ready to deal with SEH stuff, it may leak. What can I do? In my case I controlled the signature of the callback, so I just changed the signature to return an HRESULT, so that I could morph exceptions into HRESULTS, which is what the CLR does for the COM interop case:

delegate()
{
int hresult = Constants.S_OK;
try
{
DoWork()
}
catch (Exception e)
{
hresult = Marshal.GetHRException(e);
}

return hresult;
}

Are we done? Nope. What are we missing? For one, a thread abort could be induced in the 'return hresult' statement, so we would be back to square 1, the unmanaged code seeing an unexpected SEH. Another interesting thing to look at is what does the unmanaged to managed transition look like. First, let's think about what an unmanaged to unmanaged callback would look like:

Let's assume a callback with that adds the 2 arguments. This would be some code in the caller:

mov ECX, [x]
mov EDX, [y]
call [EDX]

This is what the callback code could look like

mov eax, ecx
add eax, edx
ret

What can go wrong here? Not much, we jump directly to the callee, the only thing you could hit could be a stack overflow, if you are running out of stack when the call instruction pushes the return address. Besides that, this code shouldn't fail.

Now, let's go back to our unmanaged to managed callback. It's definitely more complicated than the unmanaged to managed one. What can go wrong? For example, marshalling a LPWSTR to a string object requires memory allocations, which may fail, also, you may not be allowed to actually go into managed code because the appdomain the thread was running on is unloading, etc... So definitely, things can fail before we have any chance to take any action, and again we are in hands of the CLR. Again, for COM interop, the CLR knows the signature has an HRESULT, so it will actually trap any of these problems and return an HRESULT. However, for our callback, what can it do if the callback returns void? In Whidbey, the answer is nothing, again, a rude SEH will be raised against us.

So basically, in order to be robust here, the best we can do right now is have the C++ code wrap it's callback with a __try/__catch block, which will handle these situations and also the thread abort on return I describe above. Of course, in this case I owned the unmanaged code, so I was able to fix it, but there are a lot of these callbacks in Windows APIs, which won't do the __try/__catch, so you basically are always at risk of something bad happening. Don't you hate when you can't write reliable code?

HRESULT __stdcall DoCallback(PFN_ MYCALLBACK callback)
{
HRESULT hr;
__try
{
hr = callback();
}
__except(FilterCLRExceptions(GetExceptionCode(), GetExceptionInformation()))
{
hr = OOPS;
}

return hr;
}

I talked with my excoworkers about this and they acknowledged that this is not something they are very happy about, but there just wasn't time to address this in Whidbey , fixing this is in the TODO list for future versions of the CLR.

Note: The one thing that drove me to write this down was that not one of the tutorials or documentation that explains 'reverse pinvoke' or unmanaged to managed callbacks mentions these problems, which is something I think we can do much better about (and we really need to do a better job, since 99.999% of people out there can't see our code to see what we really do).

New job!

davidnotario — Fri, 13 Jan 2006 10:42:00 GMT

Another long time without posting anything. This time due to changing groups!!! Once we had wrapped Whidbey up, I spent some time thinking about what I would like to do next. One of the good things about Microsoft is that it is a huge company, you can work in a ton of software areas here, from experimental OSs or the Windows kernel and all the way to something like MSN web services, passing by games, productivity software, databases, search, etc, etc... It's just amazing.

I had a great time in the CLR, I spent 5 years there, working with some of the most talented people I've ever met, had the opportunity to work on cool features of the product. Working in the CLR is close to building an OS, you are building this platform upon which a ton of groups and people build their own software. Most of the time you are working at a very low level, and we must be one of the few groups left in the company that is implementing crazy x86 tricks to get some extra cycles out of some important codepath. It's cool to be at the bottom of the stack, but it also has it's bad side, everything you do you have to look at from 1000 different angles, see how you will affect everybody, make sure you are meeting everybody's needs, and no matter what you do, somebody out there you will be breaking or annoying. The result of this is that everything moves slow, it's the price of doing business when you are the platform.

So after thinking about it, I heard about an opportunity in Exchange, in Outlook Web Access. I had some friends there, plus its an app I use every day and I like, it's mostly managed code and there are a bunch of things to work on. After 4 months here, I can say I'm happy with my decision, I get to work on a lot of interesting stuff, I'm using the platform I helped to build, which is an incredible feeling and I get to use (dogfood) the app I'm building, every day, which is a really good feeling.

So anyways, I'll probably keep posting about CLR, I'm seeing a lot of things from a different perspective now. I now I'm using managed code every day in a real product, not in my home projects, and boy, it's amazing, The combination of Visual Studio + the super rich set of libraries the .NET platform has is just a massive productivity boost, you really have fun writing code. I also get to see some uglyness we didn't really get right, and that until you feel the pain, you don't realize how much a simple mistake in the platform or one of these 'postponed' features can cost everybody out there for n years.

Does the JIT take advantage of my CPU?

davidnotario — Mon, 15 Aug 2005 21:36:00 GMT

Short answer is yes. One of the advantages of generating native code at runtime is that we know what processor you are running on and we can tune the code accordingly. Why would we do that for x86? Every generation of x86 processors has its own personality. Their personality comes usually in 2 ways:

- New instructions: For example, SSE and SSE2 instruction set

- New ‘moods’: For example, Pentium 1 wanted programmer to schedule instructions by hand (in order to fill its 2 execution pipes), changes in branch prediction logic, P4s trace cache, etc, and even in the form of ‘regressions’, such as P4 preferring ADD REG, 1 vs INC/DEC instructions (which were very frequent instructions in tight loop code).

Also, AMD processors have their own personality, although in my experience, AMD’s are much more predictable and ‘well behaved’, and thus, need less work.

Note that we just don’t jump and implement in the JIT functionality to take advantage of every processor difference. The process is usually identifying something that is hurting in one of the benchmarks or user scenarios we track, evaluate the cost of the fix, the risk of the fix (every time we make a processor specific optimization, we make the life of our test team a bit harder and it’s easier to have x-proc only bugs, which are a bit harder to track down) and then, once we have all that data, we make a decision.

Examples of some of the processor specific optimizations in x86 (none of these will be a big surprise for developers that do machine level programming, all of these optimizations are called out in big fonts in the processor optimization manuals).

- Use of CMOV instruction when available (enables conditional moves, which is very useful in branches that are taken in a random (ie, non predictable by the processor) fashion

- Use of FCOMIx family of instructions (makes floating point comparisons much cheaper)

- Use of SSE2 for memory copies (memory copies are fun, you would expect something that simple to be always, the same, but I’ve witnessed 4 ‘recommended’ ways of doing it during the time I’ve worked with x86: Use string instructions (REP/MOVS/STOS), Use floating point registers for the move, Use scalar instructions (to get better pairing/parallelization) and now use SSE2). Note that we don’t use SSE2 for floating point code. The reason for this is that we don’t vectorize code (which is the real win with SSE2), SSE2 for scalar floating point is not always a win compared to the x87 (different latencies between instruction sets for adds and multiplications makes each one of them better than the other depending on the scenario) and some things like converting doubles to floats was really slow on SSE2, so we decided investing in making our x87 code better (which we were going to have to support anyways).

- Use of SSE2 for floating point to int conversion.

- Other minor instruction selection differences (such as avoiding INC and DEC instructions in hot code for P4 or to avoid store forwarding problems in P4 and Centrino processors)

We don’t take advantage of other things, such as knowing code cache sizes, etc… One of the reasons for this is that we don’t want different code on every single machine out there. As usual, there is a trade off if we did this, we may get some extra speed in some situations, but on the other hand, in a realistic world, it’s more likely for us to produce bugs that only repro in machines that meet n conditions, so introducing more processor specific optimizations has to be done carefully.

What about NGEN?

NGEN is an interesting case. In previous versions of the CLR (1.0 and 1.1), we treated NGEN compilation the same way as we treated JIT compilation, ie, given that we are compiling in the target machine, we should take advantage of it. However, in Whidbey, this is not the case. We assume a PPro instruction set and generate code as if it was targeting a P4. Why is this? There was a number of reasons:

- Increase predictability of .NET redist assemblies (makes our support life easier).

- OEMs and other big customers wanted a single image per platform (for servicing, building and managing reasons)

We could have had a command line option to generate platform specific code in ngen, but given that ngen is mainly to provide better working set behavior and that this extra option would complicate some other scenarios, we decided not to go for it

CLR and floating point: Some answers to common questions

davidnotario — Mon, 08 Aug 2005 21:50:00 GMT

Some very common questions I get from customers regarding floating point are:

- I get different results when compiling with optimizations vs without optimizations!

- My == checks are failing when the expressions are the same!

The answer for this question is most of the time ‘This is by design’, but it does seem that we’re not doing a good job explaining why it is by design or why things work this way. I’ll try to cover the most common causes for this behavior in this post.

X87 FPU

X86’s old CPU is based on the following

- 8 registers laid out in the form of a stack. The registers are 80 bit wide, although the precision at which operations happen (significant bits of the mantissa) can be modified via a control register. The range cannot be changed though (the exponent will always be 15 bit wide)

- Some control registers that control the precision, exception modes, etc of the FPU.

- Set of instructions to operate with stack registers and/or memory. Instructions that operate with memory can only use a register operand if the register is the top of the stack. Operations that involve 2 FP registers can access any element of the stack. This aspect of the FPU is what makes code generation for it more ‘interesting’

Most of the non intuitive behavior people encounter comes from the fact that the registers are 80 bit wide. Precision is set by default in VC++ and CLR apps to ‘double precision’, which means that if you are operating with operands of type float, results of operations done with floats actually exist in the x87 stack as if there were of type double. In fact, it’s even weirder than that. They will have the mantissa of a double, but the range (exponent) of an extended double (80 bit).

That means that a sequence of operations like the following

- Load float A

- Load float B

- Multiply loaded arguments

- Store float result in memory

- load float A

- load float B

- Multiply loaded arguments

- Compare with stored result in memory

May result in a result of ‘not equal’. Why? If the result of the first multiplication was not representable exactly as a float, it lost bits of precision when stored in memory, thus, when we compared what we stored in memory with what was in the FP stack, we see that the numbers are similar, but not equal, resulting in a surprising ‘not equal’ result.

Even more subtle

- load double MAX_DOUBLE

- add

- store double A

- load double A

- load double MAX_DOUBLE

- Substract

Is different than

- load double MAX_DOUBLE

- add

- load double MAX_DOUBLE

- substract

At the end of this sequence we have +Infinity in the floating point stack, because 2*MAX_DOUBLE is bigger than MAX_DOUBLE, which is the biggest number that can be represented in a double. Substracting MAX_DOUBLE from Inifinity still results in infinity. However, in the second example, everything happened in the floating point stack, which is operating in double extended range, and can represent 2*MAX_DOUBLE, so when we substract, we get MAX_DOUBLE, instead of Infinity.

What CLR has to say

How does the CLR play with this weird HW? Pulled out from the ECMA spec:

Storage locations for floating-point numbers (statics, array elements, and fields of classes) are of fixed size. The supported storage sizes are float32 and float64. Everywhere else (on the evaluation stack, as arguments, as return types, and as local variables) floating-point numbers are represented using an internal floating-point type. In each such instance, the nominal type of the variable or expression is either R4 or R8, but its value can be represented internally with additional range and/or precision. The size of the internal floating-point representation is implementation-dependent, can vary, and shall have precision at least as great as that of the variable or expression being represented. An implicit widening conversion to the internal representation from float32 or float64 is performed when those types are loaded from storage. The internal representation is typically the native size for the hardware, or as required for efficient implementation of an operation. The internal representation shall have the following characteristics:

· The internal representation shall have precision and range greater than or equal to the nominal type.

· Conversions to and from the internal representation shall preserve value.

[Note: This implies that an implicit widening conversion from float32 (or float64) to the internal representation, followed by an explicit conversion from the internal representation to float32 (or float64), will result in a value that is identical to the original float32 (or float64) value. end note]

[Rationale: This design allows the CLI to choose a platform-specific high-performance representation for floating-point numbers until they are placed in storage locations. For example, it might be able to leave floating-point variables in hardware registers that provide more precision than a user has requested. At the same time, CIL generators can force operations to respect language-specific rules for representations through the use of conversion instructions. end rationale]

When a floating-point value whose internal representation has greater range and/or precision than its nominal type is put in a storage location, it is automatically coerced to the type of the storage location. This can involve a loss of precision or the creation of an out-of-range value (NaN, +infinity, or ‑infinity). However, the value might be retained in the internal representation for future use, if it is reloaded from the storage location without having been modified. It is the responsibility of the compiler to ensure that the retained value is still valid at the time of a subsequent load, taking into account the effects of aliasing and other execution threads (see memory model section). This freedom to carry extra precision is not permitted, however, following the execution of an explicit conversion (conv.r4 or conv.r8), at which time the internal representation must be exactly representable in the associated type.

[Note: To detect values that cannot be converted to a particular storage type, a conversion instruction (conv.r4, or conv.r8) can be used, followed by a check for a non-finite value using ckfinite. Underflow can be detected by converting to a particular storage type, comparing to zero before and after the conversion. end note]

[Note: The use of an internal representation that is wider than float32 or float64 can cause differences in computational results when a developer makes seemingly unrelated modifications to their code, the result of which can be that a value is spilled from the internal representation (e.g., in a register) to a location on the stack. end note]

This spec clearly had in mind the x87 FPU. The spec is basically saying that a CLR implementation is allowed to use an internal representation (in our case, the x87 80 bit representation) as long as there is no explicit storage to a coerced location (a class or valuet type field), that forces narrowing. Also, at any point, the IL stream may have conv.r4 and conv.r8 instructions, which will force the narrowing to happen.

Why did the spec implementers decide to go down this route? Imagine they hadn’t, and said that the precision/range of FP results would always have to be of the type of their operands. In x87 it would mean having to spill to memory (in order to narrow down to the operand precision/range) after every operation. This can become a very high price to pay for that extra predictability, which is not really cross platform, and that not everybody needs.

Typical problems: It’s a language choice

Note that with current spec, it’s still a language choice to give ‘predictability’. The language may insert conv.r4 or conv.r8 instructions after every FP operation to get a ‘predictable’ behavior. Obviously, this is really expensive, and different languages have different compromises. C#, for example, does nothing, if you want narrowing, you will have to insert (float) and (double) casts by hand. On the other hand, VC++ Whidbey has a new floating point model, which by default will do narrowing on assignment boundaries (it’s more complex than that, see http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dv_vstechart/html/floapoint.asp for more details).

Let’s look what can happen with current C# semantics in a recent question we got:

int Compare(Point x, Point y)

{

float thetaX = Worker(pivot, x);

float thetaY = Worker (pivot, y);

if (thetaX == thetaY)

{

return (0);

}

else

{

return (-1);

}

Worker was a ‘pure’ function that returned a float value, as in it depended only of it’s input for it’s result, and produced no side effects. The surprise was coming from the fact that if you called Compare with an x == y, the function would still result in a -1. Let’s look at the disassembly to see what may have happened:

…. Prolog

lea EAX, bword ptr [ESI+4]

cmp ECX, dword ptr [EAX]

sub ESP, 8

movq XMM0, qword ptr [EAX]

movq qword ptr [ESP], XMM0

lea EAX, bword ptr [ESP+14H+08H]

sub ESP, 8

movq XMM0, qword ptr [EAX]

movq qword ptr [ESP], XMM0

call [Worker(struct,struct):float]

fstp dword ptr [ESP]

add ESI, 4

sub ESP, 8

movq XMM0, qword ptr [ESI]

movq qword ptr [ESP], XMM0

lea EAX, bword ptr [ESP+0CH+08H]

sub ESP, 8

movq XMM0, qword ptr [EAX]

movq qword ptr [ESP], XMM0

call [Worker(struct,struct):float]

fld dword ptr [ESP]

fcomip ST(0), ST(1)

fstp ST(0)

jpe SHORT G_M004_IG03

jne SHORT G_M004_IG03

xor EAX, EAX

…. Epilog

What is happening is that Worker returned a value in the ST(0) register of the FP stack, which was narrowed down to float precision in the fstp (memory store) following the first call. But then, for doing the comparison, it’s comparing ST(0) returned by the second call with the narrowed down to float result of the first call. So if what Worker did was not exactly representable in a float, the comparison (fcomip) would result in false.

How could this be solved? By narrowing by hand the result of the Worker() calls:

int Compare(Point x, Point y)

{

float thetaX = (float) Worker(pivot, x);

float thetaY = (float) Worker (pivot, y);

if (thetaX == thetaY)

{

return (0);

}

else

{

return (-1);

}

Which generates:

…prolog

lea EAX, bword ptr [ESI+4]

cmp ECX, dword ptr [EAX]

sub ESP, 8

movq XMM0, qword ptr [EAX]

movq qword ptr [ESP], XMM0

lea EAX, bword ptr [ESP+18H+08H]

sub ESP, 8

movq XMM0, qword ptr [EAX]

movq qword ptr [ESP], XMM0

call [ComputationalGeometryUtils.theta(struct,struct):float]

fstp dword ptr [ESP]

add ESI, 4

sub ESP, 8

movq XMM0, qword ptr [ESI]

movq qword ptr [ESP], XMM0

lea EAX, bword ptr [ESP+10H+08H]

sub ESP, 8

movq XMM0, qword ptr [EAX]

movq qword ptr [ESP], XMM0

call [ComputationalGeometryUtils.theta(struct,struct):float]

fstp dword ptr [ESP+04H] ; Narrow down

fld dword ptr [ESP+04H] ; and reload result of call

fld dword ptr [ESP] ; result of first call

fcomip ST(0), ST(1) ; comparison is now done on narrowed down results

fstp ST(0)

jpe SHORT G_M004_IG03

jne SHORT G_M004_IG03

xor EAX, EAX

.epilog

This will now be more in line with what programmer expects. All this that is happening is in 100% compliance with CLR ECMA specification. I would agree that it is a behavior that is not 100% intuitive, but the problem is that being intuitive will result in a performance penalty, which all users may not be happy with. Maybe the solution is for C# to adopt FP models just like C++ has now, which, although don’t solve the problem completely, do help in the most common situations.

Lazy init singleton

davidnotario — Wed, 20 Jul 2005 05:05:00 GMT

I’ve seen some confusion in some MS internal mailing lists about when singletons are instantiated for the pattern described in:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnbda/html/singletondespatt.asp

// .NET Singleton

sealed class Singleton

{

private Singleton() {}

public static readonly Singleton Instance = new Singleton();

}

The confusion comes from trying to understand when the singleton’s instance is created. The article states:

‘What we really care about is that we get the instance created either on or just before the first call to (in this case) the Instance property, and that we have a defined initialization order for static variables in a class’

With the code above, the correct answer is ‘Whenever the CLR decides so, as long as it happens before the first use of Singleton.Instance’. This is not what you really want if your singleton is bringing in some expensive resource and the text in the article if not wrong, may be a bit misleading.

The following counterexample initializes the singleton even if we will never call Instance.

using System;

sealed class Singleton

{

private Singleton()

{

Console.WriteLine("Singleton initialized");

}

public void Method()

{

Console.WriteLine("Method");

}

public static readonly Singleton Instance = new Singleton();

}

class Test

{

static public bool bFalse = false;

static public void Main()

{

if (bFalse)

{

Singleton.Instance.Method();

}

Why?

The compiler embeds the static initialization

public static readonly Singleton Instance = new Singleton();

inside the class constructor (.cctor) of the class Singleton. So the question now is ‘when does the .cctor get run’. I go over this in http://blogs.msdn.com/davidnotario/archive/2005/02/08/369593.aspx . If you really want a singleton that only gets created on first access, you want the Singleton type to have precise init. You would do it this way:

sealed class Singleton

{

Static Singleton()

{

}

private Singleton()

{

Console.WriteLine("Singleton initialized");

}

public void Method()

{

Console.WriteLine("Method");

}

public static readonly Singleton Instance = new Singleton();

}

Now, note this behavior may come at the expense of runtime checks (that ask the 'did my .cctor run already?' question), whereas with beforefieldinit semantics, the .cctor gets run aggressively by the JIT (at compile time) and no checks need to be inserted (so maybe you want 2 singleton classes, Singleton and ExpensiveResourceSingleton, depending on your needs).

Interested in working in the CLR?

davidnotario — Wed, 15 Jun 2005 18:15:00 GMT

Would you like to join the CLR's development team? We have 2 job openings for the Rotor project, the shared source CLR that brings managed code to BSD, Mac and Windows. You can find more about Rotor here:

http://msdn.microsoft.com/msdnmag/issues/02/07/SharedSourceCLI/

For additional information on the positions or submitting your resume, check out the following URLs

http://members.microsoft.com/careers/search/details.aspx?JobID=be2065d1-0943-4ebf-9560-ada9673a6cd6

http://members.microsoft.com/careers/search/details.aspx?JobID=b02cafac-c95a-46e1-b110-e3faef0c1db2

We're looking forward to hearing from you!!!

Other positions in the CLR team can be found in:

http://members.microsoft.com/careers/search/results.aspx?FromCP=Y&JobCategoryCodeID=&JobLocationCodeID=&JobProductCodeID=10166&JobTitleCodeID=&Divisions=&TargetLevels=&Keywords=&JobCode=&ManagerAlias=&Interval=10

What is mscorsvw.exe and why is it eating up my CPU? What is this new CLR Optimization Service?

davidnotario — Thu, 28 Apr 2005 06:36:00 GMT

Short version:

mscorsvw.exe is precompiling .NET assemblies in the background. Once it's done, it will go away. Typically, after you install the .NET Redist, it will be done with the high priority assemblies in 5 to 10 minutes and then will wait until your computer is idle to process the low priority assemblies. Once it does that it will shutdown and you won't see mscorsvw.exe. One important thing is that while you may see 100% CPU usage, the compilation happens in a process with low priority, so it tries not to steal the CPU for other stuff you are doing. Once everything is compiled, assemblies will now be able to share pages across different processes and warm start up will be typically much faster, so we're not throwing away your cycles.

If you are really want to get rid of mscorsvw.exe from your task manager, just do:

ngen.exe executequeueditems

which will drain all the queued up work.

Long version:

If you wonder why I haven't been posting much to my blog, mscorsvw.exe is the reason ;). mscorsvw.exe doubles up as both the CLR Optimization Service (or NGEN Service, as we know it internally) and as the NGEN worker. If you don't know what NGEN is you may start by reading Reid Wilkes' excellent article about NGEN in Whidbey: http://msdn.microsoft.com/msdnmag/issues/05/04/NGen/default.aspx . Reid is one of the members of the team that has been working in NGEN and the Optimization Service.

What problem is the CLR Optimization Service trying to solve?

One problem with precompiled assemblies (I will refer to them as NGEN images) is that they are very dependent on the VM and other asemblies. For example, they have Method Tables in a binary form, know what CLR helpers to call to do various tasks, may inline code from other assemblies, etc...

The downside, of course, is that whenever the runtime (for example via a Service Pack) or one of the dependencies of your ngen images changes (version upgrade or patch), your ngen image becomes invalid. In v1.0 and v1.1 of the CLR we weren't really encouraging people to use NGEN, and this was one of the reasons, nobody recompiled the assemblies if they became invalid, so anybody that NGENed an assembly had to NGEN it again after servicing.

Due to some changes in our long term schedule, we decided after shipping Beta 1 that we had to have a better story in Whidbey for this problem, hence the CLR Optimization Service was born. The CLR Optimization Service will keep track of dependencies of assemblies and will recompile them as necessary. For example, when the .NET Runtime gets serviced, the SP installer will tell the service that there have been changes in the runtime and that it should start recompiling assemblies. Of course, if you have 1000 managed applications installed, that would mean it would take some time to complete the installation of the SP, so instead of waiting to recompile the world, it will compile assemblies in the background.

How does the CLR Optimization Service work?

Reid's article already covers most of this, so I won't extend myself much on this.

There's 2 ways of waking up the service:

- Notifying the service there have been changes that may require recompilation (via ngen.exe update): The service will start up and figure out what needs to be recompiled and will queue up the work that is necessary.

- Queueing up new work (via ngen.exe install /queue): will add new work to the queue of assemblies that need to get compiled.

When does the service do work?

Work is prioritized. There's 2 big buckets of work

- High priority work: Service starts work immediately. Compilation happens in a low priority process to try to minimize impact on other apps that are running. It still can have impact on the machine performance, though, specially if the machine is doing a lot of I/O work or is very low on memory. Currently, priorities 1 and 2 are high priority, but this is subject to change.

- Low priority work: Service starts but waits until the machine is idle. Our definition of idle is something that can change in the future, but I can tell you it looks at things like the last time the user had input, if the machine is running on batteries, screen savers, etc... Once it decides the machine is idle, it will go ahead and do the work that's left.

Once all work is done, the service will shut itself down and won't come back until somebody wakes it up.

What is the typical lifetime of the CLR Optimization Service?

Let's look at the .NET Redist as an example. The .NET Redist does some compilation during setup and also queues up both high and low priority work items. Typically, when the installation completes, you will see 2 instances of mscorsvw.exe running. One is the service itself and the other is the compilation worker process, which is actually doing all the interesting work. If you are curious about which one is which, as a rule of thumb, the one using more memory and CPU is the worker. If you want to know for sure, you can do 'tasklist /svc', which will show you which mscorsvw.exe is the process.

After a while, all the high priority work will get done and you will see only one mscorsvw.exe running. The service is just waiting for the machine to be idle. During this time, the service is not doing anything interesting, so it won't consume CPU and shouldn't be using much memory. One the computer is idle (typically when you take a break or go for lunch), the service will start up new workers to complete the remaining work.

Once all work is done, the service will set itself to manual start (meaning it won't start when you restart your computer) and will shut down.

Should I use the CLR Optimization Service in my installer?

The guideline I would give is the following:

Use /queue in your installer if you have assemblies that you would like NGENed, having the assembly compiled is not necessary for you to meet your performance goals (as with /queue there is no guarantee of when the compilation will actually happen), and install time is very important for you. Rationale: High priority work can impact user experience and with low priority work you will at least have a service hanging around. Also Idle detection is not perfect (or each person will have one definition of idle), so the best way of having no negative impact on your customer's machine is not having any pending work.

What about 64 bit?

64 bit .NET Redist actually installs 2 versions of the runtime side by side, a 32 bit and a 64 bit one. You get 2 services, but they cooperate to make sure that both are not working at the same time.

Why don't you take advantage of my multiproc machine?

Because we don't want to degrade machine performance. The CLR Optimization service tries hard to do work in the background, having 2, 4 or 8 compilations happen at the same time multiplies by 2, 4 or 8 the memory/CPU/I/O resources we would need.

How do I make mscorsvw.exe go away?

I don't want to wait until my computer is idle, compile everything now!!!

ngen.exe executequeueditems

will process all pending work. Once this is done, the service will shutdown, as it has nothing else to do.

How do I know what is compiled or if my deferred compilation failed?

Look in the Application Event Log, we log there when we start and finish compiling assemblies, and any errors that can occur. We are working on improving the logging for RTM version.

ngen.exe display

Will also give you some status, but is not 100% guaranteed to be completely up to date regarding pending work.

Feedback

We've worked really hard to get this service done for Beta 2. One of the mean reasons to do this is getting feedback from our customers, so let us know your opinion, concerns, or if you see anything suspicious or that looks like a bug let us know about it: http://lab.msdn.microsoft.com/productfeedback/ (we keep better track of bugs/suggestions that come from there, so while you're free to use the blog to do that, I recommend you doing it via the feedback site)

I'm posting this now because I have seen already questions in the newsgroups. Let me know if you have any other questions. I'll also be updating this post as things come up

[Update Jun/17/2005]
After investigating some machines that were having issues with mscorsvw.exe, we've found a problem that occurs when you have 2 builds of the CLR installed (typically a Whidbey Beta 2 and a CTP). This configuration was never supported (we don't support side by side builds of Whidbey) and other things are broken. The underlying cause of the problem is that we're not COM compatible between builds (only between major versions, for example: Everett and Whidbey's COM interfaces are compatible) and setup is not updating type libraries correctly in this scenario. I asked our Setup team to address this issue (by telling the user that side by side builds is not a supported scenario or disallowing the installation) and it will be addressed in the future (due to schedule limitations, it's not possible for them to do this work in Whidbey timeframe). We also have done work to ensure that even if we run into the scenario, we terminate gracefully

Typical symptoms of this problem include mscorsvw.exe permanently taking 100% CPU. If you try doing 'ngen.exe display' (using the Whidbey ngen.exe) and get a E_NOINTERFACE error you are hitting this issue. I recommend you cleaning your machine (as there may be other CLR stuff that is broken), but as a workaround you can go to the services panel (services.msc) and disable the service or do it via command line:

Disabling NGEN Service for Beta 2 build (for other builds, substitute 50215 with the number of your build)

sc.exe stop clr_optimization_v2.0.50215_32

sc.exe config clr_optimization_v2.0.50215_32 start= disabled

sc.exe stop clr_optimization_v2.0.50215_64

sc.exe config clr_optimization_v2.0.50215_64 start= disabled

I apologize for any trouble this may have caused.

[Update Sep/19/2005]
Some folks have also reported mscorsvw.exe problems in Vista Beta 1. There's 2 problems people are seeing here:

- Same as problem described above, but caused by installing Betas, CTP or RC builds on top of Vista. Vista has already installed a Whidbey build, so if you install another CLR build on top of it, you may break the Whidbey build in LH (or viceversa). Once Vista has final Whidbey CLR bits, the problem will go away, as trying to install RTM Whidbey on top of Vista will result in a no op. We tried to get in some changes so that the MSI installers refused to install on Vista (it's a non supported scenario to have any other Whidbey build than the one that comes with the OS), unfortunately, it was too late to do this. Again, as a workaround, you may want to disable the service and make the problem go away, but there is possibly other stuff broken (such as the debuggers or any other COM based infrastructure the CLR has)

- mscorsvw.exe kicks in when the machine is not idle. Due to recent changes in Vista in how services run (they are now run in a different session), the idle time detection of NGEN Service doesn't work well in Vista Beta 1, as it cannot see user's input, declaring the machine as idle before it should. This has been addressed in our RTM release (which will probably make it for Vista Beta 2). Also, note that not all service work is meant to be done at idle time. In Vista, some of the high priority work happens during setup/first boot, then the low priority work happens at idle time.

JIT compiler and type constructors (.cctors)

davidnotario — Wed, 09 Feb 2005 06:37:00 GMT

I get this question a lot

‘When do class constructors (.cctor) get run’

My answer is usually:

‘It depends’

From Partition 1 of the ECMA spec (http://msdn.microsoft.com/netframework/programming/clr/default.aspx)

The semantics of when, and what triggers execution of such type initialization methods, is as follows:

1.             A type may have a type-initializer method, or not.

2.             A type may be specified as having a relaxed semantic for its type-initializer method (for convenience below, we call this relaxed semantic BeforeFieldInit)

3.             If marked BeforeFieldInit then the type’s initializer method is executed at, or sometime before, first access to any static field defined for that type

4.             If not marked BeforeFieldInit then that type’s initializer method is executed at (i.e., is triggered by):

·         first access to any static or instance field of that type, or

·         first invocation of any static, instance or virtual method of that type

5.             Execution of any type's initializer method will not trigger automatic execution of any initializer methods defined by its base type, nor of any interfaces that the type implements

Note: BeforeFieldInit behavior is intended for initialization code with no interesting side-effects, where exact timing does not matter. Also, under BeforeFieldInit semantics, type initializers are allowed to be executed at or before first access to any static field of that Type -- at the discretion of the CLI

If a language wishes to provide more rigid behavior -- e.g. type initialization automatically triggers execution of parents initializers, in a top-to-bottom order, then it can do so by either:

·              defining hidden static fields and code in each class constructor that touches the hidden static field of its parent and/or interfaces it implements, or

·              by making explicit calls to System.Runtime.CompilerServices.Runtime-Helpers.RunClassConstructor (see Partition IV).

Besides the difference in the number of ‘events’ we look at (static access vs static access + any method) there is the difference in timing, for beforefieldinit classes, the CLR is allowed to run the .cctor at any point before it’s actually required by the beforefieldinit semantics, whereas for precise types, we have to run them exactly at the trigger points mentioned in the spec (where ‘exactly’ means that we have to maintain the order of all visible side effects).

What does this mean for your code?

- For beforefieldinit types, you can’t control the exact point where your .cctor is run. On the other side, it gives the CLR a better opportunity to optimize

- With precise types, you know exactly when the .cctor is going to be called.

Let’s see it with a C# example. In C#, a class has precise semantics when it has an explicit .cctor (ie, static Type() { } ). If it doesn’t have an explicit .cctor, the class gets tagged with the BeforeFieldInit attributee.

using System;

class Precise

{

public static int i = CCtorHelper();

public static int CCtorHelper()

{

Console.WriteLine("Running inside Precise .cctor");

return 20;

}

static Precise()

{

}

class BeforeFieldInit

{

public static int CCtorHelper()

{

Console.WriteLine("Running inside BeforeFieldInit .cctor");

return 20;

}

public static int i = CCtorHelper();

}

class Test

{

static public void Main()

{

Console.WriteLine("Visible side effect");

Console.WriteLine("Precise.i = {0}" , Precise.i);

Console.WriteLine("BeforeFieldInit.i = {0}", BeforeFieldInit.i);

}

Now lets take a look at the code that gets generated:

G_M398_IG01:

push EDI

push ESI

G_M398_IG02:

mov ECX, gword ptr [01A53270H] 'Visible side effect' // Use WriteLine as a side effect

call System.Console.WriteLine(ref)

mov EDI, gword ptr [01A53274H]

mov ECX, 0x31fb688

call CORINFO_HELP_NEWSFAST

mov ESI, EAX

mov EDX, 1 // call .cctor for precise

mov ECX, 0x30b0d18

call CORINFO_HELP_CCTOR

mov EAX, dword ptr [classVar[0x30b0ef8]]

mov dword ptr [ESI+4], EAX

mov EDX, ESI

mov ECX, EDI

call [System.Console.WriteLine(ref,ref)]

mov EDI, gword ptr [01A53278H] 'BeforeFieldInit.i = {0}'

mov ECX, 0x31fb688

call CORINFO_HELP_NEWSFAST

mov ESI, EAX

mov EAX, dword ptr [classVar[0x30b10e8]]

mov dword ptr [ESI+4], EAX

mov EDX, ESI

mov ECX, EDI

call [System.Console.WriteLine(ref,ref)]

__epilog:

pop ESI

pop EDI

ret

You may notice that there is only one .cctor call, you’ll have to take my word that it is the call for Precise’s .cctor. So where is BeforeFieldInit’s cctor? it’s certainly not in Main(). Is it not going to run? Is this a bug?

Let’s take look at the output

Running inside BeforeFieldInit .cctor

Visible side effect

Running inside Precise .cctor

Precise.i = 20

BeforeFieldInit.i = 20

So it does appear that the .cctor is running, and before the Precise .cctor. The CLR is allowed to run BeforeFieldInit’s cctor anytime before the access to BeforeFieldInit.i, and it’s taking advantage of that by doing it at the time the JIT compiles Main(). This way you don’t spend any instructions (size and size matters here) calling a helper to make sure the .cctor has already run. You could argue that this is a JIT compiler and that we could be generating a new function when the Precise .cctor ran that doesn’t call the helper. While this may be ok in server scenarios (where throughput is all that matters), client scenarios have different requirements (low number of private pages and working set are more important than in server), and Precise .cctor classes are not frequent (you have to go out of your way to get them).

So when should you be using Precise semantics? When do you need precise control of where the .cctor gets run? One example can be a Singleton that protects an expensive resource.

For example

Class Singleton

{

Public Foo f;

Static Singleton()

{

F = new VeryExpensiveObject()

}

Int main()

{

If ( DoStuff() == FAILED )

{

Singleton.f.ReportError();

}

…

}

If going through the fail path is not frequent, getting Singleton’s cctor to run when main() is jitted may be a waste of resources, better be lazy about it, which is what the precise semantics give you.

Now that I’ve told you a bit about how this works, just some final words of my personal experience about .cctors. While I see how they make code easier to write, .cctors, as any code that gets run behind your back (and that does not have an explicit call in your code) can get tricky to debug. Given the nature of beforefieldinit semantics, .cctors can run at any time, so you can find that an unrelated change in your codebase changes the order that .cctors get run, causing subtle bugs. This is specially true in sets classes that have circular dependencies. So while .cctors provide a nice invariant that can avoid some simple bugs, it’s very easy to get into a place where you will have some other subtle bugs, that depend on timing or what other code ran before you, so if you feel that you really need them, keep them simple.

The world is a better place if you generate verifiable IL

davidnotario — Sat, 05 Feb 2005 06:34:00 GMT

If you are writing a compiler that targets IL or just emitting IL, you may find this an interesting read:

The JIT compiler will always try to generate code, even if the IL is bad. From the JIT’s point of view, IL code falls in 3 categories:

1) Verifiable IL. Most of the code can be verifiable. You can look for bugs in it using offline tools (PEVerify)

2) Non verifiable but correct IL

3) Non verifiable and incorrect IL

It is difficult for us to tell the difference between 2) and 3). Whenever we see something that looks really wrong (Bad EH regions, IL stack imbalances, etc..) we bail out throwing an InvalidProgramException. The JIT could probably do a better job here by being stricter on things that are obviously wrong, but even if we did, bad IL will get through (we can’t prove it’s incorrect for all cases). Historically, it’s difficult to be in the 2) camp and be in shape, I’ve had to debug many more bugs coming from people that generate unverifiable code than from let’s say , the C# compiler, which generates verifiable code (unless you use the unsafe keyword).

The problem is that It is very easy to go from 2) to 3). What can happen when you are in 3)?

- If you get very lucky, you will immediately crash, with some clear indication of what method had a problem, you will inspect the IL and find the problem.

- If you are less lucky, you will break some CLR invariant and crash in a way you will have a hard time to figure out what went wrong.

- If you are not very lucky, you will have a bug that won’t reproduce deterministically, that will behave different ways in different machines (specially in your customer’s machines), and that will need debugging by somebody with expertise in CLR internals. Trust me, you don’t want to be here.

What’s my recommendation? It’s very likely that 100% of your code could be in camp 1). If not, try to get close to 100%. Once you’re there separate verifiable from non verifiable code into different assemblies and make sure the code you think is verifiable passes verification. You can do this by saving your assembly to disk and running PEVerify on it, or refusing the SkipVerification permission, in which case the JIT compiler will verify your methods as you go and will throw a nice an easy to debug exception when it encounters code that isn’t verifiable.

Some examples on how easy and how non obvious it is to get into really hard to debug problems. The CLR Garbage Collector performs all its work on a huge dynamic graph of objects. To put it in a simple form, the GC follows pointers from one object to the other in order to obtain the set of objects that are currently alive. For example, if your have a live object of the following class

Class Foo

{

Int i

Foo next

}

To see what other live objects this one is keeping alive, the GC has to walk the object looking for pointers. To do that, the GC obtains the layout of the object and follows the managed pointers, in this case, ‘next’.

Now, what happens if next is a bad pointer? Most likely you will crash. Why? To figure out what objects is ‘next’ keeping alive, we need to know what type ‘next’ is. The GC figures this out by looking at ‘next’s method table, which is, let’s say, in *( (MethodTable**) next). Now, if next is a bad pointer, you are basically reading from a random location, if you get lucky you will crash right there, if not more dangerous things can happen: heap corruption, type safety violation, etc… This class of bug is what we call here a ‘GC Hole’

How can you get into this mess by emitting IL? Let’s write a silly example (at the right of each instruction I show what’s in the IL stack after the instruction executes) that will show you that not even inspecting the x86 assembly we generate from your IL will be enough proof that you are correct (obviously, for simplicity, this is not real code. Real code can be more subtle and more innocent looking). Let’s hope it can convince you to only generate verifiable code ;)

newobj instance void Test::.ctor() ( Test*)

castclass Test ( Test*)

ret ()

This would generate the following code:

IN0001: mov ECX, 0x2cb0dd4 (GC regs: - )

IN0002: call CORINFO_HELP_NEWSFAST (GC regs: EAX)

IN0003: mov EDX, EAX (GC regs: EDX)

IN0004: mov ECX, 0x2cb0dd4 (GC regs: EDX)

IN0005: call CORINFO_HELP_CHKCASTCLASS (GC regs: EAX)

This code is fine. Note the ‘GC regs’ on each line The JIT emits GC information, which tells the GC what registers hold GC pointers for every instruction (again, we don’t do this always, but let’s say we do for this discussion), so basically, after instructions 2,3 and 4 EDX is holding a GC reference (the Test object we just created) and after instruction 5 the result of the case is in EAX

Now, let’s introduce a small change (that will make our code unverifiable)

newobj instance void Test::.ctor()( Test*)

conv.i ( I )

castclass Test ( Test*)

ret ()

This yields the following code:

IN0001: mov ECX, 0x2cb0dd4 (GC regs: - )

IN0002: call CORINFO_HELP_NEWSFAST (GC regs: EAX)

IN0003: mov EDX, EAX (GC regs: None!)

IN0004: mov ECX, 0x2cb0dd4 (GC regs: None!)

IN0005: call CORINFO_HELP_CHKCASTCLASS (GC regs: EAX)

What is the difference? The generated code is exactly the same, but the GC info isn’t. In our second version, after the 3rd instruction EAX is no longer a GC pointer, because conv.i told us to treat it as a pointer size integer. What does this mean to us? It means that if a GC happens while in instructions 3 and 4, the GC won’t think there is an object in EDX, which is the only reference we have to our newly create object, so it will reclaim it’s memory back and possibly reuse it. So now the GC returns control to the code and we call into CORINFO_HELP_CHKCASTCLASS with a bogus pointer, oops! And what’s worse, you will only see the bug if a GC happens in that 2 instruction window. Combine the above with multithread code (let’s say you are running in ASP.NET) and you have the perfect GC hole, which will cause unexplained crashes that you’ll just see in production machines or while demoing your product.

[Edit: Fixed fonts and a typo]

Jit Optimizations: Inlining (II)

davidnotario — Mon, 01 Nov 2004 09:31:00 GMT

In a JIT compiler, inlining can become an expensive optimization (compile time wise): it can involve loading other classes or assemblies, doing security checks, etc... What's worse, even after doing all this expensive work, you may find out that the candidate for inlining wasn't really worth it, so you just have to throw away work you did, wasting not only time, but possibly affecting other things like working set (you loaded a class that you didn't really need).

Another reason the JIT has to be smart about what it inlines is due to how the JIT compiler works. To give you an idea, consider a function f() that will generate an optimal solution for a problem in O(N^2) steps and a function g() that solves the same problem, but not optimally, in O(N) steps. If you have a limited time to find the solution, a good approach could be doing the optimal solution for small Ns, and fallback to the non optimal, but fast, solver for larger Ns. In our case, the problem is generating good code, and N is a measure of the complexity of our input (code size, complexity of flowgraph, number of variables etc...). What does this have to do with inlining? Well, with inlining you are just making that N number bigger, which can result in us crossing the line (which in practice is not as well defined as in my example) that will make us generate less optimal code. This is a problem the VC team encountered with their IL generation (they do a lot of optimizations at an IL level, among others, inlining and they found out that very aggressive inlining was hurting the quality of the code our JIT generates)

From an engineering point of view, it makes sense to aproach this optimization with a 'Best Bang for your Buck' attitude, which means spending your compile time resources (time and space) and developer resources covering the most common cases where there is benefit inlining, but not a big risk of making things worse. A typical example of a really good candidate for inlining is a property getter/setter. These are usually really small methods that usually just do a memory fetch or store, so it's usually a size and speed win to inline them.

These are some of the reasons for which we won't inline a method:

- Method is marked as not inline with the CompilerServices.MethodImpl attribute.

- Size of inlinee is limited to 32 bytes of IL: This is a heuristic, the rationale behind it is that usually, when you have methods bigger than that, the overhead of the call will not be as significative compared to the work the method does. Of course, as a heuristic, it fails in some situations. There have been suggestions for us adding an attribute to control these threshold. For Whidbey, that attribute has not been added (it has some very bad properties: it's x86 JIT specific and it's longterm value, as compilers get smarter, is dubious).

- Virtual calls: We don't inline across virtual calls. The reason for not doing this is that we don't know the final target of the call. We could potentially do better here (for example, if 99% of calls end up in the same target, you can generate code that does a check on the method table of the object the virtual call is going to execute on, if it's not the 99% case, you do a call, else you just execute the inlined code), but unlike the J language, most of the calls in the primary languages we support, are not virtual, so we're not forced to be so aggressive about optimizing this case.

- Valuetypes: We have several limitations regarding value types an inlining. We take the blame here, this is a limitation of our JIT, we could do better and we know it. Unfortunately, when stack ranked against other features of Whidbey, getting some statistics on how frequently methods cannot be inlined due to this reason and considering the cost of making this area of the JIT significantly better, we decided that it made more sense for our customers to spend our time working in other optimizations or CLR features. Whidbey is better than previous versions in one case: value types that only have a pointer size int as a member, this was (relatively) not expensive to make better, and helped a lot in common value types such as pointer wrappers (IntPtr, etc).

- MarshalByRef: Call targets that are in MarshalByRef classes won't be inlined (call has to be intercepted and dispatched). We've got better in Whidbey for this scenario

- VM restrictions: These are mostly security, the JIT must ask the VM for permission to inline a method (see CEEInfo::canInline in Rotor source to get an idea of what kind of things the VM checks for).

- Complicated flowgraph: We don't inline loops, methods with exception handling regions, etc...

- If basic block that has the call is deemed as it won't execute frequently (for example, a basic block that has a throw, or a static class constructor), inlining is much less aggressive (as the only real win we can make is code size)

- Other: Exotic IL instructions, security checks that need a method frame, etc...

Jit Optimizations: Inlining (I)

davidnotario — Thu, 28 Oct 2004 11:11:00 GMT

Inlining is an optimization that can happen when you have calls. The optimization consists in substituting the call for the code of the callee.

What do you gain from doing this?

- Speed: Reduce the overhead of the call. To get to the actual instructions that do the work, we have to do the following:

- Setup arguments (in registers and the stack).
- Do the actual call instruction.
- Callee prolog
- Callee work
- Callee epilog
- return

- Size: sometimes you get smaller code, as there are situations (for example property getters) that the number of instructions generated when inlining is less than the number of instructions that is generated to do an actual call.

- Expose more optimizations (speed and size wise), as now the code you got from substituting the call can be the target of other
optimizations.

Let's take a look at a simple example:

class Test
{
static int   And(int i1, int i2)
{
  return i1 & i2;
}

static int i;
static public void Main()
{
  i = And(i, 0);
}
}

If we don't inline And(), the generated code for Main() is:

        mov     ECX, dword ptr [classVar[0x2ca0d24]] ; setup first argument (i)
      xor     EDX, EDX                             ; setup second argument (0)
      call    [Test.And(int,int):int]              ; do the call
      mov     dword ptr [classVar[0x2ca0d24]], EAX ; assign result to static
      ret                                          ; return

And the code for And() is

        and     ECX, EDX                             ; perform AND
      mov     EAX, ECX                             ; setup return register
      ret                                          ; return to caller

Note that And() is really simple, and we're not paying anything for prolog and epilog. Even with this, we still get a win when inlining:

Main() with inlining turned on:

      xor     EAX, EAX                              ; generate final result ;)
      mov     dword ptr [classVar[0x2ca0d24]], EAX ; move result to static
      ret                                           ; return

For this really simple code we got the following from the inlining optimization

- Went from 8 to 3 instructions for the codepath we have to execute, we'll be much faster.
- For Main(), we went from 20 bytes to 8 bytes of code.

Note how inlining enabled a further optimization, when we inlined we could actually fold away the And operation, as the JIT realized one side of the AND was a 0.

In my next post, I will go into more details of our current implementation, such as some limitations we currently have.

PS: I'm saving all the questions you guys are asking, I'll eventually answer (I hope most of them) on the blog, just have a bit of patience, we're really busy right now trying to get Beta 2 done!.

The CLR x86 JIT, an overview

davidnotario — Tue, 26 Oct 2004 11:33:00 GMT

I'll be doing a series of articles on how the x86 CLR JIT compiler works and the different optimizations it does for you.

Whenever a method is about to be executed for the first time, the VM realizes it doesn't have native code for it, and invokes the JIT to generate it (if you are curious about how it works and have the Rotor source code, go to CallCompileMethodWithSEHWrapper in jitinterface.cpp and backtrack from there).

Although it is designed to be a fast compiler (compiling is happening at runtime, you can't keep the user waiting) and a lot of trade offs, design decisions an heuristics are in place to make this happen, the JIT really doesn't look that different from a 'normal' compiler (where 'normal' means as per the Dragon Book ;)

Work is roughly divided into the following stages:

1. Importing: In this stage, the input IL is transformed to the JIT's internal representation. Verification (if required) also happens here. A lot of communication happens back and forth between the JIT and the VM, as the JIT has to ask a lot of questions to the VM (for example, if the IL has a LDFLD instruction, it has to make sure the VM loads the class, and ask how it can access the field, directly? need a helper (for eg MarshalByRef classes), what helper?).

2. Morphing: Here, the compiler just applies a transformations to our internal structures in order to simplify or optimize the code. Things that happen here include detecting JIT intrinsics (functions the JIT has special knowledge about, such as Math.Sin(), which will end up being an fsin x86 instruction), constant propagation, inlining, etc...

3. Flowgraph analysis: The JIT performs a traditional flowgraph analysis, to determine the liveness of variables and gen/kill sets, dominator information, loop detection, etc.... This information is used in all subsequent stages.

4. Optimization phase: In this stage, the heavyweight optimizations happen: Common Subexpression and Range Check Elimination, loop hoisting, etc...

5. Register allocation: Registers are a scarce resource on x86. Operations performed on registers are generally faster than those done on memory, hence its important to make variables live in registers. This stage tries to select the most optimal placement for each variable. To do this it takes in account the number of uses of each variable in the method. It also makes a decision about what type of frame the method will use (EBP or ESP based, will the frame be double aligned), etc...

6. Code generation: We finally get to generate the x86 code. Here we take in account what processor we're generating code for, in order to choose the best idioms for it. GC and debugger information is also recorded here.

7. Emit: This is the last stage, at this point we have the x86 code the method will use and information to generate the GC and debugger information. Everything will be packaged together and returned to the VM, which will then redirect control to the newly generated code.

Introduction

davidnotario — Mon, 25 Oct 2004 20:07:00 GMT

Hi, my name's David Notario. I'm a developer in the x86 JIT compiler team in the CLR, although we've been doing a lot of other stuff there lately.

A bit about myself: I was born and raised in the sunny Canary Islands, in Spain, went to university in Madrid, Spain, spent some years in the video game industry and then decided it would be a good idea to see what working on big software was. This week it will be 4 years since I came to the US.

I used to write in a non msdn blog before (http://www.xplsv.com/blogs/devdiary/), but haven't been very active lately on it and want to speak more about the CLR (the other one is more focused on my side projects), so that's how I ended up here.

Feel free to ask questions or suggestions for my posts. I'll answer what I can and I will try get somebody to answer what I can't.

Topics that I will be visiting will be:

JIT Compiler: Optimizations, tips, etc... One of the main reasons I want to have this blog is to answer a lot of FAQs about the JIT, so instead of resending e-mails, I can just point them to this site.
Miscellaneous CLR stuff: statics, string interning, ngen, gc, etc...
Experiences with .NET with my side projects
Floating point.
Graphics.
Other stuff