cbrumme's WebLog

A quick update on me.

cbrumme — Sat, 16 Sep 2006 01:55:00 GMT

It’s been over two years since I blogged. Although I remain happily (perhaps even ecstatically) working at Microsoft, I left the CLR team and the Developer Division about a year ago. I’m now on an incubation team, exploring evolution and revolution in operating systems. This is a fascinating area that includes devices, concurrency, scheduling, security, distribution, application model, programming model and even some aspects of user interaction (where I am totally out of my depth). And, as you might expect with my background, our effort also includes managed programming.

Anyway, this blog will remain available indefinitely. It continues to be useful for certain technical details which are unavailable elsewhere.

In the meantime, if any readers are interested in working on a deep systems incubation with me and a team of truly outstanding developers, please send me email (cbrumme). We are holding to some very high standards for this effort in terms of insight, experience and hard work. But if you are like me, I am confident you will find it a dream opportunity.

Updated Finalization and Hosting

cbrumme — Tue, 27 Apr 2004 00:58:00 GMT

My original posts on Finalization and Hosting had some hokey XXXXX markers in place of content, where that content hadn't already been disclosed in some form. Now that the Visual Studio 2005 Community Preview is available, I've gone back to those two posts and replaced the XXXXX markers with real text.

Also, it's obviously been a while since my last post. I started writing something this weekend, but the weather here has been spectacular and I was compelled to go outside and play. I'll try to have something in the next couple of weeks.

Hosting

cbrumme — Sat, 21 Feb 2004 18:28:00 GMT

Hosting

My prior three blogs were supposed to be on Hosting. Each time I got side tracked, first on Exceptions, then on Application Compatibility and finally on Finalization. I refuse to be side tracked this time… much.

Also, I need to explain why it’s taken so long to get this blog out. Part of the reason is vacation. I spent Thanksgiving skiing in Whistler. Then I took a quick side trip to Scottsdale for a friend’s surprise birthday party and to visit my parents. Finally, I spent over three weeks on Maui getting a break from the Seattle winter.

Another reason for the delay is writer’s block. This topic is so huge. The internal specification for the Whidbey Hosting Interfaces is over 100 pages. And that spec only covers the hosting interfaces themselves. There are many other aspects of hosting, like how to configure different security policy in different AppDomains, or how to use COM or managed C++ to stitch together the unmanaged host with the managed applications. There’s no way I can cover the entire landscape.

Anyway, here goes.

Mostly I was tourist overhead at the PDC. But one of the places I tried to pay for my ticket was a panel on Hosting. The other panelists included a couple of Program Managers from the CLR, another CLR architect, representatives from Avalon / Internet Explorer, SQL Server, Visual Studio / Office, and – to my great pleasure – a representative from IBM for DB2.

One thing that was very clear at that panel is that the CLR team has done a poor job of defining what hosting is and how it is done. Depending on your definition, hosting could be:

Mixing unmanaged and managed code in the same process.

Running multiple applications, each in its own specially configured AppDomain.

Using the unmanaged hosting interfaces described in mscoree.idl.

Configuring how the CLR runs in the process, like disabling the concurrent GC through an application config file.

Even though the hosting interfaces described in mscoree.idl are a small part of what could be hosting, I’m going to concentrate on those interfaces.

In V1 and V1.1 of the CLR, we provided some APIs that allowed an unmanaged process host to exercise some limited control over the CLR. This limited control included the ability to select the version of the CLR to load, the ability to create and configure AppDomains from unmanaged code, access to the ThreadPool, and a few other fundamental operations.

Also, we knew we eventually needed to support hosts which manage all the memory in the process and which use non-preemptive scheduling of tasks and perhaps even light-weight fibers rather than OS threads. So we added some rudimentary (and alas inadequate) APIs for fibers and memory control. This invariably happens when you add features that you think you will eventually need, rather than features that someone is actually using and giving feedback on.

If you look closely at the V1 and V1.1 hosting APIs, you really see what we needed to support ASP.NET and a few other scenarios, like ones involving EnterpriseServices, Internet Explorer or VSA, plus some rudimentary guesses at what we might need to coexist properly inside SQL Server.

Obviously in Whidbey we have refined those guesses about SQL Server into hard requirements. And we tried very hard to generalize each extension that we added for SQL Server, so that it would be applicable to many other hosting scenarios. In fact, it’s amazing that the SQL Server team still talks to us – whenever they ask for anything, we always say No and give them something that works a lot better for other hosts and not nearly so well for SQL Server.

In our next release (Whidbey), we’ve made a real effort to clean up the existing hosting support and to dramatically extend it for a number of new scenarios. Therefore I’m not going to spend any more time discussing those original V1 & V1.1 hosting APIs, except to the extent that they are still relevant to the following Whidbey hosting discussion.

Also I’m going to skip over all the general introductory topics like “When to host” since they were the source of my writer’s block. Instead, I’m going to leap into some of the more technically interesting topics. Maybe after we’ve studied various details we can step back and see some general guidelines.

Threading and Synchronization

One of the most interesting challenges we struggled with during Whidbey was the need to cooperate with SQL Server’s task scheduling. SQL Server can operate in either thread mode or fiber mode. Most customers run in thread mode, but SQL Server can deliver its best numbers on machines with lots of CPUs when it’s running in fiber mode. That gap between thread and fiber mode has been closing as the OS addresses issues with its own preemptive scheduler.

A few years ago, I ran some experiments to see how many threads I could create in a single process. Not surprisingly, after almost 2000 threads I ran out of address space in the process. That’s because the default stack size on NT is 1 MB and the default user address space is 2 GB. (Starting with V1.1, the CLR can load into LARGEADDRESSAWARE processes and use up to 3 GB of address space). If you shrink the default stack size, you can create more than 2000 threads before hitting the address space limit. I see stack sizes of 256 KB in the SQL Server process on my machine, clearly to reduce this impact on process address space.

Of course, address space isn’t the only limit you can hit. Even on the 4 CPU server box I was experimenting with, the real memory on the system was inadequate for the working set being used. With enough threads, I exceeded real memory and experienced paging. (Okay, it was actually thrashing). But nowadays there are plenty of servers with several GB of real – and real cheap – memory, so this doesn’t have to be an issue.

In my experiments, I simulated server request processing using an artificial work load that combined blocking, allocation, CPU-intensive computation, and a reasonable memory reference set using a mixture of both shared and per-request allocations. In the first experiments, all the threads were ready to run and all of them had equal priority. The result of this was that all threads were scheduled in a round-robin fashion on those 4 CPUs. Since the Windows OS schedules threads preemptively, each thread would execute until it either needed to block or it exceeded its quantum. With hundreds or even thousands of threads, each context switch was extremely painful. That’s because most of the memory used by that thread was so cold in the cache, having been fully displaced by the hundreds of threads that ran before it.

As we all know, modern CPUs are getting faster and faster at raw computation. And they have more and more memory available to them. But access to that memory is getting relatively slower each year. By that, I mean that a single memory access costs the equivalent of an increasing number of instructions. One of the ways the industry tries to mitigate that relative slowdown is through a cache hierarchy. Modern X86 machines have L1, L2 and L3 levels of cache, ordered from fastest and smallest to slowest and largest.

(Other ways we try to mitigate the slowdown is by increasing the locality of our data structures and by pre-fetching. If you are a developer, hopefully you already know about locality. In the unmanaged world, locality is entirely your responsibility. In the managed world, you get some locality benefits from our environment – notably the garbage collector, but also the auto-layout of the class loader. Yet even in managed code, locality remains a major responsibility of each developer).

Unfortunately, context switching between such a high number of threads will largely invalidate all those caches. So I changed my simulated server to be smarter about dispatching requests. Instead of allowing 1000 requests to execute concurrently, I would block 996 of those requests and allow 4 of them to run. This makes life pretty easy for the OS scheduler! There are four CPUs and four runnable threads. It’s pretty obvious which threads should run.

Not only will the OS keep those same four threads executing, it will likely keep them affinitized to the same CPUs. When a thread moves from one CPU to another, the new CPU needs to fill all the levels of cache with data appropriate to the new thread. However, if we can remain affinitized, we can enjoy all the benefits of a warm cache. The OS scheduler attempts to run threads on the CPU that last ran them (soft affinity). But in practice this soft affinity is too soft. Threads tend to migrate between CPUs far more than we would like. When the OS only has 4 runnable threads for its 4 CPUs, the amount of migration seemed to drop dramatically.

Incidentally, Windows also supports hard affinity. If a thread is hard affinitized to a CPU, it either runs on that CPU or it doesn’t run. The CLR can take advantage of this when the GC is executing in its server mode. But you have to be careful not to abuse hard affinity. You certainly don’t want to end up in a situation where all the “ready to run” threads are affinitized to one CPU and all the other CPUs are necessarily stalled.

Also, it’s worth mentioning the impact of hyper-threading or NUMA on affinity. On traditional SMP, our choices were pretty simple. Either our thread ran on its ideal processor, where we are most likely to see all the benefits of a warm cache, or it ran on some other processor. All those other processor choices can be treated as equally bad for performance. But with hyper-threading or NUMA, some of those other CPUs might be better choices than others. In the case of hyper-threading, some logical CPUs are combined into a single physical CPU and so they share access to the same cache memory at some level in the cache hierarchy. For NUMA, the CPUs may be arranged in partitions (e.g. hemispheres on some machines), where each partition has faster access to some memory addresses and slower access to other addresses. In all these cases, there’s some kind of gradient from the very best CPU(s) for a thread to execute on, down to the very worst CPU(s) for that particular thread. The world just keeps getting more interesting.

Anyway, remember that my simulated server combined blocking with other operations. In a real server, that blocking could be due to a web page making a remote call to get rows from a database, or perhaps it could be blocking due to a web service request. If my server request dispatcher only allows 4 requests to be in flight at any time, such blocking will be a scalability killer. I would stall a CPU until my blocked thread is signaled. This would be intolerable.

Many servers address this issue by releasing some multiple of the ideal number of requests simultaneously. If I have 4 CPUs dedicated to my server process, then 4 requests is the ideal number of concurrent requests. If there’s “moderate” blocking during the processing of a typical request, I might find that 8 concurrent requests and 8 threads is a good tradeoff between more context switching and not stalling any CPUs. If I pick too high of a multiple over the number of CPUs, then context switching and cache effects will hurt my performance. If I pick too low a multiple, then blocking will stall a CPU and hurt my performance.

If you look at the heuristics inside the managed ThreadPool, you’ll find that we are constantly monitoring the CPU utilization. If we notice that some CPU resources are being wasted, we may be starving the system by not doing enough work concurrently. When this is detected, we are likely to release more threads from the ThreadPool in order to increase concurrency and make better use of the CPUs. This is a decent heuristic, but it isn’t perfect. For instance, CPU utilization is “backwards looking.” You actually have to stall a CPU before we will notice that more work should be executed concurrently. And by the time we’ve injected extra threads, the stalling situation may already have passed.

The OS has a better solution to this problem. IO Completion Ports have a direct link to the blocking primitives in Win32. When a thread is processing a work item from a completion port, if that thread blocks efficiently through the OS, then the blocking primitive will notify the completion port that it should release another thread. (Busy waiting instead of efficient blocking can therefore have a substantial impact on the amount of concurrency in the process). This feedback mechanism with IO Completion Ports is far more immediate and effective than the CLR’s heuristic based on CPU utilization. But in fairness I should point out that if a managed thread performs managed blocking via any of the managed blocking primitives (contentious Monitor.Enter, WaitHandle.WaitOne/Any/All, Thread.Join, GC.WaitForPendingFinalizers, etc.), then we have a similar feedback mechanism. We just don’t have hooks into the OS, so we cannot track all the blocking operations that occur in unmanaged code.

Of course, in my simulated server I didn’t have to worry about “details” like how to track all OS blocking primitives. Instead, I postulated a closed world where all blocking had to go through APIs exposed by my server. This gave me accurate and immediate information about threads either beginning to block or waking up from a blocking operation. Given this information, I was able to tweak my request dispatcher so it avoided any stalling by injecting new requests as necessary.

Although it’s possible to completely prevent stalling in this manner, it’s not possible to prevent context switches. Consider what happens on a 1 CPU machine. We release exactly one request which executes on one thread. When that thread is about to block, we release a second thread. So far, it’s perfect. But when the first thread resumes from its blocking operation, we now have two threads executing concurrently. Our request dispatcher can “retire” one of those threads as soon as it’s finished its work. But until then we have two threads executing on a single CPU and this will impact performance.

I suppose we could try to get ruthless in this situation, perhaps by suspending one of the threads or reducing its priority. In practice, it’s never a good idea to suspend an executing thread. If that thread holds any locks that are required by other concurrent execution, we may have triggered a deadlock. Reducing the priority might help and I suspect I played around with that technique. To be honest, I can’t remember that far back.

We’ll see that SQL Server can even solve this context switching problem.

Oh yeah, SQL Server

So what does any of this have to do with SQL Server?

Not surprisingly, the folks who built SQL Server know infinitely more than me about how to get the best performance out of a server. And when the CLR is inside SQL Server, it must conform to their efficient design. Let’s look at their thread mode, first. Fiber mode is really just a refinement over this.

Incoming requests are carried on threads. SQL Server handles a lot of simultaneous requests, so there are a lot of threads in the process. With normal OS “free for all” scheduling, this would result in way too many context switches, as we have seen. So instead those threads are affinitized to a host scheduler / CPU combination. The scheduler tries to ensure that there is one unblocked thread available at any time. All the other threads are ideally blocked. This gives us the nirvana of 100% busy CPUs and minimal context switches. To achieve this nirvana, all the blocking primitives need to cooperate with the schedulers. Even if an event has been signaled and a thread is considered by the application to be “ready to run”, the scheduler may not choose to release it, if the scheduler’s corresponding CPU is already executing another thread. In this manner, the blocking primitive and the scheduler are tightly integrated.

When I built my simulated server, I was able to achieve an ideal “closed world” where all the synchronization primitives were controlled by me. SQL Server attempts the same thing. If a thread needs to block waiting for a data page to be read, or for a page or row latch to be released, that blocking occurs through the SQL Server scheduler. This guarantees that exactly one thread is available to run on each CPU, as we’ve seen.

Of course, execution of managed code also hits various blocking points. Monitor.Enter (‘lock’ in C# and ‘SyncLock’ in VB.NET) is a typical case. Other cases include waiting for a GC to complete, waiting for class construction or assembly loading or type loading to occur, waiting for a method to be JITted, or waiting for a remote call or web service to return. For SQL Server to hit their performance goals and to avoid deadlocks, the CLR must route all of these blocking primitives to SQL Server (or any other similar host) through the new Whidbey hosting APIs.

Leaving the Closed World

But what about synchronization primitives that are used for coordination with unmanaged code and which have precise semantics that SQL Server cannot hope to duplicate? For example, WaitHandle and its subtypes (like Mutex, AutoResetEvent and ManualResetEvent) are thin wrappers over the various OS waitable handles. These primitives provide atomicity guarantees when you perform a WaitAll operation on them. They have special behavior related to message pumping. And they can be used to coordinate activity across multiple processes, in the case of named primitives. It’s unrealistic to route operations on WaitHandle through the hosting APIs to some equivalent host-provided replacements.

This issue with WaitHandle is part of a more general problem. What happens if I PInvoke from managed code to an OS service like CoInitialize or LoadLibrary or CryptEncrypt? Do those OS services block? Well, I know that LoadLibrary will have to take the OS loader lock somewhere. I could imagine that CoInitialize might need to synchronize something, but I have no real idea. One thing I am sure of: if any blocking happens, it isn’t going to go through SQL Server’s blocking primitives and coordinate with their host scheduler. The idealized closed world that SQL Server needs has just been lost.

The solution here is to alert the host whenever a thread “leaves the runtime”. In other words, if we are PInvoking out, or making a COM call, or the thread is otherwise transitioning out to some unknown unmanaged execution, we tell the host that this is happening. If the host is tracking threads as closely as SQL Server does, it can use this event to disassociate the thread from the host scheduler and release a new thread. This ensures that the CPU stays busy. That’s because even if the disassociated thread blocks, we’ve released another thread. This newly released thread is still inside our closed world, so it will notify before it blocks so we can guarantee that the CPU won’t stall.

Wait a second. The CLR did a ton of work to re-route most of its blocking operations through the host. But we could have saved almost that entire ton of engineering effort if we had just detached the thread from the host whenever SQL Server called into managed code. That way, we could freely block and we wouldn’t disrupt the host’s scheduling decisions.

This is true, but it won’t perform as well as the alternative. Whenever a thread disassociates from a host scheduler, another thread must be released. This guarantees that the CPU is busy, but it has sacrificed our nirvana of only having a single runnable thread per CPU. Now we’ve got two runnable threads for this CPU and the OS will be preemptively context-switching between them as they run out of quantum.

If a significant amount of the processing inside a host is performed through managed code, this would have a serious impact on performance.

Indeed, if a significant amount of the processing inside a host is performed in unmanaged code, called via PInvokes or COM calls or other mechanisms that “leave the runtime”, this too can have a serious impact on performance. But, for practical purposes, we expect most execution to remain inside the host or inside managed code. The amount of processing that happens in arbitrary unmanaged code should be low, especially over time as our managed platform grows to fill in some of the current gaps.

Of course, some PInvokes or COM calls might be to services that were exported from the host. We certainly don’t want to disassociate from the host scheduler every time the in-process ADO provider performs a PInvoke back to SQL Server to get some data. This would be unnecessary and expensive. So there’s a way for the host to control which PInvoke targets perform a “leave runtime” / “return to runtime” pair and which ones are considered to remain within the closed world of our integrated host + runtime.

Even if we were willing to tolerate the substantial performance impact of considering all of the CLR to be outside the host’s closed world (i.e. we disassociated from the host’s scheduler whenever we ran managed code), this approach would be inadequate when running in fiber mode. That’s because of the nasty effects which thread affinity can have on a fiber-based system.

Fiber Mode

As we’ve seen, SQL Server and other “extreme” hosts can ensure that at any time each CPU has only a single thread within the closed world that is ready to run. But when SQL Server is in thread mode, there are still a large number of threads that aren’t ready to run. It turns out that all those blocked threads impose a modest cost upon the OS preemptive scheduler. And that cost becomes an increasing consideration as the number of CPUs increases. For 1, 2, 4 and probably 8 CPU machines, fiber mode isn’t worth the headaches we’re about to discuss. But by the time you get to a larger machine, you might achieve something like a 20% throughput boost by switching to fiber mode. (I haven’t seen real numbers in a year or two, so please take that 20% as a vague ballpark).

Fiber mode simply eliminates all those extra threads from any consideration by the OS. If you stay within the idealized nirvana (i.e. you don’t perform a “leave runtime” operation), there is only one thread for each host scheduler / CPU. Of course, there are many stacks / register contexts and each such stack / register context corresponds to an in-flight request. When a stack is ready to run, the single thread switches away from whatever stack it was running and switches to the new stack. But from the perspective of the OS scheduler, it just keeps running the only thread it knows about.

So in both thread mode and fiber mode, SQL Server uses non-preemptive host scheduling of these tasks. This scheduling happens in user mode, which is a distinct advantage over the OS preemptive scheduling which happens in kernel mode. The only difference is whether the OS scheduler is aware of all the tasks on the host scheduler, or whether they all look like a single combined thread – albeit with different stacks and register contexts.

But the impact of this difference is significant. First, it means that there is an M:N relationship between stacks (logical CLR threads) and OS threads. This is M:N because multiple stacks will execute on a single thread, and because the specially nominated thread that carries those stacks can change over time. This change in the nominated thread occurs as a consequence of those “leave runtime” calls. Remember that when a thread leaves the runtime, we inform the host which disassociates the thread from the host scheduler. A new thread is then created or obtained from a short list of already-created threads. This new thread then picks up the next stack that is ready to run. The effect is that this stack has migrated from the original disassociated thread to the newly nominated thread.

This M:N relationship between stacks and OS threads causes problems everywhere that thread affinity would normally occur. I’ve already mentioned CPU affinity when discussing how threads are associated with CPUs. But now I’m talking about a different kind of affinity. Thread affinity is the association between various programmatic operations and the thread that these operations must run on. For example, if you take an OS critical section by calling EnterCriticalSection, the resulting ownership is tied to your thread. Sometimes developers say that the OS critical section is scoped to your thread. You must call LeaveCriticalSection from that same thread.

None of this is going to work properly if your logical thread is asynchronously and apparently randomly migrating between different physical threads. You’ll successfully take the critical section on one logical thread. If you attempt to recursively acquire this critical section, you will deadlock if a migration has intervened. That’s because it will look like a different physical thread is actually the owner.

Imagine writing some hypothetical code inside the CLR:

EnterCriticalSection(pCS);

If (pGlobalBlock == NULL)

pGlobalBlock = Alloc(count);

LeaveCriticalSection(pCS);

Obviously any real CLR code would be full of error handling, including a ‘finally’ clause to release the lock. And we don’t use OS critical sections directly since we typically reflect them to an interested host as we’ve discussed. And we instrument a lot of this stuff, including spinning during lock acquisition. And we wrap the locks with lots of logic to avoid deadlocks, including GC-induced deadlocks. But let’s ignore all of the goop that would be necessary for real CLR code.

It turns out that the above code has a thread affinity problem. Even though SQL Server’s fiber scheduling is non-preemptive, scheduling decisions can still occur whenever we call into the host. For reasons that I’ll explain later, all memory allocations in the CLR have the potential to call into the host and result in scheduling. Obviously most allocations will be satisfied locally in the CLR without escalation to the host. And most escalations to the host still won’t cause a scheduling decision to occur. But from a correctness perspective, all allocations have the potential to cause scheduling.

Other places where thread affinity can bite us include:

The OS Mutex and the managed System.Threading.Mutex wrapper.

LoadLibrary and DllMain interactions. As I’ve explained in my blog entry on Shutdown, DllMain notifications occur on a thread which holds the OS loader lock.

TLS (thread local storage). It’s worth mentioning that, starting with Windows Server 2003, there are new FLS (fiber local storage) APIs. These APIs allow you to associate state with the logical rather than the physical thread. When a fiber is associated with a thread for execution (SwitchToFiber), the FLS is automatically moved from the fiber onto the thread. For managed TLS, we now move this automatically. But we cannot do this unconditionally for all the unmanaged TLS.

Thread culture or locale, the impersonation context or user identity, the COM+ transaction context, etc. In some sense, these are just special cases of thread local storage. However, for historical reasons it isn’t possible to solve these problems by moving them to FLS.

Taking control of a thread for GC, Abort, etc. via the OS SuspendThread() service.

Any use of ThreadId or Thread Handle. This includes all debugging.

“Hand-rolled” locks that we cannot discover or reason about, and which you have inadvertently based on the physical OS thread rather than the logical thread or fiber.

Various PInvokes or COM calls that might end up in unmanaged code with affinity requirements. For instance, MSHTML can only be called on STA threads which are necessarily affinitized. Of course, there is no list of all the APIs that have odd threading behavior. It’s a minefield out there.

Solving affinity issues is relatively simple. The hard part is identifying all the places. Note that the last two bullet items are actually the application’s responsibility to identify. Some application code might appear to execute correctly when logical threads and OS threads are 1:1. But when a host creates an M:N relationship, any latent application bugs will be exposed.

In many cases, the easiest solution to a thread affinity issue is to disassociate the thread from the host’s scheduler until the affinity is no longer required. The hosting APIs provide for this, and we’ve taken care of it for you in many places – like System.Threading.Mutex.

Before we finish our discussion of locking, there’s one more aspect worth mentioning. In an earlier blog, I have mentioned the limited deadlock detection and deadlock breaking which the CLR performs when executing class constructors or JITting.

Except for this limited case, the CLR doesn’t concern itself with application-level deadlocks. If you write some managed code that takes a set of locks in random order, resulting in a potential deadlock, we consider that to be your application bug. But some hosts may be more helpful. Indeed, SQL Server has traditionally detected deadlocks in all data accesses. When a deadlock occurs, SQL Server selects a victim and aborts the corresponding transaction. This allows the other requests implicated in the deadlock to proceed.

With the new Whidbey hosting APIs, it’s possible for the host to walk all contentious managed locks and obtain a graph of the participants. This support extends to locking through our Monitor and our ReaderWriterLock. Clearly, an application could perform locking through other means. For example, an AutoResetEvent can be used to simulate mutual exclusion. But it’s not possible for such locks to be included in the deadlock algorithms, since there isn’t a strong notion of lock ownership that we can use.

Once the host has selected a deadlock victim, it must cause that victim to abort its forward progress somehow. If the victim is executing managed code, some obvious ways to do this include failing the lock attempt (since the thread is necessarily blocking), aborting the thread, or even unloading the AppDomain. We’ll return to the implications of this choice in the section on Reliability below.

Finally, it’s interesting to consider how one might get even better performance than what SQL Server has achieved. We’ve seen how fiber mode eliminates all the extra threads, by multiplexing a number of stacks / register contexts onto a single thread. What happens if we then eliminate all those fibers? For a dedicated server, we can achieve even better performance by forcing all application code to maintain its state outside of a thread’s stack. This allows us to use a single thread per CPU which executes user requests by processing them on its single dedicated stack. All synchronous blocking is eliminated by relying on asynchronous operations. The thread never yields while holding its stack pinned. The amount of memory required to hold an in-flight request will be far less than a 256 KB stack reservation. And the cost of processing an asynchronous completion through polling can presumably be less than the cost of a fiber context switch.

If all you care about is performance, this is an excellent way to build a server. But if you need to accommodate 3^rd party applications inside the server, this approach is questionable. Most developers have a difficult time breaking their logic into segments which can be separately scheduled with no stack dependencies. It’s a tedious programming model. Also, the underlying Windows platform still contains a lot of blocking operations that don’t have asynchronous variants available. WMI is one example.

Memory Management

Servers must not page.

Like all rules, this one isn’t strictly true. It is actually okay to page briefly now and then, when the work load transitions from one steady state to another. But if you have a server that is routinely paging, then you have driven that server beyond its capacity. You need to reduce the load on the server or increase the server’s memory capacity.

At the same time, it’s important to make effective use of the memory capacity of a server. Ideally, a database would store the entire database contents in memory. This would allow it to avoid touching the disk, except to write the durable log that protects it from data loss and inconsistency in the face of catastrophic failure. Of course, the 2 or 3 GB limit of Win32 is far too restrictive for most interesting databases. (SQL Server can use AWE to escape this limit, at some cost). And even the address limits of Win64 are likely to be exceeded by databases presently. That’s because Win64 does not give you a full 64 bits of addressing and databases are already heading into the petabytes.

So a database needs to consider all the competing demands for memory and make wise decisions about which ones to satisfy. Historically, those demands have included the buffer cache which contains data pages, compiled query plans, and all those thread stacks. When the CLR is loaded into the process, significant additional memory is required for the GC heap, application code, and the CLR itself. I’m not sure what techniques SQL Server uses to trade off the competing demands for memory. Some servers carve memory up based on fixed ratios for the different broad uses, and then rely on LRU within each memory chunk. Other servers assign a cost to each memory type, which indicates how expensive it would be to regenerate that memory. For example, in the case of a data page, that cost is an IO.

Some servers use elaborate throttling of inbound requests, to keep the memory load reasonable. This is relatively easy to do when all requests are comparable in terms of their memory and CPU requirements. But if some queries access a single database page and other queries touch millions of rows, it would be hard to factor this into a throttling decision that is so far upstream from the query processor. Instead, SQL Server tends to accept a large number of incoming requests and process them “concurrently.” We’ve already seen in great detail why this concurrent execution doesn’t actually result in preemptive context switching between all the corresponding tasks. But it is still the case that each request will hold onto some reference set of memory, even when the host’s non-preemptive scheduler has that request blocked.

If enough requests are blocked while holding onto significant unshared memory, then the server process may find itself over-committed on memory. At this point, it could page – which hurts performance. Or it could kill some of the requests and free up the resources they are holding onto. This is an unfortunate situation, because we’ve presumably already devoted resources like the CPU to get the request to its current state of partial completion. If we throw away the request, all that work was wasted. And the client is likely to resubmit the request, so we will have to repeat all that work soon.

Nevertheless, if the server is over-committed and it’s not practical to recover more memory by e.g. shrinking the number of pages devoted to the buffer cache, then killing in-flight requests is a sound strategy. This is particularly reasonable in database scenarios, since the transactional nature of database operations means that we can kill requests at any time and with impunity.

Unfortunately, the world of arbitrary managed execution has no transactional foundation we can rely on. We’ll pick up this issue again below, in the section on Reliability.

It should be obvious that, if SQL Server or any other host is going to make wise decisions about memory consumption on a “whole process” basis, that host needs to know exactly how much memory is being used and for what purposes. For example, before the host unloads an AppDomain as a way of backing out of an over-committed situation, the host needs some idea of how many megabytes this unload operation is likely to deliver.

In the reverse direction, the host needs to be able to masquerade as the operating system. For instance, the CLR’s GC monitors system memory load and uses this information in its heuristics for deciding when to schedule a collection. The host needs a way to influence these collection decisions.

SQL Server and ASP.NET

Clearly a lot of work went into threading, synchronization and memory management in SQL Server. One obvious question to ask is how ASP.NET compares. They are both server products from Microsoft and they both execute managed code. Why didn’t we need to add all this support to the hosting interfaces in V1 of the CLR, so we could support ASP.NET?

I think it’s fair to say that ASP.NET took a much simpler approach to the problem of building a scalable server. To achieve efficient threading, they rely on the managed ThreadPool’s heuristics to keep the CPUs busy without driving up too many context switches. And since the bulk of memory allocations are due to the application, rather than the ASP.NET infrastructure (in other words, they aren’t managing large shared buffer pools for data pages), it’s not really possible for ASP.NET to act as a broker for all the different memory consumers. Instead, they just monitor the total memory load, and recycle the worker process if a threshold is exceeded.

(Incidentally, V1 of ASP.NET and the CLR had an unfortunate bug with the selection of this threshold. The default point at which ASP.NET would recycle the process was actually a lower memory load than the point at which the CLR’s GC would switch to a more aggressive schedule of collections. So we were actually killing the worker process before the CLR had a chance to deliver more memory back to the application. Presumably in Whidbey this selection of default thresholds is now coordinated between the two systems.)

How can ASP.NET get away with this simpler approach?

It really comes down to their fundamental goals. ASP.NET can scale out, rather than having to scale up. If you have more incoming web traffic, you can generally throw more web servers at the problem and load balance between them. Whereas SQL Server can only scale out if the data supports this. In some cases, it does. There may be a natural partitioning of the data, like access to the HotMail mailbox for a particular incoming user. But in too many other cases, the data cannot be sufficiently partitioned and the server must be scaled up. On X86 Windows, the practical limit is a 32-way CPU with a hard limit of 3 GB of user address space. If you want to keep increasing your work load on a single box, you need to use every imaginative trick – like fibers or AWE – to eke out all possible performance.

There’s also an availability issue. ASP.NET can recycle worker processes quite quickly. And if they have scaled out, recycling a worker process on one of the computers in the set will have no visible effect on the availability of the set of servers. But SQL Server may be limited to a single precious process. If that process must be recycled, the server is unavailable. And recycling a database is more expensive than recycling a stateless ASP.NET worker process, because transaction logs must be replayed to move the database forwards or backwards to a consistent state.

The short answer is, ASP.NET didn’t have to do all the high tech fancy performance work. Whereas SQL Server was forced down this path by the nature of the product they must build.

Reliability

Well, if you haven’t read my earlier blogs on asynchronous exceptions, or if – like me – you read the Reliability blog back in June and don’t remember what it said – you might want to review it quickly at http://blogs.msdn.com/cbrumme/archive/2003/06/23/51482.aspx.

The good news is that we’ve revisited the rules for ThreadAbortException in Whidbey, so that there is now a way to abort a thread without disturbing any backout code that it is currently running. But it’s still the case that asynchronous exceptions can intrude at fairly arbitrary spots in the execution.

Anyway, the availability goals of SQL Server place some rather difficult requirements on the CLR. Sure, we were pretty solid in V1 and V1.1. We ran a ton of stress and – if you avoided stack overflow, running out of memory, and any asynchronous exceptions like Thread.Abort – we could run applications indefinitely. We really were very clean.

One problem with this is that “indefinitely” isn’t long enough for SQL Server. They have a noble goal of chasing 5 9’s and you can’t get there with loose statements like “indefinitely”. Another problem is that we can no longer exclude OutOfMemoryException and ThreadAbortException from our reliability profile. We’ve already seen that SQL Server tries to use 100% of memory, without quite triggering paging. The effect is that SQL Server is always on the brink of being out of memory, so allocation requests are frequently being denied. Along the same lines, if the server is loaded it will allow itself to become over-committed on all resources. One strategy for backing out of an over-commitment is to abort a thread (i.e. kill a transaction) or possibly unload one or more AppDomains.

Despite this stressful abuse, at no time can the process terminate.

The first step to achieve this was to harden the CLR so that it was resilient to any resource failures. Fortunately we have some extremely strong testers. One tester built a system to inject a resource failure in every allocator, for every unique logical call stack. This tests every distinct backout path in the product. This technique can be used for unmanaged and managed (FX) code. That same tester is also chasing any unmanaged leaks by applying the principles of a tracing garbage collector to our unmanaged CLR data structures. This technique has already exposed a small memory leak that we shipped in V1 of the CLR – for the “Hello World” application!

With testers like that, you better have a strong development team too. At this point, I think we’ve annotated the vast majority of our unmanaged CLR methods with reliability contracts. These are a bit like Eiffel pre- and post-conditions and they provide machine-verifiable statements about each method’s behavior with respect to GC, exceptions, and other fundamental operations. These contracts can be used during test coverage (and, in some cases, during static scans of the binary images) to test for conformance.

The bottom line is that the next release of CLR should be substantially more robust in the face of resource errors. Leaving aside stack overflows and focusing entirely on the unmanaged runtime, we are shooting for perfection. Even for stack overflow, we expect to get very, very close. And we have the mechanisms in place that allow us to be rigorous in chasing after these goals.

But what about all of the managed code?

Will FX be as robust as the unmanaged CLR? And how can we possibly hold 3^rd party authors of stored procedures or user defined functions to that same high bar? We want to enable a broad class of developers to write this sort of code, and we cannot expect them to perform many hundreds of hours of stress testing and fault injection on each new stored procedure. If we’re chasing 5 9’s by requiring every external developer to write perfect code, we should just give up now.

Instead, SQL Server relies on something other than perfect code. Consider how SQL Server worked before it started hosting the CLR:

The vast majority of execution inside SQL Server was via Transact SQL or TSQL. Any application written in TSQL is inherently scalable, fiber-aware, and robust in the face of resource errors. Any computation in TSQL can be terminated with a clean transaction abort.

Unfortunately, TSQL isn’t expressive enough to satisfy all application needs. So the remaining applications were written in extended stored procedures or xprocs. These are typically unmanaged C++. Their authors must be extremely sophisticated, because they are responsible for integrating their execution with the unusual threading environment and resource rules that exist inside SQL Server. Throw in the rules for data access and security (which I won’t be discussing in this blog) and it takes superhuman knowledge and skill to develop a bug-free xproc.

In other words, you had a choice of well-behaved execution and limited expression (TSQL), or the choice of arbitrary execution coupled with a very low likelihood that you would get it right (xprocs).

One of the shared goals of the SQL Server and CLR teams in Whidbey was to eliminate the need for xprocs. We wanted to provide a spectrum of choices to managed applications. In Whidbey, that spectrum consists of three buckets for managed code:

Safe

Code in this bucket is the most constrained. In fact, the host constrains it beyond what the CLR would normally allow to code that’s only granted SecurityPermissionFlag.Execution. So this code must be verifiably typesafe and has a reduced grant set. But it is further constrained from defining mutable static fields, from creating or controlling threads, from using the threadpool, etc. The goal here is to guide the code to best practices for scalability and robustness within the SQL Server or similar hosted environments. In the case of SQL Server, this means that all state should be stored in the database and that concurrency is controlled through transactions against the data. However, it’s important to realize that these additional constraints are not part of the Security system and they may well be subvertible. The constraints are simply speedbumps (not roadblocks) which guide the application code away from potentially non-scalable coding techniques and which encourage best practices.

External Access

Code in this bucket should be sufficient for replacing most xprocs. Such code must also be verifiably typesafe, but it is granted some additional permissions. The exact set of permissions is presumably subject to change until Yukon ships, but it’s likely to allow access to the registry, the file system, and the network.

Unsafe

This is the final managed escape hatch for writing code inside SQL Server. This code does not have to be verifiable. It has FullTrust (with the possible exception of UIPermission, which makes no sense within the database). This means that it can do anything the most arbitrary xproc can do. However, it is much more likely to work properly, compared to that xproc. First, it sits on top of a framework that has been designed to work inside the database. Second, the code has all the usual benefits of managed code, like a memory manager that’s based on accurate reachability rather than on programmer correctness. Finally, it is executing on a runtime that understands the host’s special rules for resource management, synchronization, threading, security, etc.

For code in the Safe bucket, you may be wondering how a host could constrain code beyond SecurityPermissionFlag.Execution. There are two techniques available for this:

1) Any assembly in the ‘Safe’ subset could be scanned by a host-provided pre-verifier, to check for any questionable programming constructs like the definition of mutable static fields, or the use of reflection. This raises the obvious question of how the host can interject itself into the binding process and guarantee that only pre-verified assemblies are loaded. The new Whidbey hosting APIs contain a Fusion loader hook mechanism, which allows the host to abstract the notion of an assembly store, without disturbing all our normal loader policy. You can think of this as the natural evolution of the AppDomain.AssemblyResolve event. SQL Server can use this mechanism to place all application assemblies into the database and then deliver them to the loader on demand. In addition to enabling pre-verification, the loader hooks can also be used to ensure that applications inside the database are not inadvertently broken or influenced by changes outside the database (e.g. changes to the GAC). In fact, you could even copy a database from one machine to another and theoretically this could automatically transfer all the assemblies required by that database.

2) The Whidbey hosting APIs provide controls over a new Host Protection Attribute (HPA) feature. Throughout our frameworks, we’ve decorated various unprotected APIs with an appropriate HPA. These HPAs indicate that the decorated API performs a sensitive operation like Synchronization or Thread Control. For instance, use of the ThreadPool isn’t considered a secure operation. (At some level, it is a risk for Denial of Service attacks, but DOS remains an open design topic for our managed platform). If code is running outside of a host that enables these HPAs, they have no effect. Partially trusted code, including code that only has Execution permission, can still call all these APIs. But if a host does enable these attributes, then code with insufficient trust can no longer call these APIs directly. Indirect calls are still permitted, and in this sense the HPA mechanism is similar to the mechanism for LinkDemands.

Although HPAs use a mechanism that is similar to LinkDemands, it’s very important to distinguish the HPA feature – which is all about programming model guidance – from any Security feature. A great way to illustrate this distinction is Monitor.Enter.

Ignoring HPAs, any code can call Monitor.Enter and use this API to synchronize with other threads. Naturally, SQL Server would prefer that most developers targeting their environment (including all the naïve ones) should rely on database locks under transaction control for this sort of thing. Therefore they activate the HPA on this class:

[HostProtection(Synchronization=true, ExternalThreading=true)]

public sealed class Monitor

{

...

[MethodImplAttribute(MethodImplOptions.InternalCall)]

public static extern void Enter(Object obj);

However, devious code in the ‘Safe’ bucket could use a HashTable as an alternate technique for locking. If you create a synchronized HashTable and then perform inserts or lookups, your Object.Equals and GetHashCode methods will be called within the lock that synchronizes the HashTable. The BCL developers were smart enough to realize this, and they added another HPA:

public class Hashtable : IDictionary, ISerializable,

IDeserializationCallback, ICloneable

{

...

[HostProtection(Synchronization=true)]

public static Hashtable Synchronized(Hashtable table) {

if (table==null)

throw new ArgumentNullException("table");

return new SyncHashtable(table);

}

Are there other places inside the frameworks where it’s possible to trick an API into providing synchronization for its caller? Undoubtedly there are, but we aren’t going to perform exhaustive audits of our entire codebase to discover them all. As we find additional APIs, we will decorate them with HPAs, but we make no guarantees here.

This would be an intolerable situation for a Security feature, but it’s perfectly acceptable when we’re just trying to increase the scalability and reliability of naively written database applications.

Escalation Policy

I chose the HPA on System.Threading.Monitor for a reason, in the above example. If you’ve read my earlier blogs on Thread.Abort, you know that it’s dangerous to asynchronously abort another thread. That thread could be executing a class constructor, in which case that class is now unavailable throughout the AppDomain. That thread could be in the middle of an update to some shared application state, which would leave the application in an inconsistent state.

In V1 & V1.1, it was not really possible to write code that is robust in the face of asynchronous exceptions like Abort. In Whidbey, we’re now introducing some constructs (Constrained Execution Regions and Critical Finalization) which make it possible to do this. I’m not going to discuss those constructs in this blog. But suffice it to say that, although it makes it possible to write entirely robust code, it doesn’t make it easy. Without a higher level programmatic construct, like transactions, it’s very difficult to write entirely robust code. You must acquire all the resources required for forward progress, tolerating exceptions during this acquisition phase. Then you enter a forward progress phase, which either cannot fail or which unconditionally triggers some compensating backout code upon failure. If compensation is triggered, it must guarantee that the system is returned to a consistent state before it completes.

If you’ve successfully written that sort of code, you know that it’s an onerous discipline. There’s no way that we can expect the greater population of developers to write large bodies of bug-free code based on this plan.

That’s why, in V1 & V1.1, we recommend either using Abort on the current thread (in which case it is not asynchronous) or we recommend using it in conjunction with an AppDomain.Unload (in which case any inconsistent application state is likely to be discarded).

In Whidbey, it is possible to avoid inducing asynchronous Aborts onto threads that are performing backout (i.e. filter, finally, catch or fault blocks) or that hold locks. Our definition of a lock is pretty broad. It includes execution of a class constructor, since all .cctor execution is synchronized according to elaborate rules by the CLR. It also includes Monitor.Enter, Mutex, ReaderWriterLock, etc. Finally, it includes any “hand-rolled” locks that you build, so long as you properly identify them to us.

Our rationale here is that any thread holding a lock may be updating shared state. If a thread isn’t holding a lock, then any update it performs against shared state must be atomic or at least it never leaves that shared state in an inconsistent state. This is strictly a heuristic, but it’s a pretty good one.

If we believe this heuristic, it means that we can use Abort without consequently unloading an AppDomain, if that thread doesn’t hold any locks and isn’t performing any backout. And it just so happens that the bulk of all managed code executing inside SQL Server is in the ‘Safe’ subset – which coincidentally is highly discouraged via HPAs from taking or holding locks.

In other words, code in the ‘Safe’ subset can almost always take an asynchronous exception without affecting any of the execution on other threads in the same AppDomain. This is the case, even though that code was written by developers who don’t understand the deep issues involved with asynchronous exceptions. It further means that if we should catch such a thread at a point where it isn’t safe to inject an asynchronous exception without also unloading the AppDomain, we can identify this window. Once this window is identified, we can either hold off from injecting the exception until this unsafe window has closed, or we can unload the entire AppDomain to eliminate the application inconsistency. The host can decide whether to hold off on the injection or alternatively to proceed with an AppDomain unload, based on criteria like how resource-constrained the host is.

The hosting APIs for making these decisions imperatively would be rather complicated. So the Whidbey hosting APIs provide a declarative mechanism called an escalation policy. This allows the host to express transitions and timeouts that take effect during error conditions. For instance, SQL Server might state that any attempt to Abort a thread should delay if the victim thread holds a lock. But if that delay exceeds 30 seconds, the Abort attempt should be escalated to an AppDomain.Unload. Of course, the feature is more general than SQL Server’s needs. Indeed, the V1 ASP.NET process recycling feature should now be expressible as a particular Whidbey escalation policy.

Winding down

As usual, I didn’t get around to many of the interesting topics. For instance, those guidelines on when and how to host are noticeably absent. And I didn’t explain how to do any simple stuff, like picking concurrent vs. non-concurrent vs. server GC. The above text is completely free of any specific details of what our hosting APIs look like (partly because they are subject to change until Whidbey ships). And I didn’t touch on any hosting topics outside of the hosting APIs, like all of the AppDomain considerations. As you can imagine, there’s also plenty I could have said about Security. For instance, the hosting APIs allow the host to participate in role-based security and impersonation of Windows identities… Oh well.

Fortunately, one of the PMs involved in the Whidbey hosting effort is apparently writing a book on the general topic of hosting. Presumably all these missing topics will be covered there. And hopefully he won’t run into the same issues with writer’s block that I experienced on this topic.

(Indeed, the event that ultimately resolved my writer’s block was that my wife got the flu. When she’s not around, my weekends are boring enough for me to think about work. The reason I’m posting two blogs this weekend is that Kathryn has gone to Maui for the week and has left me behind.)

Finally, the above blog talks about SQL Server a lot.

Hopefully it’s obvious that the CLR wants to be a great execution environment for a broad set of servers. In V1, we focused on ASP.NET. Based on that effort, we automatically worked well in many other servers with no additional work. For example, EnterpriseServices dropped us into their server processes simply by selecting the server mode of our GC. Nothing else was required to get us running efficiently. (Well, we did a ton of other work in the CLR to support EnterpriseServices. But that work was related to the COM+ programming model and infrastructure, rather than their server architecture. We had to do that work whether we ran in their server process or were instead loading EnterpriseServices into the ASP.NET worker process or some other server).

In Whidbey we focused on extending the CLR to meet SQL Server’s needs. But at every opportunity we generalized SQL Server’s requirements and tried to build something that would be more broadly useful. Just as our ASP.NET work enabled a large number of base server hosting scenarios, we hope that our SQL Server work will enable a large number of advanced server hosting scenarios.

If you have a “commercially significant” hosting problem, whether on the server or the client, and you’re struggling with how to incorporate managed code, I would be interested in hearing from you directly. Feel free to drop me an email with the broad outline of what you are trying to achieve, and I’ll try to get you supported. That support might be something as lame as some suggestions from me on how I would tackle the problem. Or at the other extreme, I could imagine more formal support and conceivably some limited feature work. That other extreme really depends on how commercially significant your product is and on how well our business interests align. Obviously decisions like that are far outside my control, but I can at least hook you up with the right people if this seems like a sensible approach.

Okay, one more ‘Finally’. From time to time readers of my blog send me emails asking if there are jobs available on the CLR team. At this moment, we do. Drop me an email if you are interested. It’s an extremely challenging team to work on, but the problems are truly fascinating.

Finalization

cbrumme — Sat, 21 Feb 2004 06:27:00 GMT

Earlier this week, I wrote an internal email explaining how Finalization works in V1 / V1.1, and how it has been changed for Whidbey. There’s some information here that folks outside of Microsoft might be interested in.

Costs

Finalization is expensive. It has the following costs:

1) Creating a finalizable object is slower, because each such object must be placed on a RegisteredForFinalization queue. In some sense, this is a bit like having an extra pointer-sized field in your object that the system initializes for you. However, our current implementation uses a slower allocator for every finalizable object, and this impact can be measured if you allocate small objects at a high rate.

2) Each GC must do a weak pointer scan of this queue, to find out whether any finalizable objects are now collectible. All such objects are then moved to a ReadyToFinalize queue. The cost here is small.

3) All objects in the ReadyToFinalize queue, and all objects reachable from them, are then marked. This means that an entire graph of objects which would normally die in one generation can be promoted to the next generation, based on a single finalizable root to this graph. Note that the size of this graph is potentially huge.

4) The older generation will be collected at some fraction of the frequency of the younger generation. (The actual ratio depends on your application, of course). So promotion of the graph may have increased the time to live of this graph by some large multiple. For large graphs, the combined impact of this item and #3 above will dominate the total cost of finalization.

5) We currently use a single high priority Finalizer thread to walk the ReadyToFinalize queue. This thread dequeues each object, executes its Finalize method, and proceeds to the next object. This is the one cost of finalization which customers actually expect.

6) Since we dedicate a thread to calling finalizers, we inflict an expense on every managed process. This can be significant in Terminal Server scenarios where the high number of processes multiplies the number of finalizer threads.

7) Since we only use a single thread for finalization, we are inherently non-scalable if a process is allocating finalizable objects at a high rate. One CPU performing finalization might not keep up with 31 other CPUs allocating those finalizable objects.

8) The single finalizer thread is a scarce resource. There are various circumstances where it can become blocked indefinitely. At that point, the process will leak resources at some rate and eventually die. See http://blogs.msdn.com/cbrumme/archive/2004/02/02/66219.aspx for extensive details.

9) Finalization has a conceptual cost to managed developers. In particular, it is difficult to write correct Finalize methods as I shall explain.

Eventually we would like to address #5 thru #8 above by scheduling finalization activity over our ThreadPool threads. We have also toyed with the idea of reducing the impact of #3 and #4 above, by pruning the graph based on reachability from your Finalize method and any code that it might call. Due to indirections that we cannot statically explore, like interface and virtual calls, it’s not clear whether this approach will be fruitful. Also, this approach would cause an observable change in behavior if resurrection occurs. Regardless, you should not expect to see any of these possible changes in our next release.

Reachability

One of the guidelines for finalization is that a Finalize method shouldn’t touch other objects. People sometimes incorrectly assume that this is because those other objects have already been collected. Yet, as I have explained, the entire reachable graph from a finalizable object is promoted.

The real reason for the guideline is to avoid touching objects that may have already been finalized. That’s because finalization is unordered.

So, like most guidelines, this one is made to be broken under certain circumstances. For instance, if your object “contains” a private object that is not itself finalizable, clearly you can refer to it from your own Finalize method without risk.

In fact, a sophisticated developer might even create a cycle between two finalizable objects and coordinate their finalization behavior. Consider a buffer and a file. The Finalize method of the buffer will flush any pending writes. The Finalize method of the file will close the handle. Clearly it’s important for the buffer flush to precede the handle close. One legitimate but brittle solution is to create a cycle of references between the buffer and the file. Whichever Finalize method is called first will execute a protocol between the two objects to ensure that both side effects happen in order. The subsequent Finalize call on the second object should do nothing.

I should point out that Whidbey solves the buffer and file problem differently, relying on the semantics of critical finalization. And I should also point out that any protocol for sequencing the finalization of two objects should anticipate that one day we may execute these two Finalize methods concurrently on two different threads. In other words, the protocol must be thread-safe.

Ordering

This raises the question of why finalization is unordered.

In many cases, no natural order is even possible. Finalizable objects often occur in cycles. You could imagine decorating some references between objects, to indicate the direction in which finalization should proceed. This would add a sorting cost to finalization. It would also cause complexity when these decorated references cross generation boundaries. And in many cases the decorations would not fully eliminate cycles. This is particularly true in component scenarios, where no single developer has sufficient global knowledge to create an ordering:

Your component would achieve its guarantees when tested by you, prior to deployment. Then in some customer application, additional decorated references would create cycles and your guarantees would be lost. This is a recipe for support calls and appcompat issues.

Unordered finalization is substantially faster. Not only do we avoid sorting (which might involve metadata access and marking through intermediate objects), but we can also efficiently manage the RegisteredForFinalization and ReadyToFinalize queues without ever having to memcpy. Finally, there’s value in forcing developers to write Finalize methods with minimal dependencies on any other objects. This is key to our eventual goal of making Finalization scalable by distributing it over multiple threads.

Based on the above and other considerations like engineering complexity, we made a conscious decision that finalization should be unordered.

Partial Trust

There are no security permissions associated with the definition of a Finalize method. As we’ve seen, it’s possible to mount a denial of service attack via finalization. However, many other denial of service attacks are possible from partial trust, so this is uninteresting.

Customers and partners sometimes ask why partially trusted code is allowed to participate in finalization. After all, Finalize methods are typically used to release unmanaged resources. Yet partially trusted code doesn’t have direct access to unmanaged resources. It must always go through an API provided by an assembly with UnmanagedCodePermission or some other effective equivalent to FullTrust.

The reason is that finalization can also be used to control pure managed resources, like object pools or caches. I should point out that techniques based on weak handles can be more efficient than techniques based on finalization. Nevertheless, it’s quite reasonable for partially trusted code to use finalization for pure managed resources.

SQL Server has a set of constraints that they place on partially trusted assemblies that are loaded into their environment. I believe that these constraints prevent definition of static fields (except for initonly and literal static fields), use of synchronization, and the definition of Finalize methods. However, these constraints are not related to security. Rather, they are to improve scalability and reliability of applications by simplifying the threading model and by moving all shared state into the database where it can be transacted.

It’s hard to implement Finalize perfectly

Even when all Finalize methods are authored by fully trusted developers, finalization poses some problems for processes with extreme availability requirements, like SQL Server. In part, this is because it’s difficult to write a completely reliable Finalize method – or a completely reliable anything else.

Here are some of the concerns specifically related to finalization. I’ll explain later how some of these concerns are addressed in the context of a highly available process like SQL Server.

Your Finalize method must tolerate partially constructed instances

It’s possible for partially trusted code to subtype a fully trusted finalizable object (with APTCA) and throw an exception from the constructor. This can be done before chaining to the base class constructor. The result is that a zero-initialized object is registered for finalization.

Even if partially trusted code isn’t intentionally causing finalization of your partially constructed instances, asynchronous problems like StackOverflowException, OutOfMemoryException or AppDomainUnloadException can cause your constructor to be interrupted at a fairly arbitrary location.

Your Finalize method must consider the consequence of failure

It’s possible for partially trusted code to subtype a fully trusted finalizable object (with APTCA) and fail to chain to the base Finalize method. This causes the fully trusted encapsulation of the resource to leak.

Even if partially trusted code isn’t intentionally causing finalization of your object to fail, the aforementioned asynchronous exceptions can cause your Finalize method to be interrupted at a fairly arbitrary location.

In addition, the CLR exposes a GC.SuppressFinalize method which can be used to prevent finalization of any object. Arguably we should have made this a protected method on Object or demanded a permission, to prevent abuse of this method. However, we didn’t want to add a member to Object for such an obscure feature. And we didn’t want to add a demand, since this would have prevented efficient implementation of IDisposable from partial trust.

Your object is callable after Finalization

We’ve already seen how all the objects in a closure can access each other during finalization. Indeed, if any one of those objects re-establishes its reachability from a root (e.g. it places itself into a static field or a handle), then all the other objects it reaches will also become re-established. This is referred to as resurrection. If you have a finalizable object that is publicly exposed, you cannot prevent your object from becoming resurrected. You are at the mercy of all the other objects in the graph.

One possible solution here is to set a flag to indicate that your object has been finalized. You can pepper your entire API with checks to this flag, throwing an ObjectDisposedException if you are subsequently called. Yuck.

Your object is callable during Finalization

It’s true that the finalizer thread is currently single-threaded (though this may well change in the future). And it’s true that the finalizer thread will only process instances that – at some point – were discovered to be unreachable from the application. However, the possibility of resurrection means that your object may become visible to the application before its Finalize method is actually called. This means that application threads and the finalizer thread can simultaneously be active in your object.

If your finalizable object encapsulates a protected resource like an OS handle, you must carefully consider whether you are exposed to threading attacks. Shortly before we shipped V1, we fixed a number of handle recycling attacks that were due to race conditions between the application and Finalization. See http://blogs.msdn.com/cbrumme/archive/2003/04/19/51365.aspx for more details.

Your Finalizer could be called multiple times

Just as there is a GC.SuppressFinalize method, we also expose a GC.ReRegisterForFinalize method. And the same arguments about protected accessibility or security demands apply to the ReRegisterForFinalize method.

Your Finalizer runs in a delicate security context

As I’ve explained in prior blogs, the CLR flows the compressed stack and other security information around async points like ThreadPool.QueueUserWorkItem or Control.BeginInvoke. Indeed, in Whidbey we include more security information by default. However, we do not flow any security information from an object’s constructor to an object’s Finalize method. So (to use an absurd example) if you expose a fully trusted type that accepts a filename string in its constructor and subsequently opens that file in its Finalize method, you have created a security bug.

Clearly it’s hard to write a correct Finalize method. And the managed platform is supposed to make hard things easier. I’ll return to this when I discuss the new Whidbey features of SafeHandles, Critical Finalizers and Constrained Execution Regions.

But what guarantees do I get if I don’t use any of those new gizmos? What happens in a V1 or V1.1 process?

V1 & V1.1 Finalization Guarantees

If you allocate a finalizable object, we guarantee that it will be registered for finalization. Once this has happened, there are several possibilities:

1) As part of the natural sequence of garbage collection and finalization, the finalizer thread dequeues your object and finalizes it.

2) The process can terminate without cooperating with the CLR’s shutdown code. This can happen if you call TerminateProcess or ExitProcess directly. In those cases, the CLR’s first notification of the shutdown is via a DllMain DLL_PROCESS_DETACH notification. It is not safe to call managed code at that time, and we will leak all the finalizers. Of course, the OS will do a fine job of reclaiming all its resources (including abandonment of any cross-process shared resources like Mutexes). But if you needed to flush some buffers to a file, your final writes have been lost.

3) The process can terminate in a manner that cooperates with the CLR’s shutdown code. This includes calling exit() or returning from main() in any unmanaged code built with VC7 or later. It includes System.Environment.Exit(). It includes a shutdown triggered from a managed EXE when all the foreground threads have completed. And it includes shutdown of processes that are CLR-aware, like VisualStudio. In these cases, the CLR attempts to drain both the ReadyToFinalize and the RegisteredForFinalization queues, processing all the finalizable objects.

4) The AppDomain containing your object is unloaded. Prior to Whidbey, the AppDomain will not unload until we have scanned the ReadyToFinalize and the RegisteredForFinalization queues, processing all the finalizable objects that live in the doomed AppDomain.

There are a few points to note here.

· Objects are always finalized in the AppDomain they were created in. A special case exists for any finalizable objects that are agile with respect to AppDomains. To my knowledge, the only such type that exists is System.Threading.Thread.

· I have heard that there is a bug in V1 and V1.1, where we get confused on AppDomain transitions in the ReadyToFinalize queue. The finalization logic attempts to minimize AppDomain transitions by noticing natural partitions in the ReadyToFinalize queue. I’m told there is a bug where we may occasionally skip finalizing the first object of a partition. I don’t believe any customers have noticed this and it is fixed in Whidbey.

· Astute readers will have noticed that during process shutdown and AppDomain unloading we actually finalize objects in the RegisteredForFinalization queue. Such objects are still reachable and would not normally be subject to finalization. Normally a Finalize method can rely on safely accessing finalizable state that is rooted via statics or some other means. You can detect when this is no longer safe by checking AppDomain.IsFinalizingForUnload or Environment.HasShutdownStarted.

· Since there is no ordering of finalization, critical infrastructure is being finalized along with application objects. This means that WaitHandles, remoting infrastructure and even security infrastructure is disappearing underneath you. This is a potential security concern and a definite reliability concern. We have spot-fixed a few cases of this. For example, we never finalize our Thread objects during process shutdown.

· Finalization during process termination will eventually timeout. If a particular Finalize method gets stuck, or if the queue isn’t reducing in size over time (i.e. you create 2 new finalizable instances out of each execution of your Finalize method), we will eventually timeout and terminate the process. The exact timeouts depend on whether a profiler is attached and other details.

· The thread that initiates process shutdown performs the duties of “watchdog.” It is responsible for detecting timeouts during process termination. If this thread is an STA thread, we cause it to pump COM calls in and out of the STA while it blocks as watchdog. If the application has a deadlock that implicates the STA thread while it is executing these unmanaged COM calls, then the timeout mechanism is defeated and the process will hang. This is fixed in Whidbey.

· Subject to all of the above, we guarantee that we will dequeue your object and initiate a call to the Finalize method. We do not guarantee that your Finalize method can be JITted without running out of stack or memory. We do not guarantee that the execution of your Finalize method will complete without being aborted. We do not guarantee that any types you require can be loaded and have their .cctors run. All you get is a “best effort” attempt. We’ll soon see how Whidbey extensions allow you to do better than this and guarantee full execution.

· (If you want to know more about the shutdown of managed processes, see http://blogs.msdn.com/cbrumme/archive/2003/08/20/51504.aspx.)

SafeHandle

Whidbey contains some mechanisms that address many of the V1 and V1.1 issues with finalization. Let’s start with SafeHandle, since it’s the easiest to understand. Conceptually, this is just an encapsulation of an OS handle. You should read the documentation of this feature for details. Briefly, SafeHandle provides the following benefits:

1) Someone else wrote it and is maintaining it. Using it is much easier than building equivalent functionality yourself.

2) It prevents races between an application thread and the finalizer thread in unmanaged code. And it does this in a manner that leverages the type system. Specifically, clients are forced to deal with SafeHandles rather than IntPtrs or value types which don’t have strong identity and lifetime semantics.

3) It prevents handle-recycling attacks. You can read more details about finalization races (#2 above) and this bullet on handle-recycling attacks by reading http://blogs.msdn.com/cbrumme/archive/2003/04/19/51365.aspx. In that blog from last April, I allude to the existence of SafeHandle without giving details.

4) It discourages promotion of large graphs of objects, by placing the finalizable resources in a tiny leaf instance.

5) It participates with the PInvoke marshaler to ensure that unmarshaled instances will be registered for finalization.

6) For the handful of bizarre APIs that aren’t covered by our standard marshaling styles, Constrained Execution Regions (CERs) can be used to guarantee that unmarshaled instances will be registered for finalization.

7) It uses the new Critical Finalization mechanism to guarantee that leaks cannot occur. This means that we not only guarantee we will initiate execution of your Finalize method, but we also make some strong guarantees that allow you to ensure that it actually completes execution.

8) In order to guarantee that there will be no leaks, we necessarily leave the system open to denial of service and hangs. This is the halting problem. The Critical Finalization mechanism addresses this dilemma by making the leak protection explicit, restricting it to small regions of carefully written code, and by using the security system. Only trusted code can achieve strong guarantees about leakage. Such code is trusted not to create denial of service problems, whether maliciously or inadvertently, over small blocks of explicitly identified code.

9) Since SafeHandle uses Critical Finalization, it solves the problem of sequencing buffer flushing before handle closing that I mentioned earlier.

So what is this Critical Finalization thing?

Critical Finalization (CF) and CERs

Any object that derives from CriticalFinalizerObject (including SafeHandle) obtains critical finalization. This means:

1) Before any objects of this type are created, the CLR will “prepare” any resources that will be necessary for the Finalize method to run. Preparation includes JITting the code, running class constructors and – most importantly – traversing the static reachability of other methods and types that will be required during execution and making sure that they are likewise prepared. However, the CLR cannot statically traverse through indirections like interface calls and virtual calls. So there is a mechanism for the developer to guide the CLR through these opaque indirections.

2) The CLR will never timeout on the execution of one of these Finalize methods. As I mentioned, we rely on the limited amount of code written via this discipline combined with the trust decisions of the security system to avoid hangs here.

3) When the Finalize method is called, it is called in a protected state that prevents the CLR from injecting Thread.Aborts or other optional asynchronous exceptions. Because of our special preparation, we also prevent other asynchronous exceptions like OutOfMemoryExceptions due to JITting or type loading and TypeInitializationExceptions due to .cctors failures. Of course, if the application tries to allocate an object it may get an OutOfMemoryException. This is application-induced rather than system-induced and therefore is not considered the CLR’s responsibility. The Finalize method can use standard exception handling to protect itself here.

4) All normal finalizable objects are either executed or discarded without finalization, before any critical finalizers are executed. This means that a buffer flush can precede the close of the underlying handle.

The first 3 bullet points above are not restricted to CF. These bullet points apply to all CERs. The fundamental difference between CF and other CERs is the funky flow control from the instantiation of an object to the execution of its Finalize method via registration on our finalization queues. Other CERs can use normal block scopes in a single method to express the same reliability concepts. For normal CERs, the preparation phase, the forward execution phase and the backout phases are all contained in a single method.

A full description of CERs is beyond the scope of a note that is ostensibly about finalization. However, a brief description makes sense.

Essentially, CERs address issues with asynchronous exceptions. I have already mentioned asynchronous exceptions, which is the CLR’s term for all the pesky problems that manifest themselves as surprising exceptions. These are distinct from the application-level exceptions, which presumably are anticipated by and handled by the application.

You can read about asynchronous exceptions and the novel problems introduced by a managed execution environment that virtualizes resources so aggressively at http://blogs.msdn.com/cbrumme/archive/2003/06/23/51482.aspx.

In V1 and V1.1, the CLR does a poor job of distinguishing asynchronous exceptions from application exceptions. In Whidbey, we are starting to make this separation but it remains one of the weak design points for our hosting and exception stories.

Anyway, I’m sure that many readers are familiar with the difficulty of writing reliable unmanaged code that is guaranteed to complete in the face of limited resources (e.g. memory or stack), free threading, and other facts of life. And by now, if you’ve read all the blog articles I’ve mentioned, you are also familiar with the additional problems caused by a highly virtualized execution environment.

CERs allow you to declare regions of code where the CLR is constrained from injecting any system-generated errors. And the author of the code is constrained from performing certain actions if he wants to avoid additional exceptions. An obvious example is that he shouldn’t new up an object if he is not prepared to deal with an OutOfMemoryException from that operation.

In addition to CERs, Whidbey provides reliability contracts. These contracts can be used to annotate methods with their guarantees and requirements with respect to reliability. Using these contracts, it’s possible to compose reliable execution out of components written by different authors. This is necessary for building reliable applications that make use of framework services. If the reliability requirements and guarantees of the framework services were not themselves explicit, the client applications could not remain reliable on top of them.

Finalization in SQL Server and other high availability hosts

Back to finalization.

In a normal unhosted process, there isn’t a strong distinction between normal and critical finalization. Normal processes won’t run out of memory, and if they do they should probably Fail Fast. It’s unlikely that the risk of trying to continue execution after resource exhaustion is worth the increased risk of subsequent crashes, hangs or other corruptions. Normal processes won’t experience Thread.Aborts that are injected across threads. (As opposed to aborting the current thread, which is no more dangerous than throwing any other exception).

So the only real concern is whether all the finalizable objects will drain during process exit, before the timeouts kick in. The timeouts are quite generous and in practice this is not a concern.

However, a hosted process like SQL Server is quite different. Because of SQL Server’s availability requirements, it is vital that the process not FailFast for something innocuous like OutOfMemoryExceptions. Indeed, SQL Server tries to run on the brink of memory exhaustion for performance reasons, so these memory exceptions are a constant fact of life in that environment. Furthermore, SQL Server uses Thread.Abort explicitly across threads to terminate long-running requests and it uses Thread.Abort implicitly to unload AppDomains. On a heavily loaded system, AppDomains may be unloaded to relieve resource pressure.

I have a lengthy blog on this topic, but I have not been able to post it because it talks about undisclosed Whidbey features. At some point (no later than shipping Beta1), you will find it at http://blogs.msdn.com/cbrumme with a title of Hosting. Until then, I’ll just mention that the Whidbey APIs support an escalation policy. This is a declarative mechanism by which the host can express timeouts for normal finalization, normal AppDomain unload, normal Abort attempts, etc. In addition to timeouts, the escalation policy can indicate appropriate actions whenever these timeouts expire. So a normal AppDomain unload could (for example) be escalated to a rude AppDomain unload or a normal process exit or a rude process exit.

The distinction between polite/normal and rude involves several aspects beyond finalization. If we just consider finalization, polite/normal means that we execute both normal and critical finalization. Contrast this with a rude scenario where we will ignore the normal finalizers, which are discarded, and only execute the critical finalizers. As you might expect, a similar distinction occurs between executing normal exception backout on threads, vs. restricting ourselves to any backout that is associated with CERs.

This allows a host to avoid solving the halting problem when performing normal finalization and exception backout, without putting the process at risk with respect to (critical) resource leakage or inconsistent state.

Apartments and Pumping in the CLR

cbrumme — Mon, 02 Feb 2004 19:52:00 GMT

I’ve already written the much-delayed blog on Hosting, but I can’t post it yet because it mentions a couple of new Whidbey features, which weren’t present in the PDC bits. Obviously Microsoft doesn’t want to make product disclosures through my random blog articles.

I’m hoping this will be sorted out in another week or two.

While we’re waiting, I thought I would talk briefly(!) about pumping and apartments. The CLR made some fundamental decisions about OLE, thread affinity, reentrancy and finalization. These decisions have a significant impact on program correctness, server scalability, and compatibility with legacy (i.e. unmanaged) code. So this is going to be a blog like the one on Shutdown from last August (see http://blogs.msdn.com/cbrumme/archive/2003/08/20/51504.aspx). There will be more detail than you probably care to know about one of the more frustrating parts of the Microsoft software stack.

First, an explanation of my odd choice of terms. I’m using OLE as an umbrella which includes the following pieces of technology:

COM – the fundamental object model, like IUnknown and IClassFactory

DCOM – remoting of COM using IDL, NDR pickling and the SCM

Automation – IDispatch, VARIANT, Type Libraries, etc.

Active/X – Protocols for controls and their containers

Next, some disclaimers:

I am not and have never been a GUI programmer. So anything I know about Windows messages and pumping is from debugging GUI applications, not from writing them. I’m not going to talk about WM_PENCTL notifications or anything else that requires UI knowledge.

Also, I’m going to point out a number of problems with OLE and apartments. The history of the CLR and OLE are closely related. In fact, at one point COM+ 1.0 was known internally as COM98 and the CLR was known internally as COM99. We had some pretty aggressive ship targets back then!

In general, I love OLE and the folks who work on it. Although it is inappropriate for the Internet, DCOM is still the fastest and most enterprise-ready distributed object system out there. In a few ways the architecture of .NET Remoting is superior to DCOM, but we never had the time or resources to even approach the engineering effort that has gone into DCOM. Presumably Indigo will eventually change this situation. I also love COM’s strict separation of contract from implementation, the ability to negotiate for contracts, and so much more.

The bottom line is that OLE has had at least as much impact on Microsoft products and the industry, in its day, as .NET is having now.

But, like anything else, OLE has some flaws. In contrast to the stark architectural beauty of COM and DCOM, late-bound Automation is messy. At the time this was all rolled out to the world, I was at Borland and then Oracle. As an outsider, it was hard for me to understand how one team could have produced such a strange combination.

Of course, Automation has been immensely successful – more successful than COM and DCOM. My aesthetic taste is clearly no predictor of what people want. Generally, people want whatever gets the job done, even if it does so in an ad hoc way. And Automation has enabled an incredible number of application scenarios.

Apartments

If there’s another part of OLE that I dislike, it’s Single Threaded Apartments. Presumably everyone knows that OLE offers three kinds of apartments:

Single Threaded Apartment (STA) – one affinitized thread is used to call all the objects residing in the apartment. Any call on these objects from other threads must perform cross-thread marshaling to this affinitized thread, which dispatches the call. Although a process can have an arbitrary number of STAs (with a corresponding number of threads), most client processes have a single Main STA and the GUI thread is the affinitized thread that owns it.

Multiple Threaded Apartment (MTA) – each process has at most one MTA at a time. If the current MTA is not being used, OLE may tear it down. A different MTA will be created as necessary later. Most people think of the MTA as not having thread affinity. But strictly speaking it has affinity to a group of threads. This group is the set of all the threads that are not affinitized to STAs. Some of the threads in this group are explicitly placed in the MTA by calling CoInitializeEx. Other threads in this group are implicitly in the MTA because the MTA exists and because these threads haven’t been explicitly placed into STAs. So, by the strict rules of OLE, it is not legal for STA threads to call on any objects in the MTA. Instead, such calls must be marshaled from the calling STA thread over to one of the threads in the MTA before the call can legally proceed.

Neutral Apartment (NA) – this is a recent invention (Win2000, I think). There is one NA in the process. Objects contained in the NA can be called from any thread in the process (STA or MTA threads). There are no threads associated with the NA, which is why it isn’t called NTA. Calls into NA objects can be relatively efficient because no thread marshaling is ever required. However, these cross-apartment calls still require a proxy to handle the transition between apartments. Calls from an object in the NA to an object in an STA or the MTA might require thread marshaling. This depends on whether or not the current thread is suitable for calling into the target object. For example, a call from an STA object to an NA object and from there to an MTA object will require thread marshaling during the transition out of the NA into the MTA.

Threading

The MTA is effectively a free-threaded model. (It’s not quite a free-threaded model, because STA threads aren’t strictly allowed to call on MTA objects directly). From an efficiency point of view, it is the best threading model. Also, it imposes the least semantics on the application, which is also desirable. The main drawback with the MTA is that humans can’t reliably write free-threaded code.

Well, a few developers can write this kind of code if you pay them lots of money and you don’t ask them to write very much. And if you code review it very carefully. And you test it with thousands of machine hours, under very stressful conditions, on high-end MP machines like 8-ways and up. And you’re still prepared to chase down a few embarrassing race conditions once you’ve shipped your product.

But it’s not a good plan for the rest of us.

The NA model is truly free-threaded, in the sense that any thread in the process can call on these objects. All such threads must still transition through a proxy layer that maintains the apartment boundary. But within the NA all calls are direct and free-threaded. This is the only apartment that doesn’t involve thread affinity.

Although the NA is free-threaded, it is often used in conjunction with a lock to achieve rental threading. The rental model says that only one thread at a time can be active inside an object or a group of objects, but there is no restriction on which thread this might be. This is efficient because it avoids thread marshaling. Rather than marshaling a call from one thread to whatever thread is affinitized to the target objects, the calling thread simply acquires the lock (to rent the context) and then completes the call on the current thread. When the thread returns back out of the context, it releases the lock and now other threads can make calls.

If you call out of a rental context into some other object (as opposed to the return pathway), you have a choice. You can keep holding the lock, in which case other threads cannot rent the context until you fully unwind. In this mode, the rental context supports recursion of the current thread, but it does not support reentrancy from other threads. Alternatively, the thread could release the lock when it calls out of the rental context, in which case it must reacquire the lock when it unwinds back and returns to the rental context. In this mode, the rental context supports full reentrancy.

Throughout this blog, we’ll be returning to this fundamental decision of whether to support reentrancy. It’s a complex issue.

If only recursion is supported on a rental model, it’s clear that this is a much more forgiving world for developers than a free-threaded model. Once a thread has acquired the rental lock, no other threads can be active in the rented objects until the lock has been released. And the lock will not be released until the thread fully unwinds from the call into the context.

Even with reentrancy, the number of places where concurrency can occur is limited. Unless the renting thread calls out of the context, the lock won’t be released and the developer knows that other threads aren’t active within the rented objects. Unfortunately, it might be hard for the developer to know all the places that call out of the current context, releasing the lock. Particularly in a componentized world, or a world that combines application code with frameworks code, the developer can rarely have sufficient global knowledge.

So it sounds like limiting a rental context to same-thread recursion is better than allowing reentrancy during call outs, because the developer doesn’t have to worry about other threads mutating the state of objects in the rental context. This is true. But it also means that the resulting application is subject to more deadlocks. Imagine what can happen if two rental contexts are simultaneously making calls to each other. Thread T1 holds the lock to rent context C1. Thread T2 holds the lock to rent context C2. If T1 calls into C2 just as T2 calls into C1, and we are on the recursion plan, we have a classic deadlock. Two locks have been taken in different sequences by two different threads. Alternatively, if we are on a reentrancy plan, T1 will release the lock for C1 before contending for the lock on C2. And T2 will release the lock for C2 before contending for the lock on C1. The deadlock has been avoided, but T1 will find that the objects in C1 have been modified when it returns. And T2 will find similar surprises when it returns to C2.

Affinity

Anyway, we now understand the free-threaded model of the MTA and NA and we understand how to build a rental model on top of these via a lock. How about the single-threaded affinitized model of STAs? It’s hard to completely describe the semantics of an STA, because the complete description must incorporate the details of pages of OLE pumping code, the behavior of 3^rd party IMessageFilters, etc. But generally an STA can be thought of as an affinitized rental context with reentrancy and strict stacking. By this I mean:

It is affinitized rental because all calls into the STA must marshal to the correct thread and because only one logical call can be active in the objects of the apartment at any time. (This is necessarily the case, since there is only ever one thread).
It has reentrancy because every callout from the STA thread effectively releases the lock held by the logical caller and allows other logical callers to either enter or return back to the STA.
It has strict stacking because one stack (the stack of the affinitized STA thread) is used to process all the logical calls that occur in the STA. When these logical calls perform a callout, the STA thread reentrantly picks up another call in, and this pushes the STA stack deeper. When the first callout wants to return to the STA, it must wait for the STA thread’s stack to pop all the way back to the point of its own callout.

That point about strict stacking is a key difference between true rental and the affinitized rental model of an STA. With true rental, we never marshal calls between threads. Since each call occurs on its own thread, the pieces of stack for different logical threads are never mingled on an affinitized thread’s actual stack. Returns back into the rental context after a callout can be processed in any order. Returns back into an STA after a callout must be processed in a highly constrained order.

We’ve already seen a number of problems with STAs due to thread affinity, and we can add some more. Here’s the combined list:

Marshaling calls between threads is expensive, compared to taking a lock.

Processing returns from callouts in a constrained fashion can lead to inefficiencies. For instance, if the topmost return isn’t ready for processing yet, should the affinitized thread favor picking up a new incoming call (possibly leading to unconstrained stack growth) or should it favor waiting for the topmost return to complete (possibly idling the affinitized thread completely and conceivably resulting in deadlocks).

Any conventional locks held by an affinitized thread are worthless. The affinitized thread is processing an arbitrary number of logical calls, but a conventional lock (like an OS CRITICAL_SECTION or managed Monitor) will not distinguish between these logical calls. Instead, all lock acquisitions are performed by the single affinitized thread and are granted immediately as recursive acquisitions. If you are thinking of building a more sophisticated lock that avoids this issue, realize that you are making that classic reentrancy vs. deadlock decision all over again.

Imagine a common server situation. The first call comes in from a particular client, creates a few objects (e.g. a shopping cart) and returns. Subsequent calls from that client manipulate that initial set of objects (e.g. putting some items into the shopping cart). A final call checks out the shopping cart, places the order, and all the objects are garbage collected. Now imagine that all those objects are affinitized to a particular thread. As a consequence, the dispatch logic of your server must ensure that all calls from the same client are routed to the same thread. And if that thread is busy doing other work, the dispatch logic must delay processing the new request until the appropriate affinitized thread is available. This is complicated and it has a severe impact on scalability.

STAs must pump. (How did I get this far without mentioning pumping?)

Any STA code that assumed a single-threaded world for the process, rather than just for the apartment, might not pump. Such code breaks when we introduce the CLR into the process, as we will see.

Failure to Pump

Let’s look at those last two bullet points in more detail. When your STA thread is doing nothing else, it needs to be checking to see if any other threads want to marshal some calls into it. This is done with a Windows message pump. If the STA thread fails to pump, these incoming calls will be blocked. If the incoming calls are GUI SendMessages or PostMessages (which I think of as synchronous or asynchronous calls respectively), then failure to pump will produce an unresponsive UI. If the incoming calls are COM calls, then failure to pump will result in calls timing out or deadlocking.

If processing one incoming call is going to take a while, it may be necessary to break up that processing with intermittent visits to the message pump. Of course, if you pump you are allowing reentrancy to occur at those points. So the developer loses all his wonderful guarantees of single threading.

Unfortunately, there’s a whole lot of STA code out there which doesn’t pump adequately. For the most part, we see this in non-GUI applications. If you have a GUI application that isn’t pumping enough, it’s obvious right there on the screen. Those bugs tend to get fixed.

For non-GUI applications, a failure to pump may not be noticed in unmanaged code. When that code is moved to managed (perhaps by re-compiling some VB6 code as VB.NET), we start seeing bugs. Let’s look at a couple of real-world cases that we encountered during V1 of the CLR and how the lingering effects of these cases are still causing major headaches for managed developers and for Microsoft Support. I’ll describe a server case first, and then a client case.

ADO and ASP Compatibility Mode

ADO.NET and ASP.NET are a winning combination. But ASP.NET also supports an ASP compatibility mode. In this mode, legacy ASP pages can be served up by the managed ASP.NET pipeline. Such pages were written before we invented our managed platform, so they use ADO rather than ADO.NET for any data access. Also, in this mode the DCOM threadpool is used rather than the managed System.Threading.ThreadPool. Although all the threads in the managed ThreadPool are explicitly placed in the MTA (as you might hope and expect), the DCOM threadpool actually contains STA threads.

The purpose of this STA threadpool was to allow legacy STA COM objects in general, and VB6 objects in particular, to be moved from the client to the server. The result suffers from the scaling problems I alluded to before, since requests are dispatched on up to 100 STA threads with careful respect for any affinity. Also, VB6 has a variable scope which corresponds to “global” (I forget its name), but which is treated as per-thread when running on the server. If there are more than 100 clients using a server, multiple clients will share a single STA thread based on the whims of the request dispatch logic. This means that global variables are shared between sets of clients in a surprising fashion, based on the STA that they happen to correspond to.

A typical ASP page written in VBScript would establish a (hopefully pooled) database connection from ADO, query up a row, modify a field, and write the row back to the database. Since the page was likely written in VB, any COM AddRef and Release calls on the ADO row and field value objects were supplied through the magic of the VB6 runtime. This means they occur on the same thread and in a very deterministic fashion.

The ASP page contains no explicit pumping code. Indeed, at no point was the STA actually pumped. Although this is a strict violation of the rules, it didn’t cause any problems. That’s because there are no GUI messages or inter-apartment COM calls that need to be serviced.

This technique of executing ASP pages on STAs with ADO worked fairly well – until we tried to extend the model to ASP.NET running in ASP compatibility mode. The first problem that we ran into was that all managed applications are automatically multi-threaded. For any application of reasonable complexity, there are sure to be at least a few finalizable objects. These objects will have their Finalize methods called by one or more dedicated finalizer threads that are distinct from the application threads.

(It’s important that finalization occurs on non-application threads, since we don’t want to be holding any application locks when we call the Finalize method. And today the CLR only has a single Finalizer thread, but this is an implementation detail. It’s quite likely that in the future we will concurrently call Finalize methods on many objects, perhaps by moving finalization duties over to the ThreadPool. This would address some scalability concerns with finalization, and would also allow us to make stronger guarantees about the availability of the finalization service).

Our COM Interop layer ensures that we almost only ever call COM objects in the correct apartment and context. The one place where we violate COM rules is when the COM object’s apartment or context has been torn down. In that case, we will still call IUnknown::Release on the pUnk to try to recover its resources, even though this is strictly illegal. We’ve gone backwards and forwards on whether this is appropriate, and we provide a Customer Debug Probe so that you can detect whether this is happening in your application.

Anyway, let’s pretend that we absolutely always call the pUnk in the correct apartment and context. In the case of an object living in an STA, this means that the Finalizer thread will marshal the call to the affinitized thread of that STA. But if that STA thread is not pumping, the Finalizer thread will block indefinitely while attempting to perform the cross-thread marshaling.

The effect on a server is crippling. The Finalizer thread makes no progress. The number of unreleased pUnks grows without bounds. Eventually some resource (usually memory) is exceeded and the process crashes.

One solution is to edit the original ASP page to pump the underlying STA thread that it is executing on. A light-weight way to pump is to call Thread.CurrentThread.Join(0). This causes the current thread to block until the current thread dies (which isn’t going to happen) or until 0 milliseconds have elapsed – whichever happens first. I’ll explain later why this also performs some pumping and why this is a controversial aspect of the CLR. A heavier-weight way to pump is to call GC.WaitForPendingFinalizers. This not only performs pumping, but it also waits for the Finalization queue to drain.

If you are porting a page that produces a modest number of COM objects, doing a simple Join on each page may be sufficient. If your page performs elaborate processing, perhaps creating an unbounded number of COM objects in a loop, then you may need to either add a Join within the loop or WaitForPendingFinalizers at the end of the page processing. The only way to really know is to experiment with both techniques, measuring the growth of the Finalization queue and the impact on server throughput.

ADO’s Threading Model

There was another problem with using ADO from ASP.NET’s ASP compatibility mode. Do you know what the threading model of ADO is? Well, if you check the registry for some ADO CLSIDs on your machine, you may find them registered as ThreadingModel=Single or you may find them registered as ThreadingModel=Both.

If these classes are registered as Single, OLE will carefully ensure that their instances can only be called from the thread that they were created on. This implies that the objects can assume a single-threaded view of the world and they do not need to be written in a thread-safe manner. If these classes are registered as Both, OLE will ensure that their instances are only called from threads in the right apartment. But if that apartment is the MTA, these objects better have been written in a thread-safe manner. For example, they had better be using InterlockedIncrement and Decrement, or an equivalent, for reference counting.

Unfortunately, the ADO classes are not thread-safe. Strictly speaking, they should never be registered as anything but Single. You may find them registered as Both on your machine because this improves scalability and throughput for some key scenarios. And those key scenarios happen to limit themselves to “one thread at a time” because of how ASP and VB6 work.

In fact, the legacy ADO classes don’t even support single-threaded access if there is reentrancy. They will randomly crash when used in this manner (and this is exactly the manner in which ADO was driven in the early days of V1). Here are the steps:

The page queries up an ADO row object, which enters managed code via COM Interop as an RCW (runtime-callable wrapper).
By making a COM call on this RCW, the page navigates to a field value. This field value also enters managed code via COM Interop as an RCW.
The page now makes a COM call via ADO which results in a call out to the remote database. At this point, the STA thread is pumped by the DCOM remote call. Since this is a remote call, it’s going to take a while before it returns.
The garbage collector decides that it’s time to collect. At this point, the RCW for the field value is still reachable and is reported. The RCW for the row object is no longer referenced by managed code and is collected.
The Finalizer thread notices that the pUnk underlying the row’s RCW is no longer in use, and it makes the cross-apartment call from the Finalizer thread’s apartment (MTA) to the ADO row object’s apartment (STA).
Recall that the STA thread is pumping for the duration of the remote database call (#3 above). It picks up the cross-thread call from the Finalizer (#5 above) and performs the Release on the Row object. This is the final Release and ADO deletes the unmanaged Row object from memory. This logical call unwinds and the Finalizer thread is unblocked (hurray). The STA thread returns to pumping.
The remote database call returns back to the server machine. The STA thread picks it up from its pumping loop and returns back to the page, unwinding the thread.
The page now updates the field value, which involves a COM call to the underlying ADO object.
ADO crashes or randomly corrupts memory.

What happened? The ADO developers made a questionable design decision when they implemented COM reference counting throughout their hierarchy. The field values refer to their owning row object, but they don’t hold a reference count on that row. Instead, they assume that the row will live as long as all of its associated field values. And yet, whenever the application makes an ADO call on a field value, the field value will access that (hopefully present) row.

This assumption worked fine in the days of ASP and VB6. So nobody even noticed the bug until the CLR violated those threading assumptions – without violating the underlying OLE rules, of course.

It was impractical to fix this by opening up ADO and rewriting the code. There are many different versions of ADO in existence, and many products that distribute it. Another option was to add GC.KeepAlive(row) calls at the bottom of each page, to extend the lifetime of the row objects until the field values were no longer needed. This would have been a nightmare for Support.

Instead, the ADO team solved the problem for managed code with a very elegant technique. (I invented it, so of course I think it was elegant). They opened up the assembly that was created by TlbImp’ing ADO. Then they added managed references from the RCWs of the field values to the RCWs of their owning rows. These managed references are completely visible to the garbage collector. Now the GC knows that if the field values are reachable then the row values must also be reachable. Problem solved.

No Typelib Registered

Incidentally, we ran into another very common problem when we moved existing client or server COM applications over to managed code. Whenever an application uses a COM object, it tries hard to match the thread of the client to the ThreadingModel of the server. In other words, if the application needs to use a ThreadingModel=Main COM object, the application tries to ensure that the creating thread is in an STA. Similarly, if the application needs to use a ThreadingModel=Free COM object, it tries to create this object from an MTA thread. Even if a COM object is ThreadingModel=Both, the application will try to access the object from the same sort of thread (STA vs. MTA) as the thread that created the object.

One reason for doing this is performance. If you can avoid an apartment transition, your calls will be much faster. Another reason has to do with pumping and reentrancy. If you make a cross-apartment call into an STA, the STA better be pumping to pick up your call. And if you make a cross-apartment call out of an STA, your thread will start pumping and your application becomes reentrant. This is a small dose of free-threading, and many application assumptions start to break. A final reason for avoiding apartment transitions is that they often aren’t supported. For instance, most ActiveX scenarios require that the container and the control are in the same STA. If you introduce an apartment boundary (even between two STAs), bizarre cases like Input Synchronous messages stop working properly.

The net result is that a great many applications avoid using COM objects across apartment boundaries. And this means that – even if that COM object is nominally marshalable across an apartment boundary – this often isn’t being tested. So an application might install itself without ensuring that the typelib of the COM component is actually registered.

When the application is moved to managed code, developers are frustrated to see InvalidCastExceptions on the managed side. A typical sequence is that they successfully ‘new’ the COM object, implying that the CoCreate returned a pUnk which was wrapped in an RCW. Then when they cast it to one of the interfaces that they know is supported, a casting exception is thrown. This casting exception is due to a QueryInterface call failing with E_NOINTERFACE. Yet this HRESULT is not returned by the COM object, which does indeed support the interface. Instead, it is returned by a COM apartment proxy which sits between the RCW and that COM object. The COM apartment proxy is simply failing to marshal the interface across the apartment boundary – usually because the COM object is using the OLEAUT marshaler and the Typelib has not been properly registered.

This is a common failure, and it’s unfortunate that a generic E_NOINTERFACE doesn’t lead to better debuggability for this case.

Finally, I can’t help but mention that the COM Interop layer added other perturbations to many unmanaged COM scenarios that seemed to be working just fine. Common perturbations from managed code include garbage collection, a Finalizer thread, strict conformance to OLE marshaling rules, and the fact that managed objects are agile with respect to COM apartments and COM+ contexts (unless they derive from ServicedComponent).

For instance, Trident required that all calls on its objects occur on the correct thread. But Trident also had an extension model where 3^rd party objects could be aggregated onto their base objects. Unfortunately, the aggregator performed blind delegation to the 3^rd party objects. And – even more unfortunate – this blind delegation did not exclude QI’s for IMarshal. Of course, managed objects implement IMarshal to achieve their apartment and context agility. So if Trident aggregated a managed object as an extention, the containing Trident object would attempt to become partially agile in a very broken way.

Hopefully we found and dealt with most of these issues before we shipped V1.

Not Pumping a Client

I said I would describe two cases where non-pumping unmanaged code caused problems when we moved to managed code. The above explains, in great detail, how ADO and ASP compatibility mode caused us problems on the server. Now let’s look at the non-GUI client case.

We all know that a WinForms GUI client is going to put the main GUI thread into an STA. And we know that there’s a lot of pumping in a GUI application, or else not much is going to show on the screen.

Assume for a moment that a Console application also puts its main thread into an STA. If that main thread creates any COM objects via COM Interop, and if those COM objects are ThreadingModel=Main or Both, then the application better be pumping. If it fails to pump, we’ll have exactly the same situation with our server running ASP compatibility mode. The Finalizer thread won’t be able to marshal calls into the STA to Release any pUnks.

On a well-loaded server, that failure is quickly noticed by the developer or by the folks in operations. But on a client, this might be just a mild case of constipation. The rate of creation of finalizable objects may be low enough that the problem is never noticed. Or it may be noticed as a gradual build up of resources. If the problem is reported to Microsoft Support, the customer generally categorizes it as a garbage collection bug.

So what is the apartment of a Console application’s main thread? Well, it depends.

If you build a Console application in Notepad, the main thread is likely to start off in the MTA. If you build a Console application with Visual Studio, then if you pick C# or VB.NET your main thread is likely to be in an STA. If you build a Console application with Visual Studio and you choose managed C++, your main thread is likely to be in an MTA for V1 or V1.1. I think it’s likely to be in an STA for our next release.

Wow. Why are we all over the place on this? Mostly, it’s because there is no correct answer. Either the developer is not going to use any COM objects in his Console application, in which case the choice doesn’t really matter, or the developer is going to use some COM objects and this should inform his decision.

For instance, if the developer will use COM objects with ThreadingModel=Main, he probably wants to put his main thread into an STA so he can use the COM objects directly without cross-thread marshaling and all the issues that this would imply. This means he should also pump that thread, if there are other threads (like the Finalizer!) active in the process. Alternatively, if the developer intends to use COM objects with ThreadingModel=Free, he probably wants to put his main thread in the MTA so he can access those objects directly. Now he doesn’t need to pump, but he does need to consider the implications of writing free-threaded code.

Either way, the developer has some responsibility.

Unfortunately, the choice of a default is typically made by the project type that he selects in Visual Studio, or is based on the CLR’s default behavior (which favors MTA). And realistically the subtleties of apartments and pumping are beyond the knowledge (or interest) of most managed developers. Let’s face it: nobody should have to worry about this sort of thing.

The Managed CoInitialize Mess

There are three ways to select an apartment choice for the main thread of your Console application. All three of these techniques have concerns associated with them.

1) You can place either an STAThreadAttribute or MTAThreadAttribute onto the main method.

2) You can perform an assignment to System.Threading.CurrentThread.ApartmentState as one of the first statements of your main method (or of your thread procedure if you do a Thread.Start).

3) You can accept the CLR’s default of MTA.

So what’s wrong with each of these techniques?

The first technique is the preferred method, and it works very well for C#. After some tweaks to the VB.NET compiler before we shipped V1, it worked well for VB too. Managed C++ still doesn’t properly support this technique. The reason is that the entrypoint of a managed C++ EXE isn’t actually your ‘main’ routine. Instead, it’s a method inside the C-runtime library. That method eventually delegates to your ‘main’ routine. But the CLR doesn’t scan through the closure of calls from the entrypoint when looking for the custom attribute that defines the threading model. If the CLR doesn’t find it on the method that is the EXE’s entrypoint, it stops looking. The net result is that your attribute is quietly ignored for C++.

I’m told that this will be addressed in Whidbey, by having the linker propagate the attribute from ‘main’ to the CRT entrypoint. And indeed this is how the VB.NET compiler works today.

What’s wrong with the second technique? Unfortunately, it is subject to a race condition. Before the CLR can actually call your thread procedure, it may first call some module constructors, class constructors, AssemblyLoad notifications and AssemblyResolve notifications. All of this execution occurs on the thread that was just created. What happens if some of these methods set the thread’s ApartmentState before you get a chance? What happens if they call Windows services like the clipboard that also set the apartment state? A more likely scenario is that one of these other methods will make a PInvoke call that marshals a BSTR, SAFEARRAY or VARIANT. Even these innocuous operations can force a CoInitializeEx on your thread and limit your ability to configure the thread from your thread procedure.

When you are developing your application, none of the above is likely to occur. The real nightmare scenario is that a future version of the CLR will provide a JIT that inlines a little more aggressively, so some extra class constructors execute before your thread procedure. In other words, you will ship an application that is balanced on a knife edge here, and this will become an App Compatibility issue for all of us. (See http://blogs.msdn.com/cbrumme/archive/2003/11/10/51554.aspx for more details on the sort of thing we worry about here).

In fact, for the next release of the CLR we are seriously considering making it impossible to set the apartment state on a running thread in this manner. At a minimum, you should expect to see a Customer Debug Probe warning of the risk here.

And the third technique from above has a similar problem. Recall that threads in the MTA can be explicitly placed there through a CoInitializeEx call, or they can be implicitly treated as being in the MTA because they haven’t been placed into an STA. The difference between these two cases is significant.

If a thread is explicitly in the MTA, any attempt to configure it as an STA thread will fail with an error of RPC_E_CHANGED_MODE. By contrast, if a thread is implicitly in the MTA it can be moved to an STA by calling CoInitializeEx. This is more likely than it may sound. If you attempt a clipboard operation, or you call any number of other Windows services, the code you call may attempt to place your thread in the STA. And when you accept the CLR default behavior, it currently leaves the thread implicitly in the MTA and therefore is subject to reassignment.

This is another place where we are seriously considering changing the rules in the next version of the CLR. Rather than place threads implicitly in the MTA, we are considering making this assignment explicit and preventing any subsequent reassignment. Once again, our motivation is to reduce the App Compat risk for applications after they have been deployed.

Speaking of race conditions and apartments, the CLR has a nasty bug which was introduced in V1 and which we have yet to remove. I’ve already mentioned that any threads that aren’t in STAs or explicitly in the MTA are implicitly in the MTA. That’s not strictly true. These threads are only in the MTA if there is an MTA for them to be in.

There is an MTA if OLE is active in the process and if at least one thread is explicitly in the MTA. When this is the case, all the other unconfigured threads are implicitly in the MTA. But if that one explicit thread should terminate or CoUninitialize, then OLE will tear down the MTA. A different MTA may be created later, when a thread explicitly places itself into it. And at that point, all the unconfigured threads will implicitly join it.

But this destruction and recreation of the MTA has some serious impacts on COM Interop. In fact, any changes to the apartment state of a thread can confuse our COM Interop layer, cause deadlocks on downlevel platforms, and lead to memory leaks and violation of OLE rules.

Let’s look at how this specific race condition occurs first, and then I’ll talk about the larger problems here.

An unmanaged thread CoInitializes itself for the MTA and calls into managed code.
While in managed code, that thread introduces some COM objects to our COM Interop layer in the form of RCWs, perhaps by ‘new’ing them from managed code.
The CLR notices that the current thread is in the MTA, and realizes that it must “keep the MTA alive.” We signal the Finalizer thread to put itself explicitly into the MTA via CoInitializeEx.
The unmanaged thread returns out to unmanaged code where it either dies or simply calls CoUninitialize. The MTA is torn down.
The Finalizer thread wakes up and explicitly CoInitializes itself into the MTA. Oops. It’s too late to keep the original MTA alive and it has the effect of creating a new MTA. At least this one will live until the end of the process.

As far as I know, this is the only race condition in the CLR that we haven’t fixed. Why have we ignored it all these years? First, we’ve never seen it reported from the field. This isn’t so surprising when you consider that the application often shares responsibility for keeping the MTA alive. Many applications are aware of this obligation and – if they use COM – they always keep an outstanding CoInitialize on one MTA thread so the apartment won’t be torn down. Second, I generally resist fixing bugs by adding inter-thread dependencies. It would be all too easy to create a deadlock by making step 3 wait for the Finalizer thread to CoInitialize itself, rather than just signaling it to do so. This is particularly true since the causality of calls from the Finalizer to other threads is often opaque to us, as I’ll explain later. And we certainly don’t want to create a dedicated thread for this purpose. Dedicated threads have a real impact on Terminal Server scenarios, where the cost of one thread in a process is multiplied by all the processes that are running. Even if we were prepared to pay this cost, we would want to create this thread lazily. But synchronizing with the creation of another thread is always a dangerous proposition. Thread creation involves taking the OS loader lock and making DLL_THREAD_ATTACH notifications to all the DllMain routines that didn’t explicitly disable these calls.

The bottom line is that the fix is expensive and distasteful. And it speaks to a more general problem, where many different components in a process may be individually spinning up threads to keep the MTA from being recycled. A better solution is for OLE to provide an API to keep this apartment alive, without requiring all those dedicated threads. This is the approach that we are pursuing for the long term.

In our general cleanup of the CLR’s treatment of CoInitialize, we are also likely to change the semantics of assigning the current thread’s ApartmentState to Unknown. In V1 & V1.1 of the CLR, any attempt to set the state to Unknown would throw an ArgumentOutOfRangeException, so we’re confident that we can make this change without breaking applications.

If the CLR has performed an outstanding CoInitializeEx on this thread, we may treat the assignment to Unknown as a request to perform a CoUninitialize to reverse the operation. Currently, the only way you can CoUninitialize a thread is to PInvoke to the OLE32 service. And such changes to the apartment state are uncoordinated with the CLR.

Now why does it matter if the apartment state of a thread changes, without the CLR knowing? It matters because:

1) The CLR may hold RCWs over COM objects in the apartment that is about to disappear. Without a notification, we cannot legally release those pUnks. As I’ve already mentioned, we break the rules here and attempt to Release anyway. But it’s still a very bad situation and sometimes we will end up leaking.

2) The CLR will perform limited pumping of STA threads when you perform managed blocking (e.g. WaitHandle.WaitOne). If we are on a recent OS, we can use the IComThreadingInfo interface to efficiently determine whether we should pump or not. But if we are on a downlevel platform, we would have to call CoInitialize prior to each blocking operation and check for a failure code to absolutely determine the current state of the thread. This is totally impractical from a performance point of view. So instead we cache what we believe is the correct apartment state of the thread. If the application performs a CoInitialize or CoUninitialize without informing us, then our cached knowledge is stale. So on downlevel platforms we might neglect to pump an STA (which can cause deadlocks). Or we may attempt to pump an MTA (which can cause deadlocks).

Incidentally, if you ever run managed applications under a diagnostic tool like AppVerifier, you may see complaints from that tool at process shutdown that we have leaked one or more CoInitialize calls. In a well-behaved application, each CoInitialize would have a balancing CoUninitialize. However, most processes are not so well-behaved. It’s typical for applications to terminate the process without unwinding all the threads of the process. There’s a very detailed description of the CLR’s shutdown behavior at http://blogs.msdn.com/cbrumme/archive/2003/08/20/51504.aspx.

The bottom line here is that the CLR is heavily dependent on knowing exactly when apartments are created and destroyed, or when threads become associated or disassociated with those apartments. But the CLR is largely out of the loop when these operations occur, unless they occur through managed APIs. Unfortunately, we are rarely informed. For an extreme example of this, the Shell has APIs which require an STA. If the calling thread is implicitly in the MTA, these Shell APIs CoInitialize that calling thread into an STA. As the call returns, the API will CoUnitialize and rip down the apartment.

We would like to do better here over time. But there are some pretty deep problems and most solutions end up breaking an important scenario here or there.

Back to Pumping

Enough of the CoInitialize mess. I mentioned above that managed blocking will perform some pumping when called on an STA thread.

Managed blocking includes a contentious Monitor.Enter, WaitHandle.WaitOne, WaitHandle.WaitAny, GC.WaitForPendingFinalizers, our ReaderWriterLock and Thread.Join. It also includes anything else in FX that calls down to these routines. One noticeable place where this happens is during COM Interop. There are pathways through COM Interop where a cache miss occurs on finding an appropriate pUnk to dispatch a call. At those points, the COM call is forced down a slow path and we use this as an opportunity to pump a little bit. We do this to allow the Finalizer thread to release any pUnks on the current STA, if the application is neglecting to pump. (Remember those ASP Compat and Console client scenarios?) This is a questionable practice on our part. It causes reentrancy at a place where it normally could never occur in pure unmanaged scenarios. But it allows a number of applications to successfully run without clogging up the Finalizer thread.

Anyway, managed blocking does not include PInvokes directly to any of the OS blocking services. And keep in mind that if you PInvoke to the OS blocking services directly, the CLR will no longer be able to take control of your thread. Operations like Thread.Interrupt, Thread.Abort and AppDomain.Unload will be indefinitely delayed.

Did you notice that I neglected to mention WaitHandle.WaitAll in the list of managed blocking opeprations? That’s because we don’t allow you to call WaitAll from an STA thread. The reason is rather subtle. When you perform a pumping wait, at some level you need to call MsgWaitForMultipleObjectsEx, or a similar Msg* based variant. But the semantics of a WAIT_ALL on an OS MsgWaitForMultipleObjectsEx call is rather surprising and not what you want at all. It waits for all the handles to be signaled AND for a message to arrive at the message queue. In other words, all your handles could be signaled and the application will keep blocking until you nudge the mouse! Ugh.

We’ve toyed with some workarounds for this case. For example, you could imagine spinning up an MTA thread and having it perform the blocking operation on the handles. When all the handles are signaled, it could set another event. The STA thread would do a WaitHandle.WaitOne on that other event. This gives us the desired behavior that the STA thread wakes up when all handles are signaled, and it still pumps the message queue. However, if any of those handles are “thread-owned”, like a Mutex, then we have broken the semantics. Our sacrificial MTA thread now owns the Mutex, rather than the STA thread.

Another technique would be to put the STA thread into a loop. Each iteration would ping the handles with a brief timeout to see if it could acquire them. Then it would check the message queue with a PeekMessage or similar technique, and then iterate. This is a terrible solution for battery-powered devices or for Terminal Server scenarios. What used to be efficient blocking is now busily spinning in a loop. And if no messages actually arrive, we have disturbed the fairness guarantees of the OS blocking primitives by pinging.

A final technique would be to acquire the handles one by one, using WaitOne. This is probably the worst approach of all. The semantics of an OS WAIT_ALL are that you will either get no handles or you will get all of them. This is critical to avoiding deadlocks, if different parts of the application block on the same set of handles – but fill the array of handles in random order.

I keep saying that managed blocking will perform “some pumping” when called on an STA thread. Wouldn’t it be great to know exactly what will get pumped? Unfortunately, pumping is a black art which is beyond mortal comprehension. On Win2000 and up, we simply delegate to OLE32’s CoWaitForMultipleHandles service. And before we wrote the initial cut of our pumping code for NT4 and Win9X, I thought I would glance through CoWaitForMultipleHandles to see how it is done. It is many, many pages of complex code. And it uses special flags and APIs that aren’t even available on Win9X.

The code we finally wrote for the downlevel platforms is relatively simple. We gather the list of hidden OLE windows associated with the current STA thread and try to restrict our pumping to the COM calls which travel through them. However, a lot of the pumping complexity is in USER32 services like PeekMessage. Did you know that calling PeekMessage for one window will actually cause SendMessages to be dispatched on other windows belonging to the same thread? This is another example of how someone made a tradeoff between reentrancy and deadlocks. In this case, the tradeoff was made in favor of reentrancy by someone inside USER32.

By now you may be thinking “Okay. Pump more and I get reentrancy. Pump less and I get deadlocks.” But of course the world is more complicated than that. For instance, the Finalizer thread may synchronously call into the main GUI STA thread, perhaps to release a pUnk there, as we have seen. The causality from the Finalizer thread to the main GUI STA thread is invisible to the CLR (though the CLR Security Lead recently suggested using OLE channel hooks as a technique for making this causality visible). If the main GUI STA thread now calls GC.WaitForPendingFinalizers in order to pump, there’s a possibility of a deadlock. That’s because the GUI STA thread must wait for the Finalizer thread to drain its queue. But the Finalizer thread cannot drain its queue until the GUI thread has serviced its incoming synchronous call from the Finalizer.

Reentrancy, Avalon, Longhorn and the Client

Ah, reentrancy again. From time to time, customers inside or outside the company discover that we are pumping messages during managed blocking on an STA. This is a legitimate concern, because they know that it’s very hard to write code that’s robust in the face of reentrancy. In fact, one internal team completely avoids managed blocking, including almost any use of FX, for this reason.

Avalon was very upset, too. I’m not sure how much detail they have disclosed about their threading model. And it’s certainly not my place to reveal what they are doing. Suffice it to say that their model is an explicit rental model that does not presume thread affinity. If you’ve read this far, I’m sure you approve of their decision.

Avalon must necessarily coexist with STAs, but Avalon doesn’t want to require them. The CLR and Avalon have a shared long term goal of driving STAs out of the platform. But, realistically, this will take decades. Avalon’s shorter term goal is to allow some useful GUI applications to be written without STAs. Even this is quite difficult. If you call the clipboard today, you will have an STA.

Avalon also has made a conscious design choice to favor deadlocks over reentrancy. In my opinion, this is an excellent goal. Deadlocks are easily debugged. Reentrancy is almost impossible to debug. Instead, it results in odd inconsistencies that manifest over time.

In order to achieve their design goals, Avalon requires the ability to control the CLR’s pumping. And since we’ve had similar requests from other teams inside and outside the company, this is a reasonable feature for us to provide.

V1 of the CLR had a conscious goal of making as much legacy VB and C++ code work as was possible. When we saw the number of applications that failed to pump, we had no choice but to insert pumping for them – even at the cost of reentrancy. Avalon is in a completely different position. All Avalon code is new code. They are in a great position to define an explicit model for pumping, and then require that all new applications rigorously conform to that model.

Indeed, as much as I dislike STAs, I have a bigger concern about Longhorn and its client focus. Historically, Microsoft has built a ton of great functionality and added it to the platform. But that functionality is often mixed up with various client assumptions. STAs are probably the biggest of those assumptions. The Shell is an example of this. It started out as a user-focused set of services, like the namespace. But it’s growing into something that’s far more generally useful. To the extent that the Shell wants to take its core concepts and make them part of the base managed Longhorn platform, it needs to shed the client focus. The same is true of Office.

For instance, I want to write some code that navigates to a particular document through some namespace and then processes it in some manner. And I want that exact same code to run correctly on the client and on the server. On the client, my processing of that document should not make the UI unresponsive. On the server, my processing of that document should not cause problems with scalability or throughput.

Historically, this just hasn’t been the case. We have an opportunity to correct this problem once, with the major rearchitecture that is Longhorn. But although Longhorn will have both client and server releases, I worry that we might still have a dangerous emphasis on the client.

This may be one of the biggest risks we face in Longhorn.

Winding Down

Finally, I feel a little bad about picking something I don’t like and writing about it. But there’s a reason that this topic came up. Last week, a customer in Japan was struggling with using mshtml.dll to crack some HTML files from inside ASP.NET. It’s the obvious thing to do. Clearly ‘mshtml’ stands for Microsoft HTML and clearly this is how we expect customers to process files in this format.

Unfortunately, MSHTML was written as client-side functionality. In fact, I’m told that it drives its own initialization by posting Windows messages back to itself and waiting for them to be pumped. So if you aren’t pumping an STA, you aren’t going to get very far.

There’s that disturbing historical trend at Microsoft to combine generally useful functionality with a client bias again!

We explained to the customer the risks of using client components on a server, and the pumping behavior that is inherent in managed blocking on an STA. After we had been through all the grisly details, the customer made the natural observation: None of this is written down anywhere.

Well, I still never talked about a mysterious new flag to CoWaitForMultipleHandles. Or how custom implementations of IMessageFilter can cause problems. Or the difference between Main and Single. Or the relationship between apartments and COM+ contexts and ServicedComponents. Or the amazing discovery that OLE32 sometimes requires you to pump the MTA if you have DCOM installed on Win9X.

But I’m sure that at this point I’ve said far more than most people care to hear about this subject.

Access to old blogs

cbrumme — Mon, 10 Nov 2003 20:26:00 GMT

By default, old blogs are truncated from this web site. If you want to read old entries that have scrolled off, go to the CATEGORIES section at the right hand side of the web page. Select CLR (rss) and you'll see the full list.

The PDC and Application Compatibility, but still no Hosting

cbrumme — Mon, 10 Nov 2003 20:00:00 GMT

The PDC has happened, which means two things. I can post some of my (slightly self-censored) reactions to the show, and I can talk about what we ve disclosed about Whidbey and Longhorn more freely. In this particular case, I had promised to talk about the deep changes we re making in Whidbey to allow you to host the CLR in your process. As you ll see, I got side tracked and ended up discussing Application Compatibility instead.

But first, my impressions of the PDC:

The first keynote, with Bill, Jim & Longhorn, was guaranteed to be good. It had all the coolness of Avalon, WinFS and Indigo, so of course it was impressive. In fact, throughout all the sessions I attended, I was surprised by the apparent polish and maturity of Longhorn. In my opinion, Avalon looked like it is the most mature and settled. Indigo also looked surprisingly real. WinFS looked good in the keynote, where it was all about the justification for the technology. But in the drill-down sessions, I had the sense that it s not as far along as the others.

Hopefully all the attendees realize that Longhorn is still a long way off. It s hard to see from the demos, but a lot of fundamental design issues and huge missing pieces remain.

Incidentally, I still can t believe that we picked WinFX to describe the extended managed frameworks and WinFS to describe the new storage system. One of those names has got to go.

I was worried that the Whidbey keynote on Tuesday would appear mundane and old-fashioned by comparison. But to an audience of developers, Eric's keynote looked very good indeed. Visual Studio looked better than I've ever seen it. The device app was so easy to write that I feel I could build a FedEx-style package tracking application in a weekend. The high point of this keynote was ASP.NET. I hadn't been paying attention to what they've done recently, so I was blown away by the personalization system and by the user-customizable web pages. If I had seen a site like that, I would have assumed the author spent weeks getting it to work properly. It s hard to believe this can all be done with drag-and-drop.

In V1, ASP.NET hit a home run by focusing like a laser beam on the developer experience. Everyone put so much effort into building apps, questioning why each step was necessary, and refining the process. It's great to see that they continue to follow that same discipline. In the drill-down sessions, over and over again I saw that focus resulting in a near perfect experience for developers. There are some other teams, like Avalon, that seem to have a similar religion and are obtaining similar results. (Though Avalon desperately needs some tools support. Notepad is fine for authoring XAML in demos, but I wouldn t want to build a real application this way).

Compared to ASP.NET, some other teams at Microsoft are still living in the Stone Age. Those teams are still on a traditional cycle of building features, waiting for customers to build applications with those features, and then incorporating any feedback. Beta is way too late to find out that the programming model is clumsy. We shouldn t be shirking our design responsibilities like this.

Anyway, the 3rd keynote (from Rick Rashid & Microsoft Research) should have pulled it all together. I think the clear message should have been something like:

Whidbey is coming next and has great developer features. After that, Longhorn will arrive and will change everything. Fortunately, Microsoft Research is looking 10+ years out, so you can be sure we will increasingly drive the whole industry.

This should have been an easy story to tell. The fact is that MSR is a world class research institution. Browse the Projects, Topics or People categories at http://research.microsoft.com and you ll see many name brand researchers like Butler Lampson and Jim Gray. You will see tremendous breadth on the areas under research, from pure math and algorithms to speech, graphics and natural language. There are even some esoterica like nanotech and quantum computing. We should have used the number of published papers and other measurements to compare MSR with other research groups in the software industry, and with major research universities. And then we should have shown some whiz-bang demos of about 2 minutes each.

Unfortunately, I think instead we sent a message that Interesting technology comes from Microsoft product groups, while MSR is largely irrelevant. Yet nothing could be further from the truth. Even if I restrict consideration to the CLR, MSR has had a big impact. Generics is one of the biggest feature added to the CLR, C# or the base Frameworks in Whidbey. This feature was added to the CLR by MSR team members, who now know at least as much about our code base as we do. All the CLR s plans for significantly improved code quality and portable compilers depend on a joint venture between MSR and the compiler teams. To my knowledge, MSR has used the CLR to experiment with fun things like transparent distribution, reorganizing objects based on locality, techniques for avoiding security stack crawls, interesting approaches to concurrency, and more. SPOT (Smart Object Personal Technology) is a wonderful example of what MSR has done with the CLR s basic IL and metadata design, eventually leading to a very cool product.

In my opinion, Microsoft Research strikes a great balance between long term speculative experimentation and medium term product-oriented improvements. I wish this had come across better at the PDC.

Trends

In the 6+ years I ve been at Microsoft, we ve had 4 PDCs. This is the first one I ve actually attended, because I usually have overdue work items or too many bugs. (I ve missed all 6 of our mandatory company meetings for the same reason). So I really don t have a basis for comparison.

I guess I had expected to be beaten up about all the security issues of the last year, like Slammer and Blaster. And I had expected developers to be interested in all aspects of security. Instead, the only times the topic came up in my discussions is when I raised it.

However, some of my co-workers did see a distinct change in the level of interest in security. For example, Sebastian Lange and Ivan Medvedev gave a talk on managed security to an audience of 700-800. They reported a real upswing in awareness and knowledge on the part of all PDC attendees.

But consider a talk I attended on Application Compatibility. At a time when most talks were overflowing into the hallways, this talk filled less than 50 seats of a 500 to 1000 seat meeting room. I know that AppCompat is critically important to IT. And it s a source of friction for the entire industry, since everyone is reluctant to upgrade for fear of breaking something. But for most developers this is all so boring compared to the cool visual effects we can achieve with a few lines of XAML.

Despite a trend to increased interest in security on the part of developers, I suspect that security remains more of an IT operations concern than it does a developer concern. And although the events of the last year or two have got more developers excited about security (including me!), I doubt that we will ever get developers excited about more mundane topics like versioning, admin or compatibility. This latter stuff is dead boring.

That doesn t mean that the industry is doomed. Instead, it means that modern applications must obtain strong versioning, compatibility and security guarantees by default rather than through deep developer involvement. Fortunately, this is entirely in keeping with our long term goals for managed code.

With the first release of the CLR, the guarantees for managed applications were quite limited. We guaranteed memory safety through an accurate garbage collector, type safety through verification, binding safety through strong names, and security through CAS. (However, I think we would all agree that our current support for CAS still involves far too much developer effort and not enough automated guarantees. Our security team has some great long-term ideas for addressing this.)

More importantly, we expressed programs through metadata and IL, so that we could expand the benefits of reasoning about these programs over time. And we provided metadata extensibility in the form of Custom Attributes and Custom Signature Modifiers, so that others could add to the capabilities of the managed environment without depending on the CLR team s schedule.

FxCop (http://www.gotdotnet.com/team/fxcop/) is an obvious example of how we can benefit from this ability to reason about programs. All teams developing managed code at Microsoft are religious about incorporating this tool into their build process. And since FxCop supports adding custom rules, we have added a large number of Microsoft-specific or product-specific checks.

Churn and Application Breakage

We also have some internal tools that allow us to compare different versions of assemblies so we can discover inadvertent breaking changes. Frankly, these tools are still maturing. Even in the Everett timeframe, they did a good job of blatant violations like the removal of a public method from a class or addition of a method to an interface. But they didn t catch changes in serialization format, or changes to representation after marshaling through PInvoke or COM Interop. As a result, we shipped some unintentional breaking changes in Everett , and until recently we were on a path to do so again in Whidbey.

As far as I know, these tools still don t track changes to CAS constructs, internal dependency graphs, thread-safety expectations, exception flow (including a static replacement for the checked exceptions feature), reliability contracts, or other aspects of execution. Some of these checks will probably be added over time, perhaps by adding additional metadata to assemblies to reveal the developer s intentions and to make automated validation more tractable. Other checks seem like research projects or are more appropriate for dynamic tools rather than static tools. It s very encouraging to see teams inside and outside of Microsoft working on this.

I expect that all developers will eventually have access to these or similar tools from Microsoft or 3^rd parties, which can be incorporated into our build processes the way FxCop has been.

Sometimes applications break when their dependencies are upgraded to new versions. The classic example of this is Win95 applications which broke when the operating system was upgraded to WinXP. Sometimes this is because the new versions have made breaking changes to APIs. But sometimes it s because things are just different . The classic case here is where a test case runs perfectly on a developer s machine, but fails intermittently in the test lab or out in the field. The difference in environment might be obvious, like a single processor box vs. an 8-way. Yet all too often it s something truly subtle, like a DLL relocating when it misses its preferred address, or the order of DllMain notifications on a DLL_THREAD_ATTACH. In those cases, the change in environment is not the culprit. Instead, the environmental change has finally revealed an underlying bug or fragility in the application that may have been lying dormant for years.

The managed environment eliminates a number of common fragilities, like the double-free of memory blocks or the use of a file handle or Event that has already been closed. But it certainly doesn t guarantee that a multi-threaded program which appears to run correctly on a single processor will also execute without race conditions on a 32-way NUMA box. The author of the program must use techniques like code reviews, proof tools and stress testing to ensure that his code is thread-safe.

The situation that worries me the most is when an application relies on accidents of current FX and CLR implementations. These dependencies can be exceedingly subtle.

Here are some examples of breakage that we have encountered, listed in the random order they occur to me:

Between V1.1 and Whidbey, the implementation of reflection has undergone a major overhaul to improve access times and memory footprint. One consequence is that the order of members returned from APIs like Type.GetMethods has changed. The old order was never documented or guaranteed, but we ve found programs including our own tests which assumed stability here.

Structs and classes can specify Sequential, Explicit or AutoLayout. In the case of AutoLayout, the CLR is free to place members in any order it chooses. Except for alignment packing and the way we chunk our GC references, our layout here is currently quite predictable. But in the future we hope to use access patterns to guide our layout for increased locality. Any applications that predict the layout of AutoLayout structs and classes via unsafe coding techniques are at risk if we pursue that optimization.

Today, finalization occurs on a single Finalizer thread. For scalability and robustness reasons, this is likely to change at some point. Also, the GC already perturbs the order of finalization. For instance, a collection can cause a generation boundary to intervene between two instances that are normally allocated consecutively. Within a given process run, there will likely be some variation in finalization sequence. But for two objects that are allocated consecutively by a single thread, there is a high likelihood of predictable ordering. And we all know how easy it is to make assumptions about this sort of thing in our code.

In an earlier blog (http://blogs.gotdotnet.com/cbrumme/PermaLink.aspx/e55664b4-6471-48b9-b360-f0fa27ab6cc0), I talked about some of the circumstances that impact when the JIT will stop reporting a reference to the GC. These include inlining decisions, register allocation, and obvious differences like X86 vs. AMD64 vs. IA64. Clearly we want the freedom to chase better code quality with JIT compilers and NGEN compilers in ways that will substantially change these factors. Just yesterday an internal team reported a GC bug on multi-processor machines only that we quickly traced to confusion over lifetime rules and bad practice in the application. One finalizable object was accessing some state in another finalizable object, in the expectation that the first object was live because it was the this argument of an active method call.

During V1.1 Beta testing, a customer complained about an application we had broken. This application contained unmanaged code that reached back into its caller s stack to retrieve a GCHandle value at an offset that had been empirically discovered. The unmanaged code then transitioned into managed and redeemed the supposed handle value for the object it referenced. This usually worked, though it was clearly dependent on filthy implementation details. Unfortunately, the System.EnterpriseServices pathways leading to the unmanaged application were somewhat variable. Under certain circumstances, the stack was not what the unmanaged code predicted. In V1, the value at the predicted spot was always a 0 and the redemption attempt failed cleanly. In V1.1, the value at that stack location was an unrelated garbage value. The consequence was a crash inside mscorwks.dll and Fail Fast termination of the process.

In V1 and V1.1, Object.GetHashCode() can be used to obtain a hashcode for any object. However, our implementation happened to return values which tended to be small ascending integers. Furthermore, these values happened to be unique across all reachable instances that were hashed in this manner. In other words, these values were really object identifiers or OIDs. Unfortunately, this implementation was a scalability killer for server applications running on multi-processor boxes. So in Whidbey Object.GetHashCode() is now all we ever promised it would be: an integer with reasonable distribution but no uniqueness guarantees. It s a great value for use in HashTables, but it s sure to disappoint some existing managed applications that relied on uniqueness.

In V1 and V1.1, all string literals are Interned as described in http://blogs.gotdotnet.com/cbrumme/PermaLink.aspx/7943b9be-cca9-41e1-8a83-3d7a0dbba270. I noted there that it is a mistake to depend on Interning across assemblies. That s because the other assembly might start to compose a String value which it originally specified as a literal. In Whidbey, assemblies can opt-in or opt-out of our Interning behavior. This new freedom is motivated by a desire to support faster loading of assemblies (particularly assemblies that have been NGEN ed). We ve seen some tests fail as a result.

I ve seen some external developers use a very fragile technique based on their examination of Rotor sources. They navigate through one of System.Threading.Thread s private fields (DONT_USE_InternalThread) to an internal unmanaged CLR data structure that represents a running managed thread. From there, they can pluck interesting information like the Thread::ThreadState bit field. None of these data structures are part of our contract with managed applications and all of them are sure to change in future releases. The only reason the ThreadState field is at a stable offset in our internal Thread struct today is that its frequency of access merits putting it near the top of the struct for good cache-line filling behavior.

Reflection allows highly privileged code to access private members of arbitrary types. I am aware of dozens of teams inside and outside of Microsoft which rely on this mechanism for shipping products. Some of these uses are entirely justified, like the way Serialization accesses private state that the type author marked as [Serializable()]. Many other uses rather questionable, and a few are truly heinous. Taken to the extreme, this technique converts every internal implementation detail into a publicly exposed API, with the obvious consequences for evolution and application compatibility.

Assembly loading and type resolution can happen on very different schedules, depending on how your application is running. We ve seen applications that misbehave based on NGEN vs. JIT, domain-neutral vs. per-domain loading, and the degree to which the JIT inlines methods. For example, one application created an AppDomain and started running code in it. That code subsequently modified the private application directory and then attempted to load an assembly from that directory. Of course, because of inlining the JIT had already attempted to load the assembly with the original application directory and had failed. The correct solution here is to disallow any changes to an AppDomain s application directory after code starts executing inside that AppDomain. This directory should only be modifiable during the initialization of the AppDomain.

In prior blogs, I ve talked about unhandled exceptions and the CLR s default policy for dealing with them. That policy is quite involved and hard to defend. One aspect of it is that exceptions that escape the Finalizer thread or any ThreadPool threads are swallowed. This keeps the process running, but it often leaves the application in an inconsistent state. For example, locks may not have been released by the thread that took the exception, leading to subsequent hangs. Now that the technology for reporting process crashes via Watson dumps is maturing, we really want to change our default policy for unhandled exceptions so that we Fail Fast with a process crash and a Watson upload. However, any change to this policy will undoubtedly cause many existing applications to stop working.

Despite the flexibility of CAS, most applications still run with Full Trust. I truly believe that this will change over time. For example, in Whidbey we will have ClickOnce permission elevation and in Longhorn we will deliver the Secure Execution Environment or SEE. Both of these features were discussed at the PDC. When we have substantial code executing in partial trust, we re going to see some unfortunate surprises. For example, consider message pumping. If a Single Threaded Apartment thread has some partial trust code on its stack when it blocks (e.g. Monitor.Enter on a contentious monitor), then we will pump messages on that thread while it is blocked. If the dispatching of a message requires a stack walk to satisfy a security Full Demand, then the partially trusted code further back on the stack may trigger a security exception. Another example is related to class constructors. As you probably know, .cctor methods execute on the first thread that needs access to a class in a particular AppDomain. If the .cctor must satisfy a security demand, the success of the .cctor now depends on the accident of what other code is active on the thread s stack. Along the same lines, the .cctor method may fail if there is insufficient stack space left on the thread that happens to execute it. These are all well understood problems and we have plans for fixing them. But the fixes will necessarily change observable behavior for a class of applications.

I could fill a lot more pages with this sort of list. And our platform is still in its infancy. Anyway, one clear message from all this is that things will change and then applications will break.

But can we categorize these failures and make some sense of it all? For each failure, we need to decide whether the platform or the application is at fault for each case. And then we need to identify some rules or mechanisms that can avoid these failures or mitigate them. I see four categories.

Category 1: The application explicitly screws itself

The easiest category to dispense with is the one where a developer intentionally and explicitly takes advantage of a behavior that s/he knows is guaranteed to change. A perfect example of this is #8 above. Anyone who navigates through private members to unmanaged internal data structures is setting himself up for problems in future versions. The responsibility (or irresponsibility in this case) lies with the application. In my opinion, the platform should have no obligations.

But consider #5 above. It s clearly in this same category, and yet opinions on our larger team were quite divided on whether we needed to fix the problem. I spoke to a number of people who definitely understood the incredible difficulty of keeping this application running on new versions of the CLR and EnterpriseServices. But they consistently argued that the operating system has traditionally held itself to this sort of compatibility bar, that this is one of the reasons for Windows ubiquity, and that the managed platform must similarly step up.

Also, we have to be realistic here. If a customer issue like this involves one of our largest accounts, or has been escalated through a very senior executive (a surprising number seem to reach Steve Ballmer), then we re going to pull out all the stops on a fix or a temporary workaround.

In many cases, our side-by-side support is an adequate and simple solution. Customers can continue to run problematic applications on their old bits, even though a new version of these bits has also been installed. For instance, the config file for an application can specify an old version of the CLR. Or binding redirects could roll back a specific assembly. But this technique falls apart if the application is actually an add-in that is dynamically loaded into a process like Internet Explorer or SQL Server. It s unrealistic to lock back the entire managed stack inside Internet Explorer (possibly preventing newer applications that use generics or other Whidbey features from running there), just so older questionable applications can keep running.

It s possible that we could provide lock back at finer-grained scopes than the process scope in future versions of the CLR. Indeed, this is one of the areas being explored by our versioning team.

Anyway, if we were under sufficient pressure I could imagine us building a one-time QFE (patch) for an important customer in this category, to help them transition to a newer version and more maintainable programming techniques. But if you aren t a Fortune 100 company or Steve Ballmer s brother-in-law, I personally hope we would be allowed to ignore any of your applications that are in this category.

Category 2: The platform explicitly screws the application

I would put #6, #7 and #11 above in a separate category. Here, the platform team wants to make an intentional breaking change for some valid reason like performance or reliability. In fact, #10 above is a very special case of this category. In #10, we would like to break compatibility in Whidbey so that we can provide a stronger model that can avoid subsequent compatibility breakage. It s a paradoxical notion that we should break compatibility now so we can increase future compatibility, but the approach really is sensible.

Anyway, if the platform makes a conscious decision to break compatibility to achieve some greater goal, then the platform is responsible for mitigation. At a minimum, we should provide a way for broken applications to obtain the old behavior, at least for some transition period. We have a few choices in how to do this, and we re likely to pick one based on engineering feasibility, the impact of a breakage, the likelihood of a breakage, and schedule pressure:

Rely on side-by-side and explicit administrator intervention. In other words, the admin notices the application no longer works after a platform upgrade, so s/he authors a config file to lock the application back to the old platform bits. This approach is problematic because it requires a human being to diagnose a problem and intervene. Also, it has the problems I already mentioned with using side-by-side on processes like Internet Explorer or SQL Server.

For some changes, it shouldn t be necessary to lock back the entire platform stack. Indeed, for many changes the platform could simultaneously support the old and new behaviors. If we change our default policy for dealing with unhandled exceptions, we should definitely retain the old policy& at least for one release cycle.

If we expect a significant percentage of applications to break when we make a change, we should consider an opt-in policy for that change. This eliminates the breakage and the human involvement in a fix. In the case of String Interning, we require each assembly to opt-in to the new non-intern ed behavior.

In some cases, we ve toyed with the idea of having the opt-in be implicit with a recompile. The logic here is that when an application is recompiled against new platform bits, it is presumably also tested against those new bits. The developer, rather than the admin, will deal with any compatibility issues that arise. We re well set up for this, since managed assemblies contain metadata giving us the version numbers of the CLR and the dependent assemblies they were compiled against. Unfortunately, execution models like ASP.NET work against us here. As you know, ASP.NET pages are recompiled automatically by the system based on dependency changes. There is no developer available when this happens.

Windows Shimming

Before we look at the next two categories of AppCompat failure, it s worth taking a very quick look at one of the techniques that the operating system has traditionally used to deal with these issues. Windows has an AppCompat team which has built something called a shimming engine.

Consider what happened when the company tried to move consumers from Win95/Win98/WinMe over to WinXP. They discovered a large number of programs which used the GetVersion or the preferred GetVersionEx APIs in such a way that the programs refused to run on NT-based systems.

In fact, WinXP did such a good job of achieving compatibility with Win9X systems that in many cases the only reason the application wouldn t run was the version check that the program made at start up. The fix was to change GetVersion or GetVersionEx to lie about the version number of the current operating system. Of course, this lie should only be told to programs that need the lie in order to work properly.

I ve heard that this shim which lies about the operating system version is the most commonly applied shim we have. As I understand it, at process launch the shimming engine tries to match the current process against any entries in its database. This match could be based on the name, timestamp or size of the EXE, or of other files found relative to that EXE like a BMP for the splash screen in a subdirectory. The entry in the database lists any shims that should be applied to the process, like the one that lies about the version. The shimming engine typically bashes the IAT (import address table) of a DLL or EXE in the process, so that its imports are bound to the shim rather than to the normal export (e.g. Kernel32!GetVersionEx). In addition, the shimming engine has other tricks it perform less frequently, like wrapping COM objects up with intercepting proxies.

It s easy to see how this infrastructure can allow applications for Win95 to execute on WinXP. However, this approach has some drawbacks. First, it s rather labor-intensive. Someone has to debug the application, determine which shims will fix it, and then craft some suitable matching criteria that will identify this application in the shimming database. If an appropriate shim doesn t already exist, it must be built.

In the best case, the application has some commercial significance and Microsoft has done all the testing and shimming. But if the application is a line of business application that was created in a particular company s IT department, Microsoft will never get its hands on it. I ve heard we re now allowing sophisticated IT departments to set up their own shimming databases for their own applications but this only allows them to apply existing shims to their applications.

And from my skewed point of view the worst part of all this is that it really won t work for managed applications. For managed apps, binding is achieved through strong names, Fusion and the CLR loader. Binding is practically never achieved through DLL imports.

So it s instructive to look at some of the techniques the operating system has traditionally used. But those techniques don t necessarily apply directly to our new problems.

Anyway, back to our categories&

Category 3: The application accidentally screws itself

Category 4: The platform accidentally screws the application

Frankly, I m having trouble distinguishing these two cases. They are clearly distinct categories, but it s a judgment call where to draw the line. The common theme here is that the platform has accidentally exposed some consistent behavior which is not actually a guaranteed contract. The application implicitly acquires a dependency on this consistent behavior, and is broken when the consistency is later lost.

In the nirvana of some future fully managed execution environment, the platform and tools would never expose consistent behavior unless it was part of a guarantee. Let s look at some examples and see how practical this is.

In example #1 above, reflection used to deliver members in a stable order. In Whidbey, that order changes. In hindsight, there s a simple solution here. V1 of the product could have contained a testing mode that randomized the returned order. This would have exposed the developer to our actual guarantees, rather than to a stronger accidental consistency. Within the CLR, we ve used this sort of technique to force us down code paths that otherwise wouldn t be exercised. For example, developers on the CLR team all use NT-based (Unicode) systems and avoid Win9X (Ansi) systems. So our Win9X Ansi/Unicode wrappers wouldn t typically get tested by developers. To address this, our checked/debug CLR build originally considered the day of the week and used Ansi code paths every other day. But imagine chasing a bug at 11:55 PM . When the bug magically disappears on your next run at 1:03 AM the next morning, you are far too frazzled to think clearly about the reason. Today, we tend to use low order bits in the size of an image like mscorwks.dll or the assembly being tested, so our randomization is now more friendly to testing.

In example #2 above, you could imagine a similar perturbation on our AutoLayout algorithms when executing a debug version of an application, or when launched from inside a tool like Visual Studio.

For example #4, the CLR already has internal stress modes that force different and aggressive GC schedules. These can guarantee compaction to increase the likelihood of detecting stale references. They can perform extensive checks of the integrity of the heap, to ensure that the write barrier and other mechanisms are effective. And they can ensure that every instruction of JITted managed code that can synchronize with the GC will synchronize with the GC. I suspect that these modes would do a partial job of eradicating assumptions about lifetimes reported by the JIT. However, we will remain exposed to significantly different code generators (like Rotor s FJIT) or execution on significantly different architectures (like CPUs with dramatically more registers).

In contrast with the above difficulty, it s easy to imagine adding a new GC stress mode that perturbs the finalization queues, to uncover any hidden assumptions about finalization order. This would address example #3.

Customer Debug Probes, AppVerifier and other tools

It turns out that the CLR already has a partial mechanism for enabling perturbation during testing and removing it on deployed applications. This mechanism is the Customer Debug Probes feature that we shipped in V1.1. Adam Nathan s excellent blog site has a series of articles on CDPs, which are collected together at http://blogs.gotdotnet.com/anathan/CategoryView.aspx/Debugging. The original goal of CDPs was to counteract the black box nature of debugging certain failures of managed applications, like corruptions of the GC heap or crashes due to incorrect marshaling directives. These probes can automatically diagnose common application errors, like failing to keep a marshaled delegate rooted so it won t be collected. This approach is so much easier than wading through dynamically generated code without symbols, because we tell you exactly where your bugs are. But we re now realizing that we can also use CDPs to increase the future compatibility of managed applications if we can perturb current behavior that is likely to change in the future.

Unfortunately, example #6 from above reveals a major drawback with the technique of perturbation. When we built the original implementation of Object.GetHashCode, we simply never considered the difference between what we wanted to guarantee (hashing) and what we actually delivered (OIDs). In hindsight, it is obvious. But I m not convinced that we aren t falling into similar traps in our new features. We might be a little smarter than we were five years ago, but only a little.

Example #10 worries me for similar reasons. I just don t think we were smart enough to predict that changing the binding configuration of an AppDomain after starting to execute code in that AppDomain would be so fragile. When a developer delivers a feature, s/he needs to consider security, thread-safety, programming model, key invariants of the code base like GC reporting, correctness, and so many other aspects. It would be amazing if a developer consistently nailed each of these aspects for every new feature. We re kidding ourselves if we think that evolution and unintentional implicit contracts will get adequate developer attention on every new feature.

Even if we had perfect foresight and sufficient resources to add perturbation for all operations, we would still have a major problem. We can t necessarily rely on 3^rd party developers to test their applications with perturbation enabled. Consider the unmanaged AppVerifier experience.

The operating system has traditionally offered a dynamic testing tool called AppVerifier which can diagnose many common unmanaged application bugs. For example, thanks to uploads of Watson process dumps from the field, most unmanaged application crashes can now be attributed to incorrect usage of dynamically allocated memory. Yet AppVerifier can use techniques like placing each allocation in its own page or leaving pages unmapped after release, to deterministically catch overruns, double frees, and reads or writes of freed memory.

In other words, there is hard evidence that if every unmanaged application had just used the memory checking support of AppVerifier, then two out of every three application crashes would be eliminated. Clearly this didn t happen.

Of course, AppVerifier can diagnose far more than just memory problems. And it s very easy and convenient to use.

Since testing with AppVerifier is part of the Windows Logo compliance program, you would expect that it s used fairly rigorously by ISVs. And, given its utility, you would expect that most IT organizations would use this tool for their internal applications. Unfortunately, this isn t the case. Many applications submitted for the Windows Logo actually fail to launch under AppVerifier. In other words, they violate at least one of the rules before they finish initializing.

The Windows AppCompat team recognizes that proactive tools like AppVerifier are so much better than reactive mitigation like shimming broken applications out in the field. That s why they made the AppVerifier tool a major focus of their poorly attended Application Compatibility talk that I sat in on at the PDC. (Aha! I really was going somewhere with all this.)

There s got to be a reason why developers don t use such a valuable tool. In my opinion, the reason is that AppVerifier is not integrated into Visual Studio. If the Debug Properties in VS allowed you to enable AppVerifier and CDP checks, we would have much better uptake. And if an integrated project system and test system could monitor code coverage numbers, and suggest particular test runs with particular probes enabled, we would be approaching nirvana.

Winding Down

Looking at development within Microsoft, one trend is very clear: Automated tools and processes are a wonderful supplement for human developers. Whether we re talking about security, reliability, performance, application compatibility or any other measure of software quality, we re now seeing that static and dynamic analysis tools can give us guarantees that we will never obtain from human beings. Bill Gates touched on this during his PDC keynote, when he described our new tools for statically verifying device driver correctness, for some definition of correctness.

This trend was very clear to me during the weeks I spent on the DCOM / RPCSS security fire drill. I spent days looking at some clever marshaling code, eventually satisfying myself that it worked perfectly. Then someone else wrote an automated attacker and discovered real flaws in just a few hours. Other architects and senior developers scrutinized different sections of the code. Then some researchers from MSR who are focused on automatic program validation ran their latest tools over the same code and gave us step-by-step execution models that led up to crashes. Towards the end of the fire drill, a virtuous cycle was established. The code reviewers noticed new categories of vulnerabilities. Then the researchers tried to evolve their tools to detect those vulnerabilities. Aspects of this process were very raw, so the tools sometimes produced a great deal of noise in the form of false positives. But it s clear that we were getting real value from Day One and the future potential here is enormous.

One question that always comes up, when we talk about adding significant value to Visual Studio through additional tools, is whether Microsoft should give away these tools. It s a contentious issue, and I find myself going backwards and forwards on it. One school of thought says that we should give away tools to promote the platform and improve all the programs in the Windows ecology. In the case of tools that make our customers applications more secure or more resilient to future changes in the platform, this is a compelling argument. Another school of thought says that Visual Studio is a profit center like any other part of the company, and it needs the freedom to charge what the market will bear.

Given that my job is building a platform, you might expect me to favor giving away Visual Studio. But I actually think the profit motive is a powerful mechanism for making our tools competitive. If Visual Studio doesn t have P&L responsibility, their offering will deteriorate over time. The best way to know whether they ve done all they can to make the best tools possible, is to measure how much their customers are willing to pay. I want Borland to compete with Microsoft on building the best tools at the best price, and I want to be able to measure the results of that competition through revenue and market penetration.

In all this, I have avoided really talking about the issues of versioning. Of course, versioning and application compatibility are enormously intertwined. Applications break for many reasons, but the typical reason is that one component is now binding to a new version of another component. We have a whole team of architects, gathered from around the company, who have been meeting regularly for about a year to grapple with the problems of a complete managed versioning story. Unlike managed AppCompat, the intellectual investment in managed versioning has been enormous.

Anyway, Application Compatibility remains a relatively contentious subject over here. There s no question that it s a hugely important topic which will have a big impact on the longevity of our platform. But we are still trying to develop techniques for achieving compatibility that will be more successful than what Windows has done in the past, without limiting our ability to innovate on what is still a very young execution engine and set of frameworks. I have deliberately avoided talking about what some of those techniques might be, in part because our story remains incomplete.

Also, we won t realize how badly AppCompat will bite us until we can see a lot of deployed applications that are breaking as we upgrade the platform. At that point, it s easier to justify throwing more resources at the problem. But by then the genie is out of the bottle& the deployed applications will already depend on brittle accidents of implementation, so recovery will be painfully breaking. In a world where we are always under intense resource and schedule pressure, the needs of AppCompat must be balanced against performance, security, developer productivity, reliability, innovation and all the other must haves .

You know, I really do want to talk about Hosting. It is a truly fascinating subject. I m much more comfortable talking about non-preemptive fiber scheduling than I am talking about uninteresting topics like implicit contracts and compatibility trends.

But Hosting is going to have to wait at least a few more weeks.

The Exception Model

cbrumme — Wed, 01 Oct 2003 20:18:00 GMT

I had hoped this article would be on changes to the next version of the CLR which allow it to be hosted inside SQL Server and other “challenging” environments. This is more generally interesting than you might think, because it creates an opportunity for other processes (i.e. your processes) to host the CLR with a similar level of integration and control. This includes control over memory usage, synchronization, threading (including fibers), extended security models, assembly storage, and more.

However, that topic is necessarily related to our next release, and I cannot talk about deep details of that next release until those details have been publicly disclosed. In late October, Microsoft is holding its PDC and I expect us to disclose many details at that time. In fact, I’m signed up to be a member of a PDC panel on this topic. If you work on a database or an application server or a similarly complicated product that might benefit from hosting the CLR, you may want to attend.

After we’ve disclosed the hosting changes for our next release, you can expect a blog on hosting in late October or some time in November.

Instead, this blog is on the managed exception model. This is an unusual topic for me. In the past, I’ve picked topics where I can dump information without having to check any of my facts or do any research. But in the case of exceptions I keep finding questions I cannot answer. At the top level, the managed exception model is nice and simple. But – as with everything else in software – the closer you look, the more you discover.

So for the first time I decided to have some CLR experts read my blog entry before I post it. In addition to pointing out a bunch of my errors, all the reviewers were unanimous on one point: I should write shorter blogs.

Of course, we can’t talk about managed exceptions without first considering Windows Structured Exception Handling (SEH). And we also need to look at the C++ exception model. That’s because both managed exceptions and C++ exceptions are implemented on top of the underlying SEH mechanism, and because managed exceptions must interoperate with both SEH and C++ exceptions.

Windows SEH

Since it’s at the base of all exception handling on Windows, let’s look at SEH first. As far as I know, the definitive explanation of SEH is still Matt Pietrek’s excellent 1997 article for Microsoft Systems Journal: http://www.microsoft.com/msj/0197/exception/exception.aspx. There have been some extensions since then, like vectored exception handlers, some security enhancements, and the new mechanisms to support IA64 and AMD64. (It’s hard to base exceptions on FS:[0] chains if your processor doesn’t have an FS segment register). We’ll look at all these changes shortly. But Matt’s 1997 article remains a goldmine of information. In fact, it was very useful to the developers who implemented exceptions in the CLR.

The SEH model is exposed by MSVC via two constructs:

__try {…} __except(filter_expression) {…}
__try {…} __finally {…}

Matt’s article explains how the underlying mechanism of two passes over a chain of single callbacks is used to provide try/except/finally semantics. Briefly, the OS dispatches an exception by retrieving the head of the SEH chain from TLS. Since the head of this chain is at the top of the TIB/TEB (Thread Information Block / Thread Environment Block, depending on the OS and the header file you look at), and since the FS segment register provides fast access to this TLS block on X86, the SEH chain is often called the FS:[0] chain.

Each entry consists of a next or a prev pointer (depending on how you look at it) and a callback function. You can add whatever data you like after that standard entry header. The callback function is called with all sorts of additional information related to the exception that’s being processed. This includes the exception record and the register state of the machine which was captured at the time of the exception.

To implement the 1^st form of MSVC SEH above (__try/__except), the callback evaluates the filter expression during the first pass over the handler chain. As exposed by MSVC, the filter expression can result in one of three legal values:

EXCEPTION_CONTINUE_EXECUTION = -1

EXCEPTION_CONTINUE_SEARCH = false 0

EXCEPTION_EXECUTE_HANDLER = true 1

Of course, the filter could also throw its own exception. That’s not generally desirable, and I’ll discuss that possibility and other flow control issues later.

But if you look at the underlying SEH mechanism, the handler actually returns an EXCEPTION_DISPOSITION:

typedef enum _EXCEPTION_DISPOSITION

{

ExceptionContinueExecution,

ExceptionContinueSearch,

ExceptionNestedException,

ExceptionCollidedUnwind

} EXCEPTION_DISPOSITION;

So there’s some mapping that MSVC is performing here. Part of that mapping is just a trivial conversion between the MSVC filter values and the SEH handler values. For instance ExceptionContinueSearch has the value 1 at the SEH handler level but the equivalent EXCEPTION_CONTINUE_SEARCH has the value 0 at the MSVC filter level. Ouch.

But the other part of the mapping has to do with a difference in functionality. For example, ExceptionNestedException and ExceptionCollidedUnwind are primarily used by the OS dispatch mechanism itself. We’ll see the circumstances in which they arise later. More importantly, MSVC filters can indicate that the __except clause should run by returning EXCEPTION_EXECUTE_HANDLER. But we shall see that at the SEH level this decision is achieved by having the exception dispatch routine fix up the register context and then resuming execution at the right spot.

The EXCEPTION_CONTINUE_EXECUTION case supports a rather esoteric use of SEH. This return value allows the filter to correct the problem that caused the exception and to resume execution at the faulting instruction. For example, an application might be watching to see when segments are being written to so that it can log this information. This could be achieved by marking the segment as ReadOnly and waiting for an exception to occur on first write. Then the filter could use VirtualProtect to change the segment containing the faulting address to ReadWrite and then restart the faulting instruction. Alternatively, the application could have two VirtualAllocs for each region of memory. One of these could be marked as ReadOnly and the second could be a shadow that is marked as ReadWrite. Now the exception filter can simply change the register state of the CPU that faulted, so that the register containing the faulting address is changed from the ReadOnly segment to the shadowed ReadWrite segment.

Obviously anyone who is playing these games must have a lot of sophistication and a deep knowledge of how the program executes. Some of these games work better if you can constrain the code that’s generated by your program to only touch faulting memory using a predictable cliché like offsets from a particular register.

I’ll talk about this kind of restartable or resumable exception in the context of managed code later. For now, let’s pretend that the filter either returns “true – I would like my ‘except’ clause to handle this exception” or “false – my ‘except’ clause is uninterested in this exception”. If the filter returns false, the next SEH handler is fetched from the chain and it is asked this same question.

The OS is pretty paranoid about corrupt stacks during this chain traversal. It checks that all chain entries are within the bounds of the stack. (These bounds are also recorded in the TEB). The OS also checks that all entries are in ascending order on the stack. If you violate these rules, the OS will consider the stack to be corrupt and will be unable to process exceptions. This is one of the reasons that a Win32 application cannot break its stack into multiple disjoint segments as an innovative technique for dealing with stack overflow.

Anyway, eventually a handler says “true – I would like my ‘except’ clause to handle this exception”. That’s because there’s a backstop entry at the end of the chain which is placed there by the OS when the thread is created. This last entry wants to handle all the exceptions, even if your application-level handlers never do. That’s where you get the default OS behavior of consulting the unhandled exception filter list, throwing up dialog boxes for Terminate or Debug, etc.

As soon as a filter indicates that it wants to handle an exception, the first pass of exception handling finishes and the second pass begins. As Matt’s article explains, the handler can use the poorly documented RtlUnwind service to deliver second pass notifications to all the previous handlers and pop them off the handler chain.

In other words, no unwinding happened as the first pass progressed. But during the second pass we see two distinct forms of unwind. The first form involves popping SEH records from the chain that was threaded from TLS. Each such SEH record is popped before the corresponding handler gets called for the second pass. This leaves the SEH chain in a reasonable form for any nested exceptions that might occur within a handler.

The other form of unwind is the actual popping of the CPU stack. This doesn’t happen as eagerly as the popping of the SEH records. On X86, EBP is used as the frame pointer for methods containing SEH. ESP points to the top of the stack, as always. Until the stack is actually unwound, all the handlers are executed on top of the faulting exception frame. So the stack actually grows when a handler is called for the first or second pass. EBP is set to the frame of the method containing a filter or finally clause so that local variables of that method will be in scope.

The actual popping of the stack doesn’t occur until the catching ‘except’ clause is executed.

So we’ve got a handler whose filter announced in the first pass that it would handle this exception via EXCEPTION_EXECUTE_HANDLER. And that handler has driven the second pass by unwinding and delivering all the second pass notifications. Typically it will then fiddle with the register state in the exception context and resume execution at the top of the appropriate ‘except’ clause. This isn’t necessarily the case, and later we’ll see some situations where the exception propagation gets diverted.

How about the try/finally form of SEH? Well, it’s built on the same underlying notion of a chain of callbacks. During the first pass (the one where the filters execute, to decide which except block is going to catch), the finally handlers all say EXCEPTION_CONTINUE_SEARCH. They never actually catch anything. Then in the second pass, they execute their finally blocks.

Subsequent additions to SEH

All of the above – and a lot more – is in Matt’s article. There are a few things that aren’t in his article because they were added to the model later.

For example, Windows XP introduced the notion of a vectored exception handler. This allows the application to register for a first crack at an exception, without having to wait for exception handling to propagate down the stack to an embedded handler. Fortunately, Matt wrote an “Under The Hood” article on this particular topic. This can be found at http://msdn.microsoft.com/msdnmag/issues/01/09/hood/default.aspx.

Another change to SEH is related to security. Buffer overruns – whether on the stack or in heap blocks – remain a favorite attack vector for hackers. A typical buffer overrun attack is to pass a large string as an argument to an API. If that API expected a shorter string, it might have a local on the stack like “char filename[256];”. Now if the API is foolish enough to strcpy a malicious hacker’s argument into that buffer, then the hacker can put some fairly arbitrary data onto the stack at addresses higher (further back on the stack) than that ‘filename’ buffer. If those higher locations are supposed to contain call return addresses, the hacker may be able to get the CPU to transfer execution into the buffer itself. Oops. The hacker is injecting arbitrary code and then executing it, potentially inside someone else’s process or under their security credentials.

There’s a new speed bump that an application can use to reduce the likelihood of a successful stack-based buffer overrun attack. This involves the /GS C++ compiler switch, which uses a cookie check in the function epilog to determine whether a buffer overrun has corrupted the return address before executing a return based on its value.

However, the return address trick is only one way to exploit buffer overruns. We’ve already seen that SEH records are necessarily built on the stack. And in fact the OS actually checks to be sure they are within the stack bounds. Those SEH records contain callback pointers which the OS will invoke if an exception occurs. So another way to exploit a buffer overrun is to rewrite the callback pointer in an SEH record on the stack. There’s a new linker switch (/SAFESEH) that can provide its own speed bump against this sort of attack. Modules built this way declare that all their handlers are embedded in a table in the image; they do not point to arbitrary code sequences sprinkled in the stack or in heap blocks. During exception processing, the exception callbacks can be validated against this table.

Of course, the first and best line of defense against all these attacks is to never overrun a buffer. If you are writing in managed code, this is usually pretty easy. You cannot create a buffer overrun in managed code unless the CLR contains a bug or you perform unsafe operations (e.g. unverifiable MC++ or ‘unsafe’ in C#) or you use high-privilege unsafe APIs like StructureToPtr or the various overloads of Copy in the System.Runtime.InteropServices.Marshal class.

So, not surprisingly and not just for this reason, I recommend writing in managed code. But if you must write some unmanaged code, you should seriously consider using a String abstraction that eliminates all those by-rote opportunities for error. And if you must code each strcpy individually, be sure to use strncpy instead!

A final interesting change to the OS SEH model since Matt’s article is due to Win64. Both IA64 and AMD64 have a model for exception handling that avoids reliance on an explicit handler chain that starts in TLS and is threaded through the stack. Instead, exception handling relies on the fact that on 64-bit systems we can perfectly unwind a stack. And this ability is itself due to the fact that these chips are severely constrained on the calling conventions they support.

If you look at X86, there are an unbounded number of calling conventions possible. Sure, there are a few common well-known conventions like stdcall, cdecl, thiscall and fastcall. But optimizing compilers can invent custom calling conventions based on inter-procedural analysis. And developers writing in assembly language can make novel decisions about which registers to preserve vs. scratch, how to use the floating point stack, how to encode structs into registers, whether to back-propagate results by re-using the stack that contained in-bound arguments, etc. Within the CLR, we have places where we even unbalance the stack by encoding data after a CALL instruction, which is then addressable via the return address. This is a particularly dangerous game because it upsets the branch prediction code of the CPU and can cause prediction misses on several subsequent RET instructions. So we are careful to reserve this technique for low frequency call paths. And we also have some stubs that compute indirect JMPs to out-of-line RET ‘n’ instructions in order to rebalance the stack.

It would be impossible for a stack crawler to successfully unwind these bizarre stacks for exception purposes, without completely simulating arbitrary code execution. So on X86 the exception mechanism must rely on the existence of a chain of crawlable FS:[0] handlers that is explicitly maintained.

Incidentally, the above distinction between perfect stack crawling on 64-bit systems vs. hopeless stack crawling on X86 systems has deeper repercussions for the CLR than just exception handling. The CLR needs the ability to crawl all the managed portions of a thread’s stack on all architectures. This is a requirement for proper enforcement of Code Access Security; for accurate reporting of managed references to the GC; for hijacking return addresses in order to asynchronously take control of threads; and for various other reasons. On X86, the CLR devotes considerable resources to achieving this.

Anyway, on 64-bit systems the correspondence between an activation record on the stack and the exception record that applies to it is not achieved through an FS:[0] chain. Instead, unwinding of the stack reveals the code addresses that correspond to a particular activation record. These instruction pointers of the method are looked up in a table to find out whether there are any __try/__except/__finally clauses that cover these code addresses. This table also indicates how to proceed with the unwind by describing the actions of the method epilog.

Managed Exceptions

Okay, enough about SEH – for now. Let’s switch to the managed exception model. This model contains a number of constructs. Depending on the language you code in, you probably only have access to a subset of these.

try {…} finally {…}

This is pretty standard. All managed languages should expose this, and it should be the most common style of exception handling in user code. Of course, in the case of MC++ the semantics of ‘finally’ is exposed through auto-destructed stack objects rather than through explicit finally clauses. You should be using ‘finally’ clauses to guarantee consistency of application state far more frequently than you use ‘catch’ clauses. That’s because catch clauses increase the likelihood that developers will swallow exceptions that should be handled elsewhere, or perhaps should even be left unhandled. And if catch clauses don’t actually swallow an exception (i.e. they ‘rethrow’), they still create a poor debugging experience as we shall see.

try {…} catch (Object o) {…}

This is pretty standard, too. One thing that might surprise some developers is that you can catch any instance that’s of type Object or derived from Object. However, there is a CLS rule that only subtypes of System.Exception should be thrown. In fact, C# is so eager for you to only deal with System.Exception that it doesn’t provide any access to the thrown object unless you are catching Exception or one of its subtypes.

When you consider that only Exception and its subtypes have support for stack traces, HRESULT mapping, standard access to exception messages, and good support throughout the frameworks, then it’s pretty clear that you should restrict yourself to throwing and processing exceptions that derive from Exception.

In retrospect, perhaps we should have limited exception support to Exception rather than Object. Originally, we wanted the CLR to be a useful execution engine for more run-time libraries than just the .NET Frameworks. We imagined that different languages would execute on the CLR with their own particular run-time libraries. So we didn’t want to couple the base engine operations too tightly with CLS rules and constructs in the frameworks. Of course, now we understand that the commonality of the shared framework classes is a huge part of the value proposition of our managed environment. I suspect we would revisit our original design if we still could.

try {…} catch (Object o) if (expression) {…}

This is invented syntax, though I’m told it’s roughly what MC++ is considering. As far as I know, the only two .NET languages that currently support exception filters are VB.NET and – of course – ILASM. (We never build a managed construct without exposing it via ILDASM and ILASM in a manner that allows these two tools to round-trip between source and binary forms).

VB.NET has sometimes been dismissed as a language that’s exclusively for less sophisticated developers. But the way this language exposes the advanced feature of exception filters is a great example of why that position is too simplistic. Of course, it is true that VB has historically done a superb job of providing an approachable toolset and language, which has allowed less sophisticated developers to be highly productive.

Anyway, isn’t this cool:

Try

…try statements…

Catch e As InvalidOperationException When expressionFilter

…catch statements…

End Try

Of course, at the runtime level we cannot separate the test for the exception type expression and the filter expression. We only support a bare expression. So the VB compiler turns the above catch into something like this, where $exception_obj is the implicit argument passed to the filter.

Catch When (IsInst($exception_obj, InvalidOperationException)

&& expressionFilter)

While we’re on the topic of exception handling in VB, have you ever wondered how VB .NET implements its On Error statement?

On Error { Goto { <line> | 0 | -1 } | Resume Next }

Me neither. But I think it’s pretty obvious how to implement this sort of thing with an interpreter. You wait for something to go wrong, and then you consult the active “On Error” setting. If it tells you to “Resume Next”, you simply scan forwards to the next statement and away you go.

But in an SEH world, it’s a little more complicated. I tried some simple test cases with the VB 7.1 compiler. The resulting codegen is based on advancing a _Vb_t_CurrentStatement local variable to indicate the progression of execution through the statements. A single try/filter/catch covers execution of these statements. It was interesting to see that the ‘On Error’ command only applies to exceptions that derive from System.Exception. The filter refuses to process any other exceptions.

So VB is nicely covered. But what if you did need to use exception filters from C#? Well, in V1 and V1.1, this would be quite difficult. But C# has announced a feature for their next release called anonymous methods. This is a compiler feature that involves no CLR changes. It allows blocks of code to be mentioned inline via a delegate. This relieves the developer from the tedium of defining explicit methods and state objects that can be gathered into the delegate and the explicit sharing of this state. This and other seductive upcoming C# features are described at http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dv_vstechart/html/vbconcprogramminglanguagefuturefeatures.asp.

Using a mechanism like this, someone has pointed out that one could define delegates for try, filter and catch clauses and pass them to a shared chunk of ILASM. I love the way the C# compiler uses type inferencing to automatically deduce the delegate types. And it manufactures a state object to ensure that the locals and arguments of DoTryCatch are available to the “try statements”, “filter expression” and “catch statements”, almost as if everything was scoped in a single method body. (I say “almost” because any locals or arguments that are of byref, argiterator or typedbyref types cannot be disassociated from a stack without breaking safety. So these cases are disallowed).

I’m guessing that access to filters from C# could look something like this:

public void delegate __Try();

public Int32 delegate __Filter();

public void delegate __Catch();

// this reusable helper would be defined in ILASM or VB.NET:

void DoTryCatch(__Try t, __Filter f, __Catch c)

// And C# could then use it as follows:

void m(…arguments…)

{

…locals…

DoTryCatch(

{ …try statements…},

{ return filter_expression; },

{ …catch statements…}

);

}

You may notice that I cheated a little bit. I didn’t provide a way for the ‘catch’ clause to mention the exception type that it is catching. Of course, this could be expressed as part of the filter, but that’s not really playing fair. I suspect the solution is to make DoTryCatch a generic method that has an unbound Type parameter. Then DoTryCatch<T> could be instantiated for a particular type. However, I haven’t actually tried this so I hate to pretend that it would work. I am way behind on understanding what we can and cannot do with generics in our next release, how to express this in ILASM, and how it actually works under the covers. Any blog on that topic is years away.

While we are on the subject of interesting C# codegen, that same document on upcoming features also discusses iterators. These allow you to use the ‘yield’ statement to convert the normal pull model of defining iteration into a convenient push model. You can see the same ‘yield’ notion in Ruby. And I’m told that both languages have borrowed this from CLU, which pioneered the feature about the time that I was born.

When you get your hands on an updated C# compiler that supports this handy construct, be sure to ILDASM your program and see how it’s achieved. It’s a great example of what a compiler can do to make life easier for a developer, so long as we’re willing to burn a few more cycles compared to a more prosaic loop construct. In today’s world, this is almost always a sensible trade-off.

Okay, that last part has nothing to do with exceptions, does it? Let’s get back to the managed exception model.

try {…} fault {…}

Have you ever written code like this, to restrict execution of your finally clause to just the exceptional cases?

bool exceptional = true;

try {

…body of try…

exceptional = false;

} finally {

if (exceptional) {…}

}

Or how about a catch with a rethrow, as an alternate technique for achieving finally behavior for just the exceptional cases:

try {

…

} catch {

…

rethrow;

}

In each case, you are accommodating for the fact that your language doesn’t expose fault blocks. In fact, I think the only language that exposes these is ILASM. A fault block is simply a finally clause that only executes in the exceptional case. It never executes in the non-exceptional case.

Incidentally, the first alternative is preferable to the second. The second approach terminates the first pass of exception handling. This is a fundamentally different semantics, which has a substantial impact on debugging and other operations. Let’s look at rethrow in more detail, to see why this is the case.

Rethrow, restartable exceptions, debugging

Gee, my language has rethrow, but no filter. Why can’t I just treat the following constructs as equivalent?

try {…} filter (expression) catch (Exception e) {…}

try {…} catch (Exception e) { if (!expression) rethrow; …}

In fact, ‘rethrow’ tries hard to create the illusion that the initial exception handling is still in progress. It uses the same exception object. And it augments the stack trace associated with that exception object, so that it includes the portion of stack from the rethrow to the eventual catch.

Hmm, I guess I should have already mentioned that the stack trace of an Exception is intentionally restricted to the segment of stack from the throw to the catch. We do this for performance reasons, since part of the cost of an exception is linear with the depth of the stack that we capture. I’ll talk about the implications of exception performance later. Of course, you can use the System.Diagnostics.StackTrace class to gather the rest of the stack from the point of the catch, and then manually merge it into the stack trace from the Exception object. But this is a little clumsy and we have sometimes been asked to provide a helper to make this more convenient and less brittle to changes in the formatting of stack traces.

Incidentally, when you are playing around with stack traces (whether they are associated with exceptions, debugging, or explicit use of the StackTrace class), you will always find JIT inlining getting in your way. You can try to defeat the JIT inliner through use of indirected calls like function pointers, virtual methods, interface calls and delegates. Or you can make the called method “interesting” enough that the JIT decides it would be unproductive or too difficult to inline. All these techniques are flawed, and all of them will fail over time. The correct way to control inlining is to use the MethodImpl(MethodImplOptions.NoInlining) pseudo-custom attribute from the System.Runtime.CompilerServices namespace.

One way that a rethrow differs from a filter is with respect to resumable or restartable exceptions. We’ve already seen how SEH allows an exception filter to return EXCEPTION_CONTINUE_EXECUTION. This causes the faulting instruction to be restarted. Obviously it’s unproductive to do this unless the filter has first taken care of the faulting situation somehow. It could do this by changing the register state in the exception context so that a different value is dereferenced, or so that execution resumes at a different instruction. Or it could have modified the environment the program is running in, as with the VirtualProtect cases that I mentioned earlier.

In V1 and V1.1, the managed exception model does not support restartable exceptions. In fact, I think that we set EXCEPTION_NONCONTINUABLE on some (but perhaps not all) of our exceptions to indicate this. There are several reasons why we don’t support restartable exceptions:

In order to repair a faulting situation, the exception handler needs intimate knowledge about the execution environment. In managed code, we’ve gone to great lengths to hide these details. For example, there is no architecture-neutral mapping from the IL expression of stack-based execution to the register set of the underlying CPU.

Restartability is often desired for asynchronous exceptions. By ‘asynchronous’ I mean that the exception is not initiated by an explicit call to ‘throw’ in the code. Rather, it results from a memory fault or an injected failure like Abort that can happen on any instruction. Propagating a managed exception, where this involves execution of a managed filter, necessarily involves the potential for a GC. A JIT has some discretion over the GC-safe points that it chooses to support in a method. Certainly the JIT must gather GC information to report roots accurately at all call-sites. But the JIT normally isn’t required to maintain GC info for every instruction. If any instruction might fault, and if any such fault could be resumed, then the JIT would need GC info for all instructions in all methods. This would be expensive. Of course, ‘mov eax, ecx’ cannot fault due to memory access issues. But a surprising number of instructions are subject to fault if you consider all of memory – including the stack – to be unmapped. And even ‘mov eax, ecx’ can fault due to a Thread.Abort.

If you were paying attention to that last bullet, you might be wondering how asynchronous exceptions could avoid GC corruption even without resumption. After all, the managed filter will still execute and we know that the JIT doesn’t have complete GC information for the faulting instruction.

Our current solution to this on X86 is rather ad hoc, but it does work. First, we constrain the JIT to never flow the contents of the scratch registers between a ‘try’ clause and any of the exception clauses (‘filter’, ‘finally’, ‘fault’ and ‘catch’). The scratch registers in this case are EAX, ECX, EDX and sometimes EBP. Our JIT compiler decides, method-by-method, whether to use EBP as a stack-frame register or a scratch register. Of course, EBP isn’t really a scratch register since callees will preserve it for us, but you can see where I’m going.

Now when an asynchronous exception occurs, we can discard the state of all the scratch registers. In the case of EAX, ECX & EDX, we can unconditionally zero them in the register context that is flowed via exception propagation. In the case of EBP, we only zero it if we aren’t using EBP as a frame register. When we execute a managed handler, we can now report GC roots based on the GC information that’s associated with the handler’s instruction pointer.

The downside to this approach, other than its ad hoc nature, is that it constrains the codegen of any method that contains exception handlers. At some point we may have to model asynchronous exceptions more accurately, or expand the GC information spewed by the JIT compiler, or a combination, so that we can enable better code generation in the presence of exceptions.

We’ve already seen how VB.NET can use a filter and explicit logic flow from a catch clause to create the illusion of restartable exceptions to support ‘On Error Resume Next’. But this should not be confused with true restartability.

Before we leave the topic of rethrow, we should briefly consider the InnerException property of System.Exception. This allows one exception to be wrapped up in the state of another exception. A couple of important places where we take advantage of this are reflection and class construction.

When you perform late-bound invocation via reflection (e.g. Type.InvokeMember or MethodInfo.Invoke), exceptions can occur in two places:

1) The reflection infrastructure may decide that it cannot satisfy your request, perhaps because you passed the wrong number of arguments, or the member lookup failed, or you are invoking on someone else’s private members. That last one sounds vaguely dirty.

2) The late-bound invocation might work perfectly, but the target method you called may throw an exception back at you. Reflection must faithfully give you that exception as the result of the call. Returning it as an outbound argument, rather than throwing it at you, would be dangerous. We would lose one of the wonderful properties of exceptions, which is that they are hard to ignore. Error codes are constantly being swallowed or otherwise ignored, leading to fragile execution.

The problem is that these two sources of exceptions are ambiguous. There must be some way to tell whether the invocation attempt failed or whether the target of the invocation failed. Reflection disambiguates these cases by using an instance of System.Reflection.TargetInvocationException for the case where the invoked method threw an exception. The InnerException property of this instance is the exception that was thrown by the invoked method. If you get any exceptions from a late-bound invocation other than TargetInvocationException, those other exceptions indicate problems with the late-bound dispatch attempt itself.

Something similar happens with TypeInitializationException. If a class constructor (.cctor) method fails, we capture that exception as the InnerException of a TypeInitializationException. Subsequent attempts to use that class in this AppDomain from this or other threads will have that same TypeInitializationException instance thrown at them.

So what’s the difference between the following three constructs, where the overloaded constructor for MyExcep is placing its argument into InnerException:

try {…} catch (Exception e) { if (expr) rethrow; …}

try {…} catch (Exception e) { if (expr) throw new MyExcep(); …}

try {…} catch (Exception e) { if (expr) throw new MyExcep(e); …}

Well, the 2^nd form is losing information. The original exception has been lost. It’s hard to recommend that approach.

Between the 1^st and 3^rd forms, I suppose it depends on whether the intermediary can add important information by wrapping the original exception in a MyExcep instance. Even if you are adding value with MyExcep, it’s still important to preserve the original exception information in the InnerException so that sophisticated programs and developers can determine the complete cause of the error.

Probably the biggest impact from terminating the first pass of exception handling early, as with the examples above, is on debugging. Have you ever attached a debugger to a process that has failed with an unhandled exception? When everything goes perfectly, the debugger pops up sitting in the context of the RaiseException or trap condition.

That’s so much better than attaching the debugger and ending up on a ‘rethrow’ statement. What you really care about is the state of the process when the initial exception was thrown. But the first pass has terminated and the original state of the world may have been lost. It’s clear why this happens, based on the two pass nature of exception handling.

Actually, the determination of whether or not the original state of the world has been lost or merely obscured is rather subtle. Certainly the current instruction pointer is sitting in the rethrow rather than on the original fault. But remember how filter and finally clauses are executed with an EBP that puts the containing method’s locals in scope… and an ESP that still contains the original faulting method? It turns out that the catching handler has some discretion on whether to pop ESP before executing the catch clause or instead to delay the pop until the catch clause is complete. The managed handler currently pops the stack before calling the catch clause, so the original state of the exception is truly lost. I believe the unmanaged C++ handler delays the pop until the catch completes, so recovering the state of the world for the original exception is tricky but possible.

Regardless, every time you catch and rethrow, you inflict this bitter disappointment on everyone who debugs through your code. Unfortunately, there are a number of places in managed code where this disappointment is unavoidable.

The most unfortunate place is at AppDomain boundaries. I’ve already explained at http://blogs.gotdotnet.com/cbrumme/PermaLink.aspx/56dd7611-a199-4a1f-adae-6fac4019f11b why the Isolation requirement of AppDomains forces us to marshal most exceptions across the boundary. And we’ve just discussed how reflection and class construction terminate the first pass by wrapping exceptions as the InnerException of an outer exception.

One alternative is to trap on all first-chance exceptions. That’s because debuggers can have first crack at exceptions before the vectored exception handler even sees the fault. This certainly gives you the ability to debug each exception in the context in which it was thrown. But you are likely to see a lot of exceptions in the debugger this way!

In fact, throughout V1 of the runtime, the ASP.NET team ran all their stress suites with a debugger attached and configured to trap on first-chance Access Violations (“sxe av”). Normally an AV in managed code is converted to a NullReferenceException and then handled like any other managed exception. But ASP.NET’s settings caused stress to trap in the debugger for any such AV. So their team enforced a rule that all their suites (including all dependencies throughout FX) must avoid such faults.

It’s an approach that worked for them, but it’s hard to see it working more broadly.

Instead, over time we need to add new hooks to our debuggers so they can trap on just the exceptions you care about. This might involve trapping exceptions that are escaping your code or are being propagated into your code (for some definition of ‘your code’). Or it might involve trapping exceptions that escape an AppDomain or that are propagated into an AppDomain.

The above text has described a pretty complete managed exception model. But there’s one feature that’s conspicuously absent. There’s no way for an API to document the legal set of exceptions that can escape from it. Some languages, like C++, support this feature. Other languages, like Java, mandate it. Of course, you could attach Custom Attributes to your methods to indicate the anticipated exceptions, but the CLR would not enforce this. It would be an opt-in discipline that would be of dubious value without global buy-in and guaranteed enforcement.

This is another of those religious language debates. I don’t want to rehash all the reasons for and against documenting thrown exceptions. I personally don’t believe the discipline is worth it, but I don’t expect to change the minds of any proponents. It doesn’t matter.

What does matter is that disciplines like this must be applied universally to have any value. So we either need to dictate that everyone follow the discipline or we must so weaken it that it is worthless even for proponents of it. And since one of our goals is high productivity, we aren’t going to inflict a discipline on people who don’t believe in it – particularly when that discipline is of debatable value. (It is debatable in the literal sense, since there are many people on both sides of the argument).

To me, this is rather like ‘const’ in C++. People often ask why we haven’t bought into this notion and applied it broadly throughout the managed programming model and frameworks. Once again, ‘const’ is a religious issue. Some developers are fierce proponents of it and others find that the modest benefit doesn’t justify the enormous burden. And, once again, it must be applied broadly to have value.

Now in C++ it’s possible to ‘const-ify’ the low level runtime library and services, and then allow client code to opt-in or not. And when the client code runs into places where it must lose ‘const’ in order to call some non-const-ified code, it can simply remove ‘const’ via a dirty cast. We have all done this trick, and it is one reason that I’m not particularly in favor of ‘const’ either.

But in a managed world, ‘const’ would only have value if it were enforced by the CLR. That means the verifier would prevent you from losing ‘const’ unless you explicitly broke type safety and were trusted by the security system to do so. Until more than 80% of developers are clamoring for an enforced ‘const’ model throughout the managed environment, you aren’t going to see us added it.

Foray into C++ Exceptions

C++ exposes its own exception model, which is distinct from the __try / __except / __finally exposure of SEH. This is done through auto-destruction of stack-allocated objects and through the ‘try’ and ‘catch’ keywords. Note that there are no double-underbars and there is no support for filters other than through matching of exception types. Of course, under the covers it’s still SEH. So there’s still an FS:[0] handler (on X86). But the C++ compiler optimizes this by only emitting a single SEH handler per method regardless of how many try/catch/finally clauses you use. The compiler emits a table to indicate to a common service in the C-runtime library where the various try, catch and finally clauses can be found in the method body.

Of course, one of the biggest differences between SEH and the C++ exception model is that C++ allows you to throw and catch objects of types defined in your application. SEH only lets you throw 32-bit exception codes. You can use _set_se_translator to map SEH codes into the appropriate C++ classes in your application.

A large part of the C++ exception model is implicit. Rather than use explicit try / finally / catch clauses, this language encourages use of auto-destructed local variables. Whether the method unwinds via a non-exceptional return statement or an exception being thrown, that local object will auto-destruct.

This is basically a ‘finally’ clause that’s been wrapped up in a more useful language construct. Auto-destruction occurs during the second pass of SEH, as you would expect.

Have you noticed that the C++ exception you throw is often a stack-allocated local? And that if you explicitly catch it, this catch is also with a stack-allocated object? Did you ever wake up at night in a cold sweat, wondering whether a C++ in-flight exception resides on a piece of stack that’s already been popped? Of course not.

In fact, we’ve now seen enough of SEH to understand how the exception always remains in a section of the stack above ESP (i.e. within the bounds of the stack). Prior to the throw, the exception is stack-allocated within the active frame. During the first pass of SEH, nothing gets popped. When the filters execute, they are pushed deeper on the stack than the throwing frame.

When a frame declares it will catch the exception, the second pass starts. Even here, the stack doesn’t unwind. Then, before resetting the stack pointer, the C++ handler can copy-construct the original exception from the piece of stack that will be popped into the activation frame that will be uncovered.

If you are an expert in unmanaged C++ exceptions, you will probably be interested to learn of the differences between managed C++ exceptions and unmanaged C++ exceptions. There’s a good write-up of these differences at http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vcmex/html/vccondifferencesinexceptionhandlingbehaviorundermanagedexceptionsforc.asp.

A Single Managed Handler

We’ve already seen how the C++ compiler can emit one SEH handler per method and reuse it for all the exception blocks in that method. The handler can do this by consulting a side table that indicates how the various clauses map to instruction sequences within that method.

In the managed environment, we can take this even further. We maintain a boundary between managed and unmanaged code for many reasons, like synchronization with the garbage collector, to enable stack crawling through managed code, and to marshal arguments properly. We have modified this boundary to erect a single SEH handler at every unmanaged -> managed call in. For the most part, we must do this without compiler support since many of our transitions occur through dynamically generated machine code.

The cost of modifying the SEH chain during calls into managed code is quickly amortized as we call freely between managed methods. So the immediate cost of pushing FS:[0] handlers on method entry is negligible for managed code. But there is still an impact on the quality of the generated code. We saw part of this impact in the discussion of register usage across exception clauses to remain GC-safe.

Of course, the biggest cost of exceptions is when you actually throw one. I’ll return to this near the end of the blog.

Flow Control

Here’s an interesting scenario that came up recently.

Let’s say we drive the first pass of exception propagation all the way to the end of the handler chain and we reach the unhandled exception backstop. That backstop will probably pop a dialog in the first pass, saying that the application has suffered an unhandled exception. Depending on how the system is configured, the dialog may allow us to terminate the process or debug it. Let’s say we choose Terminate.

Now the 2^nd pass begins. During the 2^nd pass, all our finally clauses can execute.

What if one of those 2^nd pass ‘finally’ clauses throws a new exception? We’re going to start a new exception propagation from this location – with a new Exception instance. When we drive this new Exception up the chain, we may actually find a handler that will swallow the second exception.

If this is the case, the process won’t terminate due to that first exception. This is despite the fact that SEH told the user we had an unhandled exception, and the user told us to terminate the process.

This is surprising, to say the least. And this behavior is possible, regardless of whether managed or unmanaged exceptions are involved. The mechanism for SEH is well-defined and the exception model operates within those rules. An application should avoid certain (ab)uses of this mechanism, to avoid confusion.

Indeed, we have prohibited some of those questionable uses in managed code.

In unmanaged, you should never return from a finally. In an exceptional execution of a finally, a return has the effect of terminating the exception processing. The catch handler never sees its 2^nd pass and the exception is effectively swallowed. Conversely, in a non-exceptional execution of a finally, a return has the effect of replacing the method’s return value with the return value from the finally. This is likely to cause developer confusion.

So in managed code we’ve made it impossible for you to return from a finally clause. The full rules for flow control involving managed exception clauses should be found at Section 12.4.2.8 of ECMA Partition I (http://msdn.microsoft.com/net/ecma/).

However, it is possible to throw from a managed finally clause. (In general, it’s very hard to confidently identify regions of managed code where exceptions cannot be thrown). And this can have the effect of replacing the exception that was in flight with a new 1^st and 2^nd pass sweep, as described above. This is the ExceptionCollidedUnwind situation that is mentioned in the EXCEPTION_DISPOSITION enumeration.

The C++ language takes a different approach to exceptions thrown from the 2^nd pass. We’ve already seen that C++ autodestructors execute during the 2^nd pass of exception handling. If you’ve ever thrown an exception from the destructor, when that destructor is executed as part of an exception unwind, then you have already learned a painful lesson. The C++ behavior for this situation is to terminate the process via a termination handler.

In unmanaged C++, this means that developers must follow great discipline in the implementation of their destructors. Since eventually those destructors might run in the context of exception backout, those destructors should never allow an exception to escape them. That’s painful, but presumably achievable.

In managed C++, I’ve already mentioned that it’s very hard to identify regions where exceptions cannot occur. The ability to prevent (asynchronous and resource) exceptions over limited ranges of code is something we would like to enable at some point in the future, but it just isn’t practical in V1 and V1.1. It’s way too easy for an out-of-memory or type-load or class-initialization or thread-abort or appdomain-unload or similar exception to intrude.

Finally, it’s possible for exceptions to be thrown during execution of a filter. When this happens in an OS SEH context, it results in the ExceptionNestedException situation that is mentioned in the EXCEPTION_DISPOSITION enumeration. The managed exception model took a different approach here. We’ve already seen that an MSVC filter clause has three legal returns values (resume execution, continue search, and execute handler). If a managed filter throws an exception, we contain that exception and consider the filter to have replied “No, I don’t want to handle this one. Continue searching for a handler”.

This is a reasonable interpretation in all cases, but it falls out particularly well for stack overflow. With the historical OS support for stack overflow, it’s very hard to reliably execute backout code. As I’ve mentioned in other blogs, you may only have one 4K page of stack available for this purpose. If you blow that page, the process is terminated. It’s very hard to execute managed filters reliably within such a limited region. So a reasonable approach is to consider the filters to have themselves thrown a StackOverflowException and for us to interpret this as “No, I don’t want to handle this one.”

In a future version, we would like to provide a more defensible and useful mechanism for handling stack overflow from managed code.

Error Handling without Exceptions

So we’ve seen how SEH and C++ and managed exceptions all interoperate. But not all error handling is based on exceptions. When we consider Windows, there are two other error handling systems that the CLR can interoperate with. These are the Get/SetLastError mechanism used by the OS and the HRESULT / IErrorInfo mechanism used by COM.

Let’s look at the GetLastError mechanism first, because it’s relatively simple. A number of OS APIs indicate failure by returning a sentinel value. Usually this sentinel value is -1 or 0 or 1, but the details vary depending on the API. This sentinel value indicates that the client can call GetLastError() to recover a more detailed OS status code. Unfortunately, it’s sometimes hard to know which APIs participate in the GetLastError protocol. Theoretically this information is always documented in MSDN and is consistent from one version of the OS to the next – including between the NT and Win95-based OSes.

The real issue occurs when you PInvoke to one of these methods. The OS API latches any failure codes with SetLastError. Now on the return path of the PInvoke, we may be calling various OS services and managed services to marshal the outbound arguments. We may be synchronizing with a pending GC, which could involve a blocking operation like WaitForSingleObject. Somewhere in here, we may call another OS API that itself latches an error code (or the absence of an error code) through its own call to SetLastError.

So by the time we return to some managed code that can generate up a new PInvoke stub to call GetLastError, you can be sure that the original error code is long gone. The solution is to tag your PInvoke declaration to indicate that it should participate in the GetLastError protocol. This tells the PInvoke call to capture the error as part of the return path, before any other OS calls on this thread have an opportunity to erase it or replace it.

This protocol works well for PInvokes. Unfortunately, we do not have a way to tag IJW VTFixup stubs in the same way. So when you make managed -> unmanaged calls via MC++ IJW, there isn’t a convenient and reliable way to recover a detailed OS status code on the return path. Obviously this is something we would like to address in some future version, though without blindly inflicting the cost of a GetLastError on all managed -> unmanaged transitions through IJW.

COM Error Handling

To understand how the CLR interoperates with COM HRESULTs, we must first review how PreserveSig is used to modify the behavior of PInvoke and COM Interop.

Normally, COM signatures return an HRESULT error code. If the method needs to communicate some other result, this is typically expressed with an [out, retval] outbound argument. Of course, there are exceptions to this pattern. For example, IUnknown::AddRef and Release both return a count of the outstanding references, rather than an HRESULT. More importantly, HRESULTs can be used to communicate success codes as well as error codes. The two most typical success codes are S_OK and S_FALSE, though any HRESULT with the high bit reset is considered a success code.

COM Interop normally transforms the unmanaged signature to create a managed signature where the [out, retval] argument becomes the managed return value. If there is no [out, retval], then the return type of the managed method is ‘void’. Then the COM Interop layer maps between failure HRESULTs and managed exceptions. Here’s a simple example:

COM: HRESULT GetValue([out, retval] IUnknown **ppRet)

CLR: IUnknown GetValue()

However, the return value might be a DWORD-sized integer that should not be interpreted as an HRESULT. Or it might be an HRESULT – but one which must sometimes distinguish between different success codes. In these cases, PreserveSig can be specified on the signature and it will be preserved on the managed side as the traditional COM signature.

Of course, the same can happen with PInvoke signatures. Normally a DLL export like Ole32.dll’s CoGetMalloc would have its signature faithfully preserved. Presumably the transformation would be something like this:

DLL: HRESULT CoGetMalloc(DWORD c, [out, retval] IMalloc **ppRet)

CLR: DWORD CoGetMalloc(DWORD c, ref IMalloc ppRet)

If OLE32 returns some sort of failure HRESULT from this call, it will be returned to the managed caller. If instead the application would prefer to get this error case automatically converted to a managed Exception, it can use PreserveSig to indicate this.

Huh? In the COM case PreserveSig means “give me the unconverted HRESULT signature”, but in the PInvoke case PreserveSig means “convert my HRESULTs into exceptions.” Why would we use the same flag to indicate exactly opposite semantics for these two interop layers? The reasons are, ahem, historical. The best way to think of PreserveSig is “give me the unusual transformation of my signature, as opposed to what is typical for the kind of interop I am doing.”

So now we know how to obtain mappings between HRESULTs and managed exceptions for the typical COM Interop case (no PreserveSig) and the atypical PInvoke case (PreserveSig). But what are the details of that mapping?

The exception subsystem in the CLR has mappings between COM errors, OS errors, and managed exception types.

Of course, sometimes we have a situation which doesn’t have a precise mapping. In the case of an HRESULT that isn’t associated with a specific managed Exception class, we convert it to an instance of COMException. In the case of an OS status code that isn’t associated with a specific managed Exception class, we convert it to an instance of SEHException.

Even for cases where we have a correspondence between a managed and unmanaged representation, the mapping won’t necessarily roundtrip. For instance, an AV in unmanaged code results in an SEH exception of code 0xC0000005. If this is driven through managed code, it will be mapped to the corresponding NullReferenceException class. If the propagation of this exception continues through managed code and further up the stack to an unmanaged SEH handler, the unmanaged code will see the original exception code of 0xC0000005. So, when propagating through that sequence of handlers, we see a perfect roundtrip.

But let’s change the scenario slightly, so that the original AccessViolation occurs in managed code. Now we have a NullReferenceException that is being propagated out to an unmanaged SEH handler further back on the stack. But this time the NullReferenceException will be mapped to an SEH exception code of 0xE0434F4D. This is the managed exception code used for all managed exceptions.

Have you ever wondered where these exception codes come from? Well 0xE0434F4D is 0xE0+“COM”. Originally the CLR was called COM+ 2.0. When we changed the project name, we neglected to change the exception code. The unmanaged C++ exceptions use 0xE06D7363, which is 0xE0+“msc”. You might also see 0xE0524F54 for 0xE0+“ROT” on Rotor builds.

The current mapping between OS status codes and managed exception types is quite limited. It contains standard transformations like:

STATUS_FLOAT_INEXACT_RESULT

STATUS_FLOAT_INVALID_OPERATION

STATUS_FLOAT_STACK_CHECK

STATUS_FLOAT_UNDERFLOW => ArithmeticException

STATUS_FLOAT_OVERFLOW

STATUS_INTEGER_OVERFLOW => OverflowException

STATUS_FLOAT_DIVIDE_BY_ZERO

STATUS_INTEGER_DIVIDE_BY_ZERO => DivideByZeroException

STATUS_FLOAT_DENORMAL_OPERAND => FormatException

STATUS_ACCESS_VIOLATION => NullReferenceException

STATUS_ARRAY_BOUNDS_EXCEEDED => IndexOutOfRangeException

STATUS_NO_MEMORY => OutOfMemoryException

STATUS_STACK_OVERFLOW => StackOverflowException

The HRESULT mappings are far more extensive. They include standard mappings to the well-known HRESULT values like:

E_POINTER => ArgumentNullException

And they include mappings to CLR-defined HRESULTs in the 0x8013???? range that you’ve doubtless witnessed during your development and debugging. The managed platform has its own facility code for reserving a range of HRESULTs for our exclusive use.

COR_E_ENTRYPOINTNOTFOUND => EntryPointNotFoundException

And our mappings include a gathering of similar HRESULTs to a single managed exception. Here’s a particularly extensive gathering of 26 different HRESULTs to the FileLoadException class:

FUSION_E_REF_DEF_MISMATCH

FUSION_E_INVALID_PRIVATE_ASM_LOCATION

COR_E_ASSEMBLYEXPECTED

FUSION_E_SIGNATURE_CHECK_FAILED

FUSION_E_ASM_MODULE_MISSING

FUSION_E_INVALID_NAME

FUSION_E_PRIVATE_ASM_DISALLOWED

COR_E_MODULE_HASH_CHECK_FAILED

COR_E_FILELOAD

SECURITY_E_INCOMPATIBLE_SHARE

SECURITY_E_INCOMPATIBLE_EVIDENCE

SECURITY_E_UNVERIFIABLE

COR_E_FIXUPSINEXE

HRESULT_FROM_WIN32(ERROR_TOO_MANY_OPEN_FILES)

HRESULT_FROM_WIN32(ERROR_SHARING_VIOLATION)

HRESULT_FROM_WIN32(ERROR_LOCK_VIOLATION)

HRESULT_FROM_WIN32(ERROR_OPEN_FAILED)

HRESULT_FROM_WIN32(ERROR_DISK_CORRUPT)

HRESULT_FROM_WIN32(ERROR_UNRECOGNIZED_VOLUME)

HRESULT_FROM_WIN32(ERROR_FILE_INVALID)

HRESULT_FROM_WIN32(ERROR_DLL_INIT_FAILED)

HRESULT_FROM_WIN32(ERROR_FILE_CORRUPT)

FUSION_E_CODE_DOWNLOAD_DISABLED

CORSEC_E_MISSING_STRONGNAME

INIT_E_DOWNLOAD_FAILURE

MSEE_E_ASSEMBLYLOADINPROGRESS => FileLoadException

There are some more observations we can make about the COM error handling approach. First, it should be obvious that the 32-bits of an HRESULT cannot uniquely define an arbitrary set of user-extensible error conditions. COM deals with this, in part, by including the interface that returns an HRESULT in the decision of how to interpret these 32-bits. This means that 0xE3021051 returned from IMyInterface is not the same error code as 0xE3201051 returned from IYourInterface. Unfortunately, it also means that each interface must be rigorous about the bit patterns it returns. Specifically, it would be very bad if the implementation of IMyInterface::m() happens to delegate to IYourInterface::n() and blindly return ‘n’s HRESULTs. Any HRESULTs returned from ‘n’ must somehow be mapped to the bit patterns that are legal to return from IMyInterface::m(). If ‘n’ returns a bit pattern that IMyInterface::m() cannot map, then ‘m’ is obligated to convert the HRESULT to E_UNEXPECTED and return that.

In other words, the uniqueness constraint for HRESULTs forces a painful discipline on all COM implementations that return HRESULTs. And part of this discipline is to lose error information by mapping meaningful HRESULTs into E_UNEXPECTED if the context for interpreting those HRESULTs is being lost. (There is a well-defined set of system HRESULTs which are implicitly returnable from any interface. The bit pattern for E_UNEXPECTED is necessarily part of this set. The CLR facility code allows us to live in this privileged world with our own codes).

The fact that most COM developers are unaware of this painful discipline and don’t follow it, just adds to the level of pain here.

Fortunately, COM supplements the limited expressibility and uniqueness of HRESULTs by using a second mechanism: IErrorInfo. And the COM Interop layer uses this supplementary mechanism when mapping to and from managed exception objects. In fact, System.Exception implements the IErrorInfo interface. When a managed exception is thrown to a COM client, the IErrorInfo of the Exception instance is available for the COM client to query.

Adam Nathan’s excellent book “.NET and COM – The Complete Interoperability Guide” describes how the IErrorInfo state is filled in from a managed exception in Chapter 16.

There’s one more detail of COM Interop HRESULT mapping that warrants discussion. It’s good practice for all COM methods to return an HRESULT. But there are several famous violations of this rule, including IUnknown::AddRef and Release. More importantly, every developer can choose whether to follow this best practice. Some choose not to. And there are some typical cases, like event sinks, where we often see methods returning ‘void’ or ‘bool’.

This presents the COM Interop error mapping layer with a problem. If an exception occurs inside a managed implementation of a method with one of these signatures, it’s hard to convey the error information back to the COM caller. There are several choices available to that layer – none of them good:

1) Allow the managed exception to travel back through the COM caller, using the underlying SEH mechanism. This would work perfectly, but is strictly illegal. Well-behaved COM servers do not propagate exceptions out to their COM clients.

2) Swallow the managed exception. Propagate a return value with ‘0’ out to the COM client. This 0 value might get interpreted as a returned Boolean, integer, pUnk or other data type. In the case of a ‘void’ signature, it will simply be ignored.

3) Convert the exception object into an HRESULT value. Propagate that HRESULT out as the return value to the COM client. In the ‘void’ case, this will again be ignored. In the pUnk case, it will likely be dereferenced and subsequently cause an AccessViolation. (Failure HRESULTs have the high bit set. On Win32 the high 2 GB of address space are reserved for the kernel and are unavailable unless you run a /LARGEADDRESSAWARE process on a suitably booted system. On Win64, the low couple of GB of address are reserved and unavailable to detect this sort of mistake).

As you can see, all of these solutions are broken. Unfortunately, the most broken of the three is the last one… and that’s the one we currently follow. I suspect we will change our behavior here at some point. Until then, we rely on the fact that AddRef & Release are specially handled and that the other cases are rare and are typically ‘void’ or ‘bool’ returns.

Performance and Trends

Exceptions vs. error codes has always been a controversial topic. For the last 15 years, every team has argued whether their codebase should throw exceptions or return error codes. Hopefully nobody argues whether their team should mix both styles. That’s never desirable, though it often takes major surgery to migrate to a consistent plan.

With any religious controversy, there are many arguments on either side. Some of them are related to:

A philosophy of what errors mean and whether they should be expressed out-of-band with the method contract.
Performance. Exceptions have a direct cost when you actually throw and catch an exception. They may also have an indirect cost associated with pushing handlers on method entry. And they can often have an insidious cost by restricting codegen opportunities.
It’s relatively easy to forget to check for a returned error code. It’s much harder to inadvertently swallow an exception without handling it (though we still find developers doing so!)
Exceptions tend to capture far more information about the cause and location of an error, though one could envision an error code system that’s equally powerful. (IErrorInfo anybody?)

So what’s the right answer here?

Well if you are building the kernel of an operating system, you should probably use error codes. You are a programming God who rarely makes mistakes, so it’s less likely that you will forget to check your return codes. And there are sound bootstrapping and performance reasons for avoiding exceptions within the kernel. In fact, some of the OS folks here think that SEH should be reserved for terrible “take down the process” situations. That may have been the original design point. But SEH is such a flexible system, and it is so entrenched as the basis for unmanaged C++ exceptions and managed exceptions, that it is no longer reasonable to restrict the mechanism to these critical failures.

So, if you are not a programming God like those OS developers, you should consider using exceptions for your application errors. They are more powerful, more expressive, and less prone to abuse than error codes. They are one of the fundamental ways that we make managed programming more productive and less error prone. In fact, the CLR internally uses exceptions even in the unmanaged portions of the engine. However, there is a serious long term performance problem with exceptions and this must be factored into your decision.

Consider some of the things that happen when you throw an exception:

Grab a stack trace by interpreting metadata emitted by the compiler to guide our stack unwind.
Run through a chain of handlers up the stack, calling each handler twice.
Compensate for mismatches between SEH, C++ and managed exceptions.
Allocate a managed Exception instance and run its constructor. Most likely, this involves looking up resources for the various error messages.
Probably take a trip through the OS kernel. Often take a hardware exception.
Notify any attached debuggers, profilers, vectored exception handlers and other interested parties.

This is light years away from returning a -1 from your function call. Exceptions are inherently non-local, and if there’s an obvious and enduring trend for today’s architectures, it’s that you must remain local for good performance.

Relative to straight-line local execution, exception performance will keep getting worse. Sure, we might dig into our current behavior and speed it up a little. But the trend will relentlessly make exceptions perform worse.

How do I reconcile the trend to worse performance with our recommendation that managed code should use exceptions to communicate errors? By ensuring that error cases are exceedingly rare. We used to say that exceptions should be used for exceptional cases, but folks pushed back on that as tautological.

If your API fails in 10% of all calls, you better not use an exception. Instead, change the API so that it communicates its success or failure as part of the API (e.g. ‘bool TryParse(String s)’). Even if the API fails 1% of calls, this may be too high a rate for a service that’s heavily used in a server. If 1% of calls fail and we’re processing 1000 requests per second with 100 of these API calls per request, then we are throwing 1000 times a second. That’s a very disturbing rate of exceptions. On the other hand, a 1% failure rate may be quite tolerable in a client scenario, if the exception occurs when a human user presses the wrong button.

Sometimes you won’t know whether your API will be used in a client or a server. And it may be hard for you to predict failure rates when errors are triggered by bad data from the client. If you’ve provided a way for the client to check his data without triggering an exception (like the TryParse() example above) then you’ve done your part.

As usual, there’s so much more to say. I still haven’t talked about unhandled exceptions. Or about undeniable exception propagation (Thread.Abort). Or how undeniable propagation interacts with propagation through unmanaged code via PInvoke, IJW or COM Interop. And I carefully avoided explaining why we didn’t follow our own rules when defining and using the Exception class hierarchy. And there’s plenty to say about our special treatment of OutOfMemoryException and StackOverflowException.

If you are still reading and actually want to know more, perhaps you should just apply for a job on the CLR team.

Startup, Shutdown and related matters

cbrumme — Wed, 20 Aug 2003 18:51:00 GMT

Usually I write blog articles on topics that people request via email or comments on other blogs. Well, nobody has ever asked me to write anything about shutdown.

But then I look at all the problems that occur during process shutdown in the unmanaged world. These problems occur because many people don’t understand the rules, or they don’t follow the rules, or the rules couldn’t possibly work anyway.

We’ve taken a somewhat different approach for managed applications. But I don’t think we’ve ever explained in detail what that approach is, or how we expect well-written applications to survive an orderly shutdown. Furthermore, managed applications still execute within an unmanaged OS process, so they are still subject to the OS rules. And in V1 and V1.1 of the CLR we’ve horribly violated some of those OS rules related to startup and shutdown. We’re trying to improve our behavior here, and I’ll discuss that too.

Questionable APIs

Unfortunately, I can’t discuss the model for shutting down managed applications without first discussing how unmanaged applications terminate. And, as usual, I’ll go off on a bunch of wild tangents.

Ultimately, every OS process shuts down via a call to ExitProcess or TerminateProcess. ExitProcess is the nice orderly shutdown, which notifies each DLL of the termination. TerminateProcess is ruder, in that the DLLs are not informed.

The relationship between ExitProcess and TerminateProcess has a parallel in the thread routines ExitThread and TerminateThread. ExitThread is the nice orderly thread termination, whereas if you ever call TerminateThread you may as well kill the process. It’s almost guaranteed to be in a corrupt state. For example, you may have terminated the thread while it holds the lock for the OS heap. Any thread attempting to allocate or release memory from that same heap will now block forever.

Realistically, Win32 shouldn’t contain a TerminateThread service. To a first approximation, anyone who has ever used this service has injected a giant bug into his application. But it’s too late to remove it now.

In that sense, TerminateThread is like System.Threading.Thread.Suspend and Resume. I cannot justify why I added those services. The OS SuspendThread and ResumeThread are extremely valuable to a tiny subset of applications. The CLR itself uses these routines to take control of threads for purposes like Garbage Collection and – as we’ll see later – for process shutdown. As with TerminateThread, there’s a significant risk of leaving a thread suspended at a “bad” spot. If you call SuspendThread while a thread is inside the OS heap lock, you better not try to allocate or free from that same heap. In a similar fashion, if you call SuspendThread while a thread holds the OS loader lock (e.g. while the thread is executing inside DllMain) then you better not call LoadLibrary, GetProcAddress, GetModuleHandle, or any of the other OS services that require that same lock.

Even worse, if you call SuspendThread on a thread that is in the middle of exception dispatching inside the kernel, a subsequent GetThreadContext or SetThreadContext can actually produce a blend of the register state at the point of the suspension and the register state that was captured when the exception was triggered. If we attempt to modify a thread’s context (perhaps bashing the EIP – on X86 – to redirect the thread’s execution to somewhere it will synchronize with the GC or other managed suspension), our update to EIP might quietly get lost. Fortunately it’s possible to coordinate our user-mode exception dispatching with our suspension attempts in order to tolerate this race condition.

And probably the biggest gotcha with using the OS SuspendThread & ResumeThread services is on Win9X. If a Win9X box contains real-mode device drivers (and yes, some of them still do), then it’s possible for the hardware interrupt associated with the device to interact poorly with the thread suspension. Calls to GetThreadContext can deliver a register state that is perturbed by the real-mode exception processing. The CLR installs a VxD on those operating systems to detect this case and retry the suspension.

Anyway, with sufficient care and discipline it’s possible to use the OS SuspendThread & ResumeThread to achieve some wonderful things.

But the managed Thread.Suspend & Resume are harder to justify. They differ from the unmanaged equivalents in that they only ever suspend a thread at a spot inside managed code that is “safe for a garbage collection.” In other words, we can report all the GC references at that spot and we can unwind the stack and register state to reveal our caller’s execution state.

Because we are at a place that’s safe for garbage collection, we can be sure that Thread.Suspend won’t leave a thread suspended while it holds an OS heap lock. But it may be suspended while it holds a managed Monitor (‘lock’ in C# or ‘SyncLock’ in VB.NET). Or it may be suspended while it is executing the class constructor (.cctor) of an important class like System.String. And over time we intend to write more of the CLR in managed code, so we can enjoy all the benefits. When that happens, a thread might be suspended while loading a class or resolving security policy for a shared assembly or generating shared VTables for COM Interop.

The real problem is that developers sometimes confuse Thread.Suspend with a synchronization primitive. It is not. If you want to synchronize two threads, you should use appropriate primitives like Monitor.Enter, Monitor.Wait, or WaitHandle.WaitOne. Of course, it’s harder to use these primitives because you actually have to write code that’s executed by both threads so that they cooperate nicely. And you have to eliminate the race conditions.

I’m already wandering miles away from Shutdown, and I need to get back. But I can’t resist first mentioning that TerminateThread is distinctly different from the managed Thread.Abort service, both in terms of our aspirations and in terms of our current implementation.

Nobody should ever call TerminateThread. Ever.

Today you can safely call Thread.Abort in two scenarios.

You can call Abort on your own thread (Thread.CurrentThread.Abort()). This is not much different than throwing any exception on your thread, other than the undeniable manner in which the exception propagates. The propagation is undeniable in the sense that your thread will continue to abort, even if you attempt to swallow the ThreadAbortException in a catch clause. At the end-catch, the CLR notices that an abort is in progress and we re-throw the abort. You must either explicitly call the ResetAbort method – which carries a security demand – or the exception must propagate completely out of all managed handlers, at which point we reset the undeniable nature of the abort and allow unmanaged code to (hopefully) swallow it.

An Abort is performed on all threads that have stack in an AppDomain that is being unloaded. Since we are throwing away the AppDomain anyway, we can often tolerate surprising execution of threads at fairly arbitrary spots in their execution. Even if this leaves managed locks unreleased and AppDomain statics in an inconsistent state, we’re throwing away all that state as part of the unload anyway. This situation isn’t as robust as we would like it to be. So we’re investing a lot of effort into improving our behavior as part of getting “squeaky clean” for highly available execution inside SQL Server in our next release.

Longer term, we’re committed to building enough reliability infrastructure around Thread.Abort that you can reasonably expect to use it to control threads that remain completely inside managed code. Aborting threads that interleave managed and unmanaged execution in a rich way will always remain problematic, because we are limited in how much we can control the unmanaged portion of that execution.

ExitProcess in a nutshell

So what does the OS ExitProcess service actually do? I’ve never read the source code. But based on many hours of stress investigations, it seems to do the following:

1) Kill all the threads except one, whatever they are doing in user mode. On NT-based operating systems, the surviving thread is the thread that called ExitProcess. This becomes the shutdown thread. On Win9X-based operating systems, the surviving thread is somewhat random. I suspect that it’s the last thread to get around to committing suicide.

2) Once only one thread survives, no further threads can enter the process… almost. On NT-based systems, I only see superfluous threads during shutdown if a debugger attaches to the process during this window. On Win9X-based systems, any threads that were created during this early phase of shutdown are permitted to start up. The DLL_THREAD_ATTACH notifications to DllMain for the starting threads will be arbitrarily interspersed with the DLL_PROCESS_DETACH notifications to DllMain for the ensuing shutdown. As you might expect, this can cause crashes.

3) Since only one thread has survived (on the more robust NT-based operating systems), the OS now weakens all the CRITICAL_SECTIONs. This is mixed blessing. It means that the shutdown thread can allocate and free objects from the system heap without deadlocking. And it means that application data structures protected by application CRITICAL_SECTIONs are accessible. But it also means that the shutdown thread can see corrupt application state. If one thread was wacked in step #1 above while it held a CRITICAL_SECTION and left shared data in an inconsistent state, the shutdown thread will see this inconsistency and must somehow tolerate it. Also, data structures that are protected by synchronization primitives other than CRITICAL_SECTION are still prone to deadlock.

4) The OS calls the DllMain of each loaded DLL, giving it a DLL_PROCESS_DETACH notification. The ‘lpReserved’ argument to DllMain indicates whether the DLL is being unloaded from a running process or whether the DLL is being unloaded as part of a process shutdown. (In the case of the CLR’s DllMain, we only ever receive the latter style of notification. Once we’re loaded into a process, we won’t be unloaded until the process goes away).

5) The process actually terminates, and the OS reclaims all the resources associated with the process.

Well, that sounds orderly enough. But try running a multi-threaded process that calls ExitProcess from one thread and calling HeapAlloc / HeapFree in a loop from a second thread. If you have a debugger attached, eventually you will trap with an ‘INT 3’ instruction in the OS heap code. The OutputDebugString message will indicate that a block has been freed, but has not been added to the free list… It has been leaked. That’s because the ExitProcess wacked your 2^nd thread while it was in the middle of a HeapFree operation.

This is symptomatic of a larger problem. If you wack threads while they are performing arbitrary processing, your application will be left in an arbitrary state. When the DLL_PROCESS_DETACH notifications reach your DllMain, you must tolerate that arbitrary state.

I’ve been told by several OS developers that it is the application’s responsibility to take control of all the threads before calling ExitProcess. That way, the application will be in a consistent state when DLL_PROCESS_DETACH notifications occur. If you work in the operating system, it’s reasonable to consider the “application” to be a monolithic homogenous piece of code written by a single author. So of course that author should put his house in order and know what all the threads are doing before calling ExitProcess.

But if you work on an application, you know that there are always multiple components written by multiple authors from different vendors. These components are only loosely aware of each other’s implementations – which is how it should be. And some of these components have extra threads on the side, or they are performing background processing via IOCompletion ports, threadpools, or other techniques.

Under those conditions, nobody can have the global knowledge and global control necessary to call ExitProcess “safely”. So, regardless of the official rules, ExitProcess will be called while various threads are performing arbitrary processing.

The OS Loader Lock

It’s impossible to discuss the Win32 model for shutting down a process without considering the OS loader lock. This is a lock that is present on all Windows operating systems. It provides mutual exclusion during loading and unloading.

Unfortunately, this lock is held while application code executes. This fact alone is sufficient to guarantee disaster.

If you can avoid it, you must never hold one of your own locks while calling into someone else’s code. They will screw you every time.

Like all good rules, this one is made to be broken. The CLR violates this rule in a few places. For example, we hold a ‘class constructor’ lock for your class when we call your .cctor method. However, the CLR recognizes that this fact can lead to deadlocks and other problems. So we have rules for weakening this lock when we discover cycles of .cctor locks in the application, even if these cycles are distributed over multiple threads in multi-threaded scenarios. And we can see through various other locks, like the locks that coordinate JITting, so that larger cycles can be detected. However, we deliberately don’t look through user locks (though we could see through many of these, like Monitors, if we chose). Once we discover a visible, breakable lock, we allow one thread in the cycle to see uninitialized state of one of the classes. This allows forward progress and the application continues. See my earlier blog on “Initializing code” for more details.

Incidentally, I find it disturbing that there’s often little discipline in how managed locks like Monitors are used. These locks are so convenient, particularly when exposed with language constructs like C# lock and VB.NET SyncLock (which handle backing out of the lock during exceptions), that many developers ignore good hygiene when using them. For example, if code uses multiple locks then these locks should typically be ranked so that they are always acquired in a predictable order. This is one common technique for avoiding deadlocks.

Anyway, back to the loader lock. The OS takes this lock implicitly when it is executing inside APIs like GetProcAddress, GetModuleHandle and GetModuleFileName. By holding this lock inside these APIs, the OS ensures that DLLs are not loading and unloading while it is groveling through whatever tables it uses to record the state of the process.

So if you call those APIs, you are implicitly acquiring a lock.

That same lock is also acquired during a LoadLibrary, FreeLibrary, or CreateThread call. And – while it is held – the operating system will call your DllMain routine with a notification. The notifications you might see are:

DLL_THREAD_ATTACH

The thread that calls your DllMain has just been injected into the process. If you need to eagerly allocate any TLS state, this is your opportunity to do so. In the managed world, it is preferable to allocate TLS state lazily on the first TLS access on a given thread.

DLL_THREAD_DETACH

The thread that calls your DllMain has finished executing the thread procedure that it was started up with. After it finishes notifying all the DLLs of its death in this manner, it will terminate. Many unmanaged applications use this notification to de-allocate their TLS data. In the managed world, managed TLS is automatically cleaned up without your intervention. This happens as a natural consequence of garbage collection.

DLL_PROCESS_ATTACH

The thread that calls your DllMain is loading your DLL via an explicit LoadLibraryEx call or similar technique, like a static bind. The lpReserved argument indicates whether a dynamic or static bind is in progress. This is your opportunity to initialize any global state that could not be burned into the image. For example, C++ static initializers execute at this time. The managed equivalent has traditionally been a class constructor method, which executes once per AppDomain. In a future version of the CLR, we hope to provde a more convenient module constructor concept.

DLL_PROCESS_DETACH

If the process is terminating in an orderly fashion (ExitProcess), your DllMain will receive a DLL_PROCESS_DETACH notification where the lpReserved argument is non-null. If the process is terminating in a rude fashion (TerminateProcess), your DllMain will receive no notification. If someone unloads your DLL via a call to FreeLibrary or equivalent, the process will continue executing after you unload. This case is indicated by a null value for lpReserved. In the managed world, de-initialization happens through notifications of AppDomain unload or process exit, or through finalization activity.

The DLL_THREAD_ATTACH and DLL_THREAD_DETACH calls have a performance implication. If you have loaded 100 DLLs into your process and you start a new thread, that thread must call 100 different DllMain routines. Let’s say that these routines touch a page or two of code each, and a page of data. That might be 250 pages (1 MB) in your working set, for no good reason.

The CLR calls DisableThreadLibraryCalls on all managed assemblies other than certain MC++ IJW assemblies (more on this later) to avoid this overhead for you. And it’s a good idea to do the same on your unmanaged DLLs if they don’t need these notifications to manage their TLS.

Writing code inside DllMain is one of the most dangerous places to write code. This is because you are executing inside a callback from the OS loader, inside the OS loader lock.

Here are some of the rules related to code inside DllMain:

1) You must never call LoadLibrary or otherwise perform a dynamic bind.

2) You must never attempt to acquire a lock, if that lock might be held by a thread that needs the OS loader lock. (Acquiring a heap lock by calling HeapAlloc or HeapFree is probably okay).

3) You should never call into another DLL. The danger is that the other DLL may not have initialized yet, or it may have already uninitialized. (Calling into kernel32.dll is probably okay).

4) You should never start up a thread or terminate a thread, and then rendezvous with that other thread’s start or termination.

As we shall see, the CLR violates some of these rules. And these violations have resulted in serious consequences for managed applications – particularly managed applications written in MC++.

And if you’ve ever written code inside DllMain – including code that’s implicitly inside DllMain like C++ static initializers or ‘atexit’ routines – then you’ve probably violated some of these rules. Rule #3 is especially harsh.

The fact is, programs violate these rules all the time and get away with it. Knowing this, the MC++ and CLR teams made a bet that they could violate some of these rules when executing IJW assemblies. It turns out that we bet wrong.

I’m going to explain exactly how we screwed this up with IJW assemblies, but first I need to explain what IJW assemblies are.

IJW

IJW is how we internally refer to mixed managed / unmanaged images. If you compile a MC++ assembly with ‘/clr’ in V1 or V1.1, it almost certainly contains a mixture of managed and unmanaged constructs.

In future versions, I expect there will be ways to compile MC++ assemblies with compiler-enforced guarantees that the image is guaranteed pure managed, or guaranteed pure verifiable managed, or – ultimately – perhaps even pure verifiable 32-bit / 64-bit neutral managed. In each case, the compiler will necessarily have to restrict you to smaller and smaller subsets of the C++ language. For example, verifiable C++ cannot use arbitrary unmanaged pointers. Instead, it must restrict itself to managed pointers and references, which are reported to the garbage collector and which follow certain strict rules. Furthermore, 32-bit / 64-bit neutral code cannot consume the declarations strewn through the windows.h headers, because these pick a word size during compilation.

IJW is an acronym for “It Just Works” and it reflects the shared goal of the C++ and CLR teams to transparently compile existing arbitrary C++ programs into IL. I think we did an amazing job of approaching that goal, but of course not everything “just works.” First, there are a number of constructs like inline assembly language that cannot be converted to managed execution. The C++ compiler, linker and CLR ensure that these methods are left as unmanaged and that managed callers transparently switch back to unmanaged before calling them.

So inline X86 assembly language must necessarily remain in unmanaged code. Some other constructs are currently left in unmanaged code, though with sufficient effort we could provide managed equivalents. These other constructs include setjmp / longjmp, member pointers (like pointer to virtual method), and a reasonable startup / shutdown story (which is what this blog article is supposed to be about).

I’m not sure if we ever documented the constructs that are legal in a pure managed assembly, vs. those constructs which indicate that the assembly is IJW. Certainly we have a strict definition of this distinction embedded in our code, because the managed loader considers it when loading. Some of the things we consider are:

A pure managed assembly has exactly one DLL import. This import is to mscoree.dll’s _CorExeMain (for an EXE) or _CorDllMain (for a DLL). The entrypoint of the EXE or DLL must be a JMP to this import. This is how we force the runtime to load and get control whenever a managed assembly is loaded.

A pure managed assembly can have no DLL exports. When we bind to pure managed assemblies, it is always through managed Fusion services, via AssemblyRefs and assembly identities (ideally with cryptographic strong names).

A pure managed assembly has exactly one rebasing fixup. This fixup is for the JMP through the import table that I mentioned above. Unmanaged EXEs tend to strip all their rebasing fixups, since EXEs are almost guaranteed to load at their preferred addresses. However, managed EXEs can be loaded like DLLs into a running process. That single fixup is useful for cases where we want to load via LoadLibraryEx on versions of the operating system that support this.

A pure managed assembly has no TLS section and no other exotic constructs that are legal in arbitrary unmanaged PE files.

Of course, IJW assemblies can have many imports, exports, fixups, and other constructs. As with pure managed assemblies, the entrypoint is constrained to be a JMP to mscoree.dll’s _CorExeMain or _CorDllMain function. This is the “outer entrypoint”. However, the COM+ header of the PE file has an optional “inner entrypoint”. Once the CLR has proceeded far enough into the loading process on a DLL, it will dispatch to this inner entrypoint which is… your normal DllMain. In V1 and V1.1, this inner entrypoint is expressed as a token to a managed function. Even if your DllMain is written as an unmanaged function, we dispatch to a managed function which is defined as a PInvoke out to the unmanaged function.

Now we can look at the set of rules for what you can do in a DllMain, and compare it to what the CLR does when it sees an IJW assembly. The results aren’t pretty. Remember that inside DllMain:

You must never call LoadLibrary or otherwise perform a dynamic bind

With normal managed assemblies, this isn’t a concern. For example, most pure managed assemblies are loaded through Assembly.Load or resolution of an AssemblyRef – outside of the OS loader lock. Even activation of a managed COM object through OLE32’s CoCreateInstance will sidestep this issue. The registry entries for the CLSID always mention mscoree.dll as the server. A subkey is consulted by mscoree.dll – inside DllGetClassObject and outside of the OS loader lock – to determine which version of the runtime to spin up and which assembly to load.

But IJW assemblies have arbitrary DLL exports. Therefore other DLLs, whether unmanaged or themselves IJW, can have static or dynamic (GetProcAddress) dependencies on an IJW assembly. When the OS loads the IJW assembly inside the loader lock, the OS further resolves the static dependency from the IJW assembly to mscoree.dll’s _CorDllMain. Inside _CorDllMain, we must select an appropriate version of the CLR to initialize in the process. This involves calling LoadLibrary on a particular version of mscorwks.dll, violating our first rule for DllMain.

So what goes wrong when this rule is violated? Well, the OS loader has already processed all the DLLs and their imports, walking the tree of static dependencies and forming a loading plan. It is now executing on this plan. Let’s say that the loader’s plan is to first initialize an IJW assembly, then initialize its dependent mscoree.dll reference, and then initialize advapi32.dll. (By ‘initialize’, I mean give that DLL its DLL_PROCESS_ATTACH notification). When mscoree.dll decides to LoadLibrary mscorwks.dll, a new loader plan must be created. If mscorwks.dll depends on advapi32.dll (and of course it does), we have a problem. The OS loader already has advapi32.dll on its pending list. It will initialize that DLL when it gets far enough into its original loading plan, but not before.

If mscorwks.dll needs to call some APIs inside advapi32.dll, it will now be making those calls before advapi32.dll’s DllMain has been called. This can and does lead to arbitrary failures. I personally hear about problems with this every 6 months or so. That’s a pretty low rate of failure. But one of those failures was triggered when a healthy application running on V1 of the CLR was moved to V1.1 of the CLR. Ouch.

You must never attempt to acquire a lock, if that lock might be held by a thread that needs the OS loader lock

It’s not possible to execute managed code without potentially acquiring locks on your thread. For example, we may need to initialize a class that you need access to. If that class isn’t already initialized in your AppDomain, we will use a .cctor lock to coordinate initialization. Along the same lines, if a method requires JIT compilation we will use a lock to coordinate this. And if your thread allocates a managed object, it may have to take a lock. (We don’t take a lock on each allocation if we are executing on a multi-processor machine, for obvious reasons. But eventually your thread must coordinate with the garbage collector via a lock before it can proceed with more allocations).

So if you execute managed code inside the OS loader lock, you are going to contend for a CLR lock. Now consider what happens if the CLR ever calls GetModuleHandle or GetProcAddress or GetModuleFileName while it holds one of those other locks. This includes implicit calls to LoadLibrary / GetProcAddress as we fault in any lazy DLL imports from the CLR.

Unfortunately, the sequence of lock acquisition is inverted on the two threads. This yields a classic deadlock.

Once again, this isn’t a concern for pure managed assemblies. The only way a pure managed assembly can execute managed code inside the OS loader lock is if some unmanaged code explicitly calls into it via a marshaled out delegate or via a COM call from its own DllMain. That’s a bug in the unmanaged code! But with an IJW assembly, some methods are managed and some are unmanaged. The compiler, linker and CLR conspire to make this fact as transparent as possible. But any call from your DllMain (i.e. from your inner entrypoint) to a method that happened to be emitted as IL will set you up for this deadlock.

You should never call into another DLL

It’s really not possible to execute managed code without making cross-DLL calls. The JIT compiler is in a different DLL from the ExecutionEngine. The ExecutionEngine is in a different DLL from your IJW assembly.

Once again, pure managed assemblies don’t usually have a problem here. I did run into one case where one of the Microsoft language compilers was doing a LoadLibrary of mscorlib.dll. This had the side effect of spinning up the CLR inside the OS loader lock and inflicting all the usual IJW problems onto the compilation process. Since managed assemblies have no DLL exports, it’s rare for applications to load them in this manner. In the case of this language compiler, it was doing so for the obscure purpose of printing a banner to the console at the start of compilation, telling the user what version of the CLR it was bound to. There are much better ways of doing this sort of thing, and none of those other ways would interfere with the loader lock. This has been corrected.

You should never start up a thread or terminate a thread, and then rendezvous

This probably doesn’t sound like something you would do. And yet it’s one of the most common deadlocks I see with IJW assemblies on V1 and V1.1 of the CLR. The typical stack trace contains a load of an IJW assembly, usually via a DLL import. This causes mscoree.dll’s _CorDllMain to get control. Eventually, we notice that the IJW assembly has been strong name signed, so we call into WinVerifyTrust in WinTrust.dll. That API has a perfectly reasonable expectation that it is not inside the OS loader lock. It calls into the OS threadpool (not the managed CLR threadpool), which causes the OS threadpool to lazily initialize itself. Lazy initialization involves spinning up a waiter thread, and then blocking until that waiter thread starts executing.

Of course, the new waiter thread must first deliver DLL_THREAD_ATTACH notifications to any DLLs that expect such notifications. And it must obviously obtain the OS loader lock before it can deliver the first notification. The result is a deadlock.

So I’ve painted a pretty bleak picture of all the things that can go wrong with IJW assemblies in V1 and V1.1 of the CLR. If we had seen a disturbing rate of failures prior to shipping V1, we would have reconsidered our position here. But it wasn’t until later that we had enough external customers running into these difficulties. With the benefits of perfect hindsight, it is now clear that we screwed up.

Fortunately, much of this is fixable in our next release. Until then, there are some painful workarounds that might bring you some relief. Let’s look at the ultimate solution first, and then you can see how the workarounds compare. We think that the ultimate solution would consist of several parts:

Just loading an IJW assembly must not spin up a version of the CLR. That’s because spinning up a version of the CLR necessarily involves a dynamic load, and we’ve seen that dynamic loads are illegal during loading and initializing of static DLL dependencies. Instead, mscoree.dll must perform enough initialization of the IJW assembly without actually setting up a full runtime. This means that all calls into the managed portion of the IJW assembly must be bashed so that they lazily load a CLR and initialize it on first call.

Along the same lines, the inner entrypoint of an IJW assembly must either be omitted or must be encoded as an unmanaged entrypoint. Recall that the current file format doesn’t have a way of representing unmanaged inner entrypoints, since this is always in the form of a token. Even if the token refers to an unmanaged method, we would have to spin up a version of the CLR to interpret that token for us. So we’re going to need a tweak to the current file format to enable unmanaged inner entrypoints.

An unmanaged inner entrypoint is still a major risk. If that inner entrypoint calls into managed code, we will trap the call and lazily spin up the correction version of the CLR. At that point, you are in exactly the same situation as if we had left the entrypoint as managed. Ideally, assembly-level initialization and uninitialization would never happen inside the OS loader lock. Instead, they would be replaced with modern managed analogs that are unrelated to the unmanaged OS loader’s legacy behavior. If you read my old blog on “Initializing code” at http://blogs.gotdotnet.com/cbrumme/PermaLink.aspx/611cdfb1-2865-4957-9a9c-6e2655879323, I mention that we’re under some pressure to add a module-level equivalent of .cctor methods. That mechanism would make a great replacement for traditional DLL_PROCESS_ATTACH notifications. In fact, the CLR has always supported a .cctor method at a global module scope. However, the semantics associated with such a method was that it ran before any access to static members at global module scope. A more useful semantic for a future version of the CLR would be for such a global .cctor to execute before any access to members in the containing Module, whether global or contained in any of the Module’s types.

The above changes make it possible to avoid execution of managed code inside the OS loader lock. But it’s still possible for a naïve or misbehaved unmanaged application to call a managed service (like a marshaled out delegate or a managed COM object) from inside DllMain. This final scenario is not specific to IJW. All managed execution is at risk to this kind of abuse. Ideally, the CLR would be able to detect attempts to enter it while the loader lock is held, and fail these attempts. It’s not clear whether such detection / prevention should be unconditional or whether it should be enabled through a Customer Debug Probe.

If you don’t know what Customer Debug Probes are, please hunt them down on MSDN. They are a life-saver for debugging certain difficult problems in managed applications. I would recommend starting with http://www.gotdotnet.com/Community/UserSamples/Details.aspx?SampleGuid=c7b955c7-231a-406c-9fa5-ad09ef3bb37f, and then reading most of Adam Nathan’s excellent blogs at http://blogs.gotdotnet.com/anathan.

Of the above 4 changes, we’re relatively confident that the first 3 will happen in the next release. We also experimented with the 4^th change, but it’s unlikely that we will make much further progress. A key obstacle is that there is no OS-approved way that can efficiently detect execution inside the loader lock. Our hope is that a future version of the OS would provide such a mechanism.

This is all great. But you have an application that must run on V1 or V1.1. What options do you have? Fortunately, Scott Currie has written an excellent article on this very subject. If you build IJW assemblies, please read it at http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dv_vstechart/html/vcconmixeddllloadingproblem.asp.

The Pure Managed Story

If you code in a language other than MC++, you’re saying “Enough about IJW and the OS loader lock already.”

Let’s look at what the CLR does during process shutdown. I’ll try not to mention IJW, but I’ll have to keep talking about that darn loader lock.

From the point of view of a managed application, there are three types of shutdown:

1) A shutdown initiated by a call to TerminateProcess doesn’t involve any further execution of the CLR or managed code. From our perspective, the process simply disappears. This is the rudest of all shutdowns, and neither the CLR developer nor the managed developer has any obligations related to it.

2) A shutdown initiated by a direct call to ExitProcess is an unorderly shutdown from the point of view of the managed application. Our first notification of the shutdown is via a DLL_PROCESS_DETACH notification. This notification could first be delivered to the DllMain of mscorwks.dll, mscoree.dll, or any of the managed assemblies that are currently loaded. Regardless of which module gets the notification first, it is always delivered inside the OS loader lock. It is not safe to execute any managed code at this time. So the CLR performs a few house-keeping activities and then returns from its DllMain as quickly as possible. Since no managed code runs, the managed developer still has no obligations for this type of shutdown.

3) An orderly managed shutdown gives managed code an opportunity to execute outside of the OS loader lock, prior to calling ExitProcess. There are several ways we can encounter an orderly shutdown. Because we will execute managed code, including Finalize methods, the managed developer must consider this case.

Examples of an orderly managed shutdown include:

1) Call System.Environment.Exit(). I already mentioned that some Windows developers have noted that you must not call ExitProcess unless you first coordinate all your threads… and then they work like mad to make the uncoordinated case work. For Environment.Exit we are under no illusions. We expect you to call it in races from multiple threads at arbitrary times. It’s our job to somehow make this work.

2) If a process is launched with a managed EXE, then the CLR tracks the number of foreground vs. background managed threads. (See Thread.IsBackground). When the number of foreground threads drops to zero, the CLR performs an orderly shutdown of the process. Note that the distinction between foreground and background threads serves exactly this purpose and no other purpose.

3) Starting with MSVCRT 7.0, an explicit call to ‘exit()’ or an implicit call to ‘exit()’ due to a return from ‘main()’ can turn into an orderly managed shutdown. The CRT checks to see if mscorwks.dll or mscoree.dll is in the process (I forget which). If it is resident, then it calls CorExitProcess to perform an orderly shutdown. Prior to 7.0, the CRT is of course unaware of the CLR.

4) Some unmanaged applications are aware of the CLR’s requirements for an orderly shutdown. An example is devenv.exe, which is the EXE for Microsoft Visual Studio. Starting with version 7, devenv calls CoEEShutDownCOM to force all the CLR’s references on COM objects to be Release()’d. This at least handles part of the managed shutdown in an orderly fashion. It’s been a while since I’ve looked at that code, but I think that ultimately devenv triggers an orderly managed shutdown through a 2^nd API.

If you are following along with the Rotor sources, this all leads to an interesting quirk of EEShutDown in ceemain.cpp. That method can be called:

0 times, if someone calls TerminateProcess.
1 time, if someone initiates an unorderly shutdown via ExitProcess.
2 times, if we have a single-threaded orderly shutdown. In this case, the first call is made outside of the OS loader lock. Later, we call ExitProcess for the 2^nd half of the shutdown. This causes EEShutDown to be called a 2^nd time.
Even more times, if we have a multi-threaded orderly shutdown. Many threads will race to call EEShutDown the first time, outside the OS loader lock. This routine protects itself by anointing a winner to proceed with the shutdown. Then the eventual call to ExitProcess causes the OS to kill all threads except one, which calls back to EEShutDown inside the OS loader lock.

Of course, our passage through EEShutDown is quite different when we are outside the OS loader lock, compared to when we are inside it. When we are outside, we do something like this:

First we synchronize at the top of EEShutDown, to handle the case where multiple threads race via calls to Environment.Exit or some equivalent entrypoint.
Then we finalize all objects that are unreachable. This finalization sweep is absolutely normal and occurs while the rest of the application is still running.
Then we signal for the finalizer thread to finish its normal activity and participate in the shutdown. The first thing it does is raise the AppDomain.ProcessExit event. Once we get past this point, the system is no longer behaving normally. You could either listen to this event, or you could poll System.Environment.HasShutdownStarted to discover this fact. This can be an important fact to discover in your Finalize method, because it’s more difficult to write robust finalization code when we have started finalizing reachable objects. It’s no longer possible to depend on WaitHandles like Events, remoting infrastructure, or other objects. The other time we can finalize reachable objects is during an AppDomain unload. This case can be discovered by listening to the AppDomain.DomainUnload event or by polling for the AppDomain.IsFinalizingForUnload state. The other nasty thing to keep in mind is that you can only successfully listen to the ProcessExit event from the Default AppDomain. This is something of a bug and I think we would like to try fixing it for the next release.
Before we can start finalizing reachable objects, we suspend all managed activity. This is a suspension from which we will never resume. Our goal is to minimize the number of threads that are surprised by the finalization of reachable state, like static fields, and it’s similar to how we prevent entry to a doomed AppDomain when we are unloading it.
This suspension is unusual in that we allow the finalizer thread to bypass the suspension. Also, we change suspended threads that are in STAs, so that they pump COM messages. We would never do this during a garbage collection, since the reentrancy would be catastrophic. (Threads are suspended for a GC at pretty arbitrary places… down to an arbitrary machine code instruction boundary in many typical scenarios). But since we are never going to resume from this suspension, and since we don’t want cross-apartment COM activity to deadlock the shutdown attempt, pumping makes sense here. This suspension is also unusual in how we raise the barrier against managed execution. For normal GC suspensions, threads attempting to call from unmanaged to managed code would block until the GC completes. In the case of a shutdown, this could cause deadlocks when it is combined with cross-thread causality (like synchronous cross-apartment calls). Therefore the barrier behaves differently during shutdown. Returns into managed code block normally. But calls into managed code are failed. If the call-in attempt is on an HRESULT plan, we return an HRESULT. If it is on an exception plan, we throw. The exception code we raise is 0xC0020001 and the argument to RaiseException is a failure HRESULT formed from the ERROR_PROCESS_ABORTED SCODE (0x1067).
Once all objects have been finalized, even if they are reachable, then we Release() all the COM pUnks that we are holding. Normally, releasing a chain of pUnks from a traced environment like the CLR involves multiple garbage collections. Each collection discovers a pUnk in the chain and subsequently Release’s it. If that Release on the unmanaged side is the final release, then the unmanaged pUnk will be free’d. If that pUnk contains references to managed objects, those references will now be dropped. A subsequent GC may now collect this managed object and the cycle begins again. So a chain of pUnks that interleaves managed and unmanaged execution can require a GC for each interleaving before the entire chain is recovered. During shutdown, we bypass all this. Just as we finalize objects that are reachable, we also drop all references to unmanaged pUnks, even if they are reachable.

From the perspective of managed code, at this point we are finished with the shutdown, though of course we perform many more steps for the unmanaged part of the shutdown.

There are a couple of points to note with the above steps.

We never unwind threads. Every so often developers express their surprise that ‘catch’, ‘fault’, ‘filter’ and ‘finally’ clauses haven’t executed throughout all their threads as part of a shutdown. But we would be nuts to try this. It’s just too disruptive to throw exceptions through threads to unwind them, unless we have a compelling reason to do so (like AppDomain.Unload). And if those threads contain unmanaged execution on their threads, the likelihood of success is even lower. If we were on that plan, some small percentage of attempted shutdowns would end up with “Unhandled Exception / Debugger Attach” dialogs, for no good reason.

Along the same lines, developers sometimes express their surprise that all the AppDomains aren’t unloaded before the process exits. Once again, the benefits don’t justify the risk or the overhead of taking these extra steps. If you have termination code you must run, the ProcessExit event and Finalizable objects should be sufficient for doing so.

We run most of the above shutdown under the protection of a watchdog thread. By this I mean that the shutdown thread signals the finalizer thread to perform most of the above steps. Then the shutdown thread enters a wait with a timeout. If the timeout triggers before the finalizer thread has completed the next stage of the managed shutdown, the shutdown thread wakes up and skips the rest of the managed part of the shutdown. It does this by calling ExitProcess. This is almost fool-proof. Unfortunately, if the shutdown thread is an STA thread it will pump COM messages (and SendMessages), while it is performing this watchdog blocking operation. If it picks up a COM call into its STA that deadlocks, then the process will hang. In a future release, we can fix this by using an extra thread. We’ve hesitated to do so in the past because the deadlock is exceedingly rare, and because it’s so wasteful to burn a thread in this manner.

Finally, a lot more happens inside EEShutDown than the orderly managed steps listed above. We have some unmanaged shutdown that doesn’t directly impact managed execution. Even here we try hard to limit how much we do, particularly if we’re inside the OS loader lock. If we must shutdown inside the OS loader lock, we mostly just flush any logs we are writing and detach from trusted services like the profiler or debugger.

One thing we do not do during shutdown is any form of leak detection. This is somewhat controversial. There are a number of project teams at Microsoft which require a clean leak detection run whenever they shutdown. And that sort of approach to leak detection has been formalized in services like MSVCRT’s _CrtDumpMemoryLeaks, for external use. The basic idea is that if you can find what you have allocated and release it, then you never really leaked it. Conversely, if you cannot release it by the time you return from your DllMain then it’s a leak.

I’m not a big fan of that approach to finding memory leaks, for a number of reasons:

The fact that you can reclaim memory doesn’t mean that you were productively using it. For example, the CLR makes extensive use of “loader heaps” that grow without release until an AppDomain unloads. At that point, we discard the entire heap without regard for the fine-grained allocations within it. The fact that we remembered where all the heaps are doesn’t really say anything about whether we leaked individual allocations within those heaps.
In a few well-bounded cases, we intentionally leak. For example, we often build little snippets of machine code dynamically. These snippets are used to glue together pieces of JITted code, or to check security, or twiddle the calling convention, or various other reasons. If the circumstances of creation are rare enough, we might not even synchronize threads that are building these snippets. Instead, we might use a light-weight atomic compare/exchange instruction to install the snippet. Losing the race means we must discard the extra snippet. But if the snippet is small enough, the race is unlikely enough, and the leak is bounded enough (e.g. we only need one such snippet per AppDomain or process and reclaim it when the AppDomain or process terminates), then leaking is perfectly reasonable. In that case, we may have allocated the snippet in a heap that doesn’t support free’ing.
This approach certainly encourages a lot of messy code inside the DLL_PROCESS_DETACH notification – which we all know is a very dangerous place to write code. This is particularly true, given the way threads are wacked by the OS at arbitrary points of execution. Sure, all the OS CRITICAL_SECTIONs have been weakened. But all the other synchronization primitives are still owned by those wacked threads. And the weakened OS critical sections were supposed to protect data structures that are now in an inconsistent state. If your shutdown code wades into this landmine of deadlocks and trashed state, it will have a hard time cleanly releasing memory blocks. Projects often deal with this case by keeping a count of all locks that are held. If this count is non-zero when we get our DLL_PROCESS_DETACH notification, it isn’t safe to perform leak detection. But this leads to concerns about how often the leak detection code is actually executed. For a while, we considered it a test case failure if we shut down a process while holding a lock. But that was an insane requirement that was often violated in race conditions.
The OS is about to reclaim all resources associated with this process. The OS will perform a faster and more perfect job of this than the application ever could. From a product perspective, leak detection at product shutdown is about the least interesting time to discover leaks.
DLL_PROCESS_DETACH notifications are delivered to different DLLs in a rather arbitrary order. I’ve seen DLLs either depend on brittle ordering, or I’ve seen them make cross-DLL calls out of their DllMain in an attempt to gain control over this ordering. This is all bad practice. However, I must admit that in V1 of the CLR, fusion.dll & mscorwks.dll played this “dance of death” to coordinate their termination. Today, we’ve moved the Fusion code into mscorwks.dll.
I think it’s too easy for developers to confuse all the discipline surrounding this approach with actually being leak-free. The approach is so onerous that the goal quickly turns into satisfying the requirements rather than chasing leaks.

There are at least two other ways to track leaks.

One way is to identify scenarios that can be repeated, and then monitor for leaks during the steady-state of repeating those scenarios. For example, we have a test harness which can create an AppDomain, load an application into it, run it, unload the AppDomain, then rinse and repeat. The first few times that we cycle through this operation, memory consumption increases. That’s because we actually JIT code and allocate data structures to support creating a 2^nd AppDomain, or support making remote calls into the 2^nd AppDomain, or support unloading that AppDomain. More subtly, the ThreadPool might create – and retain – a waiter thread or an IO thread. Or the application may trigger the creation of a new segment in the GC heap which the GC decides to retain even after the incremental contents have become garbage. This might happen because the GC decides it is not productive to perform a compacting collection at this time. Even the OS heap can make decisions about thread-relative look-aside lists or lazy VirtualFree calls.

But if you ignore the first 5 cycles of the application, and take a broad enough view over the next 20 cycles of the application, a trend becomes clear. And if you measure over a long enough period, paltry leaks of 8 or 12 bytes per cycle can be discovered. Indeed, V1 of the CLR shipped with a leak for a simple application in this test harness that was either 8 or 12 bytes (I can never remember which). Of that, 4 bytes was a known leak in our design. It was the data structure that recorded the IDs of all the AppDomains that had been unloaded. I don’t know if we’ve subsequently addressed that leak. But in the larger scheme of things, 8 or 12 bytes is pretty impressive.

Recently, one of our test developers has started experimenting with leak detection based on tracing of our unmanaged data structures. Fortunately, many of these internal data structures are already described to remote processes, to support out-of-process debugging of the CLR. The idea is that we can walk out from the list of AppDomains, to the list of assemblies in each one, to the list of types, to their method tables, method bodies, field descriptors, etc. If we cannot reach all the allocated memory blocks through such a walk, then the unreachable blocks are probably leaks.

Of course, it’s going to be much harder than it sounds. We twiddle bits of pointers to save extra state. We point to the interiors of heap blocks. We burn the addresses of some heap blocks, like dynamically generated native code snippets, into JITted code and then otherwise forget about the heap address. So it’s too early to say whether this approach will give us a sound mechanism for discovering leaks. But it’s certainly a promising idea and worth pursuing.

Rambling Security Addendum

Finally, an off-topic note as I close down:

I haven’t blogged in about a month. That’s because I spent over 2 weeks (including weekends) on loan from the CLR team to the DCOM team. If you’ve watched the tech news at all during the last month, you can guess why. It’s security.

From outside the company, it’s easy to see all these public mistakes and take a very frustrated attitude. “When will Microsoft take security seriously and clean up their act?” I certainly understand that frustration. And none of you want to hear me whine about how it’s unfair.

The company performed a much publicized and hugely expensive security push. Tons of bugs were filed and fixed. More importantly, the attitude of developers, PMs, testers and management was fundamentally changed. Nobody on our team discusses new features without considering security issues, like building threat models. Security penetration testing is a fundamental part of a test plan.

Microsoft has made some pretty strong claims about the improved security of our products as a result of these changes. And then the DCOM issues come to light.

Unfortunately, it’s still going to be a long time before all our code is as clean as it needs to be.

Some of the code we reviewed in the DCOM stack had comments about DGROUP consolidation (remember that precious 64KB segment prior to 32-bit flat mode?) and OS/2 2.0 changes. Some of these source files contain comments from the ‘80s. I thought that Win95 was ancient!

I’ve only been at Microsoft for 6 years. But I’ve been watching this company closely for a lot longer, first as a customer at Xerox and then for over a decade as a competitor at Borland and Oracle. For the greatest part of Microsoft’s history, the development teams have been focused on enabling as many scenarios as possible for their customers. It’s only been for the last few years that we’ve all realized that many scenarios should never be enabled. And many of the remainder should be disabled by default and require an explicit action to opt in.

One way you can see this change in the company’s attitude is how we ship products. The default installation is increasingly impoverished. It takes an explicit act to enable fundamental goodies, like IIS.

Another hard piece of evidence that shows the company’s change is the level of resource that it is throwing at the problem. Microsoft has been aggressively hiring security experts. Many are in a new Security Business Unit, and the rest are sprinkled through the product groups. Not surprisingly, the CLR has its own security development, PM, test and penetration teams.

I certainly wasn’t the only senior resource sucked away from his normal duties because of the DCOM alerts. Various folks from the Developer Division and Windows were handed over for an extended period. One of the other CLR architects was called back from vacation for this purpose.

We all know that Microsoft will remain a prime target for hacking. There’s a reason that everyone attacks Microsoft rather than Apple or Novell. This just means that we have to do a lot better.

Unfortunately, this stuff is still way too difficult. It’s a simple fact that only a small percentage of developers can write thread-safe free-threaded code. And they can only do it part of the time. The state of the art for writing 100% secure code requires that same sort of super-human attention to detail. And a hacker only needs to find a single exploitable vulnerability.

I do think that managed code can avoid many of the security pitfalls waiting in unmanaged code. Buffer overruns are far less likely. Our strong-name binding can guarantee that you call who you think you are calling. Verifiable type safety and automatic lifetime management eliminate a large number of vulnerabilities that can often be used to mount security attacks. Consideration of the entire managed stack makes simple luring attacks less likely. Automatic flow of stack evidence prevents simple asynchronous luring attacks from succeeding. And so on.

But it’s still way too hard. Looking forwards, a couple of points are clear:

1) We need to focus harder on the goal that managed applications are secure, right out of the box. This means aggressively chasing the weaknesses of our present system, like the fact that locally installed assemblies by default run with FullTrust throughout their execution. It also means static and dynamic tools to check for security holes.

2) No matter what we do, hackers will find weak spots and attack them. The very best we can hope for is that we can make those attacks rarer and less effective.

I’ll add managed security to my list for future articles.

TransparentProxy

cbrumme — Mon, 14 Jul 2003 22:58:00 GMT

One of the recurring requests for a blog is related to TransparentProxy, RealProxy, Contexts, Interception, etc. As usual, I’m typing this where I don’t have access to our corporate network and the sources, so some details might be a little off. (When will my dentist provide free wireless to his customers?) And, as usual, none of this is going to help you in developing applications. In fact, I postponed writing about this topic – despite all the requests – because it seems so obscure. But if you are struggling through the Rotor source base, it might explain some of the code that you see. I say ‘might’ because I’ve never actually looked at the Rotor sources. I’m just relying on the fact that they are a pretty faithful of a cleansed snapshot of our desktop CLR sources. Anyway…

Normally, a reference to a managed object is just that: a native memory pointer. This is reported accurately to the GC so that we can track reachability and so we can update that pointer if the object is relocated during a compaction. But in the case of an object that derives from MarshalByRefObject (MBRO), it’s possible that the object instance is actually remote. If this is the case, a proxy stands in for the server instance.

The TP / RP Pair

In fact, we don’t have a single proxy for this case. Instead, we have a proxy pair. This pair consists of a System.Runtime.Remoting.Proxies.__TransparentProxy (TP) and a RealProxy (RP). The client calls on the TP; the TP forwards the calls to the RP; the RP usually delivers calls to a remote server object via a channel. I say ‘usually’ because the RP can actually do whatever it wants. There doesn’t even have to be a remote server object if you have a clever enough RP.

Why would we have both a TP and an RP? Clearly there’s a performance penalty associated with using two proxies. We have to instantiate them both and we force calls to take a doubly-indirected path. This overhead is necessary because the TP and RP are for different reasons:

The TP is pure magic. Its job is to fool all the CLR code that performs casting, field access, method dispatch, etc. into thinking that it’s dealing with a local instance of the appropriate type. In contrast, the RP has absolutely no magic. Its job is to provide an extensibility point where we can define RemotingProxy, or YourOwnProtocolProxy. It simply isn’t possible for us to combine magic with extensibility on the same object, as we shall see.

So how does the TP work its magic? Throughout the code base, whenever we are making type-based decisions (like GetType(), castclass & isinst IL opcodes), we’re careful to consider the TP type’s special ability to stand in for other types. In our method dispatch pathways, we are always careful to tease out the cases where a TP might lie in the callpath and deliver the calls to that object rather than optimizing the dispatch. Whenever we are accessing fields, creating byrefs, or otherwise referring to instance state, we ensure that these operations are delivered to the TP in a virtualized way.

Let’s look at these operations in more detail. In the discussion that follows, we are only interested in operations on potentially remote instances. Potentially remote instances include:

The MBRO type and all types that derive from it, including ContextBoundObject (CBO) and __ComObject.
Interface methods, since these interfaces could be implemented on a MBRO type.
All methods of Object since, by widening, a remote MBRO instance could be passed around formally as Object.

Significantly, types in the hierarchy below Object and disjoint from MBRO can never be remoted. In this portion of the hierarchy, which contains the vast majority of all types and includes all array types, we are free to inline methods, perform direct field access, perform calls without indirecting stubs, take byrefs, etc., without any consideration for remoting overheads. As I explained in a prior blog, this is why we don’t allow you to use a marker interface to indicate remotability – it must be boiled into the singly-inherited hierarchy. Otherwise, widening would prevent us from applying optimizations like direct field access to anything but sealed classes.

Statics

In an earlier blog, I already explained why we never remote static members. Static members don’t involve a ‘this’, so they completely bypass the proxy and proceed like local calls.

Virtuals

When the CLR is using VTables for virtual dispatch, all it needs to do is construct a VTable for the TP class that is at least as long as the longest VTable loaded by any type in the process. We do this by reserving enough virtual memory for the longest legal VTable, and then we commit and prepare pages of this VTable as required by dynamic class loads of other proxyable types. Slot ‘n’ of this VTable does something like “PUSH n; JMP CommonStub”. In other words, the purpose of this VTable is to capture which slot was called through and then perform some common processing on it. I’ll explain the common processing later.

I’ve also implied that the CLR might not use VTables for virtual dispatch. Hopefully it’s obvious that most of what I discuss is specific to a particular implementation of the CLR. Almost all of these details can change from release to release. And I would expect them to be very different in other implementations of the CLI. Whether the CLR continues to use VTables in a future release is a different rat-hole.

Anyway, the great thing about the virtual case is that the test for whether something is local or remote is completely hidden in the normal indirection of a virtual dispatch. There are no additional penalties.

Non-virtuals

Bashing VTables is a fairly standard technique. But non-virtual methods aren’t generally called through indirections. There are some exceptions to this rule, like NGEN cross-assembly calls and JITted calls to methods that haven’t yet themselves been JITted. However, in the typical case, we make non-virtual calls direct to the target method, just like static calls. Even when C# emits a ‘callvirt’ IL opcode to target a non-virtual method, this just forces us to check that the ‘this’ is non-null, as we’ve seen in an earlier blog (e.g. ‘mov eax, [ecx]’). If we are dispatching on a TP instead of a local instance, this non-null check isn’t going to help us capture the call.

Instead, for every non-virtual call on a potentially remote instance, we go through a special stub. This stub is constructed on-demand for a specific remoted method body and is then cached in case we find other callsites that need it. The purpose of the stub is to quickly check whether the server object is local to the calling thread. The details of this test depend on exactly what might be proxied. In the case of a normal MBRO, we’re just interested in whether the client’s reference is to a TP or to a local instance. In the case of a CBO, the client’s reference is always to a TP. So in that case we are interested in whether the thread is already in the server’s context or whether a context transition must be performed. More on this later, too.

Virtuals called non-virtually

In an earlier blog, we saw how it’s possible to call virtuals in a non-virtual fashion. This supports C#’s ‘base’ calls and the more flexible scope override operator ‘::’ of Managed C++. Of course, these cases are relatively rare. Virtual methods are almost always called virtually. Anyway, we handle non-virtual calls to potentially remote virtual methods by lazily creating and caching stubs as in the non-virtual method case.

Interfaces

You might think that interfaces would be handled just like virtual calls, since we place the interface methods into each type’s VTable during class layout. However, the necessities of efficient interface dispatch cause some additional problems. Let’s side-track on how interfaces work, for a moment.

The same interface can be implemented on multiple different classes. Each class will contain the methods for the interface contract at possibly different places in its VTable. The callsite is polymorphic with respect to the class of the server. So our goal is for the dispatch to discover, as quickly as possible, where the appropriate interface methods can be discovered on the receiver’s class. (There are other ways of performing interface dispatch that finesse this goal, like using fat pointers or interface proxy instances. Those other techniques have their own problems. And this is a blog on TP / RP, not interfaces, so let’s stick with the goal that I’ve stated).

We’re currently on our 4^th implementation of interface dispatch. The implementation that’s in the Rotor sources and was shipped in V1 and V1.1 was our 3^rd implementation. In that design, every interface is assigned a unique interface number during loading. Each class that implements one or more interfaces has a secondary interface table or ITable which is available via an indirection from the standard VTable. Dispatch occurs by indirecting through this ITable using the interface number of the interface. This points us back to a section of normal VTable (usually within the receiver’s normal class VTable somewhere) where the interface’s methods are laid out. Of course, the ITable is incredibly sparse. A given class might implement just the interfaces numbered 1003 and 2043. We only need two slots for that class, so it would be extremely wasteful to burn 2044 slots. Therefore a key aspect of this design is that the ITables are all allocated sparsely in a single shared heap. It’s important to find an algorithm that can efficiently pack all the ITables for all the classes we have loaded, without forcing our class loader to solve a “knapsack” problem.

The above works well for normal types, which each implement a bounded set of interfaces. But the TP class needs to implement all interfaces because it must stand in for all possible classes. In a similar way, the __ComObject class must implement all interfaces on all RCWs. That’s because a QueryInterface on some instance of a particular RCW type might one day say “Yes, I implement that interface.” (By the end of this blog you will know why __ComObject isn’t just a TP with a specialized RP implementation that understands COM rules).

Here’s what a virtual dispatch might look like on X86, with a shipping version of the CLR.

mov eax, [ecx] ; get the VTable from ‘this’

call [eax+mslot*4] ; call through some slot

And here’s an equivalent interface dispatch, which shows the indirection through the ITable.

mov eax, [ecx] ; get the VTable from ‘this’

mov eax, [eax+j] ; get the ITable at some offset from it

mov eax, [eax+islot*4] ; get the right interface VTable

call [eax+mslot*4] ; call through some slot

Leaving aside all the cache faults that might cripple us, the interface dispatch looks pretty good. It’s just a couple more instructions than the virtual dispatch. And we certainly don’t want to slow down this great interface dispatch for our typical case, in order to support the unbounded nature of TP and __ComObject interface dispatch. So we need a data structure for TP that will work for all potentially remote types and all the interfaces. The solution is to have a single ITable for these cases, which is fully populated with all the interfaces we have ever loaded. Obviously we have to grow this by committing more memory as the application faults in more interfaces. And each entry in the ITable points to a bit of VTable representing the appropriate interface, where the slots are full of stubs. The stubs contain machine code that says something like “If my ‘this’ is a TP, go do the remoting stuff. Otherwise my ‘this’ better be a __ComObject of some kind and I must go do the COM Interop stuff.” We actually use the VTables of the interface types themselves (which otherwise would contain nothing interesting in their slots) for this purpose.

If you are struggling to understand interface dispatch in the Rotor sources, the above might provide some useful guidance. The bad news is that we have switched to a somewhat different technique in our current codebase. Of course, I can’t predict when this new technique will ship in a CLR product or when it might show up in a new Rotor drop.

Constructors

Constructors are like non-virtual calls, except for the twist that – when they are mentioned in a ‘newinst’ IL opcode rather than a ‘call’ or ‘callvirt’ IL opcode – they involve an object allocation. In remoting scenarios, it’s clearly important to combine the remote allocation with the remote execution of the instance constructor method to avoid a 2^nd round-trip.

Instance Field Access and Byrefs

If a type cannot be remoted, all field access is direct. The X86 JIT tends to produce tight code like ‘mov eax, [ecx+34]’ if it is loading up a field. But this clearly won’t work if a TP is involved. Instead, the field access is turned into an implicit property access that can be remoted. That’s great for the case where the server truly is remote. But it’s an unfortunate penalty for the case where the server is local and was only potentially remote.

In the case of a byref, once again the JIT is normally efficient (e.g. ‘lea eax, [ecx+34]’). You might imagine that we could virtualize the byref the way we virtualize the implicit property access. In other words, we could generate a secret local and prime it with the value of the field (umm, property). Then we could make the call with a byref to the local as the argument. When the call returns, we could back-propagate the – perhaps updated – value of the local back into the field/property. The machinery to do this is relatively straight-forward. But it breaks the aliasing assumptions of byrefs. For example, if you were to update the field through the byref and then examine the server before unwinding the stack, the byref modification would not have been propagated back to the server object yet.

For better or for worse, the current implementation doesn’t go this route. Instead, the CLR allows you to take a byref to a field of a potentially remote object if that object is in fact local. And it throws an exception if you attempt to take such a byref on a potentially remote object that is indeed remote.

In passing, I should mention that some languages, like C#, won’t allow you to even attempt to take a byref on a potentially remote object (subjecting yourself to exceptions if the server is indeed remote). Instead, they force the developer to explicitly create the local, pass a byref to that local, and perform the back-propagation. This way there’s no magic and the developer understands exactly when and how his values get updated.

And I should also point out that if you call a method on a remote MBRO server, passing a byref to a field of a non-MBRO local object or to a static field or to a local, that same aliasing issue can be observed. In that case, we decided it was okay to allow observable aliasing discrepancies until the stack unwinds and the byref back-propagation can occur.

Casting, progressive type refinement

So far, I’ve been a little sloppy with the term VTable. Managed objects actually have a MethodTable. The MethodTable is currently implemented to have some GC info growing at a negative offset from the MethodTable (to tell the GC where all the pointers are for tracing), some “hot” metadata, and then a VTable. Part of the “hot” metadata is a parent pointer so we can traverse up the single-inheritance hierarchy and a list of implemented interfaces so we can perform interface casting.

So normally all our type tests are based on the MethodTable of the instance. But the TP has a rather uninteresting parent pointer (System.Object) and an empty list of implemented interfaces. This means that all of the type checks tend to fail. In the failure path, right before we throw an exception, we say “Oh, is this one of those weird cases like TP or __ComObject?” If it is, we vector off to an appropriate routine that understands how to perform QueryInterface calls or ManagedRemoting calls or whatever is appropriate for each case. Unless the Rotor source cleansing process performed a rename, there’s probably a routine in JITInterface.cpp called JITutil_CheckCastBizarre that’s an example of how we handle these weird cases. Note that they are typically placed into the failure paths, so the requirements of remoting don’t impact the performance of the local cases.

For the cases where we have a CBO, we can trivially know the exact type of the server instance. Everything is loaded in the same process and we can encode the server’s type with a MethodTable in the normal fashion. But if the client and server are in different AppDomains, processes, or machines then type injection becomes a consideration. In an earlier blog, I’ve talked about the security threats that depend on injecting an assembly into someone else’s AppDomain. For example, it may be possible for an assembly to escape the restrictive security policy of one AppDomain by injecting itself into the host’s AppDomain. Furthermore, inadvertent type injection across an AppDomain boundary can interfere with a host’s ability to discard types through AppDomain unloading. That’s why we return an ObjectHandle from the various AppDomain.CreateInstance and CreateInstanceFrom overloads. You must explicitly unwrap the ObjectHandle or use a CreateInstanceAndUnwrap convenience helper, to opt-in to the injecting behavior.

Another mechanism that helps you control type injection is ‘progressive type refinement’. This mechanism leverages the ability of a TP to stand-in for all different types. When you marshal back a remote MBRO, a TP is created in the client’s Context and AppDomain. (The Context is typically the default Context for the AppDomain, unless you are using CBOs). Consider the following code fragments:

AppDomain ad = …;

Object o = ad.CreateInstanceAndUnwrap(…);

SomeIface i = (SomeIface) ad.CreateInstanceAndUnwrap(…);

SomeClass c = (SomeClass) ad.CreateInstanceAndUnwrap(…);

Type t = c.GetType();

So long as the object we create in the remote AppDomain ‘is-a’ MBRO, the result of executing CreateInstanceAndUnwrap() will be a TP that masquerades as an instance of type System.Object. If we then cast the unwrapped object to SomeIface, our program obviously has mentioned that type in an early-bound manner in the client’s AppDomain. So that type is already present and doesn’t need to be injected. If the remote object can indeed be cast to SomeIface, the TP will refine its notion of the remote server’s type so that it includes SomeIface. In a similar fashion, the TP can be refined to understand that it is-a SomeClass – and all the super-classes and implemented interfaces of SomeClass.

Unfortunately, calls like c.GetType() terminate our ability to limit the type knowledge in the client’s Context / AppDomain. If you actually ask for the fully derived type of the remote concrete instance, we must obtain that remote type and attempt an injection of it into the client’s Context. However, for constrained patterns of calls, it’s possible for the host to get some real benefits from this feature.

Clearly we can only support progressive type refinement on objects that marshal by reference with a TP. Objects that marshal by value will necessarily inject the full type of the concrete instance that is produced during the unmarshal.

So we’ve seen how TP intrudes in the normal processing of calls, field access, and type checking.

Now we can understand the reasons why the TP and RP must be separated. Any call on the TP is captured into a message object and forwarded to the RP. The RP author (either our Managed Remoting team or you as a 3^rd party extension) now wants to operate on that message. Any methods and fields you define for this purpose must necessarily be on a different type than the TP. If they were defined on the TP, they would be subject to the same capture into a message. We would never get any work done until the consequent infinite recursion blows the stack.

There’s a small lie in the above. We actually have a few methods that are exempt from this automatic “capture and forward.” Object.GetHashCode(), when it isn’t overridden by the remotable subtype, is an example. But if we wanted to allow you to add arbitrary methods, fields and interfaces to your RP implementation, we would have an impossible mess of ambiguities. Slot 23 in the TP’s VTable would somehow be a captured call for every remotable type in the system and a necessary local execution of some RP behavior on that captured call.

Along the same lines, any call to GetType() or use of the castclass and isinst IL opcodes would be ambiguous if we merged the TP and RP. We wouldn’t know if we should deliver the TP semantics of pretending to be the remote server’s type, or whether we should deliver the RP semantics of your extensibility object.

The Common Stub

Let’s go back to the stubs in the VTable of the TP, which capture virtual calls. I already said that they look like “PUSH ‘n’; JMP CommonStub”. The common stub has to somehow convert the small integer in EAX into something more useful – the MethodDesc of the target method. (A MethodDesc is an abbreviation for method descriptor. It’s the piece of internal metadata that uniquely describes each method, including its metadata token, how to generate code for it, where the generated code can be found, the signature of the method, the MethodTable that contains it, and any special information like PInvoke marshaling or COM Interop information. We encode this pretty tightly and it usually fits into 32 bytes).

All virtual methods are instance methods, so we can use the ‘this’ of the call to help us obtain the desired MethodDesc. In the case of X86, we currently pass ‘this’ in ECX. So all we need to do is find the ‘n’th virtual method in the VTable of the type of the instance in ECX.

Something similar can happen in the interface dispatch case. Recall that we end up in a stub that hangs off the interface type’s VTable rather than the receiving class’ VTable. So this stub can trivially deliver up the relevant MethodDesc.

And for the non-virtual methods (and virtual methods called non-virtually), it’s even easier. In each case, we create a stub that is specific to that method. So this stub can contain the MethodDesc as an immediate argument burned into its code.

This means that all the method dispatch scenarios can obtain a MethodDesc and then jump to a common location. That common location now has enough information to capture all the arguments into a System.Runtime.Remoting.Messaging.Message which can disassociate itself from the stack for asynchronous or cross-process remoting. Or it can just use that information to efficiently access the registers and stack locations containing the arguments of the call, for the case where the interception remains inside the same process.

Unfortunately, we don’t take advantage of that 2^nd faster option as much as we should in V1 and V1.1. We have plenty of evidence that calls on TPs could be significantly faster in cross-Context and cross-AppDomain scenarios if we teased them apart from the more general remoting codepaths. By “significantly faster”, I mean at least one order of magnitude for some common and important cases. It’s likely that you’ll see at least some improvement here in our next release. And it’s also likely that even in our next release we will have ignored significant performance opportunities.

One surprising fact is that this is also why Delegate.BeginInvoke / EndInvoke are so slow compared to equivalent techniques like ThreadPool.QueueUserWorkItem (or UnsafeQueueUserWorkItem if you understand the security implications and want to be really efficient). The codepath for BeginInvoke / EndInvoke quickly turns into the common Message processing code of the general remoting pathway. That’s fine if you are making asynchronous calls on remote objects via a delegate. In that case, we can avoid the extra context switch that would occur if we didn’t coordinate the remoting code with the async code. We are careful to initiate the remote operation synchronously on the caller’s thread if that’s the case. But it means that the local case is dramatically slower than it needs to be. Of course, when it comes to performance we have much more glaring and general purpose issues to address before this one bubbles to the top of our list.

Finally, when we convert fields to implicit properties for the purposes of remoting, there is no corresponding MethodDesc. The method doesn’t exist anywhere in metadata. Instead, we go through a different pathway and use the FieldDesc as the piece of metadata to guide us.

The Hierarchy

Here’s the bit of the inheritance hierarchy which pertains to this blog:

Object Interfaces

MarshalByRefObject

/ \

__ComObject ContextBoundObject

ServicedComponent

As we’ve seen, Object and all Interfaces are potentially remote. We have to disable some optimizations to account for this. But anything which derives from Object and which does not derive from MBRO can have all optimizations enabled. Remoting cannot be a consideration for that portion of the hierarchy.

For MBRO and for classes that derive from MBRO but do not derive from CBO, we have an opportunity to add back an optimization. If we are operating on ‘this’ in a method of such a type, then we know that the instance is now local. The reasoning is that if the instance were indeed remote, then the TP should have forwarded the call elsewhere. Since that didn’t happen, we can add back all the local optimizations like direct field access.

Under CBO, the situation is a little worse. For various reasons that are all internal implementation details, we currently don’t unwrap CBO. Even when a thread is executing inside the correct Context and the instance is in that sense local, we leave it as the remote case.

Incidentally, we actually implemented CBO the other way first, where each call into a CBO would rigorously marshal the arguments via wrapping / unwrapping between server instances and TPs. But this caused terrible performance problems when performing identity comparisons of instances typed as some Interface or as System.Object. Simple operations like the fetch of a System.Object out of an Object[] required marshaling checks for the correct Context. We were penalizing typical programming operations for non-Context programming, in order to get a performance benefit if Contexts were indeed used. This was a poor trade-off and we adopted the current plan. Of course, leaving aside observable performance differences, either technique delivers the same semantics.

In my opinion, the primary benefit of the CLR is that the semantics of your program have been largely divorced from the details of execution. We could change practically everything I’ve described in this blog, without affecting your program’s behavior.

Anyway, we currently have one trick up our sleeve for adding back performance to CBO. If the JIT notices that you are performing a lot of field access on ‘this’, it creates an unwrapped temporary alias for ‘this’. Then it performs direct field access against the alias, rather than going through the remoting abstraction that converts the field accesses into property accesses. Clearly we could pursue some other optimizations here, too.

So does this explain why cross-AppDomain and cross-Context calls are so slow? We know that we have to create additional instances for proxying (the TP and RP). We know that the callpaths contain various indirections in the form of stubs. And we know that field access and method inlining and other standard optimizations are sometimes disabled because the server object is potentially remote.

With our current design, all these overheads seem unavoidable. But the real reason that cross-AppDomain and cross-Context operations are so slow is not due to these unavoidable overheads. It’s really that we simply haven’t invested enough engineering effort to make them faster. We could retain our existing design and do a much better job of separating the “same address space” cases from the “different address space” cases. As I’ve said, I think you’ll see some of these improvements in our next release. But we will still have a long way to go.

__ComObject and ServicedComponent

__ComObject derives from MBRO. It captures calls, intrudes on type negotiations, involves remoted instantiation, etc. It must be using TP and RP for all this magic, right? Actually, no.

I hinted at this when I described the stub that performs interface dispatch on TPs and __ComObjects. The stub we build on the first call to a particular interface method checks whether the server is a TP or whether it is a __ComObject and then it bifurcates all processing based on this.

If you look at the JITutil_CheckCastBizarre() routine I mentioned earlier, you see something similar. The code checks to see if it has a TP and it checks for __ComObject separately, with bifurcated processing for the two cases.

This is all historical. We built COM Interop before we realized we needed to invest in a strong managed remoting story that was distinct from DCOM. If we were to build the two services again today, we would definitely merge the two. The TP/RP remoting code is the correct abstraction for building services like COM Interop. And, indeed, if we had taken that approach then we would have been forced to address some of the “same address space” performance problems with managed remoting as a consequence of achieving our COM Interop performance goals.

This historical artifact is still more evident when you look at ServicedComponent. In some sense, ServicedComponent is a COM Interop thing. It’s a managed object that is aware of COM+ 1.0 unmanaged contexts (which are implemented in OLE32.dll on Win2000 and later OS’es). As such, it delegates creation of these managed objects through CoCreateInstance. Whenever we call on a ServicedComponent instance, we check whether we are in the correct COM+ 1.0 context. If we are not, we call through a COM+ service to transition into the correct context which then calls us back.

Yet ServicedComponent is built on top of all the TP / RP infrastructure, rather than on top of __ComObject. The reason for this is simply that we added EnterpriseServices and ServicedComponent very late in our V1 cycle. At that point, both the __ComObject pathway and the TP / RP pathway were available to us. The TP / RP pathway is simply a much cleaner and more general-purpose abstraction for this sort of thing.

If we ever go back and re-implement COM Interop, there are several important things we would change.

First, we would put COM Interop onto the TP / RP plan.

Second, we would rewrite a lot of the COM Interop code in managed. This is a long term goal for much of the CLR. Frankly, it’s very hard to write code in the CLR without abusing the exception model, or forgetting to report GC references, or inadvertently performing type-unsafe operations. All these mistakes lead to robustness and even security bugs. If we could write more of the CLR in managed code, our productivity would go up and our bug counts would go down.

Third, we would replace all our COM Interop stubs with IL. Currently, the marshaling of COM Interop calls is expressed in a language called ML (marshaling language). This was defined at a time when our IL was still in flux, and when we thought we would have a system of JIT-expanded macros that could map ML to a lower-level IL. We ended up implementing the macro aspect of IL and then dropping it in order to ship V1 sooner. This left us with the choice of either interpreting ML or writing converters to dynamically generate machine code from simple ML streams. We ended up doing both. Now that IL is stable, and now that we are targeting multiple CPU architectures in future versions of the CLR (like IA64), the fact that COM Interop depends on ML is quite a nuisance.

However, it’s not clear whether a re-implementation of COM Interop will ever be a sensible use of our time.

CBO and Interception

The COM+ 1.0 approach to call interception is very popular with MTS & COM+ programmers. Depending on your opinion of that product, this is either because they built a powerful abstraction for interception and Aspect Oriented Programming (AOP), or because when all you have is a hammer then everything looks like a nail.

Personally I think they did a great job of extending the COM model to include a powerful form of interception and AOP. But it’s tied up with COM-isms, like the fact that side effects such as transaction commit or object pooling occur when the last Release() call happens. Another COM-ism is that the COM’s rules for apartment marshaling have been extended to apply to finer-grained contexts. Some aspects of the COM+ model don’t apply well to managed code (like attaching side effects to a Release() call that might be delayed until a GC occurs). Other aspects could potentially be done better in managed code, since we have the advantage of a strong notion of class and metadata, the ability to generate code on the fly, and the ability to prevent leakage across marshaling boundaries rather than relying on developer hygiene.

During V1 of the CLR, we tried to take the great aspects of the COM+ model and adjust them so they would be more appropriate to the managed world. The result is ContextBoundObject, System.Runtime.Remoting.Contexts.Context, and all the related classes and infrastructure.

Very briefly, all CBO instances live in Contexts. All other managed instances are agile with respect to Contexts. Calls on these agile instances will never trigger a Context transition. Instead, such instances execute methods in the Context of their caller. When a Context transition does occur, there is sufficient extensibility for the application or other managed code to participate in the transition. That code can inject execution into the act of calling out of the caller’s Context, entering into the server’s Context, leaving the server’s Context when the method body completes, and returning to the caller’s Context. In addition, there is a declarative mechanism for attaching attributes to types of CBO, indicating what sort of Context they should live in. It is these attributes which are notified as Context transitions occur. The best example of code which uses this mechanism is the SynchronizationAttribute class. By attaching this attribute to a class that derives from CBO, you are declaring that any instance of your CBO should participate in a form of rental threading. Only one thread at a time can be active inside your object. Based on whether your attribute is Reentrant or not, you can either select a Single Threaded Apartment-style reentrancy (without the thread affinity of an STA, of course) or you can select a COM+ style recursive activity-threaded model. Another important aspect of Context-based programming is that the activation or instantiation of objects can be influenced through the same sort of declarative interception.

With this rich model, we expected CBO and managed Contexts to be a key extensibility point for the managed programming model. For example, we expected to reimplement ServicedComponent so that it would depend on managed contexts, rather than depending on unmanaged COM+ contexts for its features. In fact, our biggest fear was that we would build and deliver a model for managed contexts which would be quickly adopted by customers, we would then reimplement ServicedComponent and, during that process, we would discover that our initial model contained some fundamental flaws. It’s always extremely risky to deliver key infrastructure without taking the time to build systems on top of that infrastructure to prove the concept.

So what’s our current position? I don’t know the official word, but my sense is the following:

In the case of managed interception and AOP, we remain firmly committed to delivering on these core features. However, we find ourselves still debating the best way to do this.

One school holds that CBO is a model which proved itself through our customers’ COM+ experiences, which has been delivered to customers, but which suffers from poor documentation and poor performance. Given this, the solution is to put the appropriate resources into this facet of the CLR and address the documentation and performance problems.

Another school proposes that there’s an entirely different way to achieve these goals in the managed world. This other approach takes advantage of our ability to reason about managed programs based on their metadata and IL. And it takes advantage of our ability to generate programs on the fly, and to control binding in order to cache and amortize some of the cost of this dynamic generation. Given this, the solution is to maintain CBO in its current form and to invest in this potentially superior approach.

To be honest, I think we’re currently over-committed on a number of other critical deliverables. In the next release, I doubt if either of the above schools of thought will win out. Instead, we will remain focused on some of the other critical deliverables and postpone the interception decision. That’s unfortunate, because I would love to see the CLR expose a general-purpose but efficient extensibility mechanism to our customers. Such a mechanism might eliminate some of the feature requests we are currently struggling with, since teams outside the CLR could leverage our extensibility to deliver those features for us.

On the other hand, there’s something to be said for letting a major feature sit and stew for a while. We’ve gathered a lot of requirements, internally and externally, for interception. But I think we’re still puzzling through some of the implications of the two distinctly different design approaches.

Reliability

cbrumme — Mon, 23 Jun 2003 19:51:00 GMT

I’ve been putting off writing this blog, not just because I’m on vacation in Maui and have far more tempting things to do. It’s because one of my blogs has already been used on Slashdot as evidence that Windows cannot scale and won’t support distributed transactions (http://slashdot.org/comments.pl?sid=66598&cid=6122733), despite the fact that Windows scales well and does support distributed transactions. So I’m nervous that this new blog will be quoted somewhere as evidence that managed applications cannot be reliable.

The fact is that there are a lot of robust managed applications out there. During V1 of .NET, we all ran a peer-to-peer application called Terrarium. This could run as a background application and would become a screen saver during periods of inactivity. Towards the end of the release, I was running a week at a time without incident. The only reason for recycling the process after a week was to switch to a newer CLR so it could be tested.

Each team ran their own stress suite in their own labs on dedicated machines. And many teams would “borrow” machines from team members at night, to get even more machine-hours of stress coverage. These stress runs are generally more stressful than normal applications. For example, the ASP.NET team would put machines under high load with a mix of applications in a single worker process. They would then recycle various AppDomains in that worker process every two or three minutes. The worker process was required to keep running indefinitely with no memory growth, with no change in response time for servicing requests, and with no process recycling. This simulates what happens if you update individual applications of a web site over an extended period of time.

Another example of stress was our GC Stress runs. We would run our normal test suites in a mode where a GC was triggered on every thread at every point that a GC would normally be tolerated. This includes a GC at every machine instruction of JITted code. Each GC was forced to compact so that stale references could be detected.

We also have a “threaded stress” which tries to break loose as many race conditions in the CLR as possible by throwing many threads at each operation.

Our stress efforts are getting better over time. The dev lead for the 64-bit effort recently told me that we’ve already had more 64-bit process starts in our lab than all of the 32-bit process starts during the 5 years of building V1 and V1.1. We’ve got a very large (and very noisy) lab that is crammed with 64-bit boxes that are constantly running every test that we can throw at them.

Having said all that, CLR reliability in V1 and V1.1 still falls far short of where we would like it to be. When these releases of the CLR encounter a serious problem, we “fail fast” and terminate the process. Examples of serious problems include:

ExecutionEngineException
An Access Violation inside mscorwks.dll or mscoree.dll (except in a few specific bits of code, like the write barrier code, where AVs are converted into normal NullReferenceExceptions).
A corrupt GC heap
Stack overflow
Out of memory

The first three of the above examples are legitimate reasons for the process to FailFast. They represent serious bugs in the CLR, or serious bugs in (highly trusted / unsafe) portions of the frameworks and the application. It’s probably a security risk to continue execution under these circumstances, because it’s easy to imagine cases where type safety or other invariants have been violated.

But the last two cases (stack overflow and memory exhaustion) are a different matter. In a perfect world – i.e. some future version of our platform – these resource errors should be handled more gracefully. This means that the CLR should be hardened to tolerate them and the managed application should be able to trap these conditions and handle them.

There’s a lot of work involved in getting there. I want to explain some of the complexity that’s involved. But first I would like to point out why the current situation isn’t as bad as it seems.

For example, ASP.NET is able to pursue 100% reliability without worrying too much about resource errors. That’s because ASP.NET can use AppDomain recycling and process recycling to avoid gradual “decay” of the server process. Even if the server is leaking resources at a moderate rate, recycling will reclaim those resources. And, if a server application is highly stack intensive, it can be run with a larger stack or it can be rewritten to use iteration rather than recursion.

And for client processes, it’s historically been the case that excessive paging occurs before actual memory exhaustion. Since performance completely tanks when we thrash virtual memory, the user often kills the process before the CLR’s FailFast logic even kicks in. In a sense, the human is proactively recycling the process the same way ASP.NET does.

This stopped being the case for server boxes some time ago. It’s often the case that server boxes have enough physical memory to back the entire 2 or 3 GB of user address space in the server process. Even when there isn’t quite this much memory, memory exhaustion often means address space exhaustion and it occurs before any significant paging has occurred.

This is increasingly the case for client machines, too. It’s clear that many customers are bumping up against the hard limits of 32-bit processing.

Anyway, back to stack overflow & memory exhaustion errors. These are both resource errors, similar to the inability to open another socket, create another brush, or connect to another database. However, on the CLR team we categorize them as “asynchronous exceptions” rather than resource errors. The other common asynchronous exception is ThreadAbortException. It’s clear why ThreadAbortException is asynchronous: if you abort another thread, this could cause it to throw a ThreadAbortException at any machine instruction in JITted code and various other places inside the CLR. In that sense, the exception occurs asynchronously to the normal execution of the thread.

(In fact, the CLR will currently induce a ThreadAbortException while you are executing exception backout code like catch clauses and finally blocks. Sure, we’ll reliably execute your backout clauses during the processing of a ThreadAbortException – but we’ll interrupt an existing exception backout in order to induce a ThreadAbortException. This has been a source of much confusion. Please don’t nest extra backout clauses in order to protect your code from this behavior. Instead, you should assume a future version of the CLR will stop inducing aborts so aggressively.)

Now why would we consider stack overflow & memory exhaustion to be asynchronous? Surely they only occur when the application calls deeper into its execution or when it attempts to allocate memory? Well, that’s true. But unfortunately the extreme virtualization of execution that occurs with managed code works against us here. It’s not really possible for the application to predict all the places that the stack will be grown or a memory allocation will be attempted. Even if it were possible, those predictions would be version-brittle. A different version of the CLR (or an independent implementation like Mono, SPOT, Rotor or the Compact Frameworks) will certainly behave differently.

Here are some examples of memory allocations that might not be obvious to a managed developer:

Implicit boxing occurs in some languages, causing value types to be instantiated on the heap.
Class constructor (.cctor) methods are executed by the CLR prior to the first use of a class.
In the future, JITting might occur at a finer granularity than a method. For example, a rarely executed basic block containing a ‘throw’ clause might not be JITted until first use.
Although we chose to remove this from our V1 product, the CLR used to discard code and re-JIT it even during return paths.
Class loading can be delayed until the first use of a class.
For domain-neutral code, the storage for static fields is duplicated in each AppDomain. Some versions of the CLR have allocated this storage lazily, on first access.
Operations on MarshalByRefObjects must sometimes be remoted. This requires us to allocate during marshaling and unmarshaling. Along the same lines, casting a ComObject might cause a QueryInterface call to unmanaged code.
Accessing the Type of an instance, or accessing the current Thread, or accessing various other environmental state might cause us to lazily materialize that state.
Security checks are implicit to many operations. These generally involve instantiation.
Strings are immutable. Many “simple” string operations must allocate as a consequence.
Future versions of the CLR might delay allocation of portions of an instance for better cache effects. This already happens for some state, like an instance’s Monitor lock and – in some versions and circumstances – its hash code.
VTables are a space-inefficient mechanism for dispatching virtual calls and interface calls. Other popular techniques involve caching dispatch stubs which are lazily created.

The above is a very partial list, just to give a sense of how unpredictable this sort of thing is. Also, any dynamic memory allocation attempt might be the one that drives the system over the edge, because the developer doesn’t know the total memory available on the target system, and because other threads and other processes are asynchronously competing for that same unknown pool.

But a developer doesn’t have to worry about other threads and processes when he’s considering stack space. The total extent of the stack is reserved for himf when the thread is created. And he can control how much of that reservation is actually committed at that time.

It should be obvious that it’s inadequate to only reserve some address space for the stack. If the developer doesn’t also commit the space up front, then any subsequent attempt to commit a page of stack could fail because memory is unavailable. In fact, Windows has the unfortunate behavior that committing a page of stack can fail even if plenty of memory is available. This happens if the swap file needs to be extended on disk. Depending on the speed of your disk, extending the swap file can take a while. The attempt to commit your page of stack can actually time out during this period, giving you a spurious fault. If you want a robust application, you should always commit your stack reservations eagerly.

So StackOverflowException is more tractable than OutOfMemoryException, in the sense that we can avoid asynchronous competition for the resource. But stack overflows have their own special problems. These include:

The difficulty of predicting how much stack is enough.

This is similar to the difficulty of predicting where and how large the memory allocations are in a managed environment. For example, how much stack is used if a GC is triggered while you are executing managed code? Well, if the code you are executing is “while (true) continue;” the current version of the CLR might need several pages of your stack. That’s because we take control of your thread – which is executing an infinite loop – by rewriting its context and vectoring it to some code that throws an exception. I have no idea whether the Compact Frameworks would require more or less stack for this situation.

The difficulty of presenting an exception via SEH (Structured Exception Handling) when the thread is low on stack space.

If you are familiar with stack overflow handling in Windows, you know that there is a guard bit. This bit is set on all reserved but uncommitted stack pages. The application must touch stack pages a page at a time, so that these uncommitted pages can be committed in order. (Don’t worry – the JIT ensures that we never skip a page). There are 3 interesting pages at the end of the stack reservation. The very last one is always unmapped. If you ever get that far, the process will be torn down by the operating system. The one before that is the application’s buffer. The application is allowed to execute all its stack-overflow backout using this reserve. Of course, one page of reserve is inadequate for many modern scenarios. In particular, managed code has great difficulty in restricting itself to a single page.

The page before the backout reserve page is the one on which the application generates the stack overflow exception. So you might think that the application gets two pages in which to perform backout, and indeed this can sometimes be the case. But the memory access that triggers the StackOverflowException might occur in the very last bits of a page. So you can really only rely on 1 page for your backout.

A further requirement is that the guard bit must be restored on the final pages, as the thread unwinds out of its handling. However, without resorting to hijacking some return addresses on the stack, it can be difficult to guarantee that the guard bits can be restored. Failure to restore the guard bit means that stack overflow exceptions won’t be reliably generated on subsequent recursions.

The inability of the CLR (and parts of the operating system!) to tolerate stack overflows at arbitrary places.

I’m told that a stack overflow exception at just the wrong place in EnterCriticalSection or LeaveCriticalSection (I forget which) will leave the critical section in a corrupt state. Whether or not this is true, I would be amazed if the user-mode portion of the operating system is completely hardened to stack overflow conditions. And I know for a fact that the CLR has not been.

In order for us to report all GC references, perform security checks, process exceptions and other operations, we need the stack to be crawlable at all times. Unfortunately, some of the constructs we use for crawling the stack are allocated on the stack. If we should take a stack overflow exception while erecting one of these constructs, we cannot continue managed execution. Today, this situation drives us to our FailFast behavior. In the future, we need to tighten up some invariants between the JIT and the execution engine, to avoid this catastrophe. Part of the solution involves adding stack probes throughout the execution engine. This will be tedious to build and maintain.

Unwinding issues

Managed exception handling is largely on the Windows SEH plan. This means that filter clauses are called during the first pass, before any unwinding of the stack has occurred. We can cheat a little here: if there isn’t enough stack to call the managed filter safely, we can pretend that it took a nested stack overflow exception when we called it. Then we can interpret this failure as “No, I don’t want to handle this exception.”

When the first pass completes, we know where the exception will be caught (or if it will remain unhandled). The finally and fault blocks and the terminating catch clause are executed during the second pass. By the end of the second pass, we want to unwind the stack. But should we unwind the stack aggressively, giving ourselves more and more stack for subsequent clauses to execute in? If we are unwinding a StackOverflowException, we would like to be as aggressive as possible. But when we are interoperating with C++ exceptions, we must delay the unwind. That’s because the C++ exception is allocated on the stack. If we unwind and reuse that portion of the stack, we will corrupt the exception state.

(In fact, we’re reaching the edges of my understanding here. I think that a C++ rethrow effectively continues the first pass of the initial exception, looking for a new handler further up the stack. And I think that this means the stack cannot be unwound until we reach the end of a C++ catch clause. But I’m constantly surprised by the subtleties of exception handling).

So it’s hard to predict where OutOfMemoryException, StackOverflowException and ThreadAbortException might occur.

But one day the CLR will harden itself so it can tolerate these exceptions without requiring a FailFast escape hatch. And the CLR will do some (unspecified) fancy work with stack reserves and unwinding so that there’s enough stack available for managed code to process StackOverflowExceptions more like regular exceptions.

At that point, managed code could process asynchronous exceptions just like normal exceptions.

Where does that leave the application? Unfortunately, it leaves it in a rather bad spot. Consider what would happen if all application code had to be hardened against these asynchronous exceptions. We already know that they can occur pretty much anywhere. There’s no way that the application can pin-point exactly where additional stack or memory might be required – across all versions and implementations of the CLI.

As part of hardening, any updates the application makes to shared state must be transacted. Before any protecting locks are released via exception backout processing, the application must guarantee that all shared state has been returned to a consistent state. This means that the application must guarantee it can make either forward or backward process with respect to that state – without requiring new stack or memory resources.

For example, any .cctor method must preserve the invariant that either the class is fully initialized when the .cctor terminates, or that an exception escapes. Since the CLR doesn’t support restartable .cctors, any exception that escapes will indicate that the class is “off limits” in this AppDomain. This means that any attempt to use the class will receive a TypeInitializationException. The inner exception indicates what went wrong with initializing this class in this AppDomain. Since this might mean that the String class is unavailable in the Default AppDomain (which would be disastrous), we’re highly motivated to add support for restartable .cctors in a future release of the CLR.

Let’s forget about managed code for a moment, because we know that the way we virtualize execution makes it very difficult to predict where stack or memory resources might be required. Instead, imagine that you are writing this “guaranteed forward or backward progress” code in unmanaged code. I’ve done it, and I find it is very difficult. To do it right, you need strict coding rules. You need static analysis tools to check for conformance to those rules. You need a harness that performs fault injection. You need hours of directed code reviews with your brightest peers. And you need many machine-years of stress runs.

It’s a lot of work, which is only warranted for those few pieces of code that need to be 100% robust. Frankly, most applications don’t justify this level of effort. And this really isn’t the sweet spot for managed development, which targets extremely high productivity.

So today it’s not possible to write managed code and still be 100% reliable, and most developers shouldn’t even try. But our team has a broad goal of eventually supporting all unmanaged coding techniques in managed code (with some small epsilon of performance loss). Furthermore, the CLR and frameworks teams already have a need for writing some small chunks of reliable managed code. For example, we are trying to build some managed abstractions that guarantee resource cleanup. We would like to build those abstractions in managed code, so we can get automatic support for GC reporting, managed exceptions, and all that other good stuff. We’re prepared to invest all the care and effort necessary to make this code reliable – we just need it to be possible.

So I think you’ll see us delivering on this capability in the next release or two. If I had to guess, I think you’ll see a way of declaring that some portion of code must be reliable. Within that portion of code, the CLR won’t induce ThreadAbortExceptions asynchronously. As for stack overflow and memory exhaustion, the CLR will need to ensure that sufficient stack and memory resources are available to execute that portion of code. In other words, we’ll ensure that all the code has been JITted, all the classes loaded and initialized, all the storage for static fields is pre-allocated, etc. Obviously the existence of indirections / polymorphism like virtual calls and interface calls makes it difficult for the CLR to deduce exactly what resources you might need. We will need the developer to help out by indicating exactly what indirected resources must be prepared. This will make the technique onerous to use and somewhat expensive in terms of working set. In some ways, this is a good thing. Only very limited sections of managed code should be hardened in this manner. And most of the hardened code will be in the frameworks, where it belongs.

Here are my recommendations:

Application code is responsible for dealing with synchronous application exceptions. It’s everyone’s responsibility to deal with a FileNotFoundException when opening a file on disk.
Application code is not responsible for dealing with asynchronous exceptions. There’s no way we can make all our code bullet-proof when exceptions can be triggered at every machine instruction, in such a highly virtualized execution environment.
Perhaps 0.01% of code will be especially hardened using not-yet-available techniques. This code will be used to guarantee that all resources are cleaned up during an AppDomain unload, or to ensure that pending asynchronous operations never experience races when unloads occur. We’re talking tricky “systems” operations.

Well, if the application isn’t responsible for dealing with asynchronous exceptions, who is?

That’s the job of the process host. In the case of ASP.NET, it’s the piece of code that decides to recycle the process when memory utilization hits some threshold. In the case of SQL Server, it’s the piece of code that decides whether to abort a transaction, unload an AppDomain, or even suspend all managed activity. In the case of a random console application, it’s the [default] policy that might retain the FailFast behavior that you’ve seen in V1 and V1.1 of the CLR. And if Office ever builds a managed version, it’s the piece of code I would expect to see saving edits and unloading documents when memory is low or exhausted. (Don’t read anything into that last example. I have no idea when/if there will be a managed Excel). In other words, the process host knows how important the process is and what pieces can be discarded to achieve a consistent application state. In the case of ASP.NET, the only reason to keep the process running is to avoid a pause while we spin up a new process. In the case of SQL Server, the process is vitally important. Those guys are chasing 5 9’s of availability. In the case of a random console application, there’s a ton of state in the application that will be lost if the process goes down. But there probably isn’t a unit of execution that we can discard, to get back to a consistent application state. If the process is corrupt, it must be discarded.

In V1 & V1.1, it’s quite difficult for a process host to specify an appropriate reaction or set of reactions when an asynchronous exception occurs. This will get much easier in our next release.

As usual, I don’t want to get into any specifics on exactly what we’re going to ship in the future. But I do hope that I’ve painted a picture where the next release’s features will make sense as part of a larger strategy for dealing with reliability.

Infrequent blogging

cbrumme — Mon, 16 Jun 2003 19:50:00 GMT

Just a quick explanation for why there hasn't been a new blog lately. I'm partway through a 3.5 week vacation on Maui. I have wireless & broadband out by the pool, but I can't seem to find the time to blog. Things will return to normal by mid-July.

AppDomains ("application domains")

cbrumme — Sun, 01 Jun 2003 19:21:00 GMT

An AppDomain is a light-weight process. Well, if you actually measure the costs associated with an AppDomain – especially the first one you create, which has some additional costs that are amortized over all subsequent ones – then “light-weight” deserves some explanation:

A Win32 process is heavy-weight compared to a Unix process. A Win32 thread is heavy-weight compared to a Unix thread, particularly if you are using a non-kernel user threads package on Unix. A good design for Windows will create and destroy processes at a low rate, will have a small number of processes, and will have a small number of threads in each process.

Towards the end of V1, we did some capacity testing using ASP.NET. At that time, we were able to squeeze 1000 very simple applications / AppDomains into a single worker process. Presumably that process would have had 50-100 threads active in it, even under heavy load. If we had used OS processes for each application, we would have 1000 CLRs with 1000 GC heaps. More disturbing, we would have at least 10,000 threads. This would reserve 10 GB of VM just for their default 1 MB stacks (though it would only commit a fraction of that memory). All those threads would completely swamp the OS scheduler.

Also, if you execute a lot of processes, it’s key that those processes are filled with shared pages (for example, the same code loaded at the same preferred addresses) rather than private pages (like dynamically allocated data). Unfortunately, JITted code results in private pages. Our NGEN mechanism can be used to create pre-JITted images that can be shared across processes. But NGEN is not a panacea: NGEN images must be explicitly generated; if their dependencies change through versioning, modifications to security policy, etc., then the loader will reject the images as invalid and quietly fall back on JITting; NGEN images improve load time, but they actually insert a small steady-state cost to some operations, due to indirections; and NGEN can do a worse job of achieving locality than JITting and dynamically loading types (at least in the absence of a training scenario).

Over time, I think you’ll see NGEN address many of these limitations and become a core part of our execution story.

Of course, I wouldn’t recommend that you actually run a process with 1000 AppDomains either. For example, address space is an increasingly scarce resource – particularly on servers. The version of the CLR we just shipped now supports 3 GB of user address space, rather than the 2 GB that is normally available. (You need to boot the system for this, and sacrifice OS buffer space, so don’t do it unless you really need it). 64-bit systems, including a 64-bit CLR, cannot come soon enough for certain scenarios.

Compared to our goals, it still takes too long to create and destroy AppDomains. The VM and working set hits are too high. And the cost of crossing an AppDomain boundary is embarrassing. But the general architecture is sound and you should see improvements in all these areas in future releases.

It’s too simplistic to say that AppDomains are just light-weight OS processes. There is more to say in several dimensions:

Security
Instance lifetime
Type identity
Domain-neutrality
Per-AppDomain state like static fields
Instance-agility
Configuration and assembly binding
Unloading and other resource management
Programming model

Security

Code Access Security only works within an OS process. Threads freely call through AppDomain boundaries, so the CLR must be able to crawl stacks across those boundaries to evaluate permission demands. In fact, it can crawl compressed stacks that have been disassociated from their threads, accurately evaluating permissions based on AppDomains that have already been unloaded.

It’s conceivable that one day we will have a sufficiently strong notion of distributed trust that we can usefully propagate compressed stacks into other processes. However, I don’t expect we’ll see that sort of distributed security for at least another couple of releases.

It’s possible to apply different security policy or different security evidence at the granularity of an AppDomain. Any grants that would result based on AppDomain evidence and policy are intersected with what would be granted by policy at other levels, like machine or enterprise. For example, Internet Explorer attaches a different codebase to an AppDomain to indicate the origin of the code that’s running in it. There are two ways for the host to control security at an AppDomain granularity. Unfortunately, both techniques are somewhat flawed:

1) The host can pre-load a set of highly-trusted assemblies into an AppDomain. Then it can modify the security policy to be more restrictive and start loading less-trusted application code. The new restricted policy will only apply to these subsequent loads. This approach is flawed because it forces the host to form a closure of the initial highly-trusted set of assemblies. Whatever technique the host uses here is likely to be brittle, particularly in the face of versioning. Any dependent assemblies that are forgotten in the initial load will be limited by the restricted policy. Furthermore, it is unnecessarily expensive to eagerly load assemblies, just so they can escape a particular security policy.

2) The host can load the application assemblies with extra evidence. When the security system evaluates the grant set for these assemblies, this extra evidence can be considered and the application assemblies will get reduced permissions. This technique allows the host to lazily load highly trusted assemblies into the same AppDomain, since these won’t have the extra evidence attached to them. Unfortunately, this technique also has a rough edge. If an application assembly has a dependency on a second application assembly, what is going to attach extra evidence to the 2^nd assembly? I suppose the host could get the 1^st assembly’s dependencies and eagerly load them. But now we are back on a plan where transitive closures must be eagerly loaded in order to remain secure. And, in future releases, we would like to give each assembly a chance to run initialization code. There’s a risk that such initialization code might run and fault in the dependencies before the host can explicitly load them with extra evidence.

We need to do better here in a future release.

Until then, code injection remains a real concern. A host carefully prepares an AppDomain and loads some partially trusted application code there for execution. If the application code can inject itself into a different AppDomain (especially the default AppDomain, which is presumably where the fully trusted host is executing), then it can escape the policy and extra evidence that is constraining it. This is one reason that we don’t provide AppDomain enumeration services to partially trusted code. If you can find an AppDomain, you can perform an AppDomain.DoCallBack into it passing a delegate. This has the effect of marshaling the delegate into that AppDomain and then dispatching to it there. The assemblies containing the delegate and the target of the delegate will be created in the specified AppDomain.

Today, if a host exercises great care, it can use AppDomains as the basis of building a secure environment. In the future, we would like to reduce the amount of care required of the host. One obvious way to do this is to involve the host in any assembly loads that happen in any AppDomain. Unfortunately, that simple approach makes it difficult to make wise decisions on loading assemblies as domain-neutral, as we’ll see later.

Instance Lifetime

The CLR contains a tracing GC which can accurately, though non-deterministically, detect whether an object is still reachable. It is accurate because, unlike a conservative GC, it knows how to find all the references. It never leaves objects alive just because it can’t distinguish an object reference from an integer with the same coincidental set of bits. Our GC is non-deterministic because it optimizes for efficient memory utilization. It collects portions of the GC heap that it predicts will productively return memory to the heap, and only when it thinks the returned memory warrants the effort it will expend.

If the GC can see an orphaned cycle where A refers to B and B refers to A (but neither A nor B are otherwise reachable), it will collect that cycle. However, you can create cycles that the GC cannot trace through and which are therefore uncollectible. A simple way to do this is to have object A refer to object B via a GCHandle rather than a normal object reference. All handles are considered part of the root-set, so B (and thus A) is never collected.

The GC cannot trace through unmanaged memory either. Any cycles that involve COM objects will be uncollectible. It is the application’s responsibility to explicitly break the cycle by nulling a reference, or by calling ReleaseComObject, or by some other technique. Of course, this is standard practice in the COM world anyway.

Nor can the GC trace across processes. Instead, Managed Remoting uses a system of leases to achieve control over distributed lifetime. Calls on remote objects automatically extend the lease the client holds. Leases can trivially be made infinite, in which case the application is again responsible for breaking cycles so that collection can proceed. Alternatively, the application can provide a sponsor which will be notified before a remote object would be collected. This gives the application the opportunity to extend leases “on demand”, which reduces network traffic.

By default, if you don’t access a remote object for about 6 minutes, your lease will expire and your connection to that remote object is lost. You can try this yourself, with a remote object in a 2^nd process. But listen carefully: you can also try it with a remote object in a 2^nd AppDomain. If you leave your desk for a cup of tea, your cross-AppDomain references can actually timeout and disconnect!

Perhaps one day we will build a distributed GC that is accurate and non-deterministic across a group of processes or even machines. Frankly, I think it’s just as likely that we’ll continue to rely on techniques like configurable leases for cross-process or cross-machine lifetime management.

However, there’s no good reason for using that same mechanism cross-AppDomain. There’s a relatively simple way for us to trace object references across AppDomain boundaries – even in the presence of AppDomain unloading. This would be much more efficient than what we do today, and would relieve developers of a big source of problems.

We should fix this.

Type Identity

Managed objects can be marshaled across AppDomain boundaries according to one of several different plans:

Unmarshalable

This is the default for all types. If an object is not marked with the Serializable custom attribute, it cannot be marshaled. Any attempt to pass such an object across an AppDomain boundary will result in an exception.

Marshal-by-value

This is the default for all types that are marked as Serializable, unless they inherit from MarshalByRefObject. During a single marshal of a graph of objects, identity is preserved. But if the same object is marshaled on two separate calls from AppDomain1 to AppDomain2, this will result in two unrelated instances in AppDomain2.

Marshal-by-reference

Any Serializable types that inherit from System.MarshalByRefObject will marshal by reference. This causes an identity-preserving proxy to be created in the client’s AppDomain. Most calls and any field accesses on this proxy will remote the operation back to the server’s AppDomain. There are a couple of calls, defined on System.Object (like GetType), which might actually execute in the client’s AppDomain.

Marshal-by-bleed

Certain objects are allowed to bleed. For the most part, this bleeding is an optional performance optimization. For example, if you pass a String object as an argument on a call to a remoted MarshalByRefObject instance, the String is likely to bleed across the AppDomain boundary. But if you create a value type with an Object[] field, put that same String into the Object[], and pass the struct, the current marshaler might not bleed your String. Instead, it’s likely to be marshaled by value.

In other cases, we absolutely require that an instance marshal by bleed. System.Threading.Thread is a good example of this. The same managed thread can freely call between AppDomains. Since the current marshaler cannot guarantee that an instance will always bleed, we have made Thread unmarshalable by the marshaler for now. Then the CLR bleeds it without using the marshaler when you call Thread.CurrentThread.

Identity-preserving marshal-by-value

As we’ve seen, objects which marshal by value only preserve identity in a single marshaling operation, like a single remoted call. This means that, the more you call, the more objects you create. This is unacceptable for certain objects, like certain instances of System.Type. Instead, we marshal the type specifier from one AppDomain to another, effectively do a type load in the 2^nd AppDomain (finding any corresponding type that has already been loaded, of course) and then treat that type as the result of the unmarshal.

Custom marshaling

The Managed Remoting and serialization architectures are quite flexible. They contain sufficient extensibility for you to define your own marshaling semantics. Some researchers at Microsoft tried to build a system that transparently migrated objects to whatever client process was currently using them. I’m not sure how far they got.

How does all this relate to type identity? Well, instances of System.Type, and the metaobjects reachable from them like MethodInfos and PropertyInfos, can be marshaled in two different ways. If the underlying assembly was loaded as domain-neutral into the two AppDomains involved in a remote operation, then the metaobjects from that assembly will be marshaled-by-bleed. If instead the underlying assembly was loaded per-domain, then the metaobjects from that assembly will be identity-preserving marshaled-by-value.

Domain-neutrality

So what’s this domain-neutral vs. per-domain distinction? Remember when I said that a key to good performance is to have lots of shared pages and to minimize private pages? At the time, I was talking about sharing pages across processes. But the same is true of sharing pages across AppDomains. If all the AppDomains in a process can use the same JITted code, MethodTables, MethodDescs and other runtime structures, this will give us a dramatic performance boost when we create more AppDomains in that process.

If an assembly is loaded domain-neutral, we just mean that all these data structures and code are available in all the different AppDomains. If that same assembly is loaded per-domain, we have to duplicate all those structures between AppDomains.

In V1 and V1.1 of the CLR, we offer the following policies for determining which assemblies should be domain-neutral:

1) Only share mscorlib.dll. This choice is the default. We must always share mscorlib, because the operating system will only load one copy of mscorwks.dll (the CLR) into a process. And there are many 1:1 references backwards and forwards between mscorwks and mscorlib. For this reason, we need to be sure there’s only a single mscorlib.dll, shared across all the different AppDomains.

2) Share all strongly-named assemblies. This is the choice made by ASP.NET. It’s a reasonable choice for them because all ASP.NET infrastructure is strongly-named and happens to be used in all AppDomains. The code from web pages is not strongly-named and tends to be used only from a single AppDomain anyway.

3) Share all assemblies. I’m not aware of any host or application which uses this choice.

Wait a second. If sharing pages is such a great idea, why isn’t everyone using “Share all assemblies”? That’s because domain-neutral code has a couple of drawbacks. First and most importantly, domain-neutral code can never be unloaded. This is an unfortunate consequence of our implementation, though fixing it will be quite hard. It may be several more releases before we even try.

A second drawback is that domain-neutral code introduces a few inefficiencies. Usually the working set benefits quickly justify these inefficiencies, but there may be some scenarios (like single-AppDomain processes!) where this isn’t true. These inefficiencies include a 1:M lookup on all static field accesses and some high costs associated with deciding when to execute class constructors. That’s because the code is shared across all AppDomains, yet each AppDomain needs its own copy of static fields which are initialized through its own local execution of a .cctor method. You can reduce the overhead associated with .cctors (whether in domain-neutral code or not) by marking your .cctors with tdBeforeFieldInit. I’ve mentioned this in prior blogs.

Finally, in V1 & V1.1, we don’t allow you to combine NGEN with domain-neutral code. This may not be a concern for you, given the other limitations associated with NGEN today. And I’m confident that we’ll remove this particular restriction in a future release.

Okay, but this still sucks. Why are these choices so limited? Ideally a host would specify a set of its own assemblies and some FX assemblies for sharing. Since these assemblies would be intrinsic to the operation of the host, it wouldn’t matter that they can never unload. Then the application assemblies would be loaded per-domain.

We can’t support this because, if one assembly is loaded as domain-neutral, all the other assemblies in its binding closure must also be loaded as domain-neutral. This requirement is trivially satisfied by the first and third policies above. For the 2^nd policy, we rely on the fact that strong-named assemblies can only early-bind to other strong-named assemblies.

If we didn’t require an entire binding closure to be domain-neutral, then references from a domain-neutral assembly to a per-domain assembly would require a 1:M lookup, similar to what we do for static field accesses. It’s easy to see how this sort of lookup can work for static field access. But it’s much harder to see what kind of indirections would allow a domain-neutral type to inherit from a per-domain one. All the instance field offsets, base class methods, and VTable slots would need biasing via a 1:M lookup. Ouch.

In fact, long term we’re not trying to find some more flexible policies for a host to specify which assemblies can be loaded domain-neutral. It’s evil to have knobs that an application must set. We really want to reach a world where the CLR makes sensible decisions on the most appropriate way to execute any application. To get there, we would like to remove the inefficiencies and differing semantics associated with domain-neutral code and make such assemblies unloadable. Then we would like to train our loader to notice those AppDomains which will necessarily make identical binding decisions (more on this later). This will result in maximum automatic sharing.

It’s not yet clear whether/when we can achieve this ideal.

Per-AppDomain state like static fields

As stated above, domain-neutrality would ideally be a transparent optimization that the system applies on behalf of your application. There should be no observable semantics associated with this decision, other than performance.

Whether types are domain-neutral or not, each AppDomain must get its own copy of static fields. And a class constructor must run in each of those AppDomains, to ensure that these static fields are properly initialized.

Instance-agility

We just discussed how domain-neutrality refers to assemblies and how they are shared between AppDomains. Instance-agility refers to object instances and how they are allowed to flow between AppDomains.

An agile instance must necessarily be of a type we loaded as domain-neutral. However, the converse is not true. The vast majority of domain-neutral types do not have agile instances.

If an instance marshals-by-bleed or if it performs identity-preserving marshal-by-value, then by definition it is agile. The effect is the same in both cases: it’s possible to have direct references to the same instance from multiple AppDomains.

This is in contrast to normal non-agile instances which are created, live and die in a single AppDomain. We don’t bother to track which AppDomain these instances belong to, because we can infer this. If a thread is accessing an instance, then the instance is clearly in the same AppDomain that the thread is currently executing in. If we find references to an instance further back on a thread’s stack, then we can use the AppDomain transitions which are recorded on that stack to determine the correct AppDomain. And – for per-domain types – the type itself can tell us which AppDomain the instance belongs to.

Although we don’t normally track the AppDomain which contains an instance, there are some exceptions. For example, a Finalizable object must be finalized in the AppDomain it lives in. So when an instance is registered for finalization, we always record the current AppDomain at that time. And the finalizer thread(s) take care to batch up instances in the same AppDomain to minimize transitions.

For an instance to be agile, it must satisfy these rules:

It must be of a type that was loaded as domain-neutral. (Today, we restrict ourselves to types in mscorlib.dll, which is always domain-neutral).
The type must not be unloaded until the last instance has died. (Today, we never unload these types).
Instances must not have references to any other instances that are not themselves agile.

Based on these rules, it’s actually possible for the loader to identify some types as having legally agile instances. System.String is a good example, because it is sealed and has no references to other instances. However, this automatic detection would be inadequate for our purposes. We need some additional objects like System.Threading.Thread to be agile. Since Thread can contain references to many objects that are clearly not agile (like managed thread local storage, which contains arbitrary application objects), we have to be very careful here.

In this case, being careful means that we partition some of the Thread’s state in a per-AppDomain manner.

If you’ve read my earlier blogs, you know that static fields can be per-AppDomain, per-Thread, per-Context, or per-process (RVA-based statics). Now you know why the per-Thread and per-Context statics are still partitioned by AppDomain. And you understand why the per-process statics are restricted from containing arbitrary object references. They can only contain scalars, Strings (agile instances!) and value types that are themselves similarly constrained.

If you’ve done much debugging with AppDomains and exceptions, you’ve probably noticed that the first pass of exception handling is always terminated at an AppDomain boundary. It’s annoying: if the exception goes unhandled and you take your last chance as a trap to the debugger, you’ve lost the original context of the exception. But now it’s clear why this happens. If an exception instance isn’t agile, it must be marshaled from one AppDomain to the next as the dispatch occurs. (We make a special exception for an AppDomain-agile OutOfMemoryException that we pre-create, so that it’s available when we don’t have enough memory to make a per-AppDomain instance).

In fact, there’s a lot of complexity involved in ensuring that instances are only accessible from one AppDomain, or that they follow the discipline necessary for agility. You may be wondering why we care. We care because AppDomain isolation is a fundamental guarantee of the managed environment, on which many other guarantees can be built. In this sense, it is like separate address spaces for OS processes. Because of AppDomain isolation, we can build certain security guarantees and we can reclaim resources correctly when AppDomains are unloaded.

Configuration and Assembly Binding

Since each AppDomain is expected to execute a different application, each AppDomain can have its own private paths for binding to its assemblies, its own security policy, and in general its own configuration. Even worse, a host can listen to the AssemblyResolveEvent and dynamically affect binding decisions in each AppDomain. And the application can modify configuration information like the AppDomain’s private path – even as it runs. This sets up terrible data races, which rely on unfortunate side effects like the degree of inlining the JIT is performing and how lazy or aggressive the loader is in resolving dependent assemblies. Applications that rely on this sort of thing are very fragile from one release of the CLR to the next.

This also makes it very difficult for the loader to make sensible and efficient decisions about what assemblies can be shared. To do a perfect job, the loader would have to eagerly resolve entire binding closures in each AppDomain, to be sure that those AppDomains can share a single domain-neutral assembly.

Frankly, we gave the host and the application a lot of rope to hang themselves. In retrospect, we screwed up.

I suspect that in future versions we will try to dictate some reasonable limitations on what the host and the AppDomain’s configuration can do, at least in those cases where they want efficient and implicit sharing of domain-neutral assemblies to happen.

Unloading

A host or other sufficiently privileged code can explicitly unload any AppDomain it has a reference to, except for the default AppDomain which is not unloadable. The default AppDomain is the one that is created on your behalf when the process starts. This is the AppDomain a host typically chooses for its own execution.

The steps involved in an unload operation are generally as follows. As in many of these blogs, I’m describing implementation details and I’m doing so without reading any source code. Hopefully the reader can distinguish the model from the implementation details to understand which parts of the description can change arbitrarily over time.

Since the thread that calls AppDomain.Unload may itself have stack in the doomed AppDomain, a special helper thread is created to perform the unload attempt. This thread is cached, so every Unload doesn’t imply creation of a new thread. If we had a notion of task priorities in our ThreadPool, we would be using a ThreadPool thread here.

The unload thread sends a DomainUnload event to any interested listeners. Nothing bad has happened yet, when you receive this event.

The unload thread freezes the runtime. This is similar to the freeze that happens during (portions of) a garbage collection. It results in a barrier that prevents all managed execution.

While the barrier is in place for all managed execution, the unload thread erects a finer-grained barrier which prevents entry into the doomed AppDomain. Any attempt to call in will be rejected with a DomainUnloaded exception. The unload thread also examines the stacks of all managed threads to decide which ones must be unwound. Any thread with stack in the doomed AppDomain – even if it is currently executing in a different AppDomain – must be unwound. Some threads might have multiple disjoint regions of stack in the doomed AppDomain. When this is the case, we determine the base-most frame that must be unwound before this thread is no longer implicated in the doomed AppDomain.

The unload thread unfreezes the runtime. Of course, the finer-grained barrier remains in place to prevent any new threads from entering the doomed AppDomain.

The unload thread goes to work on unwinding the threads that it has identified. This is done by injecting ThreadAbortExceptions into those threads. Today we do this in a more heavy-weight but more scalable fashion than by calling Thread.Abort() on each thread, but the effect is largely the same. As with Thread.Abort, we are unable to take control of threads that are in unmanaged code. If such threads are stubborn and never return to the CLR, we have no choice but to timeout the Unload attempt, undo our partial work, and return failure to the calling thread. Therefore, we are careful to unwind the thread that called Unload only after all the others have unwound. We want to be sure we have a thread to return our failure to, if a timeout occurs!

When threads unwind with a ThreadAbortException, the Abort is propagated in the normal undeniable fashion. If a thread attempts to catch such an exception, we automatically re-raise the exception at the end of the catch clause. However, when the exception reaches that base-most frame we identified above, we convert the undeniable ThreadAbortException to a normal DomainUnloaded exception.

No threads can execute in the doomed AppDomain – except for a Finalizer thread which is now given a special privilege. We tell the Finalizer thread to scan its queue of ready-to-run finalizable objects and finalize all the ones in this AppDomain. We also tell it to scan its queue of finalizable but still reachable objects (not ready to run, under normal circumstances) and execute them, too. In other words, we are finalizing reachable / rooted objects if they are inside the doomed AppDomain. This is similar to what we do during a normal process shutdown. Obviously the act of finalization can create more finalizable objects. We keep going until they have all been eliminated.

During finalization, we are careful to skip over any agile reachable instances like Thread instances that were created in this AppDomain. They effectively escape from this AppDomain in a lazy fashion at this time. When these instances are eventually collected, they will be finalized in the default AppDomain, which is as good as anywhere else.

If we have any managed objects that were exposed to COM via CCWs, their lifetimes are partially controlled via COM reference counting rules. If the managed objects are to agile instances, we remove them from their AppDomain’s wrapper cache and install them in the default AppDomain’s wrapper cache. Like other agile objects, they have lazily survived the death of the AppDomain they were created in.

For all the non-agile CCWs (the vast majority), the managed objects are about to disappear. So we bash all the wrappers so that they continue to support AddRef and Release properly. All other calls return the appropriate HRESULT for DomainUnloadedException. The trick here, of course, is to retain enough metadata to balance the caller’s stack properly. When the caller drives the refcount to 0 on each wrapper, it will be cleaned up.

Now we stop reporting all the handles, if they refer to the doomed AppDomain, and we trigger a full GC. This should collect all the objects that live in this AppDomain. If it fails to do so, we have a corrupted GC heap and the process will soon die a terrible death.

Once this full GC has finished, we are free to unmap all the memory containing JITted code, MethodTables, MethodDescs, and all the other constructs. We also unload all the DLLs that we loaded specifically for this AppDomain.

In a perfect world, that last step returns all the memory associated with the AppDomain. During V1, we had a leak detection test that tried to verify this. Once we reached a steady-state in the test cycle, after unloading the first few AppDomains, we got pretty close to our ideal. It’s harder to measure than you might imagine, due to things like delayed coalescing of OS heap structures. According to our measurements, we were leaking 12 bytes per unloaded AppDomain – of which 4 bytes was almost by design. (It was the ID of the unloaded AppDomain). I have no idea how well we are doing these days.

In a scenario where lots of unloads are happening, it’s unfortunate that we do a full GC for each one. For those cases, we would like to defer the full GC and the reclamation of resources until the next time that the GC is actually scheduled. …One day.

There’s so much more I had intended to write about. For example, some ambiguities exist when unmanaged (process-wide) code calls into Managed C++ and has to select a target AppDomain. This can be controlled by flags in the VTFixup entries that are used by the IJW thunks. And customers often ask us for alternatives to AppDomain unloading, like unloading individual methods, unloading individual assemblies, or unloading unreferenced domain-neutral assemblies. There are many interesting programming model issues, like the reason why we have a CreateInstanceAndUnwrap method on AppDomain.

But even I think this blog is getting way too long.

Access to old blogs

cbrumme — Sun, 18 May 2003 20:40:00 GMT

Memory Model

cbrumme — Sun, 18 May 2003 01:56:00 GMT

One of the suggestions for a blog entry was the managed memory model. This is timely, because we’ve just been revising our overall approach to this confusing topic. For the most part, I write about product decisions that have already been made and shipped. In this note, I’m talking about future directions. Be skeptical.

So what is a memory model? It’s the abstraction that makes the reality of today’s exotic hardware comprehensible to software developers.

The reality of hardware is that CPUs are renaming registers, performing speculative and out-of-order execution, and fixing up the world during retirement. Memory state is cached at various levels in the system (L0 thru L3 on modern X86 boxes, presumably with more levels on the way). Some levels of cache are shared between particular CPUs but not others. For example, L0 is typically per-CPU but a hyper-threaded CPU may share L0 between the logical CPUs of a single physical CPU. Or an 8-way box may split the system into two hemispheres with cache controllers performing an elaborate coherency protocol between these separate hemispheres. If you consider caching effects, at some level all MP (multi-processor) computers are NUMA (non-uniform memory access). But there’s enough magic going on that even a Unisys 32-way can generally be considered as UMA by developers.

It’s reasonable for the CLR to know as much as possible about the cache architecture of your hardware so that it can exploit any imbalances. For example, the developers on our performance team have experimented with a scalable rendezvous for phases of the GC. The idea was that each CPU establishes a rendezvous with the CPU that is “closest” to it in distance in the cache hierarchy, and then one of this pair cascades up a tree to its closest neighbor until we reach a single root CPU. At that point, the rendezvous is complete. I think the jury is still out on this particular technique, but they have found some other techniques that really pay off on the larger systems.

Of course, it’s absolutely unreasonable for any managed developer (or 99.99% of unmanaged developers) to ever concern themselves with these imbalances. Instead, software developers want to treat all computers as equivalent. For managed developers, the CLR is the computer and it better work consistently regardless of the underlying machine.

Although managed developers shouldn’t know the difference between a 4-way AMD server and an Intel P4 hyper-threaded dual proc, they still need to face the realities of today’s hardware. Today, I think the penalty of a CPU cache miss that goes all the way to main memory is about 1/10^th the penalty of a memory miss that goes all the way to disk. And the trend is clear.

If you wanted good performance on a virtual memory system, you’ve always been responsible for relieving the paging system by getting good page density and locality in your data structures and access patterns.

In a similar vein, if you want good performance on today’s hardware, where accessing main memory is a small disaster, you must pack your data into cache lines and limit indirections. If you are building shared data structures, consider separating any data that’s subject to false sharing.

To some extent, the CLR can help you here. On MP machines, we use lock-free allocators which (statistically) guarantee locality for each thread’s allocations. Any compaction will (statistically) preserve that locality. Looking into the very far future – perhaps after our sun explodes – you could imagine a CLR that can reorganize your data structures to achieve even better performance.

This means that if you are writing single-threaded managed code to process a server request, and if you can avoid writing to any shared state, you are probably going to be pretty scalable without even trying.

Getting back to memory models, what is the abstraction that will make sense of current hardware? It’s a simplifying model where all the cache levels disappear. We pretend that all the CPUs are attached to a single shared memory. Now we just need to know whether all the CPUs see the same state in that memory, or if it’s possible for some of them to see reordering in the loads and stores that occur on other CPUs.

At one extreme, we have a world where all the CPUs see a single consistent memory. All the loads and stores expressed in programs are performed in a serialized manner and nobody perceives a particular thread’s loads or stores being reordered. That’s a wonderfully sane model which is easy for software developers to comprehend and program to. Unfortunately, it is far too slow and non-scalable. Nobody builds this.

At the other extreme, we have a world where CPUs operate almost entirely out of private cache. If another CPU ever sees anything my CPU is doing, it’s a total accident of timing. Because loads and stores can propagate to other CPUs in any random order, performance and scaling are great. But it is impossible for humans to program to this model.

In between those extremes are a lot of different possibilities. Those possibilities are explained in terms of acquire and release semantics:

A normal load or store can be freely reordered with respect to other normal load or store operations.
A load with acquire semantics creates a downwards fence. This means that normal loads and stores can be moved down past the load.acquire, but nothing can be moved to above the load.acquire.
A store with release semantics creates an upwards fence. This means that normal loads and stores can be moved above the store.release, but nothing can be moved to below the store.release.
A full fence is effectively an upwards and downwards fence. Nothing can move in either direction across a full fence.

A super-strong extreme model puts a full fence after every load or store. A super-weak extreme model uses normal loads and stores everywhere, with no fencing.

The most familiar model is X86. It’s a relatively strong model. Stores are never reordered with respect to other stores. But, in the absence of data dependence, loads can be reordered with respect to other loads and stores. Many X86 developers don’t realize that this reordering is possible, though it can lead to some nasty failures under stress on big MP machines.

In terms of the above, the memory model for X86 can be described as:

All stores are actually store.release.
All loads are normal loads.
Any use of the LOCK prefix (e.g. ‘LOCK CMPXCHG’ or ‘LOCK INC’) creates a full fence.

Historically, Windows NT has run on Alpha and MIPS computers.

Looking forwards, Microsoft has announced that Windows will support Intel’s IA64 and AMD’s AMD64 processors. Eventually, we need to port the CLR to wherever Windows runs. You can draw an obvious conclusion from these facts.

AMD64 has the same memory model as X86.

IA64 specifies a weaker memory model than X86. Specifically, all loads and stores are normal loads and stores. The application must use special ld.acq and st.rel instructions to achieve acquire and release semantics. There’s also a full fence instruction, though I can’t remember the opcode (mf?).

Be especially skeptical when you read the next paragraph:

There’s some reason to believe that current IA64 hardware actually implements a stronger model than is specified. Based on informed hearsay and lots of experimental evidence, it looks like normal store instructions on current IA64 hardware are retired in order with release semantics.

If this is indeed the case, why would Intel specify something weaker than what they have built? Presumably they would do this to leave the door open for a weaker (i.e. faster and more scalable) implementation in the future.

In fact, the CLR has done exactly the same thing. Section 12.6 of Partition I of the ECMA CLI specification explains our memory model. This explains the alignment rules, byte ordering, the atomicity of loads and stores, volatile semantics, locking behavior, etc. According to that specification, an application must use volatile loads and volatile stores to achieve acquire and release semantics. Normal loads and stores can be freely reordered, as seen by other CPUs.

What is the practical implication of this? Consider the standard double-locking protocol:

if (a == null)

{

lock(obj)

{

if (a == null) a = new A();

}

This is a common technique for avoiding a lock on the read of ‘a’ in the typical case. It works just fine on X86. But it would be broken by a legal but weak implementation of the ECMA CLI spec. It’s true that, according to the ECMA spec, acquiring a lock has acquire semantics and releasing a lock has release semantics.

However, we have to assume that a series of stores have taken place during construction of ‘a’. Those stores can be arbitrarily reordered, including the possibility of delaying them until after the publishing store which assigns the new object to ‘a’. At that point, there is a small window before the store.release implied by leaving the lock. Inside that window, other CPUs can navigate through the reference ‘a’ and see a partially constructed instance.

We could fix this code in various ways. For example, we could insert a memory barrier of some sort after construction and before assignment to ‘a’. Or – if construction of ‘a’ has no side effects – we could move the assignment outside the lock, and use an Interlocked.CompareExchange to ensure that assignment only happens once. The GC would collect any extra ‘A’ instances created by this race.

I hope that this example has convinced you that you don’t want to try writing reliable code against the documented CLI model.

I wrote a fair amount of “clever” lock-free thread-safe code in version 1 of the CLR. This included techniques like lock-free synchronization between the class loader, the prestub (which traps first calls on methods so it can generate code for them), and AppDomain unloading so that I could back-patch MethodTable slots efficiently. But I have no desire to write any kind of code on a system that’s as weak as the ECMA CLI spec.

Even if I tried to write code that is robust under that memory model, I have no hardware that I could test it on. X86, AMD64 and (presumably) IA64 are stronger than what we specified.

In my opinion, we screwed up when we specified the ECMA memory model. That model is unreasonable because:

All stores to shared memory really require a volatile prefix.
This is not a productive way to code.
Developers will often make mistakes as they follow this onerous discipline.
These mistakes cannot be discovered through testing, because the hardware is too strong.

So what would make a sensible memory model for the CLR?

Well, first we would want to have a consistent model across all CLI implementations. This would include the CLR, Rotor, the Compact Frameworks, SPOT, and – ideally – non-Microsoft implementations like Mono. So putting a common memory model into an ECMA spec was definitely a good idea.

It goes without saying that this model should be consistent across all possible CPUs. We’re in big trouble if everyone is testing on X86 but then deploying on Alpha (which had a notoriously weak model).

We would also want to have a consistent model between the native code generator (JIT or NGEN) and the CPU. It doesn’t make sense to constrain the JIT or NGEN to order stores, but then allow the CPU to reorder those stores. Or vice versa.

Ideally, the IL generator would also follow the same model. In other words, your C# compiler should be allowed to reorder whatever the native code generator and CPU are allowed to reorder. There’s some debate whether the converse is true. Arguably, it is okay for an IL generator to apply more aggressive optimizations than the native code generator and CPU are permitted, because IL generation occurs on the developer’s box and is subject to testing.

Ultimately, that last point is a language decision rather than a CLR decision. Some IL generators, like ILASM, will rigorously emit IL in the sequence specified by the source code. Other IL generators, like Managed C++, might pursue aggressive reordering based on their own language rules and compiler optimization switches. If I had to guess, IL generators like the Microsoft compilers for C# and VB.NET would decide to respect the CLR’s memory model.

We’ve spent a lot of time thinking about what the correct memory model for the CLR should be. If I had to guess, we’re going to switch from the ECMA model to the following model. I think that we will try to persuade other CLI implementations to adopt this same model, and that we will try to change the ECMA specification to reflect this.

Memory ordering only applies to locations which can be globally visible or locations that are marked volatile. Any locals that are not address exposed can be optimized without using memory ordering as a constraint since these locations cannot be touched by multiple threads in parallel.
Non-volatile loads can be reordered freely.
Every store (regardless of volatile marking) is considered a release.
Volatile loads are considered acquire.
Device oriented software may need special programmer care. Volatile stores are still required for any access of device memory. This is typically not a concern for the managed developer.

If you’re thinking this looks an awful lot like X86, AMD64 and (presumably) IA64, you are right. We also think it hits the sweet spots for compilers. Reordering loads is much more important for enabling optimizations than reordering stores.

So what happens in 10 years when these architectures are gone and we’re all using futuristic Starbucks computers with an ultra-weak model? Well, hopefully I’ll be living the good life in retirement on Maui. But the CLR’s native code generators will generate whatever instructions are necessary to keep stores ordered when executing your existing programs. Obviously this will sacrifice some performance.

The trade-off between developer productivity and computer performance is really an economic one. If there’s sufficient incentive to write code to a weak memory model so it can execute efficiently on future computers, then developers will do so. At that point, we will allow them to mark their assemblies (or individual methods) to indicate that they are “weak model clean”. This will permit the native code generator to emit normal stores rather than store.release instructions. You’ll be able to achieve high performance on weak machines, but this will always be “opt in”. And we won’t build this capability until there’s a real demand for it.

I personally believe that for mainstream computing, weak memory models will never catch on with human developers. Human productivity and software reliability are more important than the increment of performance and scaling these models provide.

Finally, I think the person asking about memory models was really interested in where he should use volatile and fences in his code. Here’s my advice:

Use managed locks like Monitor.Enter (C# lock / VB.NET synclock) for synchronization, except where performance really requires you to be “clever”.
When you’re being “clever”, assume the relatively strong model I described above. Only loads are subject to re-ordering.
If you have more than a few places that you are using volatile, you’re probably being too clever. Consider backing off and using managed locks instead.
Realize that synchronization is expensive. The full fence implied by Interlocked.Increment can be many 100’s of cycles on modern hardware. That penalty may continue to grow, in relative terms.
Consider locality and caching effects like hot spots due to false sharing.
Stress test for days with the biggest MP box you can get your hands on.
Take everything I said with a grain of salt.

Value Types

cbrumme — Sun, 11 May 2003 01:33:00 GMT

The CLR’s type system includes primitive types like signed and unsigned integers of various sizes, booleans and floating point types. It also includes partial support for types like pointers and function pointers. And it contains some rather exotic beasts, like ArgIterators and TypedByRefs. (These are exotic because their lifetimes are restricted to a scope on the stack, so they can never be boxed, embedded in a class, or otherwise appear in the GC heap). Lastly, but most importantly, the type system includes interfaces, classes and value types.

In fact, if you look at our primitive types the right way, they’re really just some value types that are so popular and intrinsic that we gave them special encoding in our type signatures and instructions.

The CLR also supports a flexible / weak kind of enumeration. Our enums are really just a specialization of normal value types which conform to some extra conventions. From the CLR’s perspective, enums are type distinct aliases that otherwise reduce to their underlying primitive type. This is probably not the way anyone else thinks of them, so I’ll explain in more detail later.

Anyway as we’ve seen our type system has value types all over the place – as structs, enums, and primitive scalars. And there are some rather interesting aspects to their design and implementation.

The principal goal of value types was to improve performance over what could be achieved with classes. There are some aspects of classes which have unavoidable performance implications:

All instances of classes live in the GC heap. Our GC allocator and our generation 0 collections are extremely fast. Yet GC allocation and collection can never be as fast as stack allocation of locals, where the compiler can establish or reclaim an entire frame of value types and primitives with a single adjustment to the stack pointer.
All instances of classes are self-describing. In today’s implementation, we use a pointer-sized data slot on every instance to tag that instance’s type. This single slot enables us to perform dynamic casting, virtual dispatch, embedded GC pointer reporting and a host of other useful operations. But sometimes you just cannot afford to burn that data slot, or to initialize it during construction. If you have an array of 10,000 value types, you really don’t want to place that tag 10,000 times through memory – especially if dirtying the CPU’s cache in this way isn’t going to improve the application’s subsequent accesses.
Instances of classes can never be embedded in other instances. All logical embedding is actually achieved by reference. This is the case because our object-oriented model allows “is-a” substitutability. It’s hard to achieve efficient execution if subtypes can be embedded into an instance, forcing all offsets to be indirected. Of course, the CLR is a virtualized execution environment so I suspect we could actually give the illusion of class embedding. However, many unmanaged structures in Win32 are composed of structs embedded in structs. The illusion of embedding would never achieve the performance of true embedding when blittable types are passed across the managed / unmanaged boundary. The performance impact of marshaling would certainly weaken our illusion.

If you look at the class hierarchy, you find that all value types derive from System.Object. Whether this is indeed true is a matter of opinion. Certainly value types have a layout that is not an extension of the parent Object’s layout. For example, they lack the self-describing tag. It’s more accurate to say that value types, when boxed, derive from System.Object. Here’s the relevant part of the class hierarchy:

System.Object

/ \

most classes System.ValueType

/ \

most value types System.Enum

all enums

Why do I use the term “most classes” in this hierarchy? Because there are several classes that don’t appear in that section of the hierarchy. System.Object is the obvious one. And, paradoxically, System.ValueType is actually a class, rather than a value type. Along the same lines System.Enum, despite being a subtype of System.ValueType, is neither a value type nor an enum. Rather it’s a base class under which all enums are parented.

Incidentally, something similar is going on with System.Array and all the array types. In terms of layout, System.Array really isn’t an array. But it does serve as the base class under which all kinds of arrays (single-dimension, multi-dimension, zero-lower-bounds and non-zero-lower-bounds) are parented.

Now is probably a good time to address one of the glaring differences between the ECMA spec and our implementation. According to the ECMA spec, it should be possible to specify either a boxed or an unboxed value type. This is indicated by using either ELEMENT_TYPE_VALUETYPE <token> or ELEMENT_TYPE_CLASS <token>. By making this distinction, you could have method arguments or array elements or fields that are of type “boxed myStruct”. The CLR actually implemented a little of this, and then cut the feature because of schedule risk. Presumably we’ll implement it properly some day, to achieve ECMA conformance. Until then, we will refuse to load applications that attempt to specify well-typed boxed value types.

I mentioned earlier that the CLR thinks of enums rather differently than the average developer. Inside the CLR, an enum is a type-distinct alias. We generally treat the enum as an alias for the underlying integral type that is the type of the enum’s __value field. This alias is type-distinct because it can be used for overloading purposes. A class can have three methods that are distinguished only by the fact that they one takes MyEnum vs. YourEnum vs. the underlying integral type as an argument.

Beyond that, the CLR should not attach any significance to the enum. In particular, we do no validation that the values of the enum ever match any of the declared enumerands.

I say the CLR “should not” attach any significance, but the model shows some rough edges if you look closely. When an enum is unboxed and is in its value type form, we only have static type information to guide us. We tend to discard this static typing information and reduce the type to its underlying integral type. You can actually assign a value of MyEnum to a variable of type YourEnum, as far as the JIT and verifier are concerned. But as soon as an enum is boxed, it becomes self-describing. At that point, cast operations and covariant array typechecks tend to be picky about whether you’ve got a boxed MyEnum or a boxed YourEnum. As one of the architects of the C# compiler remarked, “Enums are treated exactly like their underlying types, except when they aren’t.” This is unfortunate and ideally we should clean this up some day.

While we’re on the subject of using enums to create distinct overloads, it makes sense to mention custom signature modifiers. These modifiers provide an extensibility point in the type system which allows sophisticated IL generators to attach significance to types. For example, I believe Managed C++ expresses their notion of ‘const’ through a custom signature modifier that they can attach to method arguments. Custom signature modifiers come in two forms. In the first form, they simply create enough of a difference between otherwise identical signatures to support overloading. In their second form, they also express some semantics. If another IL generator doesn’t understand those semantics, it should not consume that member.

So an IL generator could attach custom signature modifiers to arguments of an integral type, and achieve the same sort of type-distinct aliasing that enums provide.

Today, custom signature modifiers have one disappointing gap. If you have a method that takes no arguments and returns void, there isn’t a type in the signature that you can modify to make it distinct. I don’t think we’ve come up with a good way to address this yet. (Perhaps we could support custom signature modifier on the calling convention?)

Back to value types. Instance methods, whether virtual or non-virtual, have an implicit ‘this’ argument. This argument is not expressed in the signature. Therefore it’s not immediately obvious that a method like “void m(int)” actually has a different true signature depending on whether the method appears on a class or on a value type. If we add back the implicit ‘this’ for illustration purposes, the true signatures are really:

void m( [ MyClass this], int arg)

void m( [ref MyStruct this], int arg)

It’s not surprising that ‘this’ is MyClass in one case and MyStruct in the other case. What may be a little surprising is that ‘this’ is actually a byref in the value type case. This is necessary if we are to support mutator methods on a value type. Otherwise any changes to ‘this’ would be through a temporary which would subsequently be discarded.

Now we get to the interesting part. Object has a number of virtual methods like Equals and GetHashCode. We now know that these methods have implicit ‘this’ arguments of type Object. It’s easy to see how System.ValueType and System.Enum can override these methods, since we’ve learned that these types are actually classes rather than value types or enums.

But what happens when MyStruct overrides GetHashCode? Somehow, the implicit ‘this’ argument needs to be ‘ref MyStruct’ when the dispatch arrives at MyStruct’s implementation. But the callsite clearly cannot be responsible for this, since the callsite calls polymorphically on boxed value types and other class instances. It should be clear that a similar situation can occur with any interface methods that are implemented by a value type.

Something must be converting the boxed value type into a byref to the unboxed value type. This ‘something’ is an unboxing stub which is transparently inserted into the call path. If an implementation uses vtables to dispatch virtual methods, one obvious way to insert an unboxing stub into the call path is to patch the vtable slot with the stub address. On X86, the unboxing stub could be very efficient:

add ecx, 4 ; bias ‘this’ past the self-describing tag

jmp <target> ; now we’re ready for the ‘ref struct’ method

Indeed, even the JMP could be removed by placing the unboxing stub right before the method body (effectively creating dual entrypoints for the method).

At polymorphic callsites, the best we can do is vector through a lightweight unboxing stub. But in many cases the callsite knows the exact type of the value type. That’s because it’s operating on a well-typed local, argument, or field reference. Remember that value types cannot be sub-typed, so substitutability of the underlying value type is not a concern.

This implies that the IL generator has two code generation strategies available to it, when dispatching an interface method or Object virtual method on a well-typed value type instance. It can box it and make the call as in the polymorphic case. Or it can try to find a method on the value type that corresponds to this contract and takes a byref to the value type, and then call this method directly.

Which technique should the IL generator favor? Well, if the method is a mutator there may be a loss of side effects if the value type is boxed and then discarded; the IL generator may need to back-propagate the changes if it goes the boxing route. Also, boxing is an efficient operation, but it necessarily involves allocating an object in the GC heap. So the boxing approach can never be as fast as the ‘byref value type’ approach.

So why wouldn’t an IL generator always favor the ‘byref value type’ approach? One disadvantage is that finding the correct method to call can be challenging. In an earlier blog (Interface layout), I revealed some of this subtlety. The compiler would have to consider MethodImpls, whether the interface is redundantly mentioned in the ‘implements’ clause, and several other points in order to predict what the class loader will do.

But let’s say our IL generator is sophisticated enough to do this. It still might prefer the boxing approach, so it can be resilient to versioning changes. If the value type is defined in a different assembly than the callsite, the value type’s implementation can evolve independently. The value type has made a contract that it will implement an interface, but it has not guaranteed which method will be used to satisfy that interface contract. Theoretically, it could use a MethodImpl to satisfy ‘I.xyz’ using a class method called ‘abc’ in one version and a method called ‘jkl’ in some future version. In practice, this is unlikely and some sophisticated compilers predict the method body to call and then hope that subsequent versions won’t invalidate the resulting program.

Given that a class or value type can re-implement a contract in subsequent versions, consider the following scenario:

class Object { public virtual int GetHashCode() {…} … }

class ValueType : Object { public override int GetHashCode() {…} … }

struct MyVT : ValueType { public override int GetHashCode() {…} …}

As we know, MyVT.GetHashCode() has a different actual signature, taking a ‘ref MyVT’ as the implicit ‘this’. Let’s say an IL generator takes the efficient but risky route of generating a call on a local directly to MyVT.GetHashCode. If a future version of MyVT decides it is satisfied with its parent’s implementation, it might remove this override. If value types weren’t involved, this would be an entirely safe change. We already saw in one of my earlier blogs (Virtual and non-virtual) that the CLR will bind calls up the hierarchy. But for value types, the signature is changing underneath us.

Today, we consider this scenario to be illegal. The callsite will fail to bind to a method and the program is rejected as invalid. Theoretically, the CLR could make this scenario work. Just as we insert unboxing stubs to match an ‘Object this’ callsite to a ‘ref MyVT this’ method body, we could also create and insert reboxing stubs to match a ‘ref MyVT’ callsite to an ‘Object this’ method body.

This would be symmetrical. And it’s the sort of magic that you would naturally expect a virtual execution environment like the CLR to do. As with so many things, we haven’t got around to even seriously considering it yet.

Request a topic

cbrumme — Fri, 09 May 2003 06:06:00 GMT

If there's a topic related to the CLR, feel free to drop me a line asking me to talk about it. I have a very time-consuming day job and a full life outside of work, so expect a long delay before I address your topic. Or I might feel I have nothing interesting to say about it... But you never know.

Obviously I can't help you with stuff like "I'm trying to get my Web Service working with Passport and..." or "While I'm waiting for a Mouse Up during a drag operation..."

I'm more suited to questions like, "How do you maintain type safety when you dispatch methods on value types, since the instances aren't self-describing?"

Security and Asynchrony

cbrumme — Fri, 09 May 2003 05:28:00 GMT

In a comment to my last ramble, about asynchronous execution and pinning, someone asked for advice on using Windows impersonation in a managed application. Unfortunately, the managed platform currently has poor abstractions and infrastructure for controlling Windows identity, and indeed for most of the unmanaged Windows security system. For example, the managed classes for WaitHandles and Streams lack overloads for specifying SECURITY_ATTRIBUTES. It’s true that we have defined some classes like System.Security.Principal.WindowsIdentity and WindowsPrincipal, but I don’t think these classes add enough value in their current form.

For now, you might even decide to avoid the managed abstractions and PInvoke to the underlying OS services, like RevertToSelf and SetThreadToken. Be aware that this technique won’t work well in a fiber-based environment like SQL Server. In that world, a logical thread (i.e. fiber) might be switched to a different operating system thread after you PInvoke to initiate impersonation. If a fiber switch does indeed happen at that time, a different logical thread will now execute inside your impersonated context.

We certainly understand the limitations of our current support and we’re working to provide better abstractions in a future release.

The reason for this poor support is perhaps obvious. In managed code, the focus of security is Code Access Security, not operating system concepts like impersonation. We put a lot of effort into capturing CAS state and propagating it automatically through asynchronous operations. For example, if Thread 1 creates and starts a new managed Thread 2, the CAS stack information from Thread 1 is automatically captured and propagated to the base of Thread 2. When you call the normal ThreadPool operations, a similar capture and propagation of stack evidence occurs. A sophisticated and trusted client of the ThreadPool can trade off that implicit security for better performance, by using ‘unsafe’ operations like UnsafeQueueUserWorkItem and UnsafeWaitForSingleObject. A similarly sophisticated and trusted client could PInvoke to CreateThread, to avoid attaching his CAS information to the new thread.

Why do we propagate the CAS information from one thread to another in this manner?

Well, System.Environment.Exit() can be used to terminate the current process. This is considered a privileged operation, so it’s protected by a Demand for UnmanagedCodePermission. (I can’t remember if it’s a FullDemand or a LinkDemand. For the purposes of this blog, let’s pretend it’s a FullDemand). Because of the demand, partially trusted code cannot call this API directly. If it does attempt the call, the security system will examine the stack and discover that partially trusted code is involved in the operation. A security exception will be thrown.

But what if the partially trusted code can find a delegate declaration with the same signature as Exit()? There are plenty of fully trusted delegate declarations, like System.Threading.ThreadStart. (I realize that ThreadStart doesn’t have quite the right signature for Environment.Exit, but you get the idea). If the partially trusted code can form one of these delegates over the Exit() method and then queue the delegate for execution on the threadpool, it can mount a security attack. That’s because a threadpool thread will now call Exit() and satisfy the security Demand. An examination of the stack would not find any code from the partially trusted attacker.

We prevent this attack by capturing the stack of the partially trusted caller when he calls ThreadPool.QueueUserWorkItem. Then when the stack crawl is initiated by the Demand on Exit(), we consider that captured stack. We discover the partially trusted code on the captured stack and fail the Demand.

In addition to creating a thread or initiating a safe ThreadPool operation, we also capture and propagate CAS information in services like System.Windows.Forms.Control.BeginInvoke.

However, we do not capture and propagate CAS information for the most common asynchronous operation of them all – finalization. I can give two reasons to rationalize this fact.

Finalization is intended for cleaning up resources rather than for arbitrary execution. The body of the finalize method should be self-contained; it should be designed so it is not subject to re-purposing attacks. For example, fully trusted code should never expose an object that will call through an arbitrary delegate from its Finalize() method.

The performance impact of capturing and propagating stack information on each finalizable object would be unacceptable. It’s an unfortunate fact of life that performance and security are often at odds with each other. The best we can hope for is to strike an appropriate balance between these competing needs.

What if you need to do something delicate in a Finalize() method? More generally, what if you are building your own ThreadPool or your own queue of server requests? (Of course, there are many good reasons for using our ThreadPool rather than writing your own, but let’s ignore this for a moment). Ultimately, any array of objects that’s shared between two threads can be a scenario where the thread inserting into the array might need to propagate its CAS information to the thread that is removing objects from that array and operating on them.

The solution to this problem is for you to call System.Threading.Thread.GetCompressedStack() and SetCompressedStack() yourself. Of course, you need to have a high level of privilege (probably ControlEvidence) before you can do this. These APIs were not public in our V1 release, but they are publicly available in 1.1.

If you go this route, there is one important detail you should be aware of. The current behavior of this API is to place the “attached” CAS compressed stack at the base of the target thread. It is not inserted into the new thread’s stack at the current stack location. For normal stack crawls, this detail won’t matter. But if your stacks contain combinations of Deny, Assert and PermitOnly statements, then position is significant. By considering these statements out of order – which is our current behavior – it’s theoretically possible to get different results.

For example, you might Assert and then pick up a request with a compressed stack which you install with a SetCompressedStack. This is a questionable practice already, because you really shouldn’t execute arbitrary code inside the scope of an Assert. You should try to contain the scope of an Assert as much as possible. Along the same lines, I’m personally uncomfortable with applications that base their security on Deny or PermitOnly statements. Such statements can always be trumped by a subsequent Assert.

Anyway, transferring a compressed stack is generally much more secure than not transferring the compressed stack. So any concerns about subtle interactions with Assert, Deny & PermitOnly based on the order in which we consider the current thread’s stack and the transferred stack are secondary.

Incidentally, Get/SetCompressedStack has a nifty merging mechanism which can avoid some common overflow scenarios. Imagine what happens if you queue an asynchronous read. The API you call will capture your compressed stack and flow it through the threadpool. When the threadpool uses one of its threads to call your completion, the caller’s CAS information is available as we have seen. A perfectly reasonable thing to do in the completion callback is to initiate a new asynchronous read, and return. Now rinse and repeat indefinitely.

In terms of the stacks of the operating system threads, they are all nicely unwound by virtue of performing asynchronous operations. But in terms of the CAS compressed stacks, their growth is unbounded. When we initiate the 1000’th asynchronous read, the prior 999 stacks are all being propagated along. What makes this particularly painful is that at least 998 of those compressed stacks are completely identical! The additional stacks typically convey no new security information.

Fortunately, the mechanism for capturing and merging compressed stacks contains a simple pattern recognizer. In this sort of scenario, the pattern recognizer will discard any obvious redundancies. The CAS information quickly finds a fixed point.

I’ve already pointed out that there’s a spectrum of asynchronous operations. At one end of the spectrum, we have obviously asynchronous scenarios like Stream.BeginRead, ThreadPool.QueueUserWorkItem, Thread.Start and raising Events. At the other end of the spectrum, we have subtly asynchronous scenarios like one thread calling through an object that was placed into a shared static reference by another thread. Ultimately, if you have shared memory and multiple threads, you have the potential for asynchrony and security attacks.

This is troubling, because there isn’t a bright line between risky operations that need securing via techniques like transferred compressed stacks versus normal safe operations which don’t warrant the overhead of stack transfers.

One scenario that’s particularly troubling to our team is events. What if we can find an event that’s raised by some fully trusted code with a signature that matches System.Environment.Exit()? Well, we could wire up the fully trusted caller (the event source) to the fully trusted but dangerous Exit service (the event sink) using a fully trusted delegate of the appropriate signature.

At that point, we just need to wait for the event to fire and the process will terminate. There is no partially trusted code on the stack.

We’ve discussed many ways to solve this problem. Most of them have a clumsy programming model. All of them have a significant performance impact. None of them do a great job of solving all the attacks possible with indirect calls (i.e. non-Event usage of delegates, and indirections through well-known interface methods or virtual methods).

Indeed, Events are probably the least susceptible to attack of all the indirect call attacks. That’s because almost all Events on our platform share an idiosyncratic signature of (Object sender, EventArgs args). An attacker isn’t going to find a lot of powerful APIs like Exit that have the same signature. Indeed, checking for dangerous methods with this sort of signature is just one of the many, many security audits that we perform throughout our frameworks before shipping a release.

Still, it’s definitely an area where we would like to do better, and where we shall continue to invest design effort.

Asynchronous operations, pinning

cbrumme — Wed, 07 May 2003 04:41:00 GMT

One thing we tried to do with the CLR and FX is provide a consistent asynchronous programming model.

To briefly recap the model, an API called XXX may also offer an async alternative composed of BeginXXX and EndXXX methods. Even if the class that implements XXX doesn’t also offer BeginXXX and EndXXX, you can define a Delegate class whose signature is consistent with the signature of XXX. On that Delegate, you will find BeginXXX and EndXXX methods defined, which can be used to call the XXX method asynchronously.

The BeginXXX method takes the inbound arguments, an optional state object and an optional callback delegate. It returns an implementation of IAsyncResult that can be used to rendezvous with the completion.

The managed asynchronous programming model provides a choice of four different ways to rendezvous with the completion:

The asynchronous provider calls the delegate callback specified in the BeginXXX call, passing the optional state object.
The initiator polls for completion, using the IAsyncResult.IsComplete property.
The initiator waits for an event to be signaled, via IAsyncResult.WaitHandle.
The initiator blocks until the asynchronous operation completes, by calling the EndXXX API.

Of these four techniques, the first is by far the most popular and arguably the easiest for developers to code to.

The second could be used in a highly scalable server, which can afford a dedicated thread to routinely poll all outstanding asynchronous operations and process any that have completed.

The third technique can be used to process each operation as it completes (WaitHandle.WaitAny) or to process all operations after the last one completes (WaitHandle.WaitAll). Because WaitHandles are expensive resources, a sophisticated implementation of IAsyncResult may delay materializing the handle until a client requests it. In most cases, the client will select a different rendezvous method and the WaitHandle is never needed.

The fourth technique is the hardest to understand. Why initiate an operation asynchronously if you intend to rendezvous with it synchronously? But this can make sense if the application is interleaving a finite amount of synchronous processing with the asynchronous operation, to reduce latency. Once the synchronous processing is complete, it may make sense to block.

Regardless of which of these techniques is used to achieve the rendezvous, the final step of the completion is to call the EndXXX API to retrieve the return value, any outbound arguments, or possibly an exception. If the rendezvous is of the first form, the EndXXX method is probably called directly out of the callback.

Once the EndXXX API returns, the operation is fully complete and the IAsyncResult serves no further purpose. Since there may be significant resources associated with the operation, the IAsyncResult implementation might treat EndXXX as the equivalent of IDisposable.Dispose(). For instance, any materialized WaitHandle can be disposed at this time.

One of the most common questions related to the managed asynchronous programming model is whether it’s strictly necessary to call EndXXX. If the operation doesn’t have any return values or outbound arguments, then it’s certainly convenient to “Fire and Forget.” However, there are a few problems with this:

1) If the operation fails, a call to EndXXX will throw the exception that signals this failure. If the application never calls EndXXX, it has no way of knowing whether the asynchronous operation actually happened.

2) As we’ve seen, EndXXX is an opportunity for resources to be eagerly disposed. If you don’t call EndXXX, those resources must be retained until the GC collects the IAsyncResult object and finalizes it. On the server, this can be a significant performance issue.

3) The last time I checked, some of the FX async APIs would misbehave if EndXXX is not called. For example, finalization of a stream and finalization of any pending IAsyncResult objects are not well ordered. Because of the subtlety involved in efficiently fixing these cases, there’s some debate over whether these are framework bugs or application bugs.

4) Skipping the EndXXX calls is sloppy. This is certainly a matter of taste, but I consider it a strong argument.

Because of the above reasons, you should always balance a successful BeginXXX call with its EndXXX counterpart.

Another common question has to do with the best way to perform a synchronous operation asynchronously. If an API offers BeginXXX / EndXXX methods, you should use them. This is definitely going to be the technique with the best performance. But if you only have an XXX API, you still have several obvious choices:

1) Create a new Thread which calls XXX and then dies.

2) ThreadPool.QueueUserWorkItem() allows a client to call XXX on a ThreadPool thread. The rendezvous model is similar to the delegate callback mechanism we already discussed.

3) Create a Delegate over XXX and then call the BeginXXX / EndXXX methods on that delegate.

The first choice is almost never the correct one. You should only create a dedicated thread if you have a long-running use for one, or if your thread must be different from all the “anonymous” threads in the threadpool. (For example, threadpool threads are all CoInitialized for the MTA. If you need an STA thread, you need to create your own thread).

The second choice will actually perform better than using a Delegate’s BeginXXX / EndXXX methods. If you are queueing work in your own AppDomain, this is the way to go. I know that with work we can narrow the performance gap between QueueUserWorkItem and asynchronous Delegates, but I don’t think we can ever achieve parity.

If your application is making asynchronous calls on remote objects, then asynchronous Delegates have an important optimization. They don’t actually switch to a different thread in this case. Instead, they synchronously initiate a remote call from the calling thread and then return. Asynchronous Delegates have the additional benefit of sharing a consistent model with explicit BeginXXX / EndXXX APIs in FX, so you may prefer them to QueueUserWorkItem for this reason.

Finally, a word on pinning. I often see applications that aggressively pin managed objects or managed delegates that have been passed to unmanaged code. In many cases, the explicit pin is unnecessary. It arises because the developer has confused the requirement of tracking an object instance via a handle with the requirement of keeping the bytes of that object at a fixed location in memory.

For normal PInvokes, a blittable type exposes the bytes of an object in the GC heap directly to unmanaged code. This obviously means that the bytes mustn’t be moved by a GC relocation until the unmanaged code has stopped accessing them. In most cases, the PInvoke layer can automatically pin the bytes for the lifetime of the call. And this layer can pin those bytes in a more efficient manner than you could with a pinned GCHandle. (The PInvoke layer is hooked into the CLR’s stack crawling mechanism for GC reporting. So it can defer all overhead related to pinning unless a GC actually occurs while the PInvoke call is in progress). Applications that explicitly pin buffers around PInvoke calls are often doing so unnecessarily.

Along the same lines, managed Delegates can be marshaled to unmanaged code, where they are exposed as unmanaged function pointers. Calls on those pointers will perform an unmanaged to managed transition; a change in calling convention; entry into the correct AppDomain; and any necessary argument marshaling. Clearly the unmanaged function pointer must refer to a fixed address. It would be a disaster if the GC were relocating that! This leads many applications to create a pinning handle for the delegate. This is completely unnecessary. The unmanaged function pointer actually refers to a native code stub that we dynamically generate to perform the transition & marshaling. This stub exists in fixed memory outside of the GC heap.

However, the application is responsible for somehow extending the lifetime of the delegate until no more calls will occur from unmanaged code. The lifetime of the native code stub is directly related to the lifetime of the delegate. Once the delegate is collected, subsequent calls via the unmanaged function pointer will crash or otherwise corrupt the process. In our recent release, we added a Customer Debug Probe which allows you to cleanly detect this – all too common – bug in your code. If you haven’t started using Customer Debug Probes during development, please take a look!

So there are lots of places where applications often pin unnecessarily. The reason I bring this up is that asynchronous operations through unmanaged code are an important and legitimate scenario for pinning. If you are passing a buffer or OverlappedStruct out to an asynchronous unmanaged API via a PInvoke, you had better be pinning that object. We have a Customer Debug Probe that attempts to validate your pinning through some stressful GC and Finalization calls around the PInvoke call. But this sort of race condition is necessarily a hard bug to provoke cleanly, and the performance impact of this probe is significant.

Whenever you pin an object like a buffer, you should consider whether the buffer is naturally long-lived. If it is not, consider whether you could build a buffer recycling cache so that the buffers become long-lived. This is worth doing because the cost of a pin in the oldest generation of the GC heap is far less than the cost of a pin in the youngest generation. Objects that have survived into the oldest generation are rarely considered for collection and they are very rarely compacted. Therefore pinning an old object is often a NOP in terms of its performance impact.

Of course, if you are calling explicit BeginXXX / EndXXX APIs in FX (like Stream.BeginRead / EndRead), then the pinning isn’t your concern. The Stream implementation is responsible for ensuring that buffers are fixed if it defers to unmanaged operations that expect fixed memory locations.

Along the same lines, if you call explicit BeginXXX / EndXXX APIs, AppDomain unloads need not concern you. But if you call asynchronous unmanaged services directly via PInvoke, you had better be sure that an AppDomain.Unload doesn’t happen while you have a request in flight. If it does, the pinning handles will be reclaimed as part of the unload. This might mean that the asynchronous operation scribbles into the GC heap where a buffer or OverlappedStruct used to be. The resulting heap corruption puts the entire process at risk.

There’s no good story for this in the current product. Somehow you must delay the unload until all your asynchronous operations have drained. One way to do this might be to block in the AppDomain.UnloadDomain event until the count of outstanding operations returns to 0. We’ll be making it easier for you to remain bullet-proof in this sort of scenario in future versions.

So if you can find specific FX asynchronous APIs to call, all this nastiness is handled for you. If instead you define your own managed asynchronous APIs over some existing unmanaged implementation, you need to be very careful.

Interface layout

cbrumme — Sun, 04 May 2003 01:56:00 GMT

The CLR has two different techniques for implementing interfaces. These two techniques are exposed with distinct syntax in C#:

interface I { void m(); }

class C : I {

public virtual void m() {} // implicit contract matching

}

class D : I {

void I.m() {} // explicit contract matching

}

At first glance, it may seem like the choice between these two forms is a stylistic one. However, there are actually deep semantic differences between the two forms.

(C# has at least one other place where a choice of semantics is encoded in what seems to be a stylistic choice. A class constructor can be expressed in C# either as a static constructor method, or as assignments in a set of static field declarations. Depending on this stylistic choice, the class will or will not be marked with tdBeforeFieldInit. This mark – shown as beforefieldinit in ILDASM – affects the semantics of when the .cctor method will be executed by the CLR. This also results in performance differences, particularly in situations like NGEN or domain-neutral code.)

In class C, we get a public class method ‘m’ that does double duty as the implementation of the interface method. This is all pretty vanilla:

.method public hidebysig newslot virtual instance void m() cil managed

{

// Code size 1 (0x1)

.maxstack 0

IL_0000: ret

} // end of method C::m

But in class D, we see something quite different:

.method private hidebysig newslot virtual final instance void I.m() cil managed

{

.override I::m

// Code size 1 (0x1)

.maxstack 0

IL_0000: ret

} // end of method D::I.m

There are several surprising things about this case:

1) The method is introduced (newslot) with the bizarre incantation of virtual, private and final.

2) The name of the method isn’t even ‘m’. It is ‘I.m’.

3) There is a mysterious ‘override’ clause associated with the method body.

The method is marked as virtual because the CLR can only implement interface contracts using virtual members. There’s a bit of a debate on our team whether this is an architectural requirement or an implementation detail. At this point, we’re comfortable that we could remove this restriction without much work. However, we have to consider the ECMA standard, the impact on other CLI implementations like Compact Frameworks, the effect on the various languages targeting the compiler, and some interesting effects on existing applications. We might be saddled with this rule indefinitely.

At the language level, C# allows non-virtuals to implement interface contracts. How do they get around the CLR restriction? Well, if the class that introduces the non-virtual is in the same assembly as the class that uses that method to implement the interface contract, C# quietly defines the base class’ method as virtual. If the base class that introduced the non-virtual is in a different assembly, then C# generates a virtual thunk in the subtype which delegates to the non-virtual base method.

Getting back to our example, I.m is declared as private because it is not available for calling via the class. It can only be called via the interface.

I.m is declared as final because C# really doesn’t want to mark the method as virtual. This was forced on them by the architectural decision / implementation restriction that interface contracts can only be implemented by virtual methods.

As for the name, C# could have picked anything that’s a legal identifier. This member isn’t available for external binding, since it is private to the class and only accessible through the interface.

Since the name ‘I.m’ is insignificant, obviously this isn’t what tells the CLR loader to use this method to satisfy the interface contract. In fact, it’s that mysterious ‘override’ clause. This is what’s known as a MethodImpl. It should not be confused with System.Runtime.CompilerServices.MethodImplAttribute, which controls a method’s eligibility for inlining, its synchronization behavior and other details.

A MethodImpl is a statement in the metadata that matches a method body to a method contract. Here it is used to match the body I.m with the interface contract I::m. Generally, you will see MethodImpls used in this way to match methods to interfaces. But MethodImpls can be used to match any method body to any contract (e.g. a class virtual slot) provided that:

1) The contract is virtual

2) The body is virtual

3) The body and the MethodImpl are defined on the same class

4) The contract is defined either on this class or somewhere up the hierarchy (including implemented interfaces).

Once again, it’s open to debate whether MethodImpls require virtual contracts and bodies for sound architectural reasons or for temporary implementation reasons.

The ECMA spec contains the rules for how interface contracts are satisfied by class methods. This explains how the base class’ layout can be at least partially re-used, and it explains the precedence of the two techniques we’ve seen above (class methods match by name and signature vs. MethodImpls which match methods of any name that have the correct signature).

It also mentions one other surprising detail of interface layout. In the example below, we would expect Derived and Redundant to have the same layout. Sure, there’s a redundant mention of interface I on class Redundant, but that seems irrelevant.

interface I { void m(); }

class A : I {

public virtual void m() {}

}

class Derived : A {

public new virtual void m() {}

}

class Redundant : A, I {

public new virtual void m() {}

}

In fact, it is highly significant. Class A has already satisfied the interface contract for I. Class Derived simply inherits that layout. The new method Derived.m is unrelated to I.m. But in class Redundant, we mention interface I in the implements list. This causes the CLR loader to satisfy the interface contract all over again. In this new layout, Redundant.m can be used to satisfy I.m.

If you’re thinking that some of this stuff is pretty subtle, you are right. Normally, developers wouldn’t concern themselves with the different ways that the CLR can satisfy interface contracts. Instead, you would happily code to your language rules and you would trust your IL generator to spit out the appropriate metadata. In fact, one of the reasons we have all these subtle rules in the CLR is so we can accommodate all the different language rules that we encounter.

Virtual and non-virtual

cbrumme — Fri, 25 Apr 2003 21:42:00 GMT

The CLR type system supports both virtual and non-virtual instance methods. And IL can contain both CALLVIRT and CALL instructions. So it makes sense that IL generators would call virtual methods using CALLVIRT and call non-virtual instance methods with CALL. In fact, this is not necessarily the case. Either kind of call instruction can be applied to either kind of instance method, resulting in four distinct semantics.

Before we look at each of those four cases, we need to look at what a general call (CALL or CALLVIRT) looks like in the IL stream:

0x68 0x06000007 // call method

0x6f 0x06000007 // callvirt method

Well, that’s not too instructive. There’s just an opcode for the call, followed by a MethodRef or MethodDef token for the method. Generally you will see a MethodDef if the method is defined in the same assembly as the callsite, though IL generators aren’t required to make this optimization.

A better way to look at this is through ILDASM:

call instance string callstyle.B::m()

callvirt instance string callstyle.B::m()

ILDASM has chased the token down for you and recovered some information from it. This consists of the name of the method, the signature & calling convention of the method, and a class where that method may be found.

It’s this class hint that’s the most interesting. Alarm bells may be going off in your head. How can I make a virtual call (where the override should be determined by the actual type of the receiver) if the IL stream statically declares the method to call? This isn’t a concern. The purpose of the class hint is to indicate the contract of the virtual call, rather than the actual override. If you think in terms of VTables, it selects the slot rather than the method body.

In fact, this class hint is still a hint in the CALL (non-virtual) case. The class that’s mentioned might not even implement this method directly. So long as this class or a base class has the method, the bind attempt will succeed.

Why would an IL generator mention a method on a class, when the class doesn’t implement that method directly? If the callsite and the target are in the same assembly, there’s little reason to do so. But if multiple assemblies are involved, versioning can intrude. The IL generator might mention a method on a class, but the method could move up to a superclass in a subsequent version. And in the case of chaining calls to virtual methods up the hierarchy (e.g. ‘base’ calls in C#), the IL generator should probably mention the immediate base class in order to increase version resiliency.

In the face of metadata directives like newslot (e.g. the way C# distinguishes between ‘virtual’, ‘new’ and ‘overrides’ keywords), some of the versioning issues become quite tricky. Each language needs to define what kinds of edits are breaking and which ones are tolerated. Based on this, the IL generator can make sane decisions about how to emit class hints in call instructions.

So, to recap, the CALL or CALLVIRT instruction gives us a token which gives us the name, signature, calling convention, and class hint for the method contract to target. Then a search is made upwards from the class hint, until we find an actual method definition. Now the contract is known.

Determination of the contract could happen at JIT time or class loading time. It can be hoisted far above the actual call.

If the call has non-virtual semantics, discovering the contract also reveals the actual method definition to execute. If the call has virtual semantics, we cannot know the actual method definition to execute until the call happens. At that time, we are given the object to invoke on, so we can use that object’s actual type to select the appropriate method body.

Finally we can explain all four legal combinations of CALL / CALLVIRT instructions on virtual / non-virtual methods.

· CALLVIRT on a virtual instance method

This is the normal virtual dispatch. Given the contract and the receiver, at call-time we select the appropriate override and dispatch the call.

· CALL on a non-virtual instance method

This is the normal non-virtual dispatch. When we discovered the contract, we discovered the appropriate method implementation. Dispatch the call to it.

· CALL on a virtual instance method

This is a scoped (non-virtual) call. An example is a ‘base’ call in C# where one virtual method is calling the inherited implementation. If it used virtual semantics for this call, an infinite recursion would result. This kind of call is available more generally via the scope resolution operator ‘::’ in C++.

· CALLVIRT on a non-virtual instance method

This is the most surprising one. Why would someone make a virtual call when the selection of the method body doesn’t depend on dynamically discovering the type of the receiver? There are two reasons.

1) Some languages allow non-virtual methods to become virtual in subsequent versions of a type. If callers are already performing virtual dispatch, they might arguably tolerate this change better.

2) The JIT performs an important side effect when making virtual calls on non-virtual instance methods. It ensures that the receiver is not null. In the case of the current X86 JIT where EAX is scratch and ‘this’ is in ECX, you’ll see code like “mov eax, [ecx]” right before the call. This moves the exception out of some random point in the method body and delivers it at the callsite. If you look at C#’s use of this, they will suppress subsequent calls on ‘this’ so that only the outer skin of non-virtual instance methods receive this treatment. It’s a good heuristic, though obviously it can be thwarted if the outer caller is using an IL generator that doesn’t follow this convention.

Nothing is ever simple.

Interning Strings and immutability

cbrumme — Wed, 23 Apr 2003 01:16:00 GMT

Managed strings are subject to ‘interning’. This is the process where the system notices that the same string is used in several places, so it can fold all the references to the same unique instance.

Interning happens two ways in the CLR.

1) It happens when you explicitly call System.String.Intern(). Obviously the string returned from this service might be different from the one you pass in, since we might already have an intern’ed instance that has been handed out to the application.

2) It happens automatically, when you load an assembly. All the string literals in the assembly are intern’ed. This is expensive and – in retrospect – may have been a mistake. In the future we might consider allowing individual assemblies to opt-in or opt-out. Note that it is always a mistake to rely on some other assembly to have implicitly intern’ed the strings it gives you. Through versioning, that other assembly might start composing a string rather than using a literal.

One thing that might not be immediately obvious is that we intern strings across all AppDomains. That’s because assemblies can be loaded as domain-neutral. When this happens, we execute the same code bytes at the same address in all AppDomains into which that assembly has been loaded. Since we can burn the addresses of string literals into our native code as immediate data, we clearly benefit from intern’ing across all AppDomains rather than using per-AppDomain indirections in the code. However, this approach does add overhead to intern’ing: we are forced to use per-AppDomain reference counts into a shared intern’ing table, so that we can unload intern’ed strings accurately when the last AppDomain using them is itself unloaded.

Normally, strings should be compared with String.Equals and similar mechanisms. Note that the String class defines operator== to be String.Equals. However, if two strings are both known to have been intern’ed, then they can be compared directly with a faster reference check. In other words, you could call Object.operator==() rather than String.operator==(). This is only recommended for highly performance-sensitive scenarios when you really know what you are doing.

Of course, string intern’ing only works if strings are immutable. If they were mutable, then the sharing of strings that is implicit in intern’ing would corrupt all kinds of application assumptions – as we will see.

The good news is that strings are immutable… mostly. And they are immutable for many good reasons that have nothing to do with intern’ing. For example, immutable strings eliminate a whole host of multi-threaded race conditions where one thread uses a string while another string mutates it. In some cases, those race conditions could be used to mount security attacks. For example, you could satisfy a FileIOPermission demand with a string pointing to an innocuous section of the file system, and then use another thread to quickly change the string to point to a sensitive file before the underlying CreateFile occurs.

So how can strings be mutated?

Well, you can certainly use C#’s ‘unsafe’ feature or equivalent unverifiable ILASM or Managed C++ code to write into a string’s buffer. In those cases, some highly trusted code is performing some clearly dirty operations. This case isn’t going to happen by accident.

A more serious concern comes with marshaling. Here’s a program that uses PInvoke to accidentally mutate a string. Since the string happens to have been intern’ed, it has the effect of changing a string literal in an unrelated part of the application. We pass ‘computerName’ to the PInvoke, but ‘otherString’ gets changed too!

using System;

using System.Runtime.InteropServices;

public class Class1

{

static void Main(string[] args)

{

String computerName = "strings are always immutable";

String otherString = "strings are always immutable";

int len = computerName.Length;

GetComputerName(computerName, ref len);

Console.WriteLine(otherString);

}

[DllImport("kernel32", CharSet=CharSet.Unicode)]

static extern bool GetComputerName(

[MarshalAs (UnmanagedType.LPWStr)] string name,

ref int len);

}

And here’s the same program written to avoid this problem:

using System;

using System.Runtime.InteropServices;

public class Class1

{

static void Main(string[] args)

{

String computerName = "strings are always immutable";

String otherString = "strings are always immutable";

int len = computerName.Length;

GetComputerName(ref computerName, ref len);

Console.WriteLine(otherString);

}

[DllImport("kernel32", CharSet=CharSet.Unicode)]

static extern bool GetComputerName(

[MarshalAs(UnmanagedType.VBByRefStr)]

ref string name,

ref int len);

}

In this second case, VBByRefStr is used for the marshaling directive. The argument is treated as ‘byref’ on the managed side, but remains ‘byval’ on the unmanaged side. If the unmanaged side scribbles into the buffer, it won’t pollute the managed string, which remains immutable. Instead, a different string is back-propagated to the managed side, thereby preserving managed string immutability.

If you are coding in VB, you can pretend that the VBByRefStr is actually byval on the managed side. The compiler works its magic on your behalf, so you don’t actually realize that you now have a different string. C# works no such magic, so I had to explicitly add the ‘ref’ keyword in all the right places.

If you’re like me, you probably find all the marshaling directives bewildering. I can’t recommend Adam Nathan’s book enough. It is “.NET and COM – The Complete Interoperability Guide”. It truly is the bible for interop.

Nevertheless, even with the book it’s easy to make a lot of mistakes. There’s a feature in the new CLR release called Customer Debug Probes. It makes finding certain kinds of bugs much easier. Fortunately for all of us, it’s particularly geared to finding bugs with marshaling and other Interop issues.

Lifetime, GC.KeepAlive, handle recycling

cbrumme — Sat, 19 Apr 2003 22:18:00 GMT

It’s not possible to state exactly when a managed object will be collected. The garbage collector schedules itself based on various heuristics. Even if a garbage collection occurs, it may only collect the younger generations of the heap. And the JIT has some freedom to lengthen or shorten the lifetime of instances, based on how it generates code and reports liveness.

class C {

IntPtr _handle;

Static void OperateOnHandle(IntPtr h) { ... }

void m() {

OperateOnHandle(_handle);

...

}

...

}

class Other {

void work() {

if (something) {

C aC = new C();

aC.m();

... // most guess here

} else {

...

}

So we can’t say how long ‘aC’ might live in the above code. The JIT might report the reference until Other.work() completes. It might inline Other.work() into some other method, and report aC even longer. Even if you add “aC = null;” after your usage of it, the JIT is free to consider this assignment to be dead code and eliminate it. Regardless of when the JIT stops reporting the reference, the GC might not get around to collecting it for some time.

It’s more interesting to worry about the earliest point that aC could be collected. If you are like most people, you’ll guess that the soonest aC becomes eligible for collection is at the closing brace of Other.work()’s “if” clause, where I’ve added the comment. In fact, braces don’t exist in the IL. They are a syntactic contract between you and your language compiler. Other.work() is free to stop reporting aC as soon as it has initiated the call to aC.m().

Another common guess is that the soonest aC could be collected is when C.m() stops executing. Or perhaps after the call to C.OperateOnHandle(). Actually, aC could become eligible for collection before C.m() even calls C.OperateOnHandle(). Once we’ve extracted _handle from ‘this’, there are no further uses of this object. In other words, ‘this’ can be collected even while you are executing an instance method on that object.

Why should you care? Well, for the example above, you don’t care. The GC’s reachability will ensure that objects won’t be collected until we are finished with them. But what if class C has a Finalize() method which closes _handle? When we call C.OperateOnHandle(), we now have a race between the application and the GC / Finalizer. Eventually, that’s a race we’re going to lose.

One way to fix this race is to add a call to GC.KeepAlive(this) right after the call to OperateOnHandle(). This indicates that we need the JIT to keep reporting ‘this’ to the GC until we get to that point in the execution. KeepAlive is just a light-weight method call that is opaque to the JIT. So the JIT cannot inline the call and recognize that the call has no real side effects and hence could be eliminated.

The reason you need to add this call is that you have really broken the encapsulation of the _handle resource. The lifetime of the enclosing object and the required lifetime of the _handle are separated when you extract the value from the object’s field.

It’s bad enough that you must use GC.KeepAlive() to tie those two lifetimes back together in your encapsulation. It would be disastrous if you required the clients of your class to be responsible for calling KeepAlive. Public fields on classes are a bad idea for many reasons. As we’ve seen, when they expose a resource that is subject to finalization, they are an exceptionally bad idea.

(You may wonder why we don’t just extend all lifetimes to the end of methods. This has a terrible impact on code quality, particularly on X86 where we are cursed with limited registers. And a change like that doesn’t really fix the problem. It’s still possible for you to return the _handle, place it in a static field, or otherwise cause its lifetime to escape the lifetime of the enclosing object).

There’s another wrinkle to this issue. So far we’ve seen how the Finalizer thread and the application can race when the resource can be separated from its enclosing object. The same sort of thing can happen when you expose IDisposable on your class. Now a multi-threaded application can simultaneously use the resource on one thread and imperatively call Dispose on another thread. GC.KeepAlive isn’t going to solve this problem, since you’ve provided a public API to disassociate the lifetime of the resource from the lifetime of the enclosing object.

This is more than application issue. It can also be used to mount security attacks. If malicious code can open a file to an uninteresting part of the filesystem, it could simultaneously Read and Dispose that file object on two different threads. In a server environment, it’s possible that some other component is opening a file to a sensitive part of the filesystem. Eventually, the malicious code could exploit the race condition to read the other component’s file. This is a handle-recycling attack.

We’ve taken care to prevent this situation in our frameworks. When we use a resource in a PInvoke to the operating system (like reading from a file handle), we place a reference count on the resource. If malicious or poorly-timed code calls Dispose, this simply removes the reference count that was created when the resource was acquired. The result is that all current uses of the resource will be drained, the resource will then be safely disposed, and subsequent attempts to use the resource will be failed gracefully.

For now, you should consider similar approaches if you are encapsulating sensitive resources like this, which are subject to recycling.

But of course this is all far too messy. It runs counter to the goals for our new managed platform to force developers to worry about this sort of thing. In the future, we hope to provide some convenient mechanisms which will allow you to build safe, efficient resource managers. These mechanisms address most of the issues noted above, and some other issues related to reliability and performance. As usual, I can’t really talk about them yet.

Managed blocking

cbrumme — Thu, 17 Apr 2003 16:57:00 GMT

What’s the difference between WaitHandle.WaitOne/WaitAny/WaitAll and just PInvoke’ing to WaitForSingleObject or WaitForMultipleObjects directly? Plenty.

There are several reasons why we prefer you to use managed blocking through WaitHandle or similar primitives, rather than calling out to the operating system via PInvoke.

First, we can blur any platform differences for you

Do you know the differences between Windows 95 and Windows Server 2003 when you have duplicate handles in the list you are waiting on? You certainly shouldn’t have to!

Second, we can do any pumping that is appropriate

While a thread in a Single-Threaded Apartment (STA) blocks, we will pump certain messages for you. Message pumping during blocking is one of the black arts at Microsoft. Pumping too much can cause reentrancy that invalidates assumptions made by your application. Pumping too little causes deadlocks. Starting with Windows 2000, OLE32 exposes CoWaitForMultipleHandles so that you can pump “just the right amount.” On lower operating systems, the CLR uses MsgWaitForMultipleHandles / PeekMessage / MsgWaitForMultipleHandlesEx and whatever else is available on that version of the operating system to try to mirror the behavior of CoWaitForMultipleHandles. The net effect is that we will always pump COM calls waiting to get into your STA. And any SendMessages to any windows will be serviced. But most PostMessages will be delayed until you have finished blocking.

The degree of pumping that’s happening has been painfully tuned to be appropriate to WindowsForms, non-GUI console apps, ASP compatibility mode using an STA threadpool on the server, and all the other traditional STA scenarios. However, in the future we know we’re going to be revisiting this. The underlying operating system is evolving and there are some big changes underway in this area. Believe me, you don’t want to be doing this stuff yourself. The CLR should be insulating you from this pain.

Third, the CLR can make wise decisions about activity

The CLR threadpool monitors CPU utilization to guide its heuristics about thread injection and retirement. It also notices GC activity, since there’s little reason to inject a thread that will immediately be suspended until a non-concurrent GC is complete. The threadpool also notices whenever one of its threads is blocked or emerges from a blocking operation. We can do this accurately if you use managed blocking. If you PInvoke to unmanaged blocking services, everything is opaque.

Fourth, we can ensure that your thread can be controlled

The operating system provides a TerminateThread() service. It should never be used under any circumstances. It will corrupt the process. The CLR provides services like Thread.Abort and Thread.Interrupt. They can take control of your thread in a reasonably safe manner. By reasonably safe, I mean that the process and the CLR remain consistent. Your application state might not remain consistent. In particular, if a thread is Aborted while it is executing a .cctor method, I’ve explained in another blog how this leaves your class in an “off limits” situation. Another example of this is that your thread might be Aborted in the middle of executing some backout code like a finally or catch clause. Once again, your application state might be corrupt.

(We’re careful to allow finally and catch clauses to execute once an Abort has been induced on your thread. But that’s subtly different from never inducing an Abort in the middle of a finally or catch execution).

Over time, we hope to provide ways for your application to remain consistent – even in the face of Thread.Abort and other asynchronous exceptions, including resource failures like OutOfMemoryException and StackOverflowException.

Until then, there are only two completely safe uses of Thread.Abort:

1) You can abort your own thread via Thread.CurrentThread.Abort.

2) You can perform an AppDomain.Unload, which internally uses Thread.Abort to unwind threads out of the doomed AppDomain.

The first usage is safe because the Abort isn’t induced asynchronously. You are inducing it directly on your own thread – almost as if you had called “throw new ThreadAbortException();”

The second usage is safe because all the application state is being discarded after the thread has been Aborted. That application state might be inconsistent, but it’s all going away anyway.

However, if you PInvoke to WaitForMultipleObject, then Thread.Abort is powerless. We cannot take control of threads that are in unmanaged code. The operating system provides no safe way to do this. A thread in unmanaged code could be holding arbitrary locks (the OS loader lock and the OS heap lock are two particularly troublesome ones).

So there are several good reasons why you should favor managed blocking over a PInvoke to unmanaged blocking. Examples of managed blocking are:

Thread.Join
WaitHandle.WaitOne/WaitAny/WaitAll
GC.WaitForPendingFinalizers
Monitor.Enter if there is enough contention for us to give up on spinning and block

Thread.Sleep is a little unusual. We can take control of threads that are inside this service. But, following the tradition of Sleep on the underlying Windows operating system, we perform no pumping.

If you need to Sleep on an STA thread, but you want to perform the standard COM and SendMessage pumping, consider Thread.CurrentThread.Join(timeout) as a replacement.

ReleaseComObject

cbrumme — Wed, 16 Apr 2003 19:51:00 GMT

Developers who are accustomed to the IDisposable pattern or to C#’s ‘using’ syntax sometimes ask why COM Interop doesn’t support IDisposable on every Runtime Callable Wrapper (RCW). That way, managed code could indicate that it is finished using the unmanaged COM resource. This would allow the resources to be cleaned up much earlier than they would be if we waited for a GC. Also, it might better approximate the way an unmanaged client would have used this COM object through explicit Release calls.

There’s a service called System.Runtime.InteropServices.Marshal.ReleaseComObject() that looks suspiciously like it could be used as a Dispose() call. However, this is misleading. ReleaseComObject is quite different from Dispose() and it’s also quite different from IUnknown::Release() as I’ll explain.

The COM Interop layer in the CLR can make do with a single reference count against the unmanaged pUnk, regardless of how many managed clients refer to that object. In other words, the Interop layer does not hold a reference count for each managed client of that pUnk. Instead, we rely on the reachability magic of the GC to determine when nobody needs that pUnk anymore. When nobody needs the pUnk, then we drop our single reference count on that pUnk.

Furthermore, negotiation for interfaces in managed code via COM Interop does not necessarily affect the unmanaged refcount of the COM object. For instance, the managed wrapper might have already cached a pUnk for this interface.

Regardless of the actual refcount that the wrapper holds on the underlying COM object, ReleaseComObject will release all these refcounts at one time.

However, the return value from ReleaseComObject reveals that there’s an additional refcounting scheme involved. This is unrelated to the COM refcount. The same pUnk might be marshaled into the managed process a number of times. We keep track of this count. You can then call ReleaseComObject that same number of times before we will call IUnknown::Release on the pUnks held by the wrapper and start giving throwing InvalidComObjectExceptions. If you are passing the pUnk backwards and forwards across the layer, this means that the “marshaling count” will be a large and arbitrary number. But, for some usage patterns, the number of times the pUnk is marshaled across may correspond to the number of distinct managed clients that have got their hands on the wrapper. If this happens to be the case, then that many managed clients can independently call ReleaseComObject before the wrapper is zombied and the underlying pUnks are Release’d.

I guess that this behavior is slightly more useful than a simple Release in some circumstances. And you can turn it into the equivalent of IUnknown::Release by calling it in a loop until it returns 0. At that point, our internal “marshaling count” has been decremented to 0 and we have Release’d the pUnks. (We really need to add a ReleaseComObjectFully() service to avoid that silly loop).

Application code can either be on the GC plan, where we track whether there are references outstanding – but in a non-deterministic manner that is guided by memory pressure – or application code can do the tracking itself. But if the application does the tracking, it is responsible for knowing whether there are other managed clients still using the COM object.

One way you might do this is by subtyping the wrapper and adding a Dispose protocol on the managed side that is reference counted. But all managed clients in the process must observe the discipline you define. A more practical approach is to ensure that you are the only client of the pUnk by creating the COM object yourself and then never sharing that reference with anyone else.

If you are using a COM object in a scoped, single-threaded manner then you can safely call ReleaseComObject on that object when you are done with it. This will eagerly release any unmanaged resources associated with that object. Subsequent calls would get an InvalidComObjectException. So don’t make subsequent calls.

But if you are using a COM object from multiple places or multiple threads in your application (or from other applications in the same process), you should not call ReleaseComObject. That’s because you will inflict InvalidComObjectExceptions on those other parts of the application.

So my advice is:

1) If you are a server application, calling ReleaseComObject may be an important and necessary requirement for getting good throughput. This is especially true if the COM objects live in a Single Threaded Apartment (STA). For example, ASP compatibility mode uses the DCOM STA threadpool. In these scenarios, you would create a COM object, use it, then eagerly call ReleaseComObject on each request.

2) If you are a client application using a modest number of COM objects that are passed around freely in your managed code, you should not use ReleaseComObject. You would likely inflict Disconnected errors on parts of the application by doing so. The performance benefit of eagerly cleaning up the resources isn’t worth the problems you are causing.

3) If you have a case where you are creating COM objects at a high rate, passing them around freely, choking the Finalizer thread, and consuming a lot of unmanaged resources… you are out of luck.