This is one of the interesting scenario in which an unmanaged application built with Non-Microsoft technology was crashing during application shutdown with BOOTUP_EXCEPTION_COMPLUS exception (c0020001). This unmanaged application happened to be using unmanaged dll (built with Microsoft compiler) which in turn used IJW (a.k.a. C++ Interop) to interact with windows forms and create some ActiveX controls.

Now as you might know BOOTUP_EXCEPTION_COMPLUS occurs if a call is made in the managed code before the execution engine has finished initializing, or after it has started shutting down. Have a look at cbrume’s blog to know more about BOOTUP_EXCEPTION_COMPLUS. To confirm the cause of the problem and troubleshoot thoroughly, WinDbg was an obvious choice.

After aligning symbols correctly for Microsoft modules, the call stack showed the exception being raised during a call to a window procedure. See frame 5 below which shows a function call to user32!UserCallWinProcCheckWow. This indicates that a window procedure registered during the window creation is trying to process a message. Observe the below call stack carefully:

0:000> .lastevent
Last event: 1084.1150: Unknown exception - code c0020001 (first/second chance not available)
  debugger time: Thu Aug 27 00:09:57.524 2009 (GMT+6)

0:000> knL100
# ChildEBP RetAddr 
00 0012f99c 7a0967c9 kernel32!RaiseException+0x53
01 0012f9b8 7a0257a8 mscorwks!COMPlusThrowBoot+0x4a
02 0012f9c8 79e71e6d mscorwks!UMThunkStubRareDisableWorker+0x25
03 0012f9e8 7739b6e3 mscorwks!UMThunkStubRareDisable+0xa
04 0012fa14 7739b874 user32!InternalCallWinProc+0x28
05 0012fa8c 7739c8b8 user32!UserCallWinProcCheckWow+0x151
06 0012fae8 7739c9c6 user32!DispatchClientMessage+0xd9
07 0012fb10 7c828556 user32!__fnDWORD+0x24
08 0012fb3c 7739d1ec ntdll!KiUserCallbackDispatcher+0x2e
09 0012fb90 77393c27 user32!NtUserMessageCall+0xc
0a 0012fbac 77393c91 user32!RealDefWindowProcA+0x47
0b 0012fbf4 0127b962 user32!DefWindowProcA+0x72
…[Snip]

Now, the next question was to know which window is processing the messages and why. To determine these details with public symbols, you have to manually see the raw call stack. This can be done by running WinDbg command dds. See below output:

0:000> dds 0012fa14
0012fa14  0012fa8c <Unloaded_.DLL>+0x124d57
0012fa18  7739b874 user32!UserCallWinProcCheckWow+0x151
0012fa1c  015b5052 <Unloaded_ure.dll>+0x15b5051
0012fa20  000202b6 <Unloaded_.DLL>+0x15581
0012fa24  0000001c <Unloaded_ure.dll>+0x1b
0012fa28  00000000
0012fa2c  0000178c <Unloaded_ure.dll>+0x178b

One of the values we are interested in is the handle to the window which is processing the message. In this case it is 000202b6. As you read further, you will come to know what this value resembles to.

Now if you analyze the disassembly for mscorwks!UMThunkStubRareDisableWorker, you will notice that even before routing the message to the window procedure the stub function first confirms if it is safe to run the managed code. It does this by calling mscorwks!CanRunManagedCode. If this call returns a value indicating “No”, the stub method throws the BOOTUP_EXCEPTION_COMPLUS. This was exactly the same in this case.

0:000> uf mscorwks!UMThunkStubRareDisableWorker
mscorwks!UMThunkStubRareDisableWorker:
7a025783 55              push    ebp
7a025784 8bec            mov     ebp,esp
7a025786 56              push    esi
7a025787 6a00            push    0
7a025789 6a01            push    1
7a02578b e8896cf2ff      call    mscorwks!CanRunManagedCode (79f4c419)
7a025790 85c0            test    eax,eax
7a025792 8b7508          mov     esi,dword ptr [ebp+8]
7a025795 7511            jne     mscorwks!UMThunkStubRareDisableWorker+0x25 (7a0257a8)

mscorwks!UMThunkStubRareDisableWorker+0x14:
7a025797 682b040780      push    8007042Bh
7a02579c c7460800000000  mov     dword ptr [esi+8],0
7a0257a3 e8ff0f0700      call    mscorwks!COMPlusThrowBoot (7a0967a7)

...[Snip]

Now if the managed code is not allowed to execute, why the managed window is still alive and registered for receiving the messages? Well, a quick look into the managed heap reveals that “Microsoft.Win32.SystemEvents” object is still alive! Ideally this object is destroyed during application cleanup phase; however this was not the case. Now, SystemEvents actually provides access to system event notifications like display setting changed, user preference changed, etc. This is achieved by creating a hidden window having a specific “windowHandle”. If you carefully analyze below command you will notice the “windowHandle” for these events is 202b6 which is exactly the same value for the window trying to process the message. This window is actually the “.NET-BroadcastEventWindow”, a special window, created for monitoring the system events.

0:000> .loadby sos mscorwks

0:000> !DumpHeap -stat
total 101243 objects
Statistics:
      MT    Count    TotalSize Class Name
7a5d2608        1           20 Microsoft.Win32.SystemEvents

…[Snip]

0:000> !DumpHeap -MT 0x7a5d2608
Address       MT     Size
017eacd8 7a5d2608       20
total 1 objects
Statistics:
      MT    Count    TotalSize Class Name
7a5d2608        1           20 Microsoft.Win32.SystemEvents
Total 1 objects

0:000> !do 017eacd8
Name: Microsoft.Win32.SystemEvents
MethodTable: 7a5d2608
EEClass: 7a4724fc
Size: 20(0x14) bytes
(C:\WINDOWS\assembly\GAC_MSIL\System\2.0.0.0__b77a5c561934e089\System.dll)
Fields:
      MT    Field   Offset                 Type VT     Attr    Value Name
793332c8  40015b8        c        System.IntPtr  1 instance    202b6 windowHandle
79330a00  40015b2      568        System.String  0   static 017ead88 className
…[Snip]

0:000> du 017ead88+c
017ead94  ".NET-BroadcastEventWindow.2.0.0."
017eadd4  "0.9585cb.0"

The next step was to find why the window did not get destroyed. Reflecting the code for “Microsoft.Win32.SystemEvents.Initialize” shows that this window gets destroyed during the call to “SystemEvents.Shutdown()”.  See below code snippet:

private void Initialize()
{
   this.consoleHandler = new NativeMethods.ConHndlr(this.ConsoleHandlerProc);
   if (!UnsafeNativeMethods.SetConsoleCtrlHandler(this.consoleHandler, 1))
    {
       this.consoleHandler = null;
    }
   this.windowHandle = this.CreateBroadcastWindow();
   AppDomain.CurrentDomain.ProcessExit += new EventHandler(SystemEvents.Shutdown);
   AppDomain.CurrentDomain.DomainUnload += new EventHandler(SystemEvents.Shutdown);
}

Further, SystemEvents.Shutdown() method is called when either AppDomain.CurrentDomain.ProcessExit() or AppDomain.CurrentDomain.DomainUnload() events are served. Now these events are not guaranteed to be called always, mostly during rude exit (application directly calling kernel32!TerminateProcess/ExitProcess or StackOverFlow). So to confirm if the application is not running into a rude exit you can set breakpoints on above methods and see if it hits. It was confirmed that the problem was indeed with ProcessExit/DomainUnload events not getting called, due to which the broadcast window remains alive and thus executes the window procedure even when CLR is about to shutdown. 

A crude way to get around with the problem is to manually call SystemEvents.Shutdown() using System.Reflection just before closing the application. However, the reliable solution to the problem is to make sure the CRT/CLR cleanup code is correctly executed. For a mixed mode application, one of the crt functions which does this is exit(). It makes sure CRT/CLR code is correctly cleaned up and ProcessExit/DomainUnload events are called thus ensuring a correct cleanup.

- Gaurav Patole,

Technical Lead, Developer Support Languages Team.