If broken it is, fix it you should

Using the powers of the debugger to solve the problems of the world - and a bag of chips    by Tess Ferrandez, ASP.NET Escalation Engineer (Microsoft)

ASP.NET 2.0 Crash case study: Unhandled exceptions

ASP.NET 2.0 Crash case study: Unhandled exceptions

Rate This
  • Comments 84

For a long time all my case studies have been on 1.1. it’s time to venture out in 2.0 land and look at what may seem like a 2.0 specific issue.

 

I say “may seem” because this case study will only directly crash if you are using 2.0, but as you’ll learn later the problem existed in 1.1 and 1.0, it was just way harder to track down.

 

Problem description:

 

Once in a while ASP.NET crashes and we see events in the system event log like this one

 

Event Type:       Warning
Event Source:    W3SVC
Event Category: None
Event ID:          1009
Date:               
2006-04-25
Time:               
09:41:22
PM User:           N/A
Computer:        
SUBSPACE1
Description:
A process serving application pool
'ASP.NET V2.0' terminated unexpectedly. The process id was ‘1732’. The process exit code was ‘0xe0434f4d’.

 

Or this one

 

Event Type:       Warning

Event Source:    W3SVC

Event Category: None

Event ID:          1011

Date:                2006-04-25

Time:                09:41:22

User:                N/A

Computer:         SUBSPACE1

Description:

A process serving application pool 'ASP.NET V2.0' suffered a fatal communication error with the World Wide Web Publishing Service. The process id was '6256'. The data field contains the error number.

 

And in the application event log we get a pretty cryptic error message like this one

 

Event Type:       Error

Event Source:    .NET Runtime 2.0 Error Reporting

Event Category: None

Event ID:          5000

Date:                2006-04-25

Time:                09:41:20

User:                N/A

Computer:         SUBSPACE1

Description:

EventType clr20r3, P1 w3wp.exe, P2 6.0.3790.1830, P3 42435be1, P4 app_code.pn5mfdcr, P5 0.0.0.0, P6 444dcf44, P7 5, P8 5, P9 system.dividebyzeroexception, P10 NIL.

 

 

Initial thoughts:

 

Ok, so what do we know about the issue?  We know that asp.net terminated unexpectedly, and that right before this we got a System.DivideByZeroException…. We also know that the process exit code was 0xe0434f4d whatever that means, hmm…

 

Usually when you get a stopped unexpectedly error message the exit code will be the type of exception that caused the crash.  For example a 0xC0000005 means you got a second chance access violation, 0x800703e9 means you suffered a StackOverflowException but what about 0xe0434f4d? 

 

0xe0434f4d is the exception code for CLR (.net) exceptions, so any managed exception like a NullReferenceException or InvalidOperationException or SQLException… basically all managed exception are natively referred to as 0xe0434f4d.  In this case, if we look closer at the application event log entry we can see that it is in fact a System.DivideByZero exception.

 

Trivia: Just a piece of info of no particular value that you might want to pull out of pocket on your next dateJ 0xe0434f4d or at least 43 4f 4d  are the ASCII values for the letters COM.

 

But hey now… should a .net exception cause the asp.net process to crash???  If you divide by zero in your page and don’t have a try catch block around it, surely you will get one of those “nice” white and yellow error pages saying that an exception occurred on your page, but the process doesn’t just exit. 

 

The answer is yes, you will get one of those pages because the asp.net global error handler will eventually catch your exception, format it for you and print it out on the screen.  But what happens if it is not on an asp.net request, so there is no-one to feedback the exception to? It’s the old paradox: If a tree falls in the forest and nobody is there, does it still make a sound? 

 

In 1.0 and 1.1 it didn’t.  For example if you throw an exception in a piece of code called on a timer, or use QueueUserWorkItem and throw an exception in code executing there, or otherwise throw exceptions in code that is not running inside the context of an asp.net request, the framework will swallow the exception and continue. Or rather it will stop that thread of execution but it won’t die.

 

Doesn’t sound all that bad right?  Really??? 

 

That thread could have been doing anything, and we will never be the wiser that it died.  It could have been holding a lock of some sort, or it could have been in the middle of cleaning up resources, or really a number of different things that will now never happen, but that may immediately or eventually have really bad side effects like hangs or crashes or memory issues, but the exception will be long gone so we cant figure out what it was.

 

The policy for unhandled exceptions was changed in ASP.NET 2.0 to the default for .net which is a process exit.  This can be changed back by adding the following to the aspnet.config in the frameworks directory, but I wouldn’t recommend it without putting in some preventive measures to take care of potential unhandled exceptions on non ASP.NET threads.

 

<configuration>

<runtime>

<legacyUnhandledExceptionPolicy enabled="true" />

</runtime>

</configuration>

 

Troubleshooting the issue:

 

The main task here is to find out where this DivideByZero exception is coming from and why it occurred so there are two ways to figure this out (short of complete code inspection). 

 

Strategy #1 – logging the exception

 

The first way, and this is the way I would probably recommend, is to create an UnhandledExceptionHandler to log the exception along with its stack trace in the event log as shown in this article http://support.microsoft.com/?id=911816

 

You add the handler like this to the web.config:

 

    <system.web>

      <httpModules>

        <add type="WebMonitor.UnhandledExceptionModule, <strong name>" name="UnhandledExceptionModule"/>

      </httpModules>

         

    </system.web>

 

And it hooks an eventhandler up to the UnhandledException event of the current app domain.

 

You don’t actually need to strong name it and add it to the GAC, however if you plan it in multiple applications you should to avoid for the dll being loaded multiple times.

 

Now the next time you get one of these unhandled exceptions, the process will still exit (unless you change the unhandled exception policy), but you have a very good chance of fixing the issue.

 

The event for the exception in this particular sample looks like this…

 

Event Type:       Error

Event Source:    ASP.NET 2.0.50727.0

Event Category: None

Event ID:         0

Date:                2006-04-25

Time:                09:41:20

User:                N/A

Computer:         SUBSPACE1

Description:

 

UnhandledException logged by UnhandledExceptionModule.dll:

 

appId=/LM/w3svc/1/ROOT/CrashMe

 

type=System.DivideByZeroException

 

message=Attempted to divide by zero.

 

stack=

   at MyFinalizerClass.Finalize()

 

.

 

Bingo!!! So the exception occurs in MyFinalizerClass.Finalize() in the CrashMe application.

 

In fact the code for the finalizer for this class looks like this, so it is pretty obvious what caused it, and our work here is done…

 

    ~MyFinalizerClass()

    {

        int i = 0;

        int j = 9;

        i = j / i;

    }

 

Setting up an UnhandledException handler like this is not limited to 2.0. You can absolutely do this in 1.1 as well to determine if you are throwing any unhandled exceptions. The only modification you need to do to the code outlined in the article is to choose a different dll to gather the version from. A prime candidate for this would be mscorlib.dll.

 

 

Strategy #2 – debugging with windbg

 

If you have read any of my previous posts you know I have a special place in my heart for debugging. If you don’t have an unhandledexception eventhandler you can still find out what the exception was by running adplus in crash mode to generate dumps when the process exits.

 

In this particular case  doing this will result in the following files being generated.

 

C:\debuggers\Crash_Mode__Date_04-25-2006__Time_10-11-3030>dir

 Volume in drive C has no label.

 Volume Serial Number is 30D7-F806

 

 Directory of C:\debuggers\Crash_Mode__Date_04-25-2006__Time_10-11-3030

 

2006-04-25  10:12    <DIR>          .

2006-04-25  10:12    <DIR>          ..

2006-04-25  10:11                      4 404 ADPlus_report.txt

2006-04-25  10:11    <DIR>          CDBScripts

2006-04-25  10:11                      9 420 059 PID-3368__W3WP.EXE__1st_chance_IntegerDivide__mini_12D8_2006-04-25_10-11-51-487_0D28.dmp

2006-04-25  10:12                      108 142 158 PID-3368__W3WP.EXE__1st_chance_Process_Shut_Down__full_12D8_2006-04-25_10-12-02-440_0D28.dmp

2006-04-25  10:12                      108 294 606 PID-3368__W3WP.EXE__2nd_chance_NET_CLR__full_12D8_2006-04-25_10-11-54-690_0D28.dmp

2006-04-25  10:12                      35 324 PID-3368__W3WP.EXE__Date_04-25-2006__Time_10-11-3030.log

2006-04-25  10:11                      9 014 Process_List.txt

               6 File(s)    225 905 565 bytes

               3 Dir(s)   4 592 435 200 bytes free

 

 

So at 10-11-51 a 1st chance IntegerDivide exception occurred, followed by a 2nd chance .NET exception (2nd chance meaning that it wasn’t handled), and finally this was followed by a process shutdown.

 

Concentrating on the 2nd chance exception dump and loading up sos.dll from the framework directory

 

0:020> .load C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\sos.dll

 

We can run !threads to give us a list of the threads along with info about which thread the exception occurred on (in this case thread 20, the finalizer thread). 

 

0:020> !threads

ThreadCount: 7

UnstartedThread: 0

BackgroundThread: 7

PendingThread: 0

DeadThread: 0

Hosted Runtime: no

                                                        PreEmptive   GC Alloc                          Lock

   ID    OSID    ThreadOBJ       State        GC          Context                  Domain   Count   APT   Exception

   16    1 c20    000b9778      3808220    Enabled  00000000:00000000 000fddc0     1 Ukn (Threadpool Worker)

  20    2 16d8 000e3bd0      b220 Enabled  0242ab08:0242bf94 000caad8     0 MTA (Finalizer) System.DivideByZeroException (02428f60)

  21    3  b2c 000fa588    80a220 Enabled  00000000:00000000 000caad8     0 MTA (Threadpool Completion Port)

  22    4 16c0 000fd690      1220 Enabled  00000000:00000000 000caad8     0 Ukn

  14    6 1570 0014a970   880a220 Enabled  00000000:00000000 000caad8     0 MTA (Threadpool Completion Port)

  23    7 1b44 00145178   180b220 Enabled  00000000:00000000 000caad8     0 MTA (Threadpool Worker)

  24    5  bac 0014a5b0   880b220 Enabled  00000000:00000000 000caad8     0 MTA (Threadpool Completion Port)

 

 

So from this we know that we are looking at an exception happening during finalization.

 

Since this dump was triggered by the exception this will also be the active thread when you load up the dump.

 

If we look at the native and managed callstack it doesn’t really bring us any closer to the solution…

 

 

0:020> kb 2000

ChildEBP RetAddr  Args to Child             

01eaf9fc 79f97065 e0434f4d 00000001 00000001 kernel32!RaiseException+0x53

01eafa5c 7a05b941 02428f60 00000000 00000000 mscorwks!RaiseTheExceptionInternalOnly+0x226

01eafa70 7a05b9b2 02428f60 00000000 01eafb88 mscorwks!RaiseTheException+0x4c

01eafa98 7a05b9ea 00000000 000e3bd0 4914714b mscorwks!RaiseTheException+0xbe

01eafac4 7a05b9f5 02428f60 00000000 7a0e0701 mscorwks!RealCOMPlusThrow+0x37

01eafad0 7a0e0701 02428f60 7740e5f5 7a36a738 mscorwks!RealCOMPlusThrow+0xa

01eafc00 7a0e16e4 00000000 01eafc38 01eafca8 mscorwks!Thread::RaiseCrossContextException+0x63

01eafcb4 79f3a09c 000fddc0 79f3a0a1 01eafeb4 mscorwks!Thread::DoADCallBack+0x25a

01eafcd0 79f3581e 01eafeb4 79f3ab82 000fddc0 mscorwks!Thread::UserResumeThread+0xf3

01eafcd8 79f3ab82 000fddc0 79f3abb0 01eafd08 mscorwks!ManagedThreadBase::FinalizerAppDomain+0x26

01eafd34 79f350ea 000e3bd0 00000000 01eafd5b mscorwks!SVR::CreateGCHeap+0x163

01eafdb0 79f3500e 00000000 00000000 01eafe6c mscorwks!SVR::CreateGCHeap+0x204

01eafdc8 79ecb4a4 01eafeb4 00000000 00000001 mscorwks!SVR::GCHeap::FinalizerThreadWorker+0xe7

01eafdd8 79ecb442 01eafeb4 01eafe60 79f93fe6 mscorwks!Thread::UserResumeThread+0xfb

01eafe6c 79ecb364 01eafeb4 7740e0c9 00000000 mscorwks!Thread::DoADCallBack+0x355

01eafea8 79ed5e8b 01eafeb4 00000000 000fddc0 mscorwks!Thread::DoADCallBack+0x541

01eafed0 79ed5e56 79f34f4a 00000008 79f6fd87 mscorwks!ManagedThreadBase_NoADTransition+0x32

01eafedc 79f6fd87 79f34f4a 7740e175 00000000 mscorwks!ManagedThreadBase::FinalizerBase+0xb

01eaff14 79ecb00b 00000000 00000000 00000000 mscorwks!SVR::GCHeap::FinalizerThreadStart+0xbb

01eaffb8 77e66063 000e4390 00000000 00000000 mscorwks!Thread::intermediateThreadProc+0x49

01eaffec 00000000 79ecafc5 000e4390 00000000 kernel32!BaseThreadStart+0x34

 

 

0:020> !clrstack

OS Thread Id: 0x16d8 (20)

ESP       EIP    

01eafaac 77e55dea [GCFrame: 01eafaac]

01eafb14 77e55dea [GCFrame: 01eafb14]

01eafd14 77e55dea [GCFrame: 01eafd14]

 

But from the !threads output we get the address of the exception (02428f60) and we can take a deeper look…

 

0:020> !do 02428f60

Name: System.DivideByZeroException

MethodTable: 7915e020

EEClass: 791e97f8

Size: 72(0x48) bytes

 (C:\WINDOWS\assembly\GAC_32\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll)

Fields:

      MT    Field   Offset                 Type VT     Attr    Value Name

790fa3e0  40000b5        4        System.String  0 instance 02429e78 _className

79109208  40000b6        8 ...ection.MethodBase  0 instance 00000000 _exceptionMethod

790fa3e0  40000b7        c        System.String  0 instance 0242a18c _exceptionMethodString

790fa3e0  40000b8       10        System.String  0 instance 02429fb4 _message

79113dfc  40000b9       14 ...tions.IDictionary  0 instance 00000000 _data

790fa9e8  40000ba       18     System.Exception  0 instance 00000000 _innerException

790fa3e0  40000bb       1c        System.String  0 instance 00000000 _helpURL

790f9c18  40000bc       20        System.Object  0 instance 00000000 _stackTrace

790fa3e0  40000bd       24        System.String  0 instance 00000000 _stackTraceString

790fa3e0  40000be       28        System.String  0 instance 0242aab0 _remoteStackTraceString

790fed1c  40000bf       34         System.Int32  0 instance        0 _remoteStackIndex

790f9c18  40000c0       2c        System.Object  0 instance 00000000 _dynamicMethods

790fed1c  40000c1       38         System.Int32  0 instance -2147352558 _HResult

790fa3e0  40000c2       30        System.String  0 instance 0242a3c4 _source

790fe160  40000c3       3c        System.IntPtr  0 instance        0 _xptrs

790fed1c  40000c4       40         System.Int32  0 instance        0 _xcode

 

In particular we can dump out the _remoteStackTraceString and find out where exactly the error occurred.

 

0:020> !do 0242aab0

Name: System.String

MethodTable: 790fa3e0

EEClass: 790fa340

Size: 88(0x58) bytes

 (C:\WINDOWS\assembly\GAC_32\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll)

String:    at MyFinalizerClass.Finalize()

 

Fields:

      MT    Field   Offset                 Type VT     Attr    Value Name

790fed1c  4000096        4         System.Int32  0 instance       36 m_arrayLength

790fed1c  4000097        8         System.Int32  0 instance       35 m_stringLength

790fbefc  4000098        c          System.Char  0 instance       20 m_firstChar

790fa3e0  4000099       10        System.String  0   shared   static Empty

    >> Domain:Value  000caad8:790d6584 000fddc0:790d6584 <<

79124670  400009a       14        System.Char[]  0   shared   static WhitespaceChars

    >> Domain:Value  000caad8:023d03f0 000fddc0:023d4374 <<

 

Voila, there we have it again… our MyFinalizerClass.Finalize()

 

If you want to take away just one thing from this post, it should be to be careful with the code running on non ASP.NET threads, making sure that you have proper try / catch blocks around any code that could cause an exception.

 

Until next time…

 





  • Hi Tess,

    Wow what a great article, one of the best I have read, and just what I needed.

    I have a W3wp web service that has recently started aborting for no apparent reason. Following you excellent tutorials I used WinDbg to try to ascertain the problem.

    Unfortunately I seem to have hit a brick wall in that when I try to trace back the exception I get stuck in the framework code and am unable to see how to trace back to the root cause. I was was wondering if you could give me any quick hints as to how next to progress in tracing back the real source of this exception.

    Many thanks for an excellent series.

    Acby

    0:009> !threads

    ThreadCount: 15

    UnstartedThread: 0

    BackgroundThread: 13

    PendingThread: 0

    DeadThread: 1

    Hosted Runtime: no

                                         PreEmptive   GC Alloc           Lock

          ID OSID ThreadOBJ    State     GC       Context       Domain   Count APT Exception

      7    1  30c 0169c3f0      8220 Enabled  00000000:00000000 01697520     0 Ukn

     16    2 21bc 016aa9d8      b220 Enabled  00000000:00000000 01697520     0 MTA (Finalizer)

     18    3  cc8 1a4203a0    80a220 Enabled  00000000:00000000 01697520     0 MTA (Threadpool Completion Port)

     19    4 1f5c 1a420cb0      1220 Enabled  00000000:00000000 01697520     0 Ukn

     20    5 1484 1a465fc8   180b220 Enabled  407862f8:4078763c 1a421200     0 MTA (Threadpool Worker)

     21    6  d38 1a466398   180b220 Enabled  20409550:20409d58 1a421200     0 MTA (Threadpool Worker)

     22    7  a54 1a466768   180b220 Enabled  387eaa20:387ec59c 1a421200     0 MTA (Threadpool Worker)

     23    8 20fc 1a466b38   180b220 Enabled  20417da0:20417db8 1a421200     0 MTA (Threadpool Worker)

      9    9 1270 1a466f08   880a220 Enabled  3cab03ac:3cab15a8 01697520     0 MTA (Threadpool Completion Port) System.InvalidOperationException (3caacbe4)

    XXXX    a    0 1a4b5d98      9820 Enabled  00000000:00000000 01697520     0 MTA

      3    c 1cb4 1a53fe80       220 Enabled  3c920744:3c922704 01697520     0 Ukn

     25    b 1064 4a8faac0   200b020 Enabled  387a90fc:387aa520 1a421200     0 MTA

     27    d 1f14 1b914a80   200b220 Enabled  20403f98:20405d58 1a421200     1 MTA

     28    e 1a84 1b7e00a8    80a220 Enabled  00000000:00000000 01697520     0 MTA (Threadpool Completion Port)

     30    f 2324 1a54c8e8   180b220 Enabled  40850890:40850a68 01697520     0 MTA (Threadpool Worker)

    0:009> !do 1a466f08

    <Note: this object has an invalid CLASS field>

    Invalid object

    0:009> !do 3caacbe4

    Name: System.InvalidOperationException

    MethodTable: 72068c50

    EEClass: 71e34b64

    Size: 72(0x48) bytes

    (C:\Windows\assembly\GAC_32\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll)

    Fields:

         MT    Field   Offset                 Type VT     Attr    Value Name

    72080b54  40000b5        4        System.String  0 instance 3caacc3c _className

    7207ffc8  40000b6        8 ...ection.MethodBase  0 instance 00000000 _exceptionMethod

    72080b54  40000b7        c        System.String  0 instance 3caadacc _exceptionMethodString

    72080b54  40000b8       10        System.String  0 instance 3caacd80 _message

    7207a4b0  40000b9       14 ...tions.IDictionary  0 instance 3caae330 _data

    72080ce8  40000ba       18     System.Exception  0 instance 3caae848 _innerException

    72080b54  40000bb       1c        System.String  0 instance 00000000 _helpURL

    72080770  40000bc       20        System.Object  0 instance 00000000 _stackTrace

    72080b54  40000bd       24        System.String  0 instance 00000000 _stackTraceString

    72080b54  40000be       28        System.String  0 instance 3caaf9b0 _remoteStackTraceString

    72082da0  40000bf       34         System.Int32  1 instance        0 _remoteStackIndex

    72080770  40000c0       2c        System.Object  0 instance 00000000 _dynamicMethods

    72082da0  40000c1       38         System.Int32  1 instance -2146233079 _HResult

    72080b54  40000c2       30        System.String  0 instance 3caade38 _source

    7208341c  40000c3       3c        System.IntPtr  1 instance        0 _xptrs

    72082da0  40000c4       40         System.Int32  1 instance        0 _xcode

    0:009> !do 3caaf9b0

    Name: System.String

    MethodTable: 72080b54

    EEClass: 71e3d65c

    Size: 2554(0x9fa) bytes

    (C:\Windows\assembly\GAC_32\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll)

    String:    at System.Web.Services.Protocols.WebClientProtocol.ProcessAsyncException(WebClientAsyncResult client, Exception e, String method)

      at System.Web.Services.Protocols.WebClientProtocol.ReadResponseAsyncCallback(IAsyncResult asyncResult)

      at System.Net.LazyAsyncResult.Complete(IntPtr userToken)

      at System.Net.ContextAwareResult.CompleteCallback(Object state)

      at System.Threading.ExecutionContext.runTryCode(Object userData)

      at System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData)

      at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)

      at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)

      at System.Net.ContextAwareResult.Complete(IntPtr userToken)

      at System.Net.LazyAsyncResult.ProtectedInvokeCallback(Object result, IntPtr userToken)

      at System.Net.Sockets.BaseOverlappedAsyncResult.CompletionPortCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* nativeOverlapped)

      at System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* pOVERLAP)

    Fields:

         MT    Field   Offset                 Type VT     Attr    Value Name

    72082da0  4000096        4         System.Int32  1 instance     1269 m_arrayLength

    72082da0  4000097        8         System.Int32  1 instance     1268 m_stringLength

    72081834  4000098        c          System.Char  1 instance       20 m_firstChar

    72080b54  4000099       10        System.String  0   shared   static Empty

       >> Domain:Value  01697520:01d801d0 1a421200:01d801d0 <<

    72081784  400009a       14        System.Char[]  0   shared   static WhitespaceChars

       >> Domain:Value  01697520:01d80728 1a421200:01d8548c <<

    0:009> !do 3caae848

    Name: System.NullReferenceException

    MethodTable: 72070128

    EEClass: 71e9e118

    Size: 72(0x48) bytes

    (C:\Windows\assembly\GAC_32\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll)

    Fields:

         MT    Field   Offset                 Type VT     Attr    Value Name

    72080b54  40000b5        4        System.String  0 instance 3caae890 _className

    7207ffc8  40000b6        8 ...ection.MethodBase  0 instance 00000000 _exceptionMethod

    72080b54  40000b7        c        System.String  0 instance 3caaf630 _exceptionMethodString

    72080b54  40000b8       10        System.String  0 instance 3caae91c _message

    7207a4b0  40000b9       14 ...tions.IDictionary  0 instance 3caaf938 _data

    72080ce8  40000ba       18     System.Exception  0 instance 00000000 _innerException

    72080b54  40000bb       1c        System.String  0 instance 00000000 _helpURL

    72080770  40000bc       20        System.Object  0 instance 00000000 _stackTrace

    72080b54  40000bd       24        System.String  0 instance 00000000 _stackTraceString

    72080b54  40000be       28        System.String  0 instance 3caaeb30 _remoteStackTraceString

    72082da0  40000bf       34         System.Int32  1 instance        0 _remoteStackIndex

    72080770  40000c0       2c        System.Object  0 instance 00000000 _dynamicMethods

    72082da0  40000c1       38         System.Int32  1 instance -2147467261 _HResult

    72080b54  40000c2       30        System.String  0 instance 3caaf8a4 _source

    7208341c  40000c3       3c        System.IntPtr  1 instance        0 _xptrs

    72082da0  40000c4       40         System.Int32  1 instance        0 _xcode

    0:009> !do 3caaeb30

    Name: System.String

    MethodTable: 72080b54

    EEClass: 71e3d65c

    Size: 2720(0xaa0) bytes

    (C:\Windows\assembly\GAC_32\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll)

    String:    at System.Web.HttpApplication.ThreadContext.Enter(Boolean setImpersonationContext)

      at System.Web.HttpApplication.OnThreadEnterPrivate(Boolean setImpersonationContext)

      at System.Web.AspNetSynchronizationContext.CallCallbackPossiblyUnderLock(SendOrPostCallback callback, Object state)

      at System.Web.AspNetSynchronizationContext.CallCallback(SendOrPostCallback callback, Object state)

      at System.Web.AspNetSynchronizationContext.Post(SendOrPostCallback callback, Object state)

      at System.ComponentModel.AsyncOperation.Post(SendOrPostCallback d, Object arg)

      at System.ComponentModel.AsyncOperation.PostOperationCompleted(SendOrPostCallback d, Object arg)

      at System.Web.Services.Protocols.HttpWebClientProtocol.OperationCompleted(Object userState, Object[] parameters, Exception e, Boolean canceled)

      at System.Web.Services.Protocols.SoapHttpClientProtocol.InvokeAsyncCallback(IAsyncResult result)

      at System.Web.Services.Protocols.WebClientAsyncResult.Complete()

      at System.Web.Services.Protocols.WebClientProtocol.ProcessAsyncResponseStreamResult(WebClientAsyncResult client, IAsyncResult asyncResult)

      at System.Web.Services.Protocols.WebClientProtocol.ReadAsyncResponseStream(WebClientAsyncResult client)

      at System.Web.Services.Protocols.WebClientProtocol.ReadResponseAsyncCallback(IAsyncResult asyncResult)

    Fields:

         MT    Field   Offset                 Type VT     Attr    Value Name

    72082da0  4000096        4         System.Int32  1 instance     1352 m_arrayLength

    72082da0  4000097        8         System.Int32  1 instance     1351 m_stringLength

    72081834  4000098        c          System.Char  1 instance       20 m_firstChar

    72080b54  4000099       10        System.String  0   shared   static Empty

       >> Domain:Value  01697520:01d801d0 1a421200:01d801d0 <<

    72081784  400009a       14        System.Char[]  0   shared   static WhitespaceChars

       >> Domain:Value  01697520:01d80728 1a421200:01d8548c <<

  • Wow.  Even after 4 years this article is still helping folks like me.

    Thanks Tess!

    Another new fan. :-)

  • Hello Tess

    trust me - i'm almost never writing any comments!

    So this here is an extraordinary exception:

    Thank you, thank you, thank you!

    For "0xe0434f4d is the exception code for CLR (.net) exceptions" for ADPlus for windbg ...

    This really made my day!

  • I'm very thankful for ALL the above information, however, im VERY computer illiterate and have no clue how to get rid of the error message saying UNHANDLED EXCEPTION. It appears everytime Im on the internet and try to close a window or windows b4 shutting down system. HELP!

  • Hope someone can help me out on this.  I'm new to debugging and am desperately trying to make sense of all of this.  Here's what I've got so far.  

    Running adplus on my failing w3wp process, I received the following output files:

    FULLDUMP_FirstChance_epr_Process_Shut_Down_w3wp.exe__0878_2011-03-15_17-33-35-486_047c.dmp

    along with 30 or so

    MINIDUMP_FirstChance_av_AccessViolation_w3wp.exe__0878_2011-03-15_14-16-47-051_047c.dmp  files.

    Using windg on theFullDump, i executed !threads and got

    PDB symbol for mscorwks.dll not loaded

    ThreadCount: 27

    UnstartedThread: 0

    BackgroundThread: 27

    PendingThread: 0

    DeadThread: 0

    Hosted Runtime: no

                                         PreEmptive   GC Alloc           Lock

          ID OSID ThreadOBJ    State     GC       Context       Domain   Count APT Exception

    XXXX    1  294 000e36a0   1808220 Enabled  00000000:00000000 000e8930     0 Ukn (Threadpool Worker)

    XXXX    2  1b8 000f6010      b220 Enabled  00000000:00000000 000e8930     0 Ukn (Finalizer)

    XXXX    3  bb0 00113610    80a220 Enabled  00000000:00000000 000e8930     0 Ukn (Threadpool Completion Port)

    XXXX    4  214 00116d88      1220 Enabled  00000000:00000000 000e8930     0 Ukn

    XXXX    5  a88 05d40ae0   180b220 Enabled  00000000:00000000 000e8930     0 Ukn (Threadpool Worker)

    XXXX    6  80c 05d75c38   180b220 Enabled  00000000:00000000 000e8930     0 Ukn (Threadpool Worker)

    XXXX    7  c28 05d709d8   180b220 Enabled  00000000:00000000 000e8930     0 Ukn (Threadpool Worker)

    XXXX    8  5c0 05e1b008   180b220 Enabled  00000000:00000000 000e8930     0 Ukn (Threadpool Worker)

    XXXX    a  780 05e1ec70       220 Enabled  00000000:00000000 000e8930     0 Ukn

    XXXX    d  dd0 0bf59e78   180b220 Enabled  00000000:00000000 109b4460     1 Ukn (Threadpool Worker)

    XXXX    e  3dc 0bf4d4b0   180b220 Enabled  00000000:00000000 000e8930     0 Ukn (Threadpool Worker)

    XXXX    f  490 0bf5a248   180b220 Enabled  00000000:00000000 000e8930     0 Ukn (Threadpool Worker)

    XXXX   10  3c4 0bf92900   880a220 Enabled  00000000:00000000 000e8930     0 Ukn (Threadpool Completion Port)

    XXXX   12  454 0bf80408       220 Enabled  00000000:00000000 000e8930     0 Ukn

    XXXX   14  e74 0bf9b1d8   180b220 Enabled  00000000:00000000 000e8930     0 Ukn (Threadpool Worker)

    XXXX   16  124 0c00d270       220 Enabled  00000000:00000000 000e8930     0 Ukn

    XXXX   17  d38 0c022738       220 Enabled  00000000:00000000 000e8930     0 Ukn

    XXXX   18  d7c 0c080cb0       220 Enabled  00000000:00000000 000e8930     0 Ukn

    XXXX   13  460 0c0a31c0       220 Enabled  00000000:00000000 000e8930     0 Ukn

    XXXX    c  de4 10989598       220 Enabled  00000000:00000000 000e8930     0 Ukn

    XXXX   15 1700 0bf8b078   200b220 Enabled  00000000:00000000 109b4460     0 Ukn

    XXXX    9  cf4 109891c8   180b220 Enabled  00000000:00000000 000e8930     0 Ukn (Threadpool Worker)

    XXXX    b 179c 0c0a3590   180b220 Enabled  00000000:00000000 000e8930     0 Ukn (Threadpool Worker)

    XXXX   19 1350 0bfdca88   180b220 Disabled 00000000:00000000 109b4460     1 Ukn (Threadpool Worker) System.StackOverflowException (026c106c)

    XXXX   11 1740 0c068d68       220 Enabled  00000000:00000000 000e8930     0 Ukn

    XXXX   1a 1144 05e1cdb0   200b220 Enabled  00000000:00000000 109b4460     1 Ukn

    XXXX   1b 1094 108ca1b0       220 Enabled  00000000:00000000 000e8930     0 Ukn

    then !do 026c106c

    which resulted in

    Name: System.StackOverflowException

    MethodTable: 04930dd8

    EEClass: 0475262c

    Size: 72(0x48) bytes

    (C:\WINDOWS\assembly\GAC_32\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll)

    Fields:

         MT    Field   Offset                 Type VT     Attr    Value Name

    04930b24  40000b5        4        System.String  0 instance 00000000 _className

    0492ff98  40000b6        8 ...ection.MethodBase  0 instance 00000000 _exceptionMethod

    04930b24  40000b7        c        System.String  0 instance 00000000 _exceptionMethodString

    04930b24  40000b8       10        System.String  0 instance 00000000 _message

    0492a480  40000b9       14 ...tions.IDictionary  0 instance 00000000 _data

    04930cb8  40000ba       18     System.Exception  0 instance 00000000 _innerException

    04930b24  40000bb       1c        System.String  0 instance 00000000 _helpURL

    04930740  40000bc       20        System.Object  0 instance 00000000 _stackTrace

    04930b24  40000bd       24        System.String  0 instance 00000000 _stackTraceString

    04930b24  40000be       28        System.String  0 instance 00000000 _remoteStackTraceString

    04932d70  40000bf       34         System.Int32  1 instance        0 _remoteStackIndex

    04930740  40000c0       2c        System.Object  0 instance 00000000 _dynamicMethods

    04932d70  40000c1       38         System.Int32  1 instance -2147023895 _HResult

    04930b24  40000c2       30        System.String  0 instance 00000000 _source

    049333ec  40000c3       3c        System.IntPtr  1 instance        0 _xptrs

    04932d70  40000c4       40         System.Int32  1 instance -532459699 _xcode

    But what do I do with a _remoteStackTraceString of 00000000?  What does this mean?  Any help would be GREATLY appreciated.

    Thanks,

    JP

  • Hi JP,

    Stackoverflows are a bit special in the sense that they are only 1st chance exceptions even though they are "fatal" which means that you wont automatically get dumps on the stackoverflow.

    Try following the blogs.msdn.com/.../net-debugging-demos-lab5-crash-review.aspx walkthrough and get a dump with the unknonwn.cfg attached to that post to get a dump of the actual stackoverflow to see if you have some infinite recursion going on.

    Tess

  • Hey Tess

    My app server has frequent w3wp crashes and I got dumps from Debugdiag for the 2nd chance exception for the crashing w3wp process but dont know how to do drill down furhter. Can I have your thoughts about this?

    Report for w3wp__PID__6676__Date__04_14_2011__Time_04_19_20AM__724__Second_Chance_Exception_E0434F4D.dmp

    Type of Analysis Performed   Crash Analysis

    Operating System   Windows Server 2003 Service Pack 2

    Number Of Processors   2

    Process ID   6676

    Process Image   c:\WINDOWS\system32\inetsrv\w3wp.exe

    System Up-Time   13 day(s) 00:43:04

    Process Up-Time   07:18:09

    Thread 19 - System ID 3948

    Entry point   mscorwks!Thread::intermediateThreadProc

    Create time   4/13/2011 9:02:01 PM

    Time spent in user mode   0 Days 0:0:39.93

    Time spent in kernel mode   0 Days 0:0:1.671

    Function     Arg 1     Arg 2     Arg 3   Source

    kernel32!RaiseException+3c     e0434f4d     00000001     00000001    

    mscorwks!RaiseTheExceptionInternalOnly+2a8     26d6d490     00000000     00000000    

    mscorwks!RaiseTheException+4e     26d6d490     00000000     0725f954    

    mscorwks!RaiseTheException+c0     00000000     069519e0     ce105099    

    mscorwks!RealCOMPlusThrow+30     26d6d490     00000000     0725f960    

    mscorwks!RealCOMPlusThrow+d     26d6d490     88bd0db9     79f50648    

    mscorwks!Thread::RaiseCrossContextException+41f     00000000     0725f998     0725fa08    

    mscorwks!Thread::DoADCallBack+2a2     00000009     79ec434e     0725fb0c    

    mscorwks!Thread::DoADCallBack+310     0725fb0c     0725fab8     79fa6abb    

    mscorwks!Thread::ShouldChangeAbortToUnload+e3     0725fb0c     88bd0fd9     00000001    

    mscorwks!Thread::ShouldChangeAbortToUnload+30a     0725fb0c     00000009     00000000    

    mscorwks!Thread::ShouldChangeAbortToUnload+33e     00000009     79f47b65     00000000    

    mscorwks!ManagedThreadBase::ThreadPool+13     00000009     79f47b65     00000000    

    mscorwks!ManagedPerAppDomainTPCount::DispatchWorkItem+e9     00000000     00000000     77e619d1    

    mscorwks!ThreadpoolMgr::ExecuteWorkRequest+af     88bd08cd     00000000     79ec47b1    

    mscorwks!ThreadpoolMgr::WorkerThreadStart+20b     00000000     00000000     0725fcb8    

    mscorwks!Thread::intermediateThreadProc+49     06934538     00000000     00000000    

    kernel32!GetModuleHandleA+df     79f7571f     06934538     00000000    

    In w3wp__PID__6676__Date__04_14_2011__Time_04_19_20AM__724__Second_Chance_Exception_E0434F4D.dmp the assembly instruction at kernel32!RaiseException+3c in C:\WINDOWS\system32\kernel32.dll from Microsoft Corporation has caused a CLR Exception of type (System.NullReferenceException) on thread 19

    This exception originated from mscorwks!RaiseTheExceptionInternalOnly+2a8. Module Information

    Image Name: C:\WINDOWS\system32\kernel32.dll   Symbol Type:  Export

    Base address: 0x77e40000   Time Stamp:  Sat Mar 21 10:08:26 2009  

    Checksum: 0x00101b44   Comments:  

    COM DLL: False   Company Name:  Microsoft Corporation

    ISAPIExtension: False   File Description:  Windows NT BASE API Client DLL

    ISAPIFilter: False   File Version:  5.2.3790.4480 (srv03_sp2_gdr.090321-1244)

    Managed DLL: False   Internal Name:  kernel32

    VB DLL: False   Legal Copyright:  © Microsoft Corporation. All rights reserved.

    Loaded Image Name:  kernel32.dll   Legal Trademarks:  

    Mapped Image Name:     Original filename:  kernel32

    Module name:  kernel32   Private Build:  

    Single Threaded:  False   Product Name:  Microsoft® Windows® Operating System

    Module Size:  1.01 MBytes   Product Version:  5.2.3790.4480

    Symbol File Name:  kernel32.dll   Special Build:  &

    Thnks

    SP

  • Have you tried following the steps in the article, dumping out the exception etc.?  that should give you the callstack of the place where you got the exception

  • Hey Tess,

    Tried debugging with windbg and got the below info but its not suggesting anything wrong about ths app,

    What do you think? Thanks for ur help in advance.

    0:018> !threads

    ThreadCount: 32

    UnstartedThread: 0

    BackgroundThread: 19

    PendingThread: 0

    DeadThread: 12

    Hosted Runtime: no

                                         PreEmptive   GC Alloc           Lock

          ID OSID ThreadOBJ    State     GC       Context       Domain   Count APT Exception

     11    1 149c 000d8f68   1808220 Enabled  00000000:00000000 000d5a90     0 MTA (Threadpool Worker)

     13    2 1988 000e4ef8      b220 Enabled  00000000:00000000 000d5a90     0 MTA (Finalizer)

     14    3 1d24 000fd988    80a220 Enabled  00000000:00000000 000d5a90     0 MTA (Threadpool Completion Port)

     15    4 1cd8 00100fd0      1220 Enabled  00000000:00000000 000d5a90     0 Ukn

      9    5 18b0 0012dbe8   180a220 Enabled  00000000:00000000 000d5a90     0 MTA (Threadpool Worker)

     17    7 15a8 05780b00   180b220 Enabled  00000000:00000000 000d5a90     0 MTA (Threadpool Worker)

     18    8  7ac 0579f660      b020 Enabled  1cee7ba8:1cee937c 000d5a90     0 MTA System.Runtime.Serialization.SerializationException (1cee606c)

     20    d 1774 058519a0   200b220 Enabled  00000000:00000000 057fc170     0 MTA

     25   20 1ba8 0832dc38   200b220 Enabled  00000000:00000000 112b5688     0 MTA

      3   12 18d4 113a15f0       220 Enabled  00000000:00000000 000d5a90     0 Ukn

      2   1b 1818 000fec40       220 Enabled  00000000:00000000 000d5a90     0 Ukn

     27   1c 1b8c 11560810   a80b220 Enabled  00000000:00000000 00101710     0 MTA (Threadpool Completion Port)

     28   10 1ce0 115b4b78   200b220 Enabled  00000000:00000000 11547d50     0 MTA

     29    e 1700 1687b9a8   200b220 Enabled  00000000:00000000 11547d50     0 MTA

     30    b 17f0 1154a0a8   200b220 Enabled  00000000:00000000 11547d50     0 MTA

      4   17 19ac 113fa338       220 Enabled  00000000:00000000 000d5a90     0 Ukn

     31   29  e98 16a202d8   880b220 Enabled  00000000:00000000 000d5a90     0 MTA (Threadpool Completion Port)

     33   2d 1c14 16a0f140   180b220 Enabled  00000000:00000000 000d5a90     0 MTA (Threadpool Worker)

    XXXX   30    0 1689eef0      9820 Enabled  00000000:00000000 000d5a90     0 Ukn

    XXXX   2f    0 1696e880   8801820 Enabled  00000000:00000000 000d5a90     0 Ukn (Threadpool Completion Port)

    XXXX   27    0 1151caa8     10820 Enabled  00000000:00000000 000d5a90     0 Ukn

    XXXX    a    0 113788e8     10820 Enabled  00000000:00000000 000d5a90     0 Ukn

    XXXX   2c    0 114a4a78     10820 Enabled  00000000:00000000 000d5a90     0 Ukn

    XXXX   34    0 168b2680     10820 Enabled  00000000:00000000 000d5a90     0 Ukn

    XXXX   38    0 16910868     10820 Enabled  00000000:00000000 000d5a90     0 Ukn

    XXXX   33    0 168e0588     10820 Enabled  00000000:00000000 000d5a90     0 Ukn

    XXXX   16    0 16959748     10820 Enabled  00000000:00000000 000d5a90     0 Ukn

    XXXX   13    0 1685d9d0      9820 Enabled  00000000:00000000 000d5a90     0 Ukn

    XXXX   21    0 0583da00      9820 Enabled  00000000:00000000 000d5a90     0 Ukn

    XXXX   1d    0 057b9bc0      9820 Enabled  00000000:00000000 000d5a90     0 Ukn

     43   2a 1f08 16a1f390   200b220 Enabled  00000000:00000000 00101710     0 MTA

     44   2e   fc 168191a0   200b220 Enabled  1ce26ce8:1ce28a18 00101710     0 MTA

    0:018> !do 1cee606c

    Name: System.Runtime.Serialization.SerializationException

    MethodTable: 79313888

    EEClass: 790e37d4

    Size: 72(0x48) bytes

    (C:\WINDOWS\assembly\GAC_32\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll)

    Fields:

         MT    Field   Offset                 Type VT     Attr    Value Name

    79330ac4  40000b5        4        System.String  0 instance 1cee60c4 _className

    7932ff38  40000b6        8 ...ection.MethodBase  0 instance 00000000 _exceptionMethod

    79330ac4  40000b7        c        System.String  0 instance 1cee6d68 _exceptionMethodString

    79330ac4  40000b8       10        System.String  0 instance 1cee6240 _message

    7932a420  40000b9       14 ...tions.IDictionary  0 instance 00000000 _data

    79330c58  40000ba       18     System.Exception  0 instance 00000000 _innerException

    79330ac4  40000bb       1c        System.String  0 instance 00000000 _helpURL

    793306e0  40000bc       20        System.Object  0 instance 00000000 _stackTrace

    79330ac4  40000bd       24        System.String  0 instance 00000000 _stackTraceString

    79330ac4  40000be       28        System.String  0 instance 1cee73a0 _remoteStackTraceString

    79332d10  40000bf       34         System.Int32  1 instance        0 _remoteStackIndex

    793306e0  40000c0       2c        System.Object  0 instance 00000000 _dynamicMethods

    79332d10  40000c1       38         System.Int32  1 instance -2146233076 _HResult

    79330ac4  40000c2       30        System.String  0 instance 1cee7250 _source

    7933338c  40000c3       3c        System.IntPtr  1 instance        0 _xptrs

    79332d10  40000c4       40         System.Int32  1 instance        0 _xcode

    79330ac4  4000e4d      2a8        System.String  0   shared   static _nullMessage

       >> Domain:Value  000d5a90:1ceb63c8 00101710:NotInit  057fc170:NotInit  058058a8:NotInit  05862878:NotInit  112b5688:NotInit  114ab458:NotInit  11547d50:NotInit  168b6648:NotInit  <<

    0:018> !do 1cee73a0

    Name: System.String

    MethodTable: 79330ac4

    EEClass: 790ed65c

    Size: 2054(0x806) bytes

    (C:\WINDOWS\assembly\GAC_32\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll)

    String:    at System.Runtime.Serialization.Formatters.Binary.WriteObjectInfo.InitSerialize(Object obj, ISurrogateSelector surrogateSelector, StreamingContext context, SerObjectInfoInit serObjectInfoInit, IFormatterConverter converter, ObjectWriter objectWriter)

      at System.Runtime.Serialization.Formatters.Binary.WriteObjectInfo.Serialize(Object obj, ISurrogateSelector surrogateSelector, StreamingContext context, SerObjectInfoInit serObjectInfoInit, IFormatterConverter converter, ObjectWriter objectWriter)

      at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.Serialize(Object graph, Header[] inHeaders, __BinaryWriter serWriter, Boolean fCheck)

      at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Serialize(Stream serializationStream, Object graph, Header[] headers, Boolean fCheck)

      at System.Runtime.Remoting.Channels.CrossAppDomainSerializer.SerializeObject(Object obj, MemoryStream stm)

      at System.AppDomain.Serialize(Object o)

      at System.AppDomain.MarshalObject(Object o)

    Fields:

         MT    Field   Offset                 Type VT     Attr    Value Name

    79332d10  4000096        4         System.Int32  1 instance     1019 m_arrayLength

    79332d10  4000097        8         System.Int32  1 instance     1018 m_stringLength

    793317a4  4000098        c          System.Char  1 instance       20 m_firstChar

    79330ac4  4000099       10        System.String  0   shared   static Empty

       >> Domain:Value  000d5a90:025f1198 00101710:025f1198 057fc170:025f1198 058058a8:025f1198 05862878:025f1198 112b5688:025f1198 114ab458:025f1198 11547d50:025f1198 168b6648:025f1198 <<

    793316f4  400009a       14        System.Char[]  0   shared   static WhitespaceChars

       >> Domain:Value  000d5a90:025f16f0 00101710:025f63c8 057fc170:028c21c0 058058a8:02a72e6c 05862878:02b413d4 112b5688:027944cc 114ab458:0315b5f4 11547d50:0301534c 168b6648:14a3be14 <<

Page 6 of 6 (84 items) «23456
Leave a Comment
  • Please add 4 and 2 and type the answer here:
  • Post