ASP.NET Crash: StackOverflowException with Server.Transfer

Dew Drop - May 6, 2008 | Alvin Ashcraft's Morning Dew

Dew Drop - May 6, 2008 | Alvin Ashcraft's Morning Dew — Tue, 06 May 2008 16:08:51 GMT

PingBack from http://www.alvinashcraft.com/2008/05/06/dew-drop-may-6-2008/

Link Listing - May 6, 2008

Christopher Steen — Wed, 07 May 2008 14:40:07 GMT

ASP.NET ASP.NET Crash: StackOverflowException with Server.Transfer [Via: Tess ] WPF Cider improvements...

re: ASP.NET Crash: StackOverflowException with Server.Transfer

Manoj — Tue, 13 May 2008 10:03:15 GMT

Hi Tess,

Thanks for excellent posts. I am facing an issue in our production envt.

We are using ASP.NET 2.0 on Win2003 server. When w3wp process shutdown due to ideal time out period elapse then there is an error in eventlog stating,

EventType clr20r3, P1 w3wp.exe, P2 6.0.3790.3959, P3 45d6968e, P4 xapidui, P5 3.0.1623.0, P6 48227f33, P7 d70, P8 51, P9 system.security.security, P10 NIL.

Another one is ,

An unhandled exception occurred and the process was terminated.

Application ID: DefaultDomain

Process ID: 3644

Exception: System.AppDomainUnloadedException

Message: Attempted to access an unloaded AppDomain.

StackTrace:

For more information, see Help and Support Center at ..

I tried resolving issue using windbg/sos but after doing some analysis I was able to get it was something related to threadabortexception that happen during appdomain unload but not sure why is it happening,

Output of dump is,

# 12 Id: d14.3c0 Suspend: 1 Teb: 7ffaa000 Unfrozen

ChildEBP RetAddr Args to Child

01cef798 78158e89 e06d7363 00000001 00000003 kernel32!RaiseException+0x53

01cef7d0 79f71262 01cef804 79f71264 5f7c56e4 msvcr80!_CxxThrowException+0x46

01cef7fc 79f711bc 05940d88 01cef8a8 79f6f4d6 mscorwks!ThrowHR+0xf5

01cef808 79f6f4d6 80131530 5f7c59b0 00000000 mscorwks!IfFailThrow+0x11

01cef8a8 793df368 025e3088 01cef904 00000000 mscorwks!AppDomainNative::Unload+0xa6

01cef8d4 66004508 01b72760 025fa024 02828da0 mscorlib_ni+0x31f368

01cef904 79407caa 79373ecd 01cef934 01fc2010 System_Web_ni+0xe4508

01cef908 79373ecd 01cef934 01fc2010 02828da0 mscorlib_ni+0x347caa

01cef91c 79407e18 02828da0 00000000 02828da0 mscorlib_ni+0x2b3ecd

01cef934 79407d90 01cef958 01cef974 000ec5f8 mscorlib_ni+0x347e18

01cef948 79e7c74b 5f7c5898 00114474 01cef9d8 mscorlib_ni+0x347d90

00000000 00000000 00000000 00000000 00000000 mscorwks!CallDescrWorker+0x33

and output of !clrStack is,

0:012> !clrstack

OS Thread Id: 0x3c0 (12)

ESP EIP

01cef85c 77e4bee7 [HelperMethodFrame: 01cef85c] System.AppDomain.nUnload(Int32)

01cef8b0 793df368 System.AppDomain.Unload(System.AppDomain)

01cef8dc 66004508 System.Web.HttpRuntime.ReleaseResourcesAndUnloadAppDomain(System.Object)

01cef90c 79407caa System.Threading._ThreadPoolWaitCallback.WaitCallback_Context(System.Object)

01cef910 79373ecd System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)

01cef928 79407e18 System.Threading._ThreadPoolWaitCallback.PerformWaitCallbackInternal(System.Threading._ThreadPoolWaitCallback)

01cef93c 79407d90 System.Threading._ThreadPoolWaitCallback.PerformWaitCallback(System.Object)

01cefacc 79e7c74b [GCFrame: 01cefacc]

01cefc18 79e7c74b [ContextTransitionFrame: 01cefc18]

Not sure why is i happening.

Pls email me at manoj.mittal@xchanging.com

re: ASP.NET Crash: StackOverflowException with Server.Transfer

Tess — Tue, 13 May 2008 10:19:52 GMT

Threadabort exceptions are normal during appdomain unloads since all threads for the appdomain will be aborted during an appdomain unload, so unfortunately the stack above doesn't say much about why you get the System.AppDomainUnloadedException. You would have to get a dump during the System.AppDomainUnloadedException

re: ASP.NET Crash: StackOverflowException with Server.Transfer

Manoj — Wed, 14 May 2008 12:43:31 GMT

Hi Tess,

Thanks for you reply, pls let me knwo how should i configure adplus config file to get dump during System.AppDomainUnloadedException.

re: ASP.NET Crash: StackOverflowException with Server.Transfer

Tess — Wed, 14 May 2008 14:18:42 GMT

Try the script in this post http://blogs.msdn.com/tess/archive/2005/11/30/498297.aspx

if you're on 2.0 use the one posted in the comments that breaks and dumps on a specific exception...

re: ASP.NET Crash: StackOverflowException with Server.Transfer

Manoj — Wed, 14 May 2008 14:48:45 GMT

Hi Tess,

Sorry to bother you again. I am able to get dump during System.AppDomainUnloadedException but clouldn't derived much from it. Here is the result of ~*kb

0 Id: 137c.ff0 Suspend: 1 Teb: 7ffdf000 Unfrozen

ChildEBP RetAddr Args to Child

0006fbd4 7c826f4b 77e41ed1 00000000 0006fc18 ntdll!KiFastSystemCallRet

0006fbd8 77e41ed1 00000000 0006fc18 5a301104 ntdll!NtDelayExecution+0xc

0006fc40 77e424ed 000003e8 00000000 0006fc68 kernel32!SleepEx+0x68

0006fc50 5a3019fa 000003e8 00000000 0027b0a0 kernel32!Sleep+0xf

0006fc68 5a301d30 5a30233d 0027ef58 00000000 w3tp!THREAD_MANAGER::DrainThreads+0xa7

0006fc80 5a3027da 5a30233d 0027ef58 0027ba30 w3tp!THREAD_MANAGER::TerminateThreadManager+0x14

0006fc90 5a302944 5a366ebe 0006fcb4 5a3b018f w3tp!THREAD_POOL::TerminateThreadPool+0x51

0006fc94 5a366ebe 0006fcb4 5a3b018f 00000000 w3tp!ThreadPoolTerminate+0x47

0006fc9c 5a3b018f 00000000 00000000 00000000 w3dt!UlAtqTerminate+0x52

0006fcb4 5a3bc347 00000000 01001418 010013e4 w3core!W3_SERVER::Terminate+0x13f

0006ff0c 0100187c 00000007 0027a740 00000000 w3core!UlW3Start+0x280

0006ff44 01001a27 00000007 0027a740 0027b1a0 w3wp!wmain+0x22a

0006ffc0 77e6f23b 00000000 00000000 7ffd7000 w3wp!wmainCRTStartup+0x12f

0006fff0 00000000 010018f8 00000000 78746341 kernel32!BaseProcessStart+0x23

1 Id: 137c.14c4 Suspend: 1 Teb: 7ffde000 Unfrozen

ChildEBP RetAddr Args to Child

0064fea0 7c827cfb 7c80e5bb 00000002 0064fef0 ntdll!KiFastSystemCallRet

0064fea4 7c80e5bb 00000002 0064fef0 00000001 ntdll!NtWaitForMultipleObjects+0xc

0064ff48 7c80e4a2 00000002 0064ff70 00000000 ntdll!EtwpWaitForMultipleObjectsEx+0xf7

0064ffb8 77e64829 00000000 00000000 00000000 ntdll!EtwpEventPump+0x27f

0064ffec 00000000 7c80e1fa 00000000 00000000 kernel32!BaseThreadStart+0x34

2 Id: 137c.c38 Suspend: 1 Teb: 7ffdd000 Unfrozen

ChildEBP RetAddr Args to Child

00cdff9c 7c826f4b 7c83d424 00000001 00cdffb0 ntdll!KiFastSystemCallRet

00cdffa0 7c83d424 00000001 00cdffb0 00000000 ntdll!NtDelayExecution+0xc

00cdffb8 77e64829 00000000 00000000 00000000 ntdll!RtlpTimerThread+0x47

00cdffec 00000000 7c83d3dd 00000000 00000000 kernel32!BaseThreadStart+0x34

3 Id: 137c.17c0 Suspend: 1 Teb: 7ffdc000 Unfrozen

ChildEBP RetAddr Args to Child

00d1ff70 7c8277db 7c839f38 000000e4 00d1ffb0 ntdll!KiFastSystemCallRet

00d1ff74 7c839f38 000000e4 00d1ffb0 00d1ffb4 ntdll!ZwRemoveIoCompletion+0xc

00d1ffb8 77e64829 00000000 00000000 00000000 ntdll!RtlpWorkerThread+0x3d

00d1ffec 00000000 7c839efb 00000000 00000000 kernel32!BaseThreadStart+0x34

4 Id: 137c.1154 Suspend: 1 Teb: 7ffdb000 Unfrozen

ChildEBP RetAddr Args to Child

00d6fe18 7c82783b 77c885ac 00000130 00d6ff74 ntdll!KiFastSystemCallRet

00d6fe1c 77c885ac 00000130 00d6ff74 00d6fe38 ntdll!NtReplyWaitReceivePortEx+0xc

00d6ff84 77c88792 00d6ffac 77c8872d 00093498 rpcrt4!LRPC_ADDRESS::ReceiveLotsaCalls+0x198

00d6ff8c 77c8872d 00093498 00000000 00000000 rpcrt4!RecvLotsaCallsWrapper+0xd

00d6ffac 77c7b110 000a2448 00d6ffec 77e64829 rpcrt4!BaseCachedThreadRoutine+0x9d

00d6ffb8 77e64829 000aeaf0 00000000 00000000 rpcrt4!ThreadStartRoutine+0x1b

00d6ffec 00000000 77c7b0f5 000aeaf0 00000000 kernel32!BaseThreadStart+0x34

5 Id: 137c.1414 Suspend: 1 Teb: 7ffd4000 Unfrozen

ChildEBP RetAddr Args to Child

0110fcec 7c827cfb 7c83c78e 00000013 0110fd34 ntdll!KiFastSystemCallRet

0110fcf0 7c83c78e 00000013 0110fd34 00000001 ntdll!NtWaitForMultipleObjects+0xc

0110ffb8 77e64829 00000000 00000000 00000000 ntdll!RtlpWaitThread+0x161

0110ffec 00000000 7c83c643 00000000 00000000 kernel32!BaseThreadStart+0x34

6 Id: 137c.16a4 Suspend: 1 Teb: 7ffae000 Unfrozen

ChildEBP RetAddr Args to Child

01b0fe18 7c82783b 77c885ac 00000130 01b0ff74 ntdll!KiFastSystemCallRet

01b0fe1c 77c885ac 00000130 01b0ff74 00000000 ntdll!NtReplyWaitReceivePortEx+0xc

01b0ff84 77c88792 01b0ffac 77c8872d 00093498 rpcrt4!LRPC_ADDRESS::ReceiveLotsaCalls+0x198

01b0ff8c 77c8872d 00093498 00000000 00000000 rpcrt4!RecvLotsaCallsWrapper+0xd

01b0ffac 77c7b110 000a2448 01b0ffec 77e64829 rpcrt4!BaseCachedThreadRoutine+0x9d

01b0ffb8 77e64829 000c0338 00000000 00000000 rpcrt4!ThreadStartRoutine+0x1b

01b0ffec 00000000 77c7b0f5 000c0338 00000000 kernel32!BaseThreadStart+0x34

7 Id: 137c.145c Suspend: 1 Teb: 7ffad000 Unfrozen

ChildEBP RetAddr Args to Child

01b4ff8c 7c826f4b 7c81943a 00000001 01b4ffac ntdll!KiFastSystemCallRet

01b4ff90 7c81943a 00000001 01b4ffac 00000000 ntdll!NtDelayExecution+0xc

01b4ffb8 77e64829 000d2940 00000000 00000000 ntdll!RtlpIOWorkerThread+0x3f

01b4ffec 00000000 7c8193fb 000d2940 00000000 kernel32!BaseThreadStart+0x34

8 Id: 137c.16d8 Suspend: 1 Teb: 7ffac000 Unfrozen

ChildEBP RetAddr Args to Child

01c6fe74 7c8277db 77e5bea2 00000258 01c6fefc ntdll!KiFastSystemCallRet

01c6fe78 77e5bea2 00000258 01c6fefc 01c6febc ntdll!ZwRemoveIoCompletion+0xc

01c6fea4 79f02f04 00000258 01c6fef4 01c6fefc kernel32!GetQueuedCompletionStatus+0x29

01c6ff14 79fc9840 00000000 00000000 f64fcc1c mscorwks!ThreadpoolMgr::CompletionPortThreadStart+0x141

01c6ffb8 77e64829 000d4348 00000000 00000000 mscorwks!ThreadpoolMgr::intermediateThreadProc+0x49

01c6ffec 00000000 79fc97fa 000d4348 00000000 kernel32!BaseThreadStart+0x34

9 Id: 137c.42c Suspend: 1 Teb: 7ffab000 Unfrozen

ChildEBP RetAddr Args to Child

01cafe38 7c826f4b 77e41ed1 00000000 01cafe7c ntdll!KiFastSystemCallRet

01cafe3c 77e41ed1 00000000 01cafe7c 00000000 ntdll!NtDelayExecution+0xc

01cafea4 79ef3dff 000001f4 00000000 8aa55cbf kernel32!SleepEx+0x68

01cafed8 79ef3e10 000001f4 00000000 8aa55d73 mscorwks!EESleepEx+0xa3

01caff14 79ef3921 000001f4 00000000 01caffb8 mscorwks!__DangerousSwitchToThread+0x75

01caff24 79f01162 000001f4 ce374190 00000235 mscorwks!__SwitchToThread+0xd

01caffb8 77e64829 00000000 00000000 00000000 mscorwks!ThreadpoolMgr::GateThreadStart+0xa1

01caffec 00000000 79f010cd 00000000 00000000 kernel32!BaseThreadStart+0x34

10 Id: 137c.15b4 Suspend: 1 Teb: 7ffaa000 Unfrozen

ChildEBP RetAddr Args to Child

01cefc78 7c827d0b 77e61d1e 00000254 00000000 ntdll!KiFastSystemCallRet

01cefc7c 77e61d1e 00000254 00000000 01cefcc0 ntdll!NtWaitForSingleObject+0xc

01cefcec 79e789c6 00000254 00027100 00000000 kernel32!WaitForSingleObjectEx+0xac

01cefd30 79e7898f 00000254 00027100 00000000 mscorwks!PEImage::LoadImage+0x1af

01cefd80 79e78944 00027100 00000000 00000000 mscorwks!CLREvent::WaitEx+0x117

01cefd94 79f014ef 00027100 00000000 00000000 mscorwks!CLREvent::Wait+0x17

01cefe14 79f591a2 000d3420 00027100 00000000 mscorwks!ThreadpoolMgr::SafeWait+0x73

01cefe3c 7a035d45 8aa15cf3 00000000 79f0207d mscorwks!ThreadpoolMgr::EnterRetirement+0x8e

01cefe94 79fc9840 00000000 00000000 00000000 mscorwks!ThreadpoolMgr::WorkerThreadStart+0x33d

01ceffb8 77e64829 000d4348 00000000 00000000 mscorwks!ThreadpoolMgr::intermediateThreadProc+0x49

01ceffec 00000000 79fc97fa 000d4348 00000000 kernel32!BaseThreadStart+0x34

11 Id: 137c.fcc Suspend: 1 Teb: 7ffa9000 Unfrozen

ChildEBP RetAddr Args to Child

01dcfe30 7c827cfb 77e6202c 00000003 01dcfe80 ntdll!KiFastSystemCallRet

01dcfe34 77e6202c 00000003 01dcfe80 00000001 ntdll!NtWaitForMultipleObjects+0xc

01dcfedc 77e62fbe 00000003 01dcff20 00000000 kernel32!WaitForMultipleObjectsEx+0x11a

01dcfef8 79f4e8d8 00000003 01dcff20 00000000 kernel32!WaitForMultipleObjects+0x18

01dcff58 79f4e831 8ab35def 00000000 00000000 mscorwks!DebuggerRCThread::MainLoop+0xe9

01dcff88 79f4e765 8ab35ddf 00000000 00000000 mscorwks!DebuggerRCThread::ThreadProc+0xe5

01dcffb8 77e64829 00000000 00000000 00000000 mscorwks!DebuggerRCThread::ThreadProcStatic+0x9c

01dcffec 00000000 79f4e71f 00000000 00000000 kernel32!BaseThreadStart+0x34

12 Id: 137c.12a8 Suspend: 1 Teb: 7ffa8000 Unfrozen

ChildEBP RetAddr Args to Child

03eafc74 7c827d0b 77e61d1e 000002e8 00000000 ntdll!KiFastSystemCallRet

03eafc78 77e61d1e 000002e8 00000000 03eafcbc ntdll!NtWaitForSingleObject+0xc

03eafce8 79e789c6 000002e8 000007d0 00000000 kernel32!WaitForSingleObjectEx+0xac

03eafd2c 79e7898f 000002e8 000007d0 00000000 mscorwks!PEImage::LoadImage+0x1af

03eafd7c 79e78944 000007d0 00000000 00000000 mscorwks!CLREvent::WaitEx+0x117

03eafd90 79ef2220 000007d0 00000000 00000000 mscorwks!CLREvent::Wait+0x17

03eafdac 79fb997b 000f17f8 03eafeb0 000f1ff0 mscorwks!WKS::WaitForFinalizerEvent+0x4a

03eafdc0 79ef3207 03eafeb0 00000000 00000000 mscorwks!WKS::GCHeap::FinalizerThreadWorker+0x79

03eafdd4 79ef31a3 03eafeb0 03eafe5c 79f91478 mscorwks!Thread::DoADCallBack+0x32a

03eafe68 79ef30c3 03eafeb0 88855cc3 00000000 mscorwks!Thread::ShouldChangeAbortToUnload+0xe3

03eafea4 79fb9643 03eafeb0 00000000 0010df48 mscorwks!Thread::ShouldChangeAbortToUnload+0x30a

03eafecc 79fb960d 79fb990e 00000008 03eaff14 mscorwks!ManagedThreadBase_NoADTransition+0x32

03eafedc 79fba09b 79fb990e 88855d73 00000000 mscorwks!ManagedThreadBase::FinalizerBase+0xd

03eaff14 79f95a2e 00000000 00000000 f4e5ac1c mscorwks!WKS::GCHeap::FinalizerThreadStart+0xbb

03eaffb8 77e64829 000f1ff0 00000000 00000000 mscorwks!Thread::intermediateThreadProc+0x49

03eaffec 00000000 79f959e8 000f1ff0 00000000 kernel32!BaseThreadStart+0x34

13 Id: 137c.638 Suspend: 1 Teb: 7ffa7000 Unfrozen

ChildEBP RetAddr Args to Child

03eef9e8 7c826f4b 77e41ed1 00000001 03eefa2c ntdll!KiFastSystemCallRet

03eef9ec 77e41ed1 00000001 03eefa2c 00000000 ntdll!NtDelayExecution+0xc

03eefa54 79ef3dff 0000000a 00000001 888158ef kernel32!SleepEx+0x68

03eefa88 79fc51ed 0000000a 00000001 03eefaac mscorwks!EESleepEx+0xa3

03eefa98 79fc51db 7a3b45ec 0000000a 00000001 mscorwks!CExecutionEngine::ClrSleepEx+0xe

03eefaac 79fc519a 0000000a 00000001 8881588f mscorwks!ClrSleepEx+0x14

03eefae8 79f6fd9e 0000000a 88815953 0010e334 mscorwks!Thread::UserSleep+0x63

03eefb34 79f6f9f2 88815e13 00000001 0010df48 mscorwks!AppDomain::UnwindThreads+0x1f2

03eefc74 79f707aa 00000001 00000001 88815edf mscorwks!AppDomain::Exit+0x142

03eefcb8 79f7061b 00000000 03eefd28 79f91478 mscorwks!AppDomain::Unload+0x1da

03eefd34 79f70597 0010df48 88815f03 ffffffff mscorwks!AppDomain::ADUnloadWorkerHelper+0x76

03eefd64 79f6f7ec 03eefdc8 79f91478 88815fb3 mscorwks!AppDomain::DoADUnloadWork+0x6c

03eefdd4 79f6f774 03eefe88 79f91478 88815cf3 mscorwks!AppDomain::DoADUnloadWork+0xde

03eefe94 79f95a2e 000f6080 00000000 00000000 mscorwks!AppDomain::ADUnloadThreadStart+0x4c6

03eeffb8 77e64829 000f5400 00000000 00000000 mscorwks!Thread::intermediateThreadProc+0x49

03eeffec 00000000 79f959e8 000f5400 00000000 kernel32!BaseThreadStart+0x34

14 Id: 137c.1334 Suspend: 1 Teb: 7ffa6000 Unfrozen

ChildEBP RetAddr Args to Child

03fbfec4 7c826f4b 77e41ed1 00000001 03fbff08 ntdll!KiFastSystemCallRet

03fbfec8 77e41ed1 00000001 03fbff08 00000000 ntdll!NtDelayExecution+0xc

03fbff30 79f00fce 00001666 00000001 03fbffa0 kernel32!SleepEx+0x68

03fbffac 79f00f5f 00000000 03fbffec 77e64829 mscorwks!ThreadpoolMgr::TimerThreadFire+0x6d

03fbffb8 77e64829 01ceeb5c 00000000 00000000 mscorwks!ThreadpoolMgr::TimerThreadStart+0x57

03fbffec 00000000 79f00f07 01ceeb5c 00000000 kernel32!BaseThreadStart+0x34

15 Id: 137c.14ec Suspend: 1 Teb: 7ffa5000 Unfrozen

ChildEBP RetAddr Args to Child

0403f460 7c827cfb 77e6202c 00000001 0403f4b0 ntdll!KiFastSystemCallRet

0403f464 77e6202c 00000001 0403f4b0 00000001 ntdll!NtWaitForMultipleObjects+0xc

0403f50c 79ed98fd 00000001 000fe3e8 00000000 kernel32!WaitForMultipleObjectsEx+0x11a

0403f574 79ed9889 00000001 000fe3e8 00000000 mscorwks!WaitForMultipleObjectsEx_SO_TOLERANT+0x6f

0403f594 79ed9808 00000001 000fe3e8 00000000 mscorwks!Thread::DoAppropriateAptStateWait+0x3c

0403f618 79ed96c4 00000001 000fe3e8 00000000 mscorwks!Thread::DoAppropriateWaitWorker+0x13c

0403f668 79ed9a62 00000001 000fe3e8 00000000 mscorwks!Thread::DoAppropriateWait+0x40

0403f6c4 79f6f725 ffffffff 00000005 00000000 mscorwks!CLREvent::WaitEx+0xf7

0403f6d4 79f6f702 0403f738 79f91478 8f6c5523 mscorwks!ADUnloadSink::WaitUnloadCompletion+0xb

0403f744 79f6f5d8 00000002 000fe3e8 8f6c55e7 mscorwks!AppDomain::UnloadWait+0x67

0403f780 79f6f4d0 00000002 00000001 00000000 mscorwks!AppDomain::UnloadById+0x126

0403f828 793df368 01dd3098 0403f884 00000000 mscorwks!AppDomainNative::Unload+0xa0

0403f854 66004508 01b72760 01de8694 02080b20 mscorlib_ni+0x31f368

0403f884 79407caa 79373ecd 0403f8b4 03fc2010 System_Web_ni+0xe4508

0403f888 79373ecd 0403f8b4 03fc2010 02080b20 mscorlib_ni+0x347caa

0403f89c 79407e18 02080b20 00000000 02080b20 mscorlib_ni+0x2b3ecd

0403f8b4 79407d90 0403f8d8 0403f8f4 001243b8 mscorlib_ni+0x347e18

0403f8c8 79e7c74b 79fc15dc f2886577 0403f958 mscorlib_ni+0x347d90

00000000 00000000 00000000 00000000 00000000 mscorwks!CallDescrWorker+0x33

16 Id: 137c.a28 Suspend: 1 Teb: 7ffa4000 Unfrozen

ChildEBP RetAddr Args to Child

0485fb78 7c827d0b 77e61d1e 00000254 00000000 ntdll!KiFastSystemCallRet

0485fb7c 77e61d1e 00000254 00000000 0485fbc0 ntdll!NtWaitForSingleObject+0xc

0485fbec 79e789c6 00000254 00027100 00000000 kernel32!WaitForSingleObjectEx+0xac

0485fc30 79e7898f 00000254 00027100 00000000 mscorwks!PEImage::LoadImage+0x1af

0485fc80 79e78944 00027100 00000000 00000000 mscorwks!CLREvent::WaitEx+0x117

0485fc94 79f014ef 00027100 00000000 00000000 mscorwks!CLREvent::Wait+0x17

0485fd14 79f591a2 000d3420 00027100 00000000 mscorwks!ThreadpoolMgr::SafeWait+0x73

0485fd3c 7a035d45 8fea5ff3 00000000 79f0207d mscorwks!ThreadpoolMgr::EnterRetirement+0x8e

0485fd94 79f95a2e 00000000 00000000 00000000 mscorwks!ThreadpoolMgr::WorkerThreadStart+0x33d

0485ffb8 77e64829 0013b058 00000000 00000000 mscorwks!Thread::intermediateThreadProc+0x49

0485ffec 00000000 79f959e8 0013b058 00000000 kernel32!BaseThreadStart+0x34

17 Id: 137c.dd8 Suspend: 1 Teb: 7ffa2000 Unfrozen

ChildEBP RetAddr Args to Child

05e5fec0 7c827cfb 77e6202c 00000003 05e5ff10 ntdll!KiFastSystemCallRet

05e5fec4 77e6202c 00000003 05e5ff10 00000001 ntdll!NtWaitForMultipleObjects+0xc

05e5ff6c 77e62fbe 00000003 769cd34c 00000000 kernel32!WaitForMultipleObjectsEx+0x11a

05e5ff88 76929e35 00000003 769cd34c 00000000 kernel32!WaitForMultipleObjects+0x18

05e5ffb8 77e64829 00000000 00000000 00000000 userenv!NotificationThread+0x5f

05e5ffec 00000000 76929dd9 00000000 00000000 kernel32!BaseThreadStart+0x34

18 Id: 137c.dc8 Suspend: 1 Teb: 7ffa0000 Unfrozen

ChildEBP RetAddr Args to Child

0612ff7c 7c8277db 71b25914 000007a4 0612ffc0 ntdll!KiFastSystemCallRet

0612ff80 71b25914 000007a4 0612ffc0 0612ffb4 ntdll!ZwRemoveIoCompletion+0xc

0612ffb8 77e64829 71b259de 00000000 00000000 mswsock!SockAsyncThread+0x69

0612ffec 00000000 71b258ab 05944120 00000000 kernel32!BaseThreadStart+0x34

19 Id: 137c.177c Suspend: 1 Teb: 7ffa1000 Unfrozen

ChildEBP RetAddr Args to Child

05e9ff0c 7739c811 7739c844 05e9ff50 00000000 ntdll!KiFastSystemCallRet

05e9ff2c 7768ffb3 05e9ff50 00000000 00000000 user32!NtUserGetMessage+0xc

05e9ff6c 7768f366 00007530 77e61c96 05927498 ole32!CDllHost::STAWorkerLoop+0x72

05e9ff88 7768f2a2 05e9ffac 776bbab4 77792c30 ole32!CDllHost::WorkerThread+0xc8

05e9ff90 776bbab4 77792c30 00000000 05927498 ole32!DLLHostThreadEntry+0xd

05e9ffac 776b1704 00000000 05e9ffec 77e64829 ole32!CRpcThread::WorkerLoop+0x26

05e9ffb8 77e64829 05927498 00000000 00000000 ole32!CRpcThreadCache::RpcWorkerThreadEntry+0x20

05e9ffec 00000000 776b16e4 05927498 00000000 kernel32!BaseThreadStart+0x34

20 Id: 137c.16e4 Suspend: 1 Teb: 7ff3e000 Unfrozen

ChildEBP RetAddr Args to Child

0670fd00 7c827d0b 77e61d1e 000008c4 00000000 ntdll!KiFastSystemCallRet

0670fd04 77e61d1e 000008c4 00000000 0670fd48 ntdll!NtWaitForSingleObject+0xc

0670fd74 77e61c8d 000008c4 00002710 00000000 kernel32!WaitForSingleObjectEx+0xac

0670fd88 4a756c3b 000008c4 00002710 00000000 kernel32!WaitForSingleObject+0x12

0670ffb8 77e64829 0594d2e8 00000000 00000000 comsvcs!PingThread+0xf6

0670ffec 00000000 4a756b45 0594d2e8 00000000 kernel32!BaseThreadStart+0x34

# 21 Id: 137c.1480 Suspend: 1 Teb: 7ffa3000 Unfrozen

ChildEBP RetAddr Args to Child

0676f800 79f071ac e0434f4d 00000001 00000001 kernel32!RaiseException+0x53

0676f860 7a10733d 021957d4 00000000 00000000 mscorwks!RaiseTheExceptionInternalOnly+0x2a8

0676f878 7a1073b1 021957d4 00000000 0676f958 mscorwks!RaiseTheException+0x4e

0676f8a0 7a1073e2 00000000 00000000 8475a76b mscorwks!RaiseTheException+0xc0

0676f8cc 7a1073f0 021957d4 00000000 0676f964 mscorwks!RealCOMPlusThrow+0x30

0676f8dc 7a14ebab 021957d4 8d195b03 0595d138 mscorwks!RealCOMPlusThrow+0xd

0676f964 7a14ed1e 00000000 0676fad0 8d1958ff mscorwks!Thread::RaiseCrossContextExceptionHelper+0x79

0676fa98 7a025c10 00000000 0676fad0 0676fb40 mscorwks!Thread::RaiseCrossContextException+0x69

0676fb4c 79f02ab7 00000002 79f0170f 0676fc44 mscorwks!Thread::DoADCallBack+0x2a2

0676fb68 79ef31a3 0676fc44 0676fbf0 79f91478 mscorwks!Thread::DoADCallBack+0x310

0676fbfc 79ef30c3 0676fc44 8d195e5f 00000000 mscorwks!Thread::ShouldChangeAbortToUnload+0xe3

0676fc38 79ef4826 0676fc44 00000002 00000000 mscorwks!Thread::ShouldChangeAbortToUnload+0x30a

0676fc60 79fc57b1 00000002 79fc57b5 0676fcd0 mscorwks!Thread::ShouldChangeAbortToUnload+0x33e

0676fc78 79fc56ac 00000002 79fc57b5 0676fcd0 mscorwks!ManagedThreadBase::KickOff+0x13

0676fd14 79f95a2e 05996378 0676fd28 00000001 mscorwks!ThreadNative::KickOffThread+0x269

0676ffb8 77e64829 05974768 00000000 00000000 mscorwks!Thread::intermediateThreadProc+0x49

0676ffec 00000000 79f959e8 05974768 00000000 kernel32!BaseThreadStart+0x34

22 Id: 137c.d9c Suspend: 1 Teb: 7ffda000 Unfrozen

ChildEBP RetAddr Args to Child

00daff70 7c8277db 7c839f38 000000e4 00daffb0 ntdll!KiFastSystemCallRet

00daff74 7c839f38 000000e4 00daffb0 00daffb4 ntdll!ZwRemoveIoCompletion+0xc

00daffb8 77e64829 00000000 00000000 00000000 ntdll!RtlpWorkerThread+0x3d

00daffec 00000000 7c839efb 00000000 00000000 kernel32!BaseThreadStart+0x34

23 Id: 137c.79c Suspend: 1 Teb: 7ff3f000 Unfrozen

ChildEBP RetAddr Args to Child

063cff70 7c8277db 7c839f38 000000e4 063cffb0 ntdll!KiFastSystemCallRet

063cff74 7c839f38 000000e4 063cffb0 063cffb4 ntdll!ZwRemoveIoCompletion+0xc

063cffb8 77e64829 00000000 00000000 00000000 ntdll!RtlpWorkerThread+0x3d

063cffec 00000000 7c839efb 00000000 00000000 kernel32!BaseThreadStart+0x34

24 Id: 137c.130 Suspend: 1 Teb: 7ff3d000 Unfrozen

ChildEBP RetAddr Args to Child

067dff70 7c8277db 7c839f38 000000e4 067dffb0 ntdll!KiFastSystemCallRet

067dff74 7c839f38 000000e4 067dffb0 067dffb4 ntdll!ZwRemoveIoCompletion+0xc

067dffb8 77e64829 00000000 00000000 00000000 ntdll!RtlpWorkerThread+0x3d

067dffec 00000000 7c839efb 00000000 00000000 kernel32!BaseThreadStart+0x34

25 Id: 137c.af8 Suspend: 1 Teb: 7ff3c000 Unfrozen

ChildEBP RetAddr Args to Child

0681ff70 7c8277db 7c839f38 000000e4 0681ffb0 ntdll!KiFastSystemCallRet

0681ff74 7c839f38 000000e4 0681ffb0 0681ffb4 ntdll!ZwRemoveIoCompletion+0xc

0681ffb8 77e64829 00000000 00000000 00000000 ntdll!RtlpWorkerThread+0x3d

0681ffec 00000000 7c839efb 00000000 00000000 kernel32!BaseThreadStart+0x34

26 Id: 137c.3d0 Suspend: 1 Teb: 7ff3b000 Unfrozen

ChildEBP RetAddr Args to Child

0685ff70 7c8277db 7c839f38 000000e4 0685ffb0 ntdll!KiFastSystemCallRet

0685ff74 7c839f38 000000e4 0685ffb0 0685ffb4 ntdll!ZwRemoveIoCompletion+0xc

0685ffb8 77e64829 00000000 00000000 00000000 ntdll!RtlpWorkerThread+0x3d

0685ffec 00000000 7c839efb 00000000 00000000 kernel32!BaseThreadStart+0x34

27 Id: 137c.da8 Suspend: 1 Teb: 7ff3a000 Unfrozen

ChildEBP RetAddr Args to Child

0689ff70 7c8277db 7c839f38 000000e4 0689ffb0 ntdll!KiFastSystemCallRet

0689ff74 7c839f38 000000e4 0689ffb0 0689ffb4 ntdll!ZwRemoveIoCompletion+0xc

0689ffb8 77e64829 00000000 00000000 00000000 ntdll!RtlpWorkerThread+0x3d

0689ffec 00000000 7c839efb 00000000 00000000 kernel32!BaseThreadStart+0x34

28 Id: 137c.103c Suspend: 1 Teb: 7ff39000 Unfrozen

ChildEBP RetAddr Args to Child

068dff08 7c826f4b 77e41ed1 00000000 068dff4c ntdll!KiFastSystemCallRet

068dff0c 77e41ed1 00000000 068dff4c 77e61c96 ntdll!NtDelayExecution+0xc

068dff74 77e424ed 0000ea60 00000000 068dffac kernel32!SleepEx+0x68

068dff84 776bbb0f 0000ea60 05941e60 776bbab4 kernel32!Sleep+0xf

068dff90 776bbab4 00000000 00000000 05941e60 ole32!CROIDTable::WorkerThreadLoop+0x14

068dffac 776b1704 00000000 068dffec 77e64829 ole32!CRpcThread::WorkerLoop+0x26

068dffb8 77e64829 05941e60 00000000 00000000 ole32!CRpcThreadCache::RpcWorkerThreadEntry+0x20

068dffec 00000000 776b16e4 05941e60 00000000 kernel32!BaseThreadStart+0x34

Thread #21 is the problem.

I also tried taking dump during all exceptions and found out that it was some cache application block that causes thread abort excepiton..here is the details

0:022> !clrstack

OS Thread Id: 0xdb8 (22)

ESP EIP

0674f030 77e4bee7 [HelperMethodFrame: 0674f030] System.Threading.Thread.SleepInternal(Int32)

0674f084 79406919 System.Threading.Thread.Sleep(Int32)

0674f088 05d64c48 Microsoft.ApplicationBlocks.Cache.CacheService.MonitorForExpirations()

0674f0f0 793b0d1f System.Threading.ThreadHelper.ThreadStart_Context(System.Object)

0674f0f8 793740ab System.Threading.ExecutionContext.runTryCode(System.Object)

0674f51c 79e7c74b [HelperMethodFrame_PROTECTOBJ: 0674f51c] System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode, CleanupCode, System.Object)

0674f584 79373ff7 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)

0674f59c 79373ede System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)

0674f5b4 793b0c68 System.Threading.ThreadHelper.ThreadStart()

0674f7e0 79e7c74b [GCFrame: 0674f7e0]

0674fad0 79e7c74b [ContextTransitionFrame: 0674fad0]

Pls guide me, where to go from here.

re: ASP.NET Crash: StackOverflowException with Server.Transfer

Tess — Wed, 14 May 2008 14:55:53 GMT

You mentioned that thread #21 is the problem, but you ran !clrstack for thread 22

re: ASP.NET Crash: StackOverflowException with Server.Transfer

Tess — Wed, 14 May 2008 15:13:16 GMT

No need to post !clrstack for #21 btw,

when you start a thread like this cache service, and it is still alive when the application domain shuts down, an appdomainunloadedexception will be thrown to notify you that the thread was aborted before it finished executing.

You could subscribe to the appdomain_unload event and destroy the thread manually there if you want, but since it is shutting down anyways this exception shouldn't really be an issue.

re: ASP.NET Crash: StackOverflowException with Server.Transfer

Manoj — Wed, 14 May 2008 15:18:05 GMT

Yaa I did tried taking 2 different dumps one with AppDomainUnloadException so for that thread 21 was the prblem and other one I treied is taking dump with all exception so in that thread 22 was the exceuting thred for that process.

Now further investigation with ALLException dump reveals following results,

0:022> !dso

OS Thread Id: 0xdb8 (22)

ESP/REG Object Name

0674eed4 0206003c System.Threading.ThreadAbortException

0674ef1c 0206003c System.Threading.ThreadAbortException

0674f090 020355e8 System.Collections.Hashtable+HashtableEnumerator

0674f094 01fb55a4 System.Collections.Hashtable

0674f09c 01fd4dcc System.Object[] (Microsoft.ApplicationBlocks.Cache.ICacheItemExpiration[])

0674f0a0 01dd5390 System.Globalization.CultureInfo

0674f0a4 01fb8950 System.Collections.ArrayList

0674f0ac 01fb5554 Microsoft.ApplicationBlocks.Cache.CacheService

0674f0b0 01fced74 System.String G4RI.Portal.Common.Caching.Items.DiaryEntryTaskType_TMR019__G4RIGetDiaryEntryTaskType

0674f0b4 01fd4dcc System.Object[] (Microsoft.ApplicationBlocks.Cache.ICacheItemExpiration[])

0674f0f0 01fb76d0 System.String TraceAppender

0674f1b0 01fb76d0 System.String TraceAppender

0674f324 01fb76d0 System.String TraceAppender

0674f4dc 01fb76d0 System.String TraceAppender

0674f4f0 01fb76d0 System.String TraceAppender

0674f500 01fb7450 System.String [

0674f55c 01fb5f00 System.Runtime.CompilerServices.RuntimeHelpers+TryCode

0674f560 01fb5f20 System.Runtime.CompilerServices.RuntimeHelpers+CleanupCode

0674f564 01fb5f40 System.Threading.ExecutionContext+ExecutionContextRunData

0674f578 01fb7450 System.String [

0674f58c 01fb7450 System.String [

0674f598 01fb5cd0 System.Threading.ThreadHelper

0674f5a4 01fb7450 System.String [

0674f804 01e684cc System.Security.Principal.WindowsPrincipal

0674f80c 01fb5ce4 System.Threading.ThreadStart

0674f814 01fb5c98 System.Threading.Thread

0:022> !clrstack

OS Thread Id: 0xdb8 (22)

ESP EIP

0674f030 77e4bee7 [HelperMethodFrame: 0674f030] System.Threading.Thread.SleepInternal(Int32)

0674f084 79406919 System.Threading.Thread.Sleep(Int32)

0674f088 05d64c48 Microsoft.ApplicationBlocks.Cache.CacheService.MonitorForExpirations()

0674f0f0 793b0d1f System.Threading.ThreadHelper.ThreadStart_Context(System.Object)

0674f0f8 793740ab System.Threading.ExecutionContext.runTryCode(System.Object)

0674f51c 79e7c74b [HelperMethodFrame_PROTECTOBJ: 0674f51c] System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode, CleanupCode, System.Object)

0674f584 79373ff7 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)

0674f59c 79373ede System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)

0674f5b4 793b0c68 System.Threading.ThreadHelper.ThreadStart()

0674f7e0 79e7c74b [GCFrame: 0674f7e0]

0674fad0 79e7c74b [ContextTransitionFrame: 0674fad0]

I guess Cache application block Expiration code doing some issue. i.e trying to access some string which may be on other appdomain and since that app domain is unloaded so its cacusing problem whicle shut down.

But how do i confirm this. As thread spawn by cache service is background thread how do i trace weather its trying to access something which is not valid. Or there is any issue with CacheApplicaionBlock code. We are using cacheapplicationblock1.1 code with ASP.NET 2.0

re: ASP.NET Crash: StackOverflowException with Server.Transfer

Manoj — Wed, 14 May 2008 15:20:24 GMT

I know this issue is not causing any business issue but it just keep logging to event logs and other logger's which is causing panic to customer admin team. Any suggestion you want to make is really helpful

re: ASP.NET Crash: StackOverflowException with Server.Transfer

Tess — Wed, 14 May 2008 15:25:18 GMT

shut down the thread in the appdomain unload method is the only thing i can think of if you want to avoid the error completely

re: ASP.NET Crash: StackOverflowException with Server.Transfer

Manoj — Wed, 14 May 2008 22:57:47 GMT

Thanks Tess! I will handle thread during appdomain unload.