Debugging a Windows 8.1 Store App Crash Dump (Part 2)

Debugging a Windows 8.1 Store App Crash Dump (Part 2)

  • Comments 0

In Part 1, we covered the debugging of a Windows Store Application crash dump that contains a Stowed Exceptions Version 1 (SE01) structure.

 

This post continues on from Part 1, covering the changes introduced in March 2014. These Windows Updates changed the way language exceptions (RoOriginateLanguageException) are recorded in Windows Store Application crash dump files. The new Stowed Exception Version 2 (SE02) structure adds additional fields that directly associate the exception with a language exception object.

 

You’ll recall from the Part 1 that the CLR Exception is loosely associated with the Stowed Exception v1 structure by comparing the HRESULT of the Stowed Exception with the HRESULT of the last CLR Exception on the default thread (the exception record thread). V2 makes this relationship direct. You’ll discover that the Last CLR Exception no longer exists in the v2 dump and that it must be referenced directly by the address stored in the Stowed Exception.

 

The direct association was added to v2 to also aid triage dump carving (done by Windows Error Reporting). It allows WER to explicitly add the memory associated with the relevant Language (CLR) Exception. This eliminates the risk of the garbage collector freeing the memory associated with the last CLR Exception before the dump is taken.  This also helps identify which exception is related to the final crash, which can be difficult when there are multiple exceptions in the dump.

 

Debug Steps

The steps to debug a v2 structure are similar to v1. You first determine the number of stowed exception entries (.exr -1), look at the header to determine the version, display the array of stowed exceptions cast to the correct type (dt -aN …), and then extract the native stack (dpS) or text (du) for each entry.

 

Instead of then comparing the HRESULT to the last CLR Exception (!sos.pe), you use the Nested Exception member to get to the innermost CLR Exception. Due to way object pointers are handled by the CLR, the address is a CCW (COM Callable Wrapper) address, not a CLR object address. To get the CLR object’s address, you use the !sos.dumpccw command. This provides the CLR object address, which can be passed to the !sos.pe command to display the exception.

 

OK, let’s do all of that, showing the commands and data fields of note along the way. (A lot of this is similar to the previous post.)

 

If not done already, set your symbol path to the Microsoft Public Symbol server:

0:003> .sympath SRV*C:\Symbols*http://msdl.microsoft.com/download/symbols

Symbol search path is: SRV*C:\Symbols*http://msdl.microsoft.com/download/symbols

Expanded Symbol search path is: srv*c:\Symbols*http://msdl.microsoft.com/download/symbols

************* Symbol Path validation summary **************

Response                         Time (ms)     Location

Deferred                                       SRV*C:\Symbols*http://msdl.microsoft.com/download/symbols

 

Force the load of the symbols using the .reload /f command:

0:003> .reload /f

...

 

The next step is to display the pointer array as the original structure type. First, we need to know what structure to cast the pointer array to. Using the Parameter[0] value from .exr -1, we will generate a dt command that will display the header of the first record. We use Parameter[0] as the address in this command.

dt  <Parameter[0]> combase!STOWED_EXCEPTION_INFORMATION_HEADER*

 

Here’s an example:

0:003> .exr -1

ExceptionAddress: 7575b152 (combase!RoFailFastWithErrorContextInternal+0x0000010b)

   ExceptionCode: c000027b

  ExceptionFlags: 00000001

NumberParameters: 2

   Parameter[0]: 00c6d3d0

   Parameter[1]: 00000002

 

0:003> dt 00c6d3d0 combase!_STOWED_EXCEPTION_INFORMATION_HEADER*

0x07a690dc

   +0x000 Size             : 0x28

   +0x004 Signature        : 0x53453032

 

The value of the Signature member (0x53453031) is converted to a string using .formats <value>.

0:003> .formats 0x53453032

Evaluate expression:

  Hex:     53453032

  Decimal: 1397043250

  Octal:   12321230062

  Binary:  01010011 01000101 00110000 00110010

  Chars:   SE02

  Time:    Wed Apr 09 04:34:10 2014

  Float:   low 8.46917e+011 high 0

  Double:  6.90231e-315

  • “SE01” maps to combase!STOWED_EXCEPTION_INFORMATION_V1
  • “SE02” maps to combase!STOWED_EXCEPTION_INFORMATION_V2

 

Now that we know the type, we can again use the values from .exr -1 to generate a dt command that will display each record. We use the Parameter[0] as the address, and Parameter[1] as the count in the command. We add a “P” to the start of the type as this is an array of pointers to the type, not structures packed next to each other.

 

In this example, there are 2 pointers, so 2 records are displayed:

dt -a<Parameter[1]> <Parameter[0]> combase!PSTOWED_EXCEPTION_INFORMATION_V2

 

Note, there is no space between the -a and <Parameter[1]>.

0:003> dt -a2 00c6d3d0 combase!PSTOWED_EXCEPTION_INFORMATION_V2

[0] @ 00c6d3d0

---------------------------------------------

0x07a690dc

   +0x000 Header           : _STOWED_EXCEPTION_INFORMATION_HEADER

   +0x008 ResultCode       : 80004001

   +0x00c ExceptionForm    : 0y01

   +0x00c ThreadId         : 0y000000000000000000100000001111 (0x80f)

   +0x010 ExceptionAddress : 0x756b3bff Void

   +0x014 StackTraceWordSize : 4

   +0x018 StackTraceWords  : 3

   +0x01c StackTrace       : 0x0619a368 Void

   +0x010 ErrorText        : 0x756b3bff  "???"

   +0x020 NestedExceptionType : 0x314f454c

   +0x024 NestedException  : 0x063a95d4 Void

 

[1] @ 00c6d3d4

---------------------------------------------

0x0619b6a8

   +0x000 Header           : _STOWED_EXCEPTION_INFORMATION_HEADER

   +0x008 ResultCode       : 80004001

   +0x00c ExceptionForm    : 0y01

   +0x00c ThreadId         : 0y000000000000000000000000000000 (0)

   +0x010 ExceptionAddress : (null)

   +0x014 StackTraceWordSize : 4

   +0x018 StackTraceWords  : 0x3f

   +0x01c StackTrace       : 0x0639bf4c Void

   +0x010 ErrorText        : (null)

   +0x020 NestedExceptionType : 0

   +0x024 NestedException  : (null)

 

Native Call Stack

Regardless of whether the error code (ResultCode) is known or unknown, it is useful to determine the location of the (native) issue by viewing the (native) call stack.

 

Symbol Pointers

If the ExceptionForm member has a value of 0y01, the structure’s union represents a call stack.

 

Unlike call stacks associated with threads, where the symbol pointers are placed throughout the stack next to local variables, these symbols pointers are packed tightly at the address specified in the StackTrace member. Think of it as an array of EBP addresses. The dpS command is used to display the call stack.

  • It is important to include a limit (L) as the call stack is regularly longer than the default 10 rows displayed by dpS. The limit’s value is in the StackTraceWords member.
  • Note that capital S is used (dps vs dpS) because we want to omit the first column normally displayed by dps; the location of the symbol pointer is irrelevant.
  • If you aren‘t using the same bitness debugger as the target’s bitness, use ddS for StackTraceWordSize = 4 (32-bit), and dqS for StackTraceWordSize = 8 (64-bit).

0:003> dt -a2 00c6d3d0 combase!PSTOWED_EXCEPTION_INFORMATION_V2

[0] @ 00c6d3d0

---------------------------------------------

0x07a690dc

   +0x000 Header           : _STOWED_EXCEPTION_INFORMATION_HEADER

   +0x008 ResultCode       : 80004001

   +0x00c ExceptionForm    : 0y01

   +0x00c ThreadId         : 0y000000000000000000100000001111 (0x80f)

   +0x010 ExceptionAddress : 0x756b3bff Void

   +0x014 StackTraceWordSize : 4

   +0x018 StackTraceWords  : 3

   +0x01c StackTrace       : 0x0619a368 Void

   +0x010 ErrorText        : 0x756b3bff  "???"

   +0x020 NestedExceptionType : 0x314f454c

   +0x024 NestedException  : 0x063a95d4 Void

...

0:003> dpS 0x619a368 L3

756ea9f1 combase!RoOriginateLanguageException+0x3b

63b2b04d clr!SetupErrorInfo+0x1e1

63bf4511 clr!MarshalNative::GetHRForException_WinRT+0x7d

 

Unicode String Pointer

If the ExceptionForm member has a value of 0y10, the structure’s union represents an error message.

 

The call stack is (hopefully) contained within the Unicode string pointed at by the ErrorText member. As the text is defined by the caller, the existence of a call stack text isn’t guaranteed.

0:003> dt –a1 13f117e0 combase!PSTOWED_EXCEPTION_INFORMATION_V1

[0] @ 13f117e0

---------------------------------------------

0x0471f3c0

   +0x000 Header           : _STOWED_EXCEPTION_INFORMATION_HEADER

   +0x008 ResultCode       : 8000ffff

   +0x00c ExceptionForm    : 0y10

   +0x00c ThreadId         : 0y000000000000000000010101110100 (0x574)

   +0x010 ExceptionAddress : 0x0de38f7c Void

   +0x014 StackTraceWordSize : 0

   +0x018 StackTraceWords  : 0

   +0x01c StackTrace       : (null)

   +0x010 ErrorText        : 0x0de38f7c  "System.Exception..   at Windows.UI.Xaml.VisualStateManager.GoToState(Control control, String stateName, Boolean useTransitions)..   at MyBadApp.Common.LayoutAwarePage.InvalidateVisualState()..   at MyBadApp.Common.LayoutAwarePage.WindowSizeChanged(Object sender, WindowSizeChangedEventArgs e)"

 

Note - These records aren’t used with v2 language exceptions (or if they are, they are extremely rare based on the Windows Error Reporting telemetry).

 

Nested Exceptions

The new fields in the v2 structure are the NestedExceptionType and NestedException members. The NestedExceptionType member is one of the following values. Much like the Signature field, you can use .formats <value> to see the characters each code represents. The possible values and their associated meaning are:

  • W32E – Win32 Exception – points to an EXCEPTION_RECORD structure
  • STOW – Stowed Exception – points to a STOWED_EXCEPTION_INFORMATION_* structure
  • CLR1 – CLR Object – points (directly) to a CLR Object
  • LEO1 – Language Exception Object – points indirectly to a CLR Exception object

 

LEO1 is the only style being generated by Windows Error Reporting for CLR Exceptions raised in Windows Store Applications.

 

Looking at the example dump file we have been using, it can be seen that the first Stowed Exception has values for the NestedException and NestedExceptionType fields, and they are NULL in the second. Using .formats tells us that the NestedExceptionType member is of type “LEO1”. Note that this is displayed backwards in the output below, in accordance with little-endian order of Intel memory layout.

0:003> dt -a2 00c6d3d0 combase!PSTOWED_EXCEPTION_INFORMATION_V2

[0] @ 00c6d3d0

---------------------------------------------

0x07a690dc

...

   +0x020 NestedExceptionType : 0x314f454c

   +0x024 NestedException  : 0x063a95d4 Void

...

0:003> .formats 0x314f454c

Evaluate expression:

  Hex:     314f454c

  Decimal: 827278668

  Octal:   06123642514

  Binary:  00110001 01001111 01000101 01001100

  Chars:   1OEL

  Time:    Tue Mar 19 16:37:48 1996

  Float:   low 3.01619e-009 high 0

  Double:  4.0873e-315

 

Passing the address to !sos.dumpccw provides the CLR Exception object’s address.

0:003> !sos.dumpccw 0x063a95d4

CCW:               0499f880

Managed object:    02517288

Outer IUnknown:    00000000

Ref count:         1

Flags:            

RefCounted Handle: 00a31478 (STRONG)

COM interface pointers:

      IP       MT Type

 

The address can be used with !sos.pe to display the CLR Exception object. The call stack that the failure investigation should focus on is in this output.

0:003> !sos.pe 02517288

Exception object: 02517288

Exception type:   System.NotImplementedException

Message:          The method or operation is not implemented.

InnerException:   <none>

StackTrace (generated):

    SP       IP       Function

    04F2E38C 00B81382 CrashStore!CrashStore.MainPage.Load_Click_1(System.Object, Windows.UI.Xaml.RoutedEventArgs)+0x62

 

StackTraceString: <none>

HResult: 80004001

 

There you have it. This is the CLR Exception that you need to find to start your code analysis or to point you in the right direction when beginning tracing.

 

But what if SOS is not available?

What do you do if SOS isn’t available? You can check if it is loaded by running the .chain command, and you can check if it is functional by running !sos.dumpccw command (without a parameter).

 

Firstly, make sure you are using the same bitness of the debugger as the bitness of the target.

 

If the dump says “x86” or “ARM (Thumb2)” in the version command or the initial debug spew, use the 32bit debugger.

Windows 8 Version 9600 MP (4 procs) Free x86 compatible

 

If the dump says “x64” in the version command or the initial debug spew, use the 64bit debugger.

Windows 8 Version 9200 MP (4 procs) Free x64

 

If you still don’t have SOS loaded (or working) after matching the bitness, or you get one of the following errors, you’ll have to debug the dump on a system with the same version of the CLR installed. Some CLR versions weren’t indexed and this causes the automatic download of sos.dll and mscordacwks.dll to fail.

0:003> !sos.dumpccw

Failed to load data access DLL, 0x80004005

Verify that 1) you have a recent build of the debugger (6.2.14 or newer)

            2) the file mscordacwks.dll that matches your version of clr.dll is

                in the version directory or on the symbol path

            3) or, if you are debugging a dump file, verify that the file

                mscordacwks_<arch>_<arch>_<version>.dll is on your symbol path.

            4) you are debugging on supported cross platform architecture as

                the dump file. For example, an ARM dump file must be debugged

                on an X86 or an ARM machine; an AMD64 dump file must be

                debugged on an AMD64 machine.

 

You can also run the debugger command .cordll to control the debugger's

load of mscordacwks.dll.  .cordll -ve -u -l will do a verbose reload.

If that succeeds, the SOS command should work on retry.

 

If you are debugging a minidump, you need to make sure that your executable

path is pointing to clr.dll as well.

 

0:003> .cordll -ve -u -l

CLRDLL: C:\Windows\Microsoft.NET\Framework\v4.0.30319\mscordacwks.dll:4.0.30319.18444 f:8

doesn't match desired version 4.0.30319.34011 f:8

CLRDLL: Unable to find mscordacwks_x86_x86_4.0.30319.34011.dll by mscorwks search

CLRDLL: Unable to find 'mscordacwks_x86_x86_4.0.30319.34011.dll' on the path

CLRDLL: Unable to get version info for 'c:\my\sym\cl\clr.dll\52968A96698000\mscordacwks_x86_x86_4.0.30319.34011.dll', Win32 error 0n87

Cannot Automatically load SOS

CLRDLL: ERROR: Unable to load DLL mscordacwks_x86_x86_4.0.30319.34011.dll, Win32 error 0n87

CLR DLL status: ERROR: Unable to load DLL mscordacwks_x86_x86_4.0.30319.34011.dll, Win32 error 0n87

 

0:003> .chain

Extension DLL search Path:

    ...

Extension DLL chain:

    C:\Windows\Microsoft.NET\Framework\v4.0.30319\sos: image 4.0.30319.18444, API 1.0.0, built Wed Oct 30 14:40:34 2013

        [path: C:\Windows\Microsoft.NET\Framework\v4.0.30319\sos.dll]

    pde.dll: image 9, 4, 0, 0, API 9.4.0, built Thu May 08 20:03:58 2014

        [path: c:\debuggers_x86\winext\pde.dll]

    dbghelp: image 6.3.9600.16384, API 6.3.6, built Wed Aug 21 20:59:03 2013

        [path: c:\debuggers_x86\dbghelp.dll]

    ext: image 6.3.9600.16384, API 1.0.0, built Wed Aug 21 21:11:11 2013

        [path: c:\debuggers_x86\winext\ext.dll]

    exts: image 6.3.9600.16384, API 1.0.0, built Wed Aug 21 21:04:14 2013

        [path: c:\debuggers_x86\WINXP\exts.dll]

    uext: image 6.3.9600.16384, API 1.0.0, built Wed Aug 21 21:04:09 2013

        [path: c:\debuggers_x86\winext\uext.dll]

    ntsdexts: image 6.3.9600.16384, API 1.0.0, built Wed Aug 21 21:04:34 2013

        [path: c:\debuggers_x86\WINXP\ntsdexts.dll]

 

Summary

As discussed in the previous article, the asynchronous and projected nature of Windows Store applications makes them significantly harder to debug than desktop applications. Stowed Exceptions v2 helps definitively determine the error code and call stack of the exception that caused the crash.

 

Solutions to some of the more common issues have been talked about on episodes of Channel 9 Defrag Tools, and also in Avoiding Windows Store App Failures talk at //build/ 2014 and the Hardcore Debugging talk at TechEd 2014.

 

If you have any questions, please feel free to email us at DefragTools@microsoft.com, we’ll be happy to help you.

Leave a Comment
  • Please add 6 and 8 and type the answer here:
  • Post