DIA based Stack Walking

DIA based Stack Walking

Rate This
  • Comments 12

Hello, I am Manish Vasani, an SDET on the VC++ compiler backend team. In this series of posts, I will talk about call stacks and how to use the DIA SDK for implementing a stack walking client that builds a call stack. If you are interested in knowing how debuggers build the call stack or want to write an application that dumps the call stack for an executing process, you may want to continue reading further.

In this post I’ll start with a brief introduction on stack walking, discuss the DIA APIs for implementing it, and then walkthrough a code sample for a DIA based stack walker. Those who are familiar with stack walking would know that apart from the DIA SDK, there are two other commonly used stack walking APIs: StackWalk64 (found in DbgHelp) and RtlVirtualUnwind (found in kernel32.lib). I’ll touch upon these two APIs at the end of the post.

 

Introduction:

When the Operating system executes a process, it calls into the program’s entry point (or the starting function) and allocates a set of resources to the process. Some of these resources such as stack memory and register context are allocated per thread basis. Every new function call in the thread is allocated a part of this stack memory and has a new register context. A stack frame or an activation record is essentially this runtime state information for an active function call. Typical attributes associated with a stack frame are:

1)      Function associated with the frame

2)      Module (binary) associated with the frame

3)      Return address

4)      Amount of stack memory allocated for arguments and locals for the function

5)      Base pointer (BP) for referencing the arguments passed to the function call

6)      Base pointer (BP) for referencing the local variables in the function (most of the time this is same as 5)

7)      Program counter or the Instruction Pointer for currently executing instruction (IP)

8)      Stack Pointer (SP)

9)      Other Register context

A sequence of active stack frames in an executing thread is generally referred to as a call stack. Call stacks help developers to debug runtime program errors and also denote the program state at a particular execution point. The process of building this call stack is called stack walking. Stack walking clients (such as debuggers) use the stack frame attributes to retrieve the values of function arguments, locals, etc.

 

DIA APIs for Stack walking:

The primary functionality of the DIA SDK is to provide access to the debug information in PDB files. Apart from that, the DIA SDK also provides a set of interfaces  that can be used to implement a platform independent stack walker. These are:

1)      IDiaStackWalker: This is the primary initialization interface for the DIA stack walker.

2)      IDiaEnumStackFrames: This is the stack frame enumerator interface that exposes the primary API to perform a stack walk to the next frame.

3)      IDiaStackFrame: This interface represents a stack frame and exposes attributes of the stack frame.

4)      IDiaStackWalkHelper: This is a helper interface for the DIA stack walker. To understand its primary purpose, let us briefly mention the two primary types of inputs required by the stack walker:

                              i.      Run-time entities: This includes the runtime aspects such as thread’s register context, properties of loaded image files such as LOADED_IMAGE, executable binary (or module) corresponding to a given virtual address, etc.

                             ii.      Compile-time generated entities: This includes the PDB records that provide static attributes for frames such as memory allocated for locals/arguments, whether the function corresponding to the frame contains SEH/C++ EH, the unwind program to execute to walk to the next frame, etc.

DIA stack walker uses other DIA interfaces to read the compile time generated entities, i.e. PDB records. It delegates fetching the runtime entities to the DIA clients. The primary reason for this being that DIA doesn’t maintain the runtime state information for the process. For example, a given virtual address in process’ address space might belong to any one of the modules (executable binaries) loaded in the process. Each module would have a different pdb file and hence a different DIA session to read the pdb. DIA isn’t aware about the loaded modules for the process and hence expects its clients to provide this mapping. Most DIA stack walker clients, such as debuggers, already need to fetch and store this runtime information. Hence, it is logical to expect the clients to provide this information to DIA through the implementation of this helper interface.

 

Code Sample Walkthrough:

I will first give the class design for the DIA based stack walker sample and then walk you through its execution flow.

Class design:

1)      MyDiaStackWalker: This is the primary DIA stack walker class. It maintains a handle to the initialization interface (IDiaStackWalker) and the stack frame enumerator (IDiaEnumStackFrames). It also contains few helper methods for other DIA operations.

2)      MyDiaStackWalkHelper: This class implements the IDiaStackWalkHelper interface. I will discuss each of its method in detail later on.

3)      Debugger: This is a Debugger class with a simple debugging engine. It launches the specified executable with debugging enabled and dumps the call stack at every debug break event in the executable. (Debug break events could be trigged by use of __debugbreak() intrinsic in your source code).

4)      ContextManager: This class manages the register context associated with the current stack frame of the current debuggee thread (debuggee is the process being debugged)

5)      ModuleManager: This class maintains the list of modules (executable binaries) loaded in the process along with their attributes.

Execution Flow:

1)      Initialization:

The first step in the application is to initialize the DIA stack walker. This is done through the IDiaStackWalker interface. IDiaStackWalker provides two platform specific APIs for initializing the DIA client: getEnumFrames (for x86) and getEnumFrames2 (for x64 and ia64 platforms). Each of these APIs take a pointer to the class that implements IDiaStackWalkHelper (MyDiaStackWalkHelper) and returns the stack frame enumerator (IDiaEnumStackFrames). You may refer to the method “MyDiaStackWalker::Initialize” in MyDiaStackWalker.cpp to see the initialization logic.

 

2)      Walking the stack:

After the initialization, we launch the specified executable under the debugger. On hitting a debug break event, we initialize the register context and loaded modules for the debuggee. Than we call “MyDiaStackWalker::WalkCallStack” method to walk the call stack and retrieve attributes such as module name, function name, Instruction pointer (IP), etc. for each stack frame:

// method to walk all the stack frames in the call stack

void MyDiaStackWalker::WalkCallStack()

  wprintf(L"\nWalking call stack...\n");

  IDiaSymbol *pFuncSym = NULL;   

  DWORD count = 0;

  BSTR szFunctionName;

  ULONGLONG ip = 0;

 

  do {

    // Get IP for current stack frame

    pContextManager->get_currentIP(&ip);

    if (!ip) {

      break;

    }

    // Get module for stack frame

    Module *pModule = pModuleManager->FindModuleByVA(ip);

    // Get DIA function symbol for stack frame

    pMyDiaStackWalkHelper->symbolForVA(ip, &pFuncSym);

    // Display stack frame attributes

    if (pFuncSym == NULL) {

      // function symbol could be null if symbols for the module were not loaded.

      // just dump the IP for the stack frame

      wprintf(L"Stack Frame %d: '%ws!%#x()'\n", ++count, pModule->name, ip);

      continue;

    }

    if (pFuncSym->get_name(&szFunctionName) == S_OK) {

      wprintf(L"Stack Frame %d: '%ws!%ws'\n", ++count, pModule->name, szFunctionName);

      SysFreeString(szFunctionName);

    }

    pFuncSym->Release();

    pFuncSym = NULL;

    // Walk to next stack frame

  } while (WalkStackFrame() == S_OK);

}

 

WalkCallStack” method calls into “WalkStackFrame” to walk each stack frame. WalkStackFrame uses the stack frame enumerator API (IDiaEnumStackFrames::Next) to do the stack walk to next frame. Following snippet shows this:

// MyDiaStackWalker.cpp

// method to walk to the next stack frame

HRESULT MyDiaStackWalker::WalkStackFrame(IDiaStackFrame **ppStackFrame)

{

  IFVERBOSE {

    wprintf(L"MyDiaStackWalker::WalkStackFrame\n");

  }

 

  ULONG celtFetched = 0

  IDiaStackFrame *pStackFrame = NULL;

 

  // walk to the next stack frame

  HRESULT hr = pEnumFrames->Next(1, &pStackFrame, &celtFetched);

 

  if (celtFetched == 1 && hr == S_OK) {

    if (ppStackFrame != NULL) {

      *ppStackFrame = pStackFrame;

    }

    else {

      pStackFrame->Release();

    }

  }

  return hr;

}

 

3)      DIA callbacks into MyDiaStackWalkHelper during stack walk:

When we walk the stack frame using the IDiaEnumStackFrames::Next, DIA makes a few callbacks into MyDiaStackWalkHelper. Let me discuss each of the methods of this class to explain its purpose:

1.        Method to read process memory from debuggee’s address space (MyDiaStackWalkHelper::readMemory): DIA can’t read into debuggee’s address space, hence it expects the client to do implement this API. I call into the ReadProcessMemory Windows API for its implementation.

 

2.       Methods to get/set debuggee register context (MyDiaStackWalkHelper::get_registerValue and put_registerValue):

                                                      i.      get_registerValue: This callback API is used by DIA to fetch the register values for the current stack frame. Prior to the WalkCallStack invocation, I use the GetThreadContext Windows API, for fetching the current thread’s register context (see “ContextManager::InitializeRegisterContext” in ContextManager.cpp). Register IDs used by DIA are defined in cvconst.h that is available in the DIA SDK.

 

                                                    ii.      put_registerValue: This callback API is used by DIA to update the register values. DIA makes a few callbacks into put_registerValue at the end of every stack walk invocation (i.e. IDiaEnumStackFrames::Next) to update the register context to the next stack frame. Note that the IDiaStackFrame object returned by IDiaEnumStackFrames::Next corresponds to the stack frame that we walked past (i.e. it corresponds to the top stack frame on the first invocation of next, while the callbacks into IDiaStackWalkHelper::put_registerValue correspond to the second stack frame)

Also, note that a call to IDiaEnumStackFrames::Reset to reset the stackwalker resets only the internal DIA stack frame enumerator. Clients should reinitialize the register context (using GetThreadContext) between a call to IDiaEnumStackFrames::Reset and subsequent IDiaEnumStackFrames::Next.

 

3.       Methods to map a virtual address (VA) to some entity:

                                                         i.            Methods to map a given VA to unwind data (MyDiaStackWalkHelper::frameForVA and  MyDiaStackWalkHelper::pDataForVA): Before getting into the specifics of these APIs, let me provide a brief description about the format of unwind data for different platforms.

x86 and x64 platforms have different ABI conventions which lead to the stack unwind information lying in different formats in different files for these architectures. x64 ABI requires the functions to setup their prolog and epilog in a standard way, so that a stack unwind can be performed from any given point in the function by just looking at function’s unwind data (pdata/xdata) from the image file. You may refer to articles about Exception Handling (x64) and x64 Unwind information for more details.

x86 platform doesn’t have this requirement of setting up the prolog/epilog in a standard manner. Hence, in order to perform a successful stack unwind from any given IP, the compiler tracks and emits additional unwind information in the PDB. This information is recorded inside the Frame Data PDB records. DIA exposes these records through the IDiaFrameData interface. We will discuss the details about IDiaFrameData and how to use it along with IDiaStackWalkFrame to walk the Frame Data records in a follow up post.

As you might have guessed by now, pDataForVA method is specific to x64 and ia64 platforms. This method parses the pdata section in the image file to find the matching pdata entry for the given VA. Structure of the mapped pdata entry that is expected by DIA is quite identical to the _IMAGE_RUNTIME_FUNCTION_ENTRY structure, except that addresses in _IMAGE_RUNTIME_FUNCTION_ENTRY structure are relative to the base virtual address of the image file, while the addresses in our custom defined IMAGE_RUNTIME_FUNCTION_ENTRY_VA structure are absolute virtual addresses (i.e. with base address relocations applied):

// ModuleManager.h

struct IMAGE_RUNTIME_FUNCTION_ENTRY_VA

{

                ULONGLONG BeginAddress;

                ULONGLONG EndAddress;

                ULONGLONG UnwindInfoAddress; 

};

 

frameForVA is an x86 specific method to fetch the Frame Data record for a given virtual address. MyDiaStackWalkHelper::frameForVA does the following mapping: given VA -> module -> pdb -> DIA session for the pdb -> frameByVA.

 

                                                       ii.            Method to map a given VA to the base virtual address of the image (exe or dll) file (MyDiaStackWalkHelper::imageForVA): I use the GetModuleInformation API to obtain the MODULEINFO structure which contains the base address for the module (see “ModuleManager::InitializeModules” in ModuleManager.cpp)

 

                                                      iii.            Method to map a given VA to DIA symbol (MyDiaStackWalkHelper::symbolForVA): I first map the VA to an appropriate DIA session and use IDiaEnumSymbolsByAddr::symbolByVA API to fetch the matching DIA symbol.

 

4.       Methods to search for return address on the stack (MyDiaStackWalkHelper::searchForReturnAddress and MyDiaStackWalkHelper::searchForReturnAddressStart): DIA will make these callbacks when walking an x86 FPO stack frame. I return E_NOTIMPL as I do not wish to override the default FPO stack search done by DIA.

Code Sample:

You can download the stack walking sample from the Visual C++ Sample page. Usage of the sample:

Ø  DIA_StackWalk_Sample.exe DIA_StackWalk_Testcase.exe

You may compile the sample with /DVERBOSE to get more descriptive logging. A sample test is also provided.

 

Alternative Stack walking APIs:

Let me talk about the two alternative stack walk APIs that I mentioned at the beginning of the post:

StackWalk64 (DbgHelp API): Similar to the DIA stack walk APIs, StackWalk64 provides a platform independent mechanism for stack walking. If you closely go through all the parameters for this API, you can map them to the corresponding IDiaStackWalkHelper methods. Given that StackWalk64 is identical to DIA stack walk APIs, how do we decide which one to pick over other? I find the following guidelines useful:

1)      If you are already using DbgHelp or are willing to add a dependency to it, than StackWalk64 should be the ideal choice for stack walking. It operates at an abstraction layer above DIA and hence provides a more high level and easy to implement interface for the client.

2)      If you are implementing your own debugger and only want to add the stack walk functionality, than using the DIA stack walker would be a good choice.

3)      If you want to tweak the low level functionality, such as override how default return address search is performed for FPO stack frames, than DIA stack walker would be a good choice.

RtlVirtualUnwind is also a similar stack walk API with identical parameters; except that unlike the above APIs:

1)      It has to be invoked in the same process and

2)      Is designed only for x64 platforms.

 

Conclusion:

Hopefully this post provided you a good overview of how to implement a stack walker client using the DIA SDK.  As always, we welcome your feedback on DIA, and look for part 2 of this series on walking the x86 Frame data records using IDiaFrameData/IDiaStackWalkFrame interfaces.

  • Does this stuff only work in debug mode, i.e. when a pdb file is present? Or could it also be used in a release build, at least to get the stack trace (without any debug symbols, obviously)?

    Thanks.

  • what about RtlCaptureStackBackTrace?

  • Colen: As long as there is a stack frame it will work regardless of debug/release build. Just remember that compiler settings like omit frame pointer can cause problems though. The pdb file isn't required, what it does is adds extra information to the debugger, it gives things like the name of the function, what source file it uses, what line on the source file an instruction refers to etc. There is nothing in the pdb file which stops you from walking the stack without it.

  • JK: Thanks for pointing it out. RtlCaptureStackBackTrace also walks the stack but returns you an array of return addresses for each frame and doesn't deal with symbols. You can use this information though to build a rich stack trace on your own.

    Colen: The response given by Crescens2k is correct. Without the pdb file, you can still get the module name and the virtual address for the frame, but not other debugging information. Stack trace without symbols for a call stack with FPO frames (omit frame pointer) has always been tricky. We are trying to improve DIA FPO stack search for this scenario, but it is still recommended to have the symbol file for an FPO'ed frame.

  • Does DIA give you the right information when crossing the boundary between native and managed code?

  • Shouldn't you be using nullptr instead of NULL?

    After all, it was your team that put that in ;)

  • Thanks. Does this only work on Windows 7, or on earlier versions too? I can't find any version info in the MSDN docs.

  • DIA only takes care of walking native frames.  Managed frames are walked by debugger itself.

    For transition from managed to native code, DIA will be asked to start from CLR generated managed-to-native thunk which usually doesn’t have any debug info since it is generated on the fly.  For frames without debug info, DIA uses its own heuristics to walk stacks, hence occasionally DIA may get wrong caller frame.

    We are improving DIA stack walking in this area.

  • Colen: DIA stack walker interfaces are supported on earlier versions of Windows as well.

    Clark: Yes, you are correct I should make that change, I need to catch up ;)

  • The sample crashes if you walk a mixed-mode app because "pModule" in the line:

    Module *pModule = pModuleManager->FindModuleByVA(ip);

    will be NULL ;)

  • Agree, there should be a null check in there first; we will update the sample source to have the result checked before using it...

  • Clark & Manish:  nullptr is not present in VC9 and below.  I assume that this code can be compiled with those compilers, so retaining NULL in this case would be most appropriate.  I mean VC10 isn't even at the Release Candidate stage yet.

Page 1 of 1 (12 items)