As a software developer, I spend much of my time looking at code, learning how it works, and figuring out how to modify or fix it. A very good tool to help examine code is the Visual Studio debugger.

 

(Even if you’re not a hard core programmer, the following tutorial shows some of the power of the Visual Studio components, such as the project system, build system, debugger, working together.)

 

At a breakpoint I can examine local variables in the Watch, Auto or Locals window to see their values and types. If it’s a class or structure, the debugger will show a “+” that indicates it can be expanded and the first couple members of that structure. Structures submembers or inherited values can be examined. These structures can get very deep. Sometimes I need to inspect a value that’s dozens of levels down in a hierarchy. That’s a lot of complicated tree navigation in the debugger. Other times I need to take a local variable name (or a member of that variable if it’s a structure/class), drag and drop it to a new line in the Watch Window, then type cast it to a value that’s more meaningful. As I step through code, the variable might go out of scope, or it might have a different name in a subroutine, so I’d have to repeat the typecasting steps in the watch window with the different variable name.

 

For example, suppose one of the variables is called VBLine and is an internal representation of a line of VB.NET code. It’s much more meaningful to see “Dim MyVar As String”, then a bunch of hex numbers in the debugger. I drag/drop it to the watch window, typecast it to a “DIM” statement, and expand/navigate the results to find “MyVar”. Then, I step into the next called function, with VBLine passed as an argument. The receiving function names the parameter VBStatement, so my watch window drilldown needs to be modified to use the different variable name.

 

This gets very cumbersome. Let’s improve it!

 

Here’s a simple demonstration of how you can control what the debugger displays.

 

  1. Start Visual Studio (2003 or 2005. (It also works in VS7, although the steps might be slightly different.)
  2. Choose File->New->Projects
  3. Choose Visual C++->Win32 Console Application, call it Test
  4. Click Finish on the wizard.

 

Now paste in some sample code to debug:

 

#include "windows.h"

int _tmain(int argc, _TCHAR* argv[])

{

      OSVERSIONINFOEX osinfo; // Declare a structure

      ZeroMemory(&osinfo, sizeof(osinfo));      // init it to 0

      osinfo.dwOSVersionInfoSize = sizeof(OSVERSIONINFOEX); // set the size

      GetVersionEx((LPOSVERSIONINFO) &osinfo);  // call WinAPI to fill it in

 

      WIN32_FIND_DATA   FAR ffd;    // Declare a structure

      FindFirstFile("c:\\windows\\system32\\k*.exe",&ffd);  // Find the first file starting with "k"

 

      return 0;   //set bpt here

}

 

This sample code just calls the Windows API functions GetVersionEx and FindFirstFile, which fill structures that we can examine in the debugger.

To make things simple, let’s use ANSI rather than Unicode chars (VS2005 defaults to Unicode). Choose Project->Properties->Configuration Properties->General->Character Set and change it to “Use Multi-Byte Character set” rather than “Use Unicode Character Set”

Let’s also remove the check for 64 bit portability issues:

Choose Project->Properties->Configuration Properties->C++->General and turn off “Detect 64 bit portability issues”

 

 

Hit F9 on the “return” line to set a breakpoint.

Then hit F5 to build and run the project.

When the breakpoint hits, the debug window shows:

 

+                      osinfo   {dwOSVersionInfoSize=284 dwMajorVersion=5 dwMinorVersion=1 ...}      _OSVERSIONINFOEXA

+                      ffd        {dwFileAttributes=32 ftCreationTime={...} ftLastAccessTime={...} ...}         _WIN32_FIND_DATAA

 

Now let’s control the string displayed for a given type.

 

Open the file called AutoExp.dat (installs with VS) in the VS editor:

File->Open->File. On my machine it’s c:\Program Files\Microsoft Visual Studio 8\Common7\Packages\Debugger\Autoexp.dat

This file describes how to customize the output of the Debug Watch, Locals, and Auto windows.  It’s formatted like an INI file.

 

Add this line to the AutoExp.Dat file in the [AutoExpand] section.

 

_OSVERSIONINFOEXA = Hi there <szCSDVersion> Build number = <dwBuildNumber>

 

 

Now hit F5 to go to the breakpoint.

Now the watch window shows:

+                      osinfo   {Hi there 0x0013fe58 "Service Pack 2" Build number = 2600}        _OSVERSIONINFOEXA

 

This is a big improvement: we’ve told the debugger which members of the structure to show and how to format them! We can still click on the “+” to drill down the member hierarchy.

 

When starting a debug session, the debugger reads the AutoExpand file and if the left of the equals matches the type in the Type column of the Locals/Watch/Auto window, then the right side will direct how to format the displayed string. The comments at the beginning of AutoExp.dat give more details, including more formatting options.

 

This is great, but it’s nothing compared to what we’ll do next!

 

You can write code that executes in the debugger process that can read the memory of the debugee! AutoExp.dat controls this feature too.

Replace the above line with these 3 lines

 

_OSVERSIONINFOEXA= $ADDIN(MyDbgEE.dll,?EE_OSVERSIONINFOEXA@@YGJKPAUtagDEBUGHELPER@@HHPADIK@Z)

_WIN32_FIND_DATAA =$ADDIN(MyDbgEE.dll,?EE_WIN32_FIND_DATAA@@YGJKPAUtagDEBUGHELPER@@HHPADIK@Z)

MyClass = $ADDIN(MyDbgEE.dll,?EE_MyClass@@YGJKPAUtagDEBUGHELPER@@HHPADIK@Z)

 

The $ADDIN(DllName,FunctionName) syntax means that the DLL named will be loaded, the FunctionName export in the DLL will be called. (Ignore the gobbledygook: it’s just C++ name decorating indicating the calling convention, the parameters, etc.) If any error occurs, like the DLL can’t be found, the export can’t be found, or the DLL caused an exception, the displayed string will be “{???}”

 

Now let’s create the project that will build MyDbg.DLL and add it to the current solution.

Choose File->New->Project->Visual C++ Win32 Project, call it MyDbgEE and choose to Add to solution (rather than to Create New Solution).

In the Win32 App Wizard that appears, change the Application Type to a DLL.

 

Change the project properties as above to non-Unicode and no 64 bit issues.

 

Add these lines:

 

#define ADDIN_API    __declspec(dllexport)

 

typedef struct tagDEBUGHELPER

{

    DWORD dwVersion;

    BOOL (WINAPI *ReadDebuggeeMemory)( struct tagDEBUGHELPER *pThis, DWORD dwAddr, DWORD nWant, VOID* pWhere, DWORD *nGot );

    // from here only when dwVersion >= 0x20000

    DWORDLONG (WINAPI *GetRealAddress)( struct tagDEBUGHELPER *pThis );

    BOOL (WINAPI *ReadDebuggeeMemoryEx)( struct tagDEBUGHELPER *pThis, DWORDLONG qwAddr, DWORD nWant, VOID* pWhere, DWORD *nGot );

    int (WINAPI *GetProcessorType)( struct tagDEBUGHELPER *pThis );

} DEBUGHELPER;

 

ADDIN_API  HRESULT WINAPI EE_OSVERSIONINFOEXA( DWORD dwAddress, DEBUGHELPER *pHelper, int nBase, BOOL bUniStrings, char *pResult, size_t max, DWORD reserved )

{

      wsprintf(pResult,"Testing Addr = %x Uni = %d base = %d %x",dwAddress,bUniStrings, nBase, *(DWORD *)dwAddress);

      return S_OK;

}

 

ADDIN_API  HRESULT WINAPI EE_WIN32_FIND_DATAA( DWORD dwAddress, DEBUGHELPER *pHelper, int nBase, BOOL bUniStrings, char *pResult, size_t max, DWORD reserved )

{

      WIN32_FIND_DATA   FAR ffd;

      DWORD nGot=0;

      pHelper->ReadDebuggeeMemory(pHelper,dwAddress,sizeof(ffd),&ffd,&nGot);

      wsprintf(pResult,"FindData found file '%s' DBG Process ID = %d",ffd.cFileName, GetCurrentProcessId());

      return S_OK;

}

 

 

Now we need to tell VS where to put the built DLL so the debugger can find it. We can use the build events in the project.

For the DLL project, choose Project->Properties->Configuration Properties->Build Events->Post Build Event->Command Line. Put in copy $(TargetPath) "$(DevEnvDir)" Make sure you have the quotes and parentheses right. If you put in a description string, then that string will be echoed to the Output Window when building. Now when you rebuild, the debug dll will be copied to the same dir as Devenv.exe.

 

Now hit F5 and see the values in the debug window! Bring up Task Manager and notice that the Process ID shown is the same as that of the Devenv.exe debugger process.

 

 

To make things more interesting, let’s see how our debug code can read the debugger memory. We’ll add some code to obscure a desired value, but we’ll dig for it in the debug dll. Add this code after the “#include windows.h”  line in the main Test code

 

struct MyClass {// normally this will go in #include file

      int mymem1; // make the 1st few members irrelevant, so debugger won't show interesting info

      int mymem2;

      int mymem3;

      int mymem4;

      int mymem5;

      short *str; // make this not a string, so debugger won't show it as a string

      MyClass * m_pNextClass; // self referential, perhaps like a linklist

};

 

 

Now add this code to just before the “return” statement:

 

 

      MyClass * pMyClass = new MyClass(); // declare a new instance of MyClass

      pMyClass->str = new short(8);       // create a heap allocated byte array

      memcpy(pMyClass->str,"NotMe!",7);   // desired value to see in debugger

 

 

      pMyClass->m_pNextClass = new MyClass();   // make a submember instance

      pMyClass->m_pNextClass->str = new short(8);     // heap allocated submember string

      memcpy(pMyClass->m_pNextClass ->str,"Bingo!",7);      // desired value to see in pMyClass

 

 

This code creates a class MyClass with a pointer to another instance of MyClass which contains the desired debug display value.

Now we need to modify the debug dll to dig for the value. Copy the same structure definition above into the debug dll code. (Typically, these definitions will be in a shared #include file.)

Add this code:

 

ADDIN_API  HRESULT WINAPI EE_MyClass( DWORD dwAddress, DEBUGHELPER *pHelper, int nBase, BOOL bUniStrings, char *pResult, size_t max, DWORD reserved )

{

      DWORD nGot=0;

      MyClass oMyClass;

      pHelper->ReadDebuggeeMemory(pHelper,dwAddress,sizeof(oMyClass),&oMyClass,&nGot); // read the debuggee's structure

      char szMainStr[100];

      char szMemberStr[100];

      *szMemberStr=0;   // init to empty string

      if (oMyClass.m_pNextClass)    // if there's a sub member

      {

            MyClass oNextClass;

            pHelper->ReadDebuggeeMemory(pHelper,(DWORD)oMyClass.m_pNextClass,sizeof(oNextClass),&oNextClass,&nGot); // read it

            pHelper->ReadDebuggeeMemory(pHelper,(DWORD)oNextClass.str,sizeof(szMemberStr),&szMemberStr,&nGot);            // read it's string

      }

      pHelper->ReadDebuggeeMemory(pHelper,(DWORD)oMyClass.str,sizeof(szMainStr),szMainStr,&nGot); // read the string of the main struct

      wsprintf(pResult,"MyClass string is '%s'. Submem = '%s'",szMainStr,szMemberStr);

      return S_OK;

}

 

Now hit the F5 button and Bingo! You can still drill down into the class manually as before, so you haven’t lost any functionality.

 

The debug DLL can be rebuilt even while debugging: it’s loaded/unloaded as needed by the debugger. That means persisting values might be cumbersome. I’ve used custom registry keys for persisting values, like global variables.

 

I’ve been using this debug expression evaluator architecture for years for huge projects, including Visual Foxpro and Visual Basic.Net, and I find it indispensable and a huge time saver.

 

See also: EEAddIn: Uses the Expression Evaluator Add-In API to extend the native debugger expression evaluator.

Very Advanced Debugging tips