What you need for this exercise:

 

  1. An out of proc COM server
  2. A client attempting to call the exe in #1
  3. #2 is getting the below hr returned by a call to #1

 

// From WinError.h

// MessageId: RPC_E_DISCONNECTED

//

// MessageText:

//

//  The object invoked has disconnected from its clients.

//

#define RPC_E_DISCONNECTED               _HRESULT_TYPEDEF_(0x80010108L)

 

  1. Both #1 and #2 are on the same machine
  2. The latest version of Debugging Tools For windows (a.k.a.) WinDBG
  3. Symbols
    1. You can download the symbol package from the WinDBG website
    2. Set your symbols path to point to the public symbol server
  4. J Patience

 

 

Let’s assume from this point forward that my particular scenario is that I’ve got many COM clients connecting to an out of proc COM server.  The clients make thousands of calls per hour.  Intermittently calls to the server will return RPC_E_DISCONNECTED.  If I try the entire process over again it may or may not be successful.  I need to figure out what my server is doing at the point that I see this HR so I can make some changes to prevent this condition from occurring.

 

Now that you have gotten to this point let’s explain what’s going to happen.  We are basically going to set a breakpoint on the hresult we are interested in and then force the debugger to take a user dump of all the different components that we’ll need to analyze the problem.  Once we have the user mode dumps we can do a post mortem analysis and make recommendations to resolve the problem.  This is just a particular RPC case, we can do this for any scenario where we want to break on a return value (typically when the value we want is moved into register eax).

 

To put a breakpoint on a particular hr and run a command:

 

1.      Choose the debugee.  This can either be the client application or the RPC service.  Just a note…debugging RPC can cause your entire system to lock up (right Niel!?). Use caution, or a VPC, when doing this.  For this situation I chose the client application.

2.      Fire up windbg and figure out where in memory ole32.dll is loaded (see the lm command, output below).  You can also use depends to figure out where this guy wants to be loaded but since we’ve got more work in windbg ahead it’s the easier thing to do.

 

start    end        module name

774e0000 7761d000   ole32     

 

3.      Search the memory range of this module for the HR we’re interested in.  the HR value was found by searching my include directory for RPC_E_DISCONNECTED.  See the s command in windbg.

 

0:005> s -d 774e0000 L(7761d000-774e0000) 80010108

77588d74  80010108 000123e9 ce8b5000 ffe61ce8  .....#...P......

77589594  80010108 04c25d5e ebc03300 909090f7  ....^]...3......

7758966c  80010108 000092e9 18458b00 0874c085  ..........E...t.

77603a34  80010108 ba3d0774 75800706 fc458b6f  ....t.=....uo.E.

 

4.      Now that you have the addresses (1st column) for each location of that hr in memory you can put a breakpoint on those locations and can associate some commands with that breakpoint.  See the bp command in windbg.  See the bl command to list your breakpoints from the windbg help docs.  See the .dump command for creating dump files.  See the adplus.doc document that discusses adplus.vbs.  Note that you can interchangeably use –hang instead of –quick if you don’t have your symbol file path set.

 

0:005> bp 77588d74 “.dump /mau –f c:\dumps\ClientDumpFile.dmp;!!start c:\program files\debugging tools for windows\adplus.vbs –quite –quick –p <RPC Server Process ID> -p <COM Server PID> -o c:\dumps;g”

0:005> bp 77589594 “” <- as above

0:005> bp 7758966c “” <- as above

0:005> bp 77603a34 “” <- as above

0:005> bl

 0 e 77588d74     0001 (0001)

 1 e 77589594     0001 (0001)

 2 e 7758966c     0001 (0001)

 3 e 77603a34     0001 (0001)

 

5.      The breakpoints from #4 may contain some false positive breakpoints as our hr may line up with some byte pattern in the memory space of the module but it’s unlikely that we’ll hit that breakpoint and if we do it should be easy enough to decipher.

 

6.      By now you should have three dump files all located in the dumps folder.  The clientdumpfile.dmp and two folders created by adplus for each exe.

 

Now that I’ve gone through that you can put most of this inside of a config file, and use the command line debugger cdb. All the steps get condensed to” cdb –p <Client PID> -c “$><custom.config”

 

Here’s the contents of the custom.config file:

 

 

bp 77588d74  “.dump /mau –f c:\dumps\ClientDumpFile.dmp;!!start c:\program files\debugging tools for windows\adplus.vbs –quite –quick –p <RPC Server Process ID> -p <COM Server PID> -o c:\dumps;g”

 

bp 77589594  “.dump /mau –f c:\dumps\ClientDumpFile.dmp;!!start c:\program files\debugging tools for windows\adplus.vbs –quite –quick –p <RPC Server Process ID> -p <COM Server PID> -o c:\dumps;g”

 

bp 7758966c  “.dump /mau –f c:\dumps\ClientDumpFile.dmp;!!start c:\program files\debugging tools for windows\adplus.vbs –quite –quick –p <RPC Server Process ID> -p <COM Server PID> -o c:\dumps;g”

 

bp 77603a34 “.dump /mau –f c:\dumps\ClientDumpFile.dmp;!!start c:\program files\debugging tools for windows\adplus.vbs –quite –quick –p <RPC Server Process ID> -p <COM Server PID> -o c:\dumps;g”

 

g

 

 

I didn’t test actually test the syntax of all of the statements above since I don’t have such a scenario handy but I did give them a twice over and look ok.  I’ll make changes to the post as problems are brought to my attention.