Configuring a Hyper-V VM For Kernel Debugging

Yesterday's blog prompted some questions about how to set up a debugger for a Windows OS running in a Hyper-V VM.  I was surprised that I wasn't able to find good, publicly available, Microsoft issued documentation for this configuration.

The first step is to configure the Windows OS in the VM to enable a kernel debugger on COM1.  One would use these same steps if you were preparing the OS to be debugged using a null modem cable.  Hyper-V will allow us to redirect the COM port so that we don't need such a cable.

1. Start an administrative command prompt.
2. Turn on debugging with this command:

   bcdedit /debug on

3. Configure the debugger to use COM1 with this command:

   bcdedit /dbgsettings SERIAL DEBUGPORT:1 BAUDRATE:115200

   Note that these are the default settings and already exist in most bcd stores.  However setting them again won't damage anything, and guards against a situation where the dbgsettings have been previously modified.

4. Reboot so that the boot loader can read the new settings and configure the OS for debugging.

Next, configure Hyper-V to redirect the COM1 port to a named pipe.  We will use this pipe in place of a traditional null modem cable.

1. Open Hyper-V Manager and browse to the settings page of the VM you configured to debug.
2. Under the Hardware list choose COM 1.
3. Change the Attachment to 'Named pipe:' and provide a pipe name.
   1. Note that the Hyper-V Manager provides the complete path to your named pipe.  Make a note of this path as you will need it in the next step.

After the OS and the VM are configured for debugging, we need to connect a debugger.

1. On the Hyper-V parent partition download and install the Debugging Tools for Windows from http://msdn.microsoft.com/en-us/windows/hardware/gg463009.
2. After installing the debugging tools you will have a ‘Debugging Tools for Windows’ entry in your start menu.
   1. From this folder right click ‘WinDbg’ and choose ‘Run as administrator’.  Windbg needs administrative rights to connect to the pipe.
3. In windbg open the File menu and choose ‘Kernel Debug’.
4. Enter a Baud Rate of 115200, to match the settings made in the VM.
5. Enter the Port that you configured in the VM settings page.
   1. To connect to the pipe remotely, substitute the '.' in the path with the Hyper-V server name.
6. Ensure that the Pipe and Reconnect boxes are checked.
7. Set Resets to 0.
8. Click OK to start debugging.
9. Windbg should display the string ' Waiting to reconnect...'

To test the debugger connection in windbg, from the ‘Debug’ menu choose ‘Break’.  This should cause the server to break into the debugger and display a kd> prompt.  Please note that breaking into the debugger will cause the OS running in the VM to halt until you tell the debugger to go, the OS will appear to be hung during this time.  The command 'g' followed by Enter will tell the debugger to ‘go’ causing the VM to resume operation.