We had two interesting days as outlined in Kernel Debugging Day 1 - Interesting snippets of information for debugging in the trenches and Kernel Debugging Day 2 - Interesting snippets of information for debugging in the trenches. Today was a journey at Mach 3 … I need the weekend to do a core brain dump and assimilate what ‘Roy’ has taught us in three very-fast paced, in-depth, guided and extremely interesting days. Therefore you will find no detailed step through today … I will continue this thread after digesting everything and once I start using the WinDBG debugger as a friend, not a strange and powerful alien :)
The commands have multiplied and I decided to combine all of the commands I used over these three days in one consolidated table. Note that this with (hang) in the description are good starting points when analysing a crash dump that is concerned with a hang condition.
So, we promised no detailed step-through, but should at least look at one of the “hang” dumps:
- We open he crash dump with WinDBG as usual.
- Run !analyze –v to get detailed debugging information.
- Run .trap … to display the trap frame register state and also sets the register context.
- Run knv to display the stack trace
- Run up @eip to show the code at the point of disaster
- Run r to display the registers
- Run !vm to display virtual memory usage, whereby we take note of the pool allocation warning:
- In the analysis we dug into the code, looked at what the cause could be by following the assembly code in reverse. In essence we have run out of pool memory and the statement “rep stos dword ptr es:[edi]”, which is a memset, fails because the pointer we are working with is invalid.
- Run !process 0 0 … and we notice that process WRSSSDK is not releasing its handles. Common mistake when creating a thread and not closing the handle once the thread has said goodbye.
That’s it …
Have a look at the www.codemachine.com and their training programs, which are definitely worth it! Tilakraj ‘Roy’ Roychoudhury had to contend with a classroom full of keen software engineers, most of whom have never and will probably never develop and debug kernel mode drivers, yet he managed to bring all of us home and share an immense amount of knowledge.
Thank you ‘Roy’ … it was a real adventure :)