Debugging Toolbox

Top Things to Consider When Troubleshooting Complex Application Issues

rafarah — Mon, 03 Oct 2011 20:10:41 GMT

1- For reactive incidents: “Bring the engineer onsite because it is going to be easier to isolate the problem.”

This is the most common misconception I’ve heard. Let me explain: most complex problems require deep debugging sessions.
Collecting the necessary information is the easy part and can be done remotely or by the customer. However, several hours or days may be necessary for debugging the dump files. Being onsite can actually slow down the process since we can be without access to our private symbols and collaborative access to
colleagues with specific technology knowledge.

Many times, a great value of being onsite is to act as ears and eyes for a remote engineer or to get a better understanding of complex problems that we can’t understand very well by e-mail or phone.

2- “We need a Code Review because our application has performance problems.”

Sometimes I receive requests for Code Review but what the customer needs, in reality, is Problem Isolation. So what is the difference and how do I know what I need?

The goal of a Code Review is to review the source code and point the parts of the code that are not following the Best Practices, or that represents security holes or yet parts of the code that can be optimized for speed.

The goal of Problem Isolation is to isolate problem(s) causing specific application symptoms. For example, crashes, hangs, memory leaks and performance bottlenecks.

Let me explain: imagine a scenario where an ASP.NET application is suffering from poor performance. If I code review the application I may find methods that can be optimized for speed. However, if the application has slow performance, because there is a bottleneck on the database side or network, the performance gain from the code review is not going to solve the problem. Worst case, it may not even be noticeable.

The Code Review is great when you want to make sure your application doesn’t have potential problems that could be avoided by implementing Best Practices or if you think the application can be further optimized to gain more speed. However, you’ll only be able to measure the speed gain if you have a baseline from when the application was not experiencing issues. Importantly, usually the performance gain is not as significant as removing the bottlenecks.

3- “So after fixing this problem the performance/memory issue is going to be normalized, right?”

The reality is that there can be different problems causing the same symptom(s) like slow performance, hangs or memory issues.

What does this mean? It means that after solving the most significant and visible problem(s) we need to monitor the application because other minor problems could be causing the same symptom. In addition, after fixing the main bottleneck(s), these other minor problems should become visible and easier to isolate. Identifying and resolving application issues are an iterative process. In case you would like to know more about it read this old post here.

4- “We’re using .NET so I don’t need to worry about memory management.”

If you have a pure .NET application, I would be inclined to agree. However, most commercial applications have some kind of interaction with the Native World, like C DLLs, COM objects or API calls.

The CLR is great to manage memory from pure .NET applications. If your application is interacting with native code it is the developer’s responsibility to make sure that resources are released/closed.

5- What information do I need to collect? How much information do I need to collect?

There is a fine line between not enough information and too much information. What is most important for us is to get the right information. One dump file from your problematic application collected when experiencing the symptom is very valuable. Five dump files from the problematic application collected when it was running fine will most likely not be very helpful.

If your application is crashing you want to collect a dump file when the application crashes. If you collect a dump file any other time it won’t have information from the exception. If you force a huge Kernel Dump file to be collected you will end up with a huge dump file from all your machine’s processes but, again, that dump file won’t have information about the exception crashing your application.

6- “We need an Architecture Review because our application has performance problems.”

This is similar to item 2 above. An Architecture Review is not the best approach to solve immediate problems.
Additionally, an Architecture Review may not even be the right approach to solve most application problems because usually these application problems are too granular. This means the customer’s application designed correctly from an Architectural point of view but problems not related to the way the architecture was designed.

Let me provide a few examples. Imagine that you haven’t installed an important update for the .NET Framework which is impacting your application. Or that your SharePoint application is not releasing the internal SharePoint objects it’s using. In these examples, an Architecture Review is not going to uncover these problems.

7- Sometimes finding the right place to start is the hardest part.

Imagine this scenario:

“We need an IIS Engineer because my W3WP.EXE is consuming too much memory. It may be an IIS bug.” How will users, administrators and developers experience the issue?

End user: I think the browser has a problem, the application is slow.
IIS Administrator: I think the problem is the ASP.NET application.
Developers: The ASP.NET application is running fine; the problem is probably on the database side.
DBA: The SQL Server is running fine; I think the bottleneck is network related.
Network Administrator: The network doesn’t have problems.

Our goal as Developer PFEs is to help our customers to isolate the problem across disparate technologies and work to provide cross group collaboration between different teams while onsite or remotely.

8- What skillset do I need to help me debug an application?

If your application requires debugging you don’t need an engineer who knows how to administer or install
the product. What you need is an engineer that knows the internals of the applications and how to debug them. The good news is that this knowledge is not application dependent.

A Microsoft Engineer can debug your application even if he/she has never seen your application before. The same applies to our own products.

If at some point we isolate the problem to one of our products then we involve an engineer from the Product Team because he/she has intimate knowledge of the problems and bugs from the product he/she
supports.

9- “I just ran !clrstack and most threads running for a long time are trying to retrieve data from the database. The bottleneck is probably on the database side.”

Let me tell you something: I used to say to our new engineers or those who want to learn more about .NET Debugging, if you want to excel at .NET Debugging you must learn Native code debugging, which implies some knowledge of C/C++ programming too.

Don’t believe me? Ask your favorite bloggers that blog about .NET Debugging if they only know .NET debugging.

With that being said !clrstack is the favorite command from people learning .NET Debugging. It’s cool; you can see the managed side of the call stack which usually is higher level than the native side. However, sometimes you still need to see the native side to really understand what the thread is doing otherwise, if focusing just on the managed side, you may come up with the wrong conclusion.

Bottom line is: If you want to improve your .NET Debugging skills learn more about Native debugging.

Here is a list of books about .NET Debugging, User Mode Debugging and Kernel Debugging.

10- “My two servers are identical but the issue happens just on server XYZ.”

When troubleshooting scenarios like that never assume the servers are identical. Instead, gather the data to prove it.

A great place to start is to run the MPSReport/SPSReport tool. This tool will collect all information from each server and compare them. In at least one occasion in which the servers really were identical, the underlying issue was one of the servers was being accessed by the application, so it was being overloaded.

11- “From the Event Log I can see the exception that crashed my application and the call stack is pointing to Windows. I think this is a Windows bug.”

This is related to item 7 above and a common misconception. Sometimes calls stacks from 2^nd chance
exceptions (exceptions not handled by your application, thus crashing the app) have DLLs from Windows as the top frames. This is normal and it doesn’t mean that Windows is causing the crash.

Example:

ChildEBP RetAddr
0013bcd0 7c90de7a ntdll!KiFastSystemCall+0x2
0013bdd0 7c81cdfe kernel32!_ExitProcess+0x62
0013bde4 79f944b0 kernel32!ExitProcess+0x14
0013c00c 79f2c09a mscorwks!SafeExitProcess+0x11b
0013c018 79eff585 mscorwks!DisableRuntime+0xd1
0013c0a8 79011628 mscorwks!CorExitProcess+0x242
0013c0b8 77c39d3c mscoree!CorExitProcess+0x46
0013c0c4 77c39e78 msvcrt!__crtExitProcess+0x29
0013c0d4 77c39e90 msvcrt!_cinit+0xee
0013c0e8 0e68d21e msvcrt!exit+0x12
0013c580 0e256834 testappl!FuTestInterface::init+0x34 <<< This is where you should start the investigation.
0013c5a4 0e1d8c01 testapp!WBNARiskReportInterface::getResults+0x442a

Therefore don’t assume that ntdll or kernel32 caused the problem. The APIs from these Operating System dlls are being called as a consequence of the exception likely caused by the application. Try to identify the latest application method call as your initial investigation point. In our example above this is testapp!FuTestInterface::init. Analyze it and, if necessary, analyze the previous frame and so on.

12- “We collected dump files from that C++ application which is crashing. We think it is a heap corruption, so the call stack should indicate the culprit, right?”

Heap corruption is not frequent as it used to be because .NET applications are more and more common. However, back in the days of COM objects and C DLLs, heap corruption was a typical problem.

In order to get call stacks from the method which actually corrupted the heap is to enable Page Heap, restart the application so it can use the new Heap Manager settings and collect a dump file. With this approach you can easily isolate the heap corruption problem.

The Page Heap can be enabled using different tools like PageHeap.exe, GFlags.exe, Application
Verifier and others. Some Page Heap settings, like Full Page Heap, create a read only page after each memory allocation, so whenever your application tries to overwrite the buffer it hits the read only page and you get an Access Violation.

Here is a didactic explanation about using GFlags to isolate Heap Corruption problems.

Note: Windows Vista/Windows Server 2008 and Windows 7 have more mechanisms to easily detect heap corruptions and minimize them.

Note 2: In some situations even PageHeap won’t crash with the culprit on the call stack but those cases are rare. (Thanks to Mario Hewardt for the reminder)

New Debugging Book – Windows Debugging Notebook: Essential User Space WinDbg Commands

rafarah — Fri, 30 Sep 2011 01:25:00 GMT

A reference book for technical support and escalation engineers troubleshooting and debugging complex software issues. The book is also invaluable for software maintenance and development engineers debugging Windows applications and services.

Do you want to know more about this book? Check out here…

[PowerShell Script] PowerDbg v6.0 – Using PowerShell to Control WinDbg

rafarah — Tue, 01 Mar 2011 01:09:00 GMT

Last October the latest version of the PowerDbg tool was released, version 6.0. This release has major changes. For example, it is now easier to write scripts or otherwise just hack around the debugger, and includes the PowerDbgConsole, a bootstrapper designed to make setting up an interactive PowerDbg debugging session a one-liner. In addition, this new version is faster, easier to use, and has full 64 bit support.

For those new to PowerDbg, with this tool you can easily create PowerShell scripts to automate a WinDbg/CDB debugging session. You can use PowerDbg for Kernel Mode or User Mode, Post-Mortem Debugging or Live Debugging, either for native or managed code.

PowerShell has several advantages over the WinDbg Scripting Language, like:

-      Increased ease in creating scripts

-      Leverage in the .NET Framework

-      Debugging and tracing features

-      Code reuse through functions and cmdlets

-      Easier maintenance

-      Increased ease in building large scripts

-      Ease in formatting and displaying important information

PowerDbg is hosted in the CodePlex website and there you can find all the information you need about the new PowerDbg, so I’m just reusing the information from CodePlex here.

Download

Download PowerDbg.

Prerequisites

In order to use PowerDbg you will need:

      PowerShell 2

      This comes ‘out of the box’ with Windows 7 and Windows 2008 R2 (though it may not be enabled by default). For other versions of Windows it can be obtained by installing the Windows Management Framework Core Package

     A recent version of the Debugging Tools for Windows.
Most of the testing has been done against the version that comes with the Windows SDK 7.1 and the previous stand-alone version 6.1.1.404 (x86 / x64).

You’ll also need to configure PowerShell to allow scripting, which is disabled by default. Typically you’d relax the execution policy to allow unsigned scripts to run from the local machine (only) [1]:

Set-ExecutionPolicy RemoteSigned

If UAC is enabled you’ll need to do this from a PowerShell prompt ‘run as administrator’.

If you’re running 64 bit Windows and plan on doing any 32 bit debugging, you’ll also need to do this in the 32 bit PowerShell console (Windows PowerShell (x86) in your Start menu).

Installation

After downloading, extract the PowerDbg.zip to a temporary directory somewhere, and use the Install_PowerDbg.ps1 script to install it: just right click on it and select ‘Run With PowerShell’.

Alternatively, you can manually drop the PowerDbg.psm1 module file into your PowerShell module directory, ensuring it gets placed in a ‘PowerDbg’ subfolder.

Establishing a PowerDbg Session

New-DbgSession

To use PowerDbg you must first establish a ‘session’. This establishes PowerDbg’s internal connection between a local debugger and either a memory dump, live process, or remote debugger.

Sessions are established using the New-DbgSession cmdlet and can be instantiated in one of four ways:

- attaching to a running process by name (-process)
- launching a process under the debugger (-command)
- opening a memory dump in the debugger (-dump)
- connecting to an existing debugger configured as a server (-remote)

Additionally, PowerDbg can be instructed to auto-load SOS (if possible), using the -sos parameter. This can only be performed if the process in question has already loaded the CLR, and so is not applicable to the -command method.

Example

The PowerDbg 'unhandled exception' demo can be run under the debugger as follows (note the quotes which allow us to pass an argument to the process being launched):

PS > New-DbgSession -command '.\blah\ManagedScenarios.exe UnhandledExceptionScenario'

PS > Send-DbgGo

...

(1104.730): CLR exception - code e0434f4d (first chance)

(1104.730): CLR exception - code e0434f4d (!!! second chance !!!)

KERNELBASE!RaiseException+0x39:

000007fe`fdedaa7d 4881c4c8000000 add     rsp,0C8h

PS >

Since the debugger immediately breaks into the launched process, we have to use Send-DbgGo to allow the process to continue on (and subsequently fail). When creating a session to a memory dump or a remote debugger already halted at the point of interest, this would be unnecessary.

Once a PowerDbg session exists, you can interact with it using any of the Get-Dbg* cmdlets, or by using Invoke-DbgCommand to send commands directly.

Notes

     Each PowerShell window running PowerDbg can only support one such session at a time.

     The session is automatically terminated when the PowerDbg module is unloaded (Remove-Module PowerDbg).

     The debugger is attached invasively, and so the debuggee will terminate when the session is ended.

Some Commands

Session Management
New-DbgSession	Establishes a PowerDbg session with the debugger
Exit-DbgSession	Terminates the current PowerDbg session
Load-DbgExtension	Loads a debugger extension into the active session
Invoke-DbgCommand	Executes an arbitrary command against the debugger
Exploration
Get-DbgArray	Expands the contents of a managed array
Get-DbgClrHeap	Locates objects on the managed heap by type or MT
Get-DbgDictionaryContents	Enumerates Dictionary<T,T> collections
Get-DbgListContents	Enumerates List<T> collections
Get-DbgObject	Views the contents of a reference-type object
Get-DbgValueClass	Views the contents of a value-type object
Process State
Get-DbgClrThreads	Lists managed threads
Get-DbgComState	Views the COM apartment state associated with each thread
Get-DbgModules	Lists modules loaded into the process
Get-DbgRunaway	Views the kernel and user time associated with each thread
Analysis
Analyze-PowerDbgThreads	Sorts all threads by CPU (User Mode) and writes in red all threads which are using third-party components, an easy way to isolate threads consuming high CPU.
Misc
Start-DebuggerOnly	Launches a debugger without creating a PowerDbg session
Get-DbgHangDump	Takes an immediate full minidump of a given process

The new PowerDbg is the effort of a great team of people from different companies and different parts of the world. Particularly for me it’s always exciting to work with people from all over the world.

With that said, I want to thank everybody who worked in this project, especially:

I want to thank Lee Holmes (SDE from the PowerShell team) for all his support and valuable tips since the first versions of PowerDbg.

I also want to thank Augusto Pedroza (SDE from the Windows 8 FileSystem team) for using PowerShell in real Microsoft scenarios and for giving valuable feedback on how to improve the tool. (Obrigado Augusto! J)

Finally, I want to send a very special thanks to Piers Williams, who not only contributed with awesome ideas but added hundreds of lines of code to PowerDbg, worked on the documentation, and turned the PowerDbg project into one in which others can easily contribute.

We are looking for developers to join the PowerDbg project. If you’re interested, use the CodePlex site and join us!

[PowerShell Script] Troubleshooting for Port Exhaustion Using NetStat

rafarah — Mon, 11 Oct 2010 19:10:00 GMT

Problem Description:

Applications that use a great deal of TCP network activity may use all of the possible port numbers -- especially if they are very “chatty”. By default, when an application closes a TCP connection, the port number used cannot be reused for the same IP address for another four minutes (TcpTimedWaitDelay). Also, by default, the possible port number is limited to a maximum of 5000 (MaxUserPort). Therefore, very chatty applications may use up all of the possible port numbers under very high loads – this is often called port exhaustion or socket burnout.

Troubleshooting:

To check for port exhaustion is simple, but tedious because NetStat –an will display all of the IP addresses, their port numbers and their statuses; however, it doesn’t count them!

One of my teammates, Frank Taglianetti (aka Tag), created a cool PowerShell script that counts the number of ports with the TIME_WAIT status, the percentage of used ports in TIME_WAIT, the total number of used ports, and the percentage of port numbers used.

The output is sorted by the number of ports used, in descending order so that you can see the IP address that is most likely exhausted or near exhaustion.

Compared to NetStat this script is easier to use and the results are easier to visualize.

Here is an example:

.\count-ports.ps1

[F:\Downloads\Tools\Scripts\NetStat]PS:3>.\Count-Ports.ps1

10/9/2010 6:31:06 PM

IPAddress PortsWaiting %Waiting PortsUsed %Used

--------- ------------ -------- --------- -----

65.55.149.121 1 33.33 % 3 0.06 %

184.85.110.98 1 33.33 % 3 0.06 %

96.17.108.146 1 100.00 % 1 0.02 %

138.108.14.10 1 50.00 % 2 0.04 %

65.54.95.216 1 11.11 % 9 0.18 %

65.54.95.86 1 50.00 % 2 0.04 %

…

Here is the source code for the Count-Ports.ps1:

## Author: Frank Taglianetti - ftag@microsoft.com

## Displays port counts per IP address

function New-Port

{

Param

(

[string] $IPAddress = [String]::EmptyString,

[int32] $PortsWaiting = 0,

[int32] $MaxUserPort = 5000

)

$newPort = New-Object PSObject

Add-Member -InputObject $newPort -MemberType NoteProperty -Name IPAddress -Value $IPAddress

Add-Member -InputObject $newPort -MemberType NoteProperty -Name PortsUsed -Value 1

Add-Member -InputObject $newPort -MemberType ScriptProperty -Name PercentUsed -Value {$this.PortsUsed / $this.MaxUserPort}

Add-Member -InputObject $newPort -MemberType NoteProperty -Name PortsWaiting -Value $portsWaiting

Add-Member -InputObject $newPort -MemberType ScriptProperty -Name PercentWaiting -Value {$this.PortsWaiting / [Math]::Max(1,$this.PortsUsed)}

Add-Member -InputObject $newPort -MemberType NoteProperty -Name MaxUserPort -Value $maxUserPort

return $newPort

}

# display current date and time

Write-Host -Object $(Get-Date)

# Display the MaxUserPort and TcpTimedWaitDelay settings in the registry if available

Get-ItemProperty -Path HKLM:\SYSTEM\CurrentControlSet\services\Tcpip\Parameters | Format-List MaxUserPort,TcpTimedWaitDelay

# Store MaxUserPort for percentage used calculations

$maxPorts = $(Get-ItemProperty -Path HKLM:\SYSTEM\CurrentControlSet\services\Tcpip\Parameters | Select-Object MaxUserPort).MaxUserPort

if($maxPorts -eq $null)

{

$maxPorts = 5000 #Windows Default

}

[System.Collections.HashTable] $ports = New-Object System.Collections.HashTable

[int32] $intWait = 0

netstat -an |

Select-String "TCP\s+.+\:.+\s+(.+)\:(\d+)\s+(\w+)" |

ForEach-Object {

$key = $_.matches[0].Groups[1].value

$Status = $_.matches[0].Groups[3].value

if("TIME_WAIT" -like $Status)

{

$intWait = 1 # incr count

}

else

{

$intWait = 0 # don't incr count

}

if(-not $ports.ContainsKey($key)) #IP Address not yet counted

{

$port = New-Port -IPAddress $key -PortsWaiting $intWait -MaxUserPort $maxPorts #intialize new tracking object

$ports.Add($key,$port) #Add the tracking object to hashtable

}

else #otherwise a tracking object exists for this IP

{

$port = $ports[$key] #retrieve the tracking object

$port.PortsUsed ++ # increment the port count (PortsUsed)

$port.PortsWaiting += $intWait # increment PortsWaiting if status is TIME_WAIT

}

# Format-Table -InputObject $ports.Values -auto

Sort-Object -InputObject $ports.Values -Property PortsUsed, PortsWaiting -Descending |

Format-Table -Property IPAddress,PortsWaiting,

@{Name='%Waiting';Expression ={"{0:P}" -f $_.PercentWaiting};Alignment="Right"},PortsUsed,

@{Name='%Used';Expression ={"{0:P}" -f $_.PercentUsed}; Alignment="Right"} -Auto

Remove-Variable -Name "ports"

[WinDbg Script] Displaying Parameters for Microsoft.ReportingServices.ReportProcessing

rafarah — Sat, 14 Aug 2010 01:45:00 GMT

Here is a new script from a PFE from Portugal, Marcio Parente.

Marcio kindly shared his source code in this blog post, so here is the story behind the script followed by its source code.

One report on a Reporting Server started to give this error:

String: Syntax error converting the varchar value 'AA' to a column of data type int.

Exploring the situation Marcio had the need to find out what was the value of one parameter, and because there were 14380 parameters an script was needed.

0x1f014e88 0x09ae5e84 28 -1 Microsoft.ReportingServices.ReportProcessing.ParameterValue

Statistics:

MT Count TotalSize Class Name

0x09ae5e84 14,380 402,640 Microsoft.ReportingServices.ReportProcessing.ParameterValue

Total 14,380 objects, Total size: 402,640

What Marcio needs is the name and value.

0:000> !dumpobj 0x10546834

Name: Microsoft.ReportingServices.ReportProcessing.ParameterValue

MethodTable 0x09ae5e84

EEClass 0x09b009cc

Size 28(0x1c) bytes

GC Generation: 2

mdToken: 0x020000e6 (c:\windows\microsoft.net\framework\v1.1.4322\temporary asp.net files\reportserver\94eb9a55\49df36c9\assembly\dl2\02d1b52f\00398a5b_d238c501\microsoft.reportingservices.processing.dll)

FieldDesc*: 0x09ae5d48

MT Field Offset Type Attr Value Name

0x09ae5e84 0x40003dc 0x4 CLASS instance 0x10546850 m_name

0x09ae5e84 0x40003dd 0x8 CLASS instance 0x1054688c m_value

0x09ae5e84 0x40003de 0xc CLASS instance 0x00000000 m_omit

0x09ae5e84 0x40003df 0x14 System.Int32 instance 1 m_exprHostID

0x09ae5e84 0x40003e0 0x10 CLASS instance 0x1c53f980 a

Content:

0:000> !do 0x10546850

String: @chrCultureCode

0:000> !do 0x1054688c

Name: Microsoft.ReportingServices.ReportProcessing.ExpressionInfo

MethodTable 0x09ae4860

EEClass 0x09b004d4

Size 60(0x3c) bytes

GC Generation: 2

mdToken: 0x02000127 (c:\windows\microsoft.net\framework\v1.1.4322\temporary asp.net files\reportserver\94eb9a55\49df36c9\assembly\dl2\02d1b52f\00398a5b_d238c501\microsoft.reportingservices.processing.dll)

FieldDesc*: 0x09ae4490

MT Field Offset Type Attr Value Name

0x09ae4860 0x40005d7 0x24 System.Int32 instance 0 m_type

0x09ae4860 0x40005d8 0x4 CLASS instance 0x00000000 m_stringValue

0x09ae4860 0x40005d9 0x34 System.Boolean instance 0 m_boolValue

0x09ae4860 0x40005da 0x28 System.Int32 instance 0 m_intValue

0x09ae4860 0x40005db 0x2c System.Int32 instance -1 m_exprHostID

0x09ae4860 0x40005dc 0x8 CLASS instance 0x00000000 a

0x09ae4860 0x40005dd 0xc CLASS instance 0x00000000 b

0x09ae4860 0x40005de 0x10 CLASS instance 0x00000000 c

0x09ae4860 0x40005df 0x14 CLASS instance 0x00000000 d

0x09ae4860 0x40005e0 0x18 CLASS instance 0x00000000 e

0x09ae4860 0x40005e1 0x1c CLASS instance 0x00000000 f

0x09ae4860 0x40005e2 0x20 CLASS instance 0x00000000 g

0x09ae4860 0x40005e3 0x30 System.Int32 instance -1 h

So Marcio decided to create the script below to get the content of the correspondent parameter.

This is the output from the script below:

Scanning all Microsoft.ReportingServices.ReportProcessing.ParameterValue ...

*****

String: @chrCultureCode

String: pt

*****

String: @chrCultureCode

String: pt

*****

String: @chrCultureCode

String: pt

*****

String: @chrCultureCode

String: pt

*****

String: @chrCultureCode

String: pt

*****

String: @chrCultureCode

String: pt

*****

String: @chrCultureCode

*****

String: @chrCultureCode

String: pt

*****

Stop scanning all Microsoft.ReportingServices.ReportProcessing.ParameterValue ...

Source code of REPORTING_SERVICES_PARAMETER.TXT:

$$ =========================================================

$$ REPORTING_SERVICES_PARAMETER.TXT

$$ Displays the values of Microsoft.ReportingServices.ReportProcessing.ParameterValue.

$$ Compatibility: Win32 (not tested on x64).

$$ Example: $$><myscripts\REPORTING_SERVICES_PARAMETER.TXT

$$ Marcio Parente

$$ All my scripts are provided "AS IS" with no warranties, and confer no rights.

$$ =========================================================

.printf /D "<b>Scanning all Microsoft.ReportingServices.ReportProcessing.ParameterValue ...</b>\n\n"

.foreach (runtime {!dumpheap -type Microsoft.ReportingServices.ReportProcessing.ParameterValue -short})

{

.foreach(obj {!dumpobj poi(${runtime}+0x4)})

{

.if(0 == $sicmp("${obj}", "@chrCultureCode"))

{

.echo *****

!dumpobj poi(${runtime}+0x4)

!dumpobj poi(poi(${runtime}+0x8)+0x4)

.echo *****

}

.printf /D "<b>Stop scanning all Microsoft.ReportingServices.ReportProcessing.ParameterValue ...</b>"

New PowerDbg – I Need Your Help

rafarah — Tue, 25 May 2010 22:55:00 GMT

First, let me fix the title. It should be: “New PowerDbg – We Need Your Help”.

I explain, a few weeks ago I received an e-mail from Piers Williams, a developer from Australia. Piers mentioned his blog article related to PowerDbg where he makes some constructive criticism.

When reading the article I was like: “Why haven’t I thought about that before?” Piers have some simple yet awesome ideas to make PowerDbg much easier to use. Right after reading the article I invited him to be my partner and implement his ideas.

So here are some of the planned features for the new version:

- No more PowerDbg for most of the cmdlets. Current cmdlets will have aliases. For example, Send-PowerDbgCommand can be called by Invoke-WinDbgCommand.

- No more CSV files. It means no more Convert-PowerDbgCSVToHashTable.

- No more Parse cmdlets. It means fewer commands to create scripts.

- Greater PowerShell experience.

- Compatible with previous version so your scripts should continue to run fine.

- More cmdlets compatible with 64 bits. Not all cmdlets are 64 bits compatible.

- Auto-detect location of Debugging Tools.

- Provide command outputs as pipeline objects.

- When debugging a dump, caches command output.

Here is a comparison between the way you use PowerDbg nowadays and how you’re going to use it for the next version.

Current version:

Send-PowerDbgCommand “!do 0x353433”

Parse-PowerDbgDUMPOBJECT

$output = Convert-PowerDbgCSVToHashTable

For the new version all you need to do is:

Dump-Object “0x353433”

There’ll be cmdlets like:

dd = Display-DWord

db = Display-Bytes

dps = Display-PointerSymbols

!DumpDomain = Dump-Domain

!DumpObj = Dump-Obj

!runaway = Run-Away

Because WinDbg has some cryptic command names not all commands are going to have a corresponding name.

We need to think about the best approach for:

!address

. . .

If you have suggestions and want to share them put them here.

From the developer perspective, I mean, people extending PowerDbg, here are some changes:

- The regular expressions are now in one place, thus reducing the size of the cmdlets, besides it’s easier to change the regular expressions.

- For people working on the PowerDbg code there’re files for each cmdlet, so the maintenance is easier. To ship the code we concatenate the files into one single file. (Piers, thanks for doing that!)

- No more global variables to save state.

The goal for the next version is simple: The new PowerDbg will be much easier to use and extend!

That said, Piers and I are very interested to have more partners who can help us to make this goal a reality.

I’ve got to tell you the number one excuse I hear when inviting PowerDbg users to help me. It’s always a variation of: “I’m new to PowerShell/I’m new to debugging/I don’t know enough about PowerShell/I don’t know enough about debugging”.

Guys, c’mon! I’m a PowerShell hobbyist and PowerDbg was my “Hello world!” application, my very first PowerShell code. Unfortunately the only opportunity I have to use PowerShell is when creating PowerDbg code or when creating PowerDbg scripts for debugging. I wished to know more about PowerShell, but I’m getting better. In fact, I wouldn’t be surprised to know that Lee Holmes when looking at my source code put his hands over his head and screams like crazy. J

So my point here is: I don’t care about your lack of knowledge. I’m interested in your ideas, in your creativity. At the end this is what matters most. PowerShell? You learn it when creating cmdlets, don’t worry.

Here is an example: my colleague Clint Huffman isn’t a developer, he works with infrastructure, specifically Windows, Biztalk and IIS administration but almost everyone that I know, knows or have been using his very helpful PAL tool. By the way, the next PAL version will be based on PowerShell.

In case you’re in doubt and need more thinking let me give you a few reasons to join Piers and myself on the next PowerDbg version:

- If you want to learn more about PowerShell this is a great opportunity.

- If you know a lot about PowerShell and think the PowerDbg code could be more optimized we’d love to have you implementing your optimizations!

- If you want to know more about debugging you can improve your knowledge creating and testing code for PowerDbg. Remember, PowerDbg works for Kernel Mode debugging, managed code debugging and User Mode debugging. Also for live debugging and post-mortem debugging.

- If you don’t want to write code but have some cool ideas to improve PowerDbg, we’d love to know them. How can we create something easier to use? Something you’d like to use? Let’s know, here.

Interested? You can contact me using this blog or writing a comment below. Or yet go to the CodePlex website here.

PSSCOR2, the Superset of SOS.DLL is Now Public!!!

rafarah — Mon, 29 Mar 2010 18:03:00 GMT

Whenever I’m debugging with customers watching it’s inevitable: they always ask me what this PSSCOR2.dll extension is. The next question is always if PSSCOR2.DLL is going to be public.

PSSCOR2.DLL is a superset of SOS.DLL and has much more commands and variations! The good news is that yes, now PSSCOR2.DLL is public, so you can download it right now from Tom's blog here.

Happy debugging!

XPerf Tool – Why Can’t You Live Without It?

rafarah — Mon, 15 Mar 2010 15:30:00 GMT

Israel Burman (Israel is one of the ADPlus creators and the guy who taught me the XPerf tool) and Mario Hewardt told me I should blog about the XPerf tool.

Although I’m new to this tool I decided to follow their suggestions because I believe you’re going to wonder how you could live without using this tool after using it for the first time. J

So, when starting to write this article I decided to browse the internet to see how many articles from Microsoft I could find because I didn’t want to be redundant. As a matter of fact I found several great articles. (Am I late to the party? J) Some of these articles are very didactic and similar to what I had in mind.

That said, I’m not going to be redundant here, likewise I’m going to give you just an overview about the tool and mention those articles that details the configuration and usage of XPerf so you can get the necessary details from these articles without having to research the internet all over again.

WHAT’S XPERF?

- XPerf is based on ETW (Event Tracing for Windows) which is a very efficient tracing infrastructure.

- The overhead caused by XPerf is about ~2.5% CPU, in other words, very minimum.

- The tracing can be dynamically enabled or disabled and it doesn’t require a reboot.

- XPerf enables you to collect logs, create reports and see charts from the collected data.

WHEN SHOULD YOU USE XPERF?

- When you need to isolate performance problems.

- To get a better understanding of the Operating System.

- Probably other scenarios where you can benefit for tracing the Windows components.

WHERE CAN YOU DOWNLOAD XPERF?

From here or here.

HOW CAN YOU START USING IT?

- Collect logs from a few seconds to 5 minutes.

- Start just before reproducing the symptom.

- Don’t forget to setup the symbols.

BASIC COMMANDS

After installing XPerf open a Command Prompt using RunAs Administrator.

Setup symbols:

set _NT_SYMBOL_PATH= srv*C:\symbols*http://msdl.microsoft.com/downloads/symbols

Start XPerf, collecting just general information:

XPerf -on DiagEasy

Reproduce the symptom.

After that use this command to stop the tracing, creating a log file:

XPerf -d trace.etl

To visualize the charts from the data you just collected use:

XPerf trace.etl

XPerf can collect different information for different scenarios.

To do that you need to change the provider. Providers are the Windows components that have the ability to log information.

Sintax:

XPerf –on <provider>

To get a list of all available providers use:

XPerf -providers k

To get call stack information you need to specify the Kernel events that should log the call stack.

This is the way to do that:

Xperf -on <provider> -stackwalk <flags>

Example using provider = diageasy and stackwalk = profile:

XPerf -on diageasy -stackwalk profile

If you want to see other types of flags use:

XPerf -help stackwalk

To send the information to a CSV file use:

XPerf -i trace.etl > output.csv

You can create a PowerShell script which parses the output from the CSV file.

It’s possible to create a CSV file with filtered information. To do that use:

XPerf –i trace.etl –a <action_name> > output.csv

Example:

XPerf –i trace.etl –a registry > output.csv

Or:

XPerf –i trace.etl –o output.txt –a registry

Notice the –o parameter above to specify the output file.

In my machine XPerf is in the C:\ETL folder.

Considering that, here is one possible way to use the tool:

C:\ETL>set _NT_SYMBOL_PATH= srv*C:\symbols*http://msdl.microsoft.com/downloads/symbols

C:\ETL>XPerf -on DiagEasy

# Reproduce symptom here…

C:\ETL>XPerf -d trace.etl

# At this point you may want to create a CSV file. See instructions above.

C:\ETL>XPerf trace.etl

Ok, you’re probably wondering where the screenshots are. I decided to not use screenshots because some of the links below have all screenshots you need.

REFERENCES

http://msdn.microsoft.com/en-us/library/cc305187.aspx

http://blogs.msdn.com/pigscanfly/archive/tags/xperf/default.aspx

http://msdn.microsoft.com/en-us/performance/default.aspx (download)

http://blogs.msdn.com/ntdebugging/archive/2008/04/03/windows-performance-toolkit-xperf.aspx

XPerf rocks!

Special Command—Editing memory with a, eb, ed, ew, eza, ezu

rafarah — Wed, 06 Jan 2010 00:55:00 GMT

When talking about editing memory, we usually think about patching code. Patching code means changing the binary code in memory for, let’s say, when you want to prove a hypothesis while debugging and you don’t have access to the source code.

This is a very exciting subject, and WinDbg has the right tools to do the job. To actually change the memory, we are going to use the following commands:

a [address] – This command assembles instruction mnemonics and puts the resulting instruction codes into memory.

The following commands change the content from a specific memory location:

e{b|d|D|f|p|q|w} <Address> [Values]
e{a|u|za|zu} <Address> <"String">
e <Address> [Values]

According to WinDbg help file:

e	Enters data in the same format as the most recent e* command. (If the most recent e* command was ea, eza, eu, or ezu, the final parameter will be String and may not be omitted.)
ea	ASCII string (not NULL-terminated).
eb	Byte values.
ed	Double-word values (4 bytes).
eD	Double-precision floating-point numbers (8 bytes).
ef	Single-precision floating-point numbers (4 bytes).
ep	Pointer-sized values. This command is equivalent to ed or eq, depending on whether the target computer's processor architecture is 32-bit or 64-bit, respectively.
eq	Quad-word values (8 bytes).
eu	Unicode string (not NULL-terminated).
ew	Word values (2 bytes).
eza	NULL-terminated ASCII string.
ezu	NULL-terminated Unicode string.

To learn how to edit memory, let’s crack the application below. We will use the Debug version compiled as 32 bits. However, this is just to make things easier because you can use the 64 bits version or Release version. The techniques are the same!

CrackMe application:

// CrackMe.cpp : Defines the entry point for the console application.

#include "stdafx.h"

#include <iostream>

using namespace std; // Necessary for cout/cin.

// Declares prototypes.

void AskPassword(void);

bool IsRightPassword(int);

void GiveAccess(void);

void DenyAccess(void);

const int PASSWORD = 357;

void AskPassword()

{

// User can try to input the password for three times.

for(int i = 0; i < 3; i++)

{

cout << "Tentative " << i + 1 << "/3" << endl;

cout << "Type the numeric secret password:" << endl;

int nPassword = 0;

cin >> nPassword;

if(IsRightPassword(nPassword))

{

GiveAccess();

break; // User got the right password. Don't need to ask more.

}

else

{

DenyAccess();

}

void GiveAccess()

{

cout << "You have access to the system." << endl;

}

void DenyAccess()

{

cout << "Wrong password!" << endl;

}

inline bool IsRightPassword(int nPasswordToValidate)

{

return PASSWORD == nPasswordToValidate;

}

int _tmain(int argc, _TCHAR* argv[])

{

AskPassword();

return 0;

}

From now on let’s pretend we have never seen the source code before and we don’t have the right password. Let’s use symbols to make things easier, but keep in mind that in real world scenarios you may not have application symbols!

Note: The answers from the tasks below are not meant to be the only right ones. They are just some of many possible answers that can be used to show the commands for memory editing.

Task #1 – Let’s find out the right password.

Run the application from Windbg and break into the debugger when it asks for the password.

Let’s disassemble the main() function:

0:001> uf CrackMe!wmain

*** WARNING: Unable to verify checksum for CrackMe.exe

CrackMe!wmain [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 59]:

59 013f1c30 55 push ebp

59 013f1c31 8bec mov ebp,esp

59 013f1c33 81ecc0000000 sub esp,0C0h

59 013f1c39 53 push ebx

59 013f1c3a 56 push esi

59 013f1c3b 57 push edi

59 013f1c3c 8dbd40ffffff lea edi,[ebp-0C0h]

59 013f1c42 b930000000 mov ecx,30h

59 013f1c47 b8cccccccc mov eax,0CCCCCCCCh

59 013f1c4c f3ab rep stos dword ptr es:[edi]

60 013f1c4e e8cbf3ffff call CrackMe!ILT+25(?AskPasswordYAXXZ) (013f101e)

61 013f1c53 33c0 xor eax,eax

62 013f1c55 5f pop edi

62 013f1c56 5e pop esi

62 013f1c57 5b pop ebx

62 013f1c58 81c4c0000000 add esp,0C0h

62 013f1c5e 3bec cmp ebp,esp

62 013f1c60 e84ef5ffff call CrackMe!ILT+430(__RTC_CheckEsp) (013f11b3)

62 013f1c65 8be5 mov esp,ebp

62 013f1c67 5d pop ebp

62 013f1c68 c3 ret

AskPasswordYAXXZ looks interesting… let’s disassemble it:

0:001> uf 013f101e

CrackMe!AskPassword [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 19]:

19 013f4190 55 push ebp

19 013f4191 8bec mov ebp,esp

19 013f4193 81ecd8000000 sub esp,0D8h

19 013f4199 53 push ebx

19 013f419a 56 push esi

19 013f419b 57 push edi

19 013f419c 8dbd28ffffff lea edi,[ebp-0D8h]

19 013f41a2 b936000000 mov ecx,36h

19 013f41a7 b8cccccccc mov eax,0CCCCCCCCh

19 013f41ac f3ab rep stos dword ptr es:[edi]

21 013f41ae c745f800000000 mov dword ptr [ebp-8],0

21 013f41b5 eb09 jmp CrackMe!AskPassword+0x30 (013f41c0)

CrackMe!AskPassword+0x27 [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 21]:

21 013f41b7 8b45f8 mov eax,dword ptr [ebp-8]

21 013f41ba 83c001 add eax,1

21 013f41bd 8945f8 mov dword ptr [ebp-8],eax

CrackMe!AskPassword+0x30 [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 21]:

21 013f41c0 837df803 cmp dword ptr [ebp-8],3

21 013f41c4 0f8dc2000000 jge CrackMe!AskPassword+0xfc (013f428c)

CrackMe!AskPassword+0x3a [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 23]:

23 013f41ca 8bf4 mov esi,esp

23 013f41cc a110a33f01 mov eax,dword ptr [CrackMe!_imp_?endlstdYAAAV?$basic_ostreamDU?$char_traitsDstd (013fa310)]

23 013f41d1 50 push eax

23 013f41d2 683c783f01 push offset CrackMe!`string' (013f783c)

23 013f41d7 8b4df8 mov ecx,dword ptr [ebp-8]

23 013f41da 83c101 add ecx,1

23 013f41dd 8bfc mov edi,esp

23 013f41df 51 push ecx

23 013f41e0 6814783f01 push offset CrackMe!`string' (013f7814)

23 013f41e5 8b150ca33f01 mov edx,dword ptr [CrackMe!_imp_?coutstd (013fa30c)]

23 013f41eb 52 push edx

23 013f41ec e872cfffff call CrackMe!ILT+350(??$?6U?$char_traitsDstdstdYAAAV?$basic_ostreamDU?$char_traitsDstd (013f1163)

23 013f41f1 83c408 add esp,8

23 013f41f4 8bc8 mov ecx,eax

23 013f41f6 ff1554a33f01 call dword ptr [CrackMe!_imp_??6?$basic_ostreamDU?$char_traitsDstdstdQAEAAV01HZ (013fa354)]

23 013f41fc 3bfc cmp edi,esp

23 013f41fe e8b0cfffff call CrackMe!ILT+430(__RTC_CheckEsp) (013f11b3)

23 013f4203 50 push eax

23 013f4204 e85acfffff call CrackMe!ILT+350(??$?6U?$char_traitsDstdstdYAAAV?$basic_ostreamDU?$char_traitsDstd (013f1163)

23 013f4209 83c408 add esp,8

23 013f420c 8bc8 mov ecx,eax

23 013f420e ff1508a33f01 call dword ptr [CrackMe!_imp_??6?$basic_ostreamDU?$char_traitsDstdstdQAEAAV01P6AAAV01AAV01ZZ (013fa308)]

23 013f4214 3bf4 cmp esi,esp

23 013f4216 e898cfffff call CrackMe!ILT+430(__RTC_CheckEsp) (013f11b3)

24 013f421b 8bf4 mov esi,esp

24 013f421d a110a33f01 mov eax,dword ptr [CrackMe!_imp_?endlstdYAAAV?$basic_ostreamDU?$char_traitsDstd (013fa310)]

24 013f4222 50 push eax

24 013f4223 68187b3f01 push offset CrackMe!`string' (013f7b18)

24 013f4228 8b0d0ca33f01 mov ecx,dword ptr [CrackMe!_imp_?coutstd (013fa30c)]

24 013f422e 51 push ecx

24 013f422f e82fcfffff call CrackMe!ILT+350(??$?6U?$char_traitsDstdstdYAAAV?$basic_ostreamDU?$char_traitsDstd (013f1163)

24 013f4234 83c408 add esp,8

24 013f4237 8bc8 mov ecx,eax

24 013f4239 ff1508a33f01 call dword ptr [CrackMe!_imp_??6?$basic_ostreamDU?$char_traitsDstdstdQAEAAV01P6AAAV01AAV01ZZ (013fa308)]

24 013f423f 3bf4 cmp esi,esp

24 013f4241 e86dcfffff call CrackMe!ILT+430(__RTC_CheckEsp) (013f11b3)

26 013f4246 c745ec00000000 mov dword ptr [ebp-14h],0

28 013f424d 8bf4 mov esi,esp

28 013f424f 8d45ec lea eax,[ebp-14h]

28 013f4252 50 push eax

28 013f4253 8b0d04a33f01 mov ecx,dword ptr [CrackMe!_imp_?cinstd (013fa304)]

28 013f4259 ff1500a33f01 call dword ptr [CrackMe!_imp_??5?$basic_istreamDU?$char_traitsDstdstdQAEAAV01AAHZ (013fa300)]

28 013f425f 3bf4 cmp esi,esp

28 013f4261 e84dcfffff call CrackMe!ILT+430(__RTC_CheckEsp) (013f11b3)

30 013f4266 8b45ec mov eax,dword ptr [ebp-14h]

30 013f4269 50 push eax

30 013f426a e8eecfffff call CrackMe!ILT+600(?IsRightPasswordYA_NHZ) (013f125d)

30 013f426f 83c404 add esp,4

30 013f4272 0fb6c8 movzx ecx,al

30 013f4275 85c9 test ecx,ecx

30 013f4277 7409 je CrackMe!AskPassword+0xf2 (013f4282)

CrackMe!AskPassword+0xe9 [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 32]:

32 013f4279 e8e9cfffff call CrackMe!ILT+610(?GiveAccessYAXXZ) (013f1267)

33 013f427e eb0c jmp CrackMe!AskPassword+0xfc (013f428c)

CrackMe!AskPassword+0xf2 [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 37]:

37 013f4282 e8dbcfffff call CrackMe!ILT+605(?DenyAccessYAXXZ) (013f1262)

38 013f4287 e92bffffff jmp CrackMe!AskPassword+0x27 (013f41b7)

CrackMe!AskPassword+0xfc [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 41]:

41 013f428c 52 push edx

41 013f428d 8bcd mov ecx,ebp

41 013f428f 50 push eax

41 013f4290 8d15b4423f01 lea edx,[CrackMe!AskPassword+0x124 (013f42b4)]

41 013f4296 e819ceffff call CrackMe!ILT+175(_RTC_CheckStackVars (013f10b4)

41 013f429b 58 pop eax

41 013f429c 5a pop edx

41 013f429d 5f pop edi

41 013f429e 5e pop esi

41 013f429f 5b pop ebx

41 013f42a0 81c4d8000000 add esp,0D8h

41 013f42a6 3bec cmp ebp,esp

41 013f42a8 e806cfffff call CrackMe!ILT+430(__RTC_CheckEsp) (013f11b3)

41 013f42ad 8be5 mov esp,ebp

41 013f42af 5d pop ebp

41 013f42b0 c3 ret

CrackMe!ILT+25(?AskPasswordYAXXZ):

013f101e e96d310000 jmp CrackMe!AskPassword (013f4190)

After analyzing the code above we see two calls like:

coutstd

And just one call like:

cinstd

This clearly suggests cout and cin from C++ Standard Template Library.

If that is true we know that cin is used to receive information from the user.

Let’s put a breakpoint after the call function just after cin.

0:001> bp 013f425f

0:001> bl

0 e 013f425f 0001 (0001) 0:**** CrackMe!AskPassword+0xcf

And continue the execution.

We type any number, like 543. Now let’s track our number and see when it is compared against the right password.

At this point the breakpoint is hit.

If you look at the disassembled code again, you will see where our password was stored:

28 013f424f 8d45ec lea eax,[ebp-14h]

28 013f4252 50 push eax

28 013f4253 8b0d04a33f01 mov ecx,dword ptr [CrackMe!_imp_?cinstd (013fa304)]

28 013f4259 ff1500a33f01 call dword ptr [CrackMe!_imp_??5?$basic_istreamDU?$char_traitsDstdstdQAEAAV01AAHZ (013fa300)]

28 013f425f 3bf4 cmp esi,esp

28 013f4261 e84dcfffff call CrackMe!ILT+430(__RTC_CheckEsp) (013f11b3)

30 013f4266 8b45ec mov eax,dword ptr [ebp-14h]

30 013f4269 50 push eax

30 013f426a e8eecfffff call CrackMe!ILT+600(?IsRightPasswordYA_NHZ) (013f125d)

Look at [ebp-14h]. It is a local variable used in cin and used again just before IsRightPassword is called.

If that is true we should get the password we typed:

0:000> dd [ebp-0x14]

001cf9dc 0000021f cccccccc cccccccc 00000000

001cf9ec cccccccc 001cfac4 013f1c53 00000000

001cf9fc 00000000 7efde000 cccccccc cccccccc

001cfa0c cccccccc cccccccc cccccccc cccccccc

001cfa1c cccccccc cccccccc cccccccc cccccccc

001cfa2c cccccccc cccccccc cccccccc cccccccc

001cfa3c cccccccc cccccccc cccccccc cccccccc

001cfa4c cccccccc cccccccc cccccccc cccccccc

0:000> ? 21f

Evaluate expression: 543 = 0000021f

Right! This is the password we just typed. Let’s follow it through the code execution.

It’s being used as argument here:

66 8b45ec mov eax,dword ptr [ebp-14h] ss:002b:001cf9dc=0000021f

013f4269 50 push eax

013f426a e8eecfffff call CrackMe!ILT+600(?IsRightPasswordYA_NHZ) (013f125d)

013f426f 83c404 add esp,4

Note: Use p to step over and t to step into.

After stepping into IsRightPassword() and disassemble it we have:

CrackMe!IsRightPassword:

013f16b0 55 push ebp

013f16b1 8bec mov ebp,esp

013f16b3 81ecc0000000 sub esp,0C0h

013f16b9 53 push ebx

013f16ba 56 push esi

013f16bb 57 push edi

013f16bc 8dbd40ffffff lea edi,[ebp-0C0h]

013f16c2 b930000000 mov ecx,30h

013f16c7 b8cccccccc mov eax,0CCCCCCCCh

013f16cc f3ab rep stos dword ptr es:[edi]

013f16ce 33c0 xor eax,eax

013f16d0 817d0865010000 cmp dword ptr [ebp+8],165h

013f16d7 0f94c0 sete al

013f16da 5f pop edi

013f16db 5e pop esi

013f16dc 5b pop ebx

013f16dd 8be5 mov esp,ebp

013f16df 5d pop ebp

013f16e0 c3 ret

The line at address number 013f16d0 is key. Why? It compares the first function parameter against a specific number.

The parameter happens to be our password, sent by argument as you can see from the disassembled listing. Now let’s see what 165h is in decimal.

Let’s put a breakpoint on address 013f16d0 and continue the execution to that point.

0:000> g

Breakpoint 1 hit

eax=00000000 ebx=7efde000 ecx=00000000 edx=6952daa4 esi=001cf90c edi=001cf900

eip=013f16d0 esp=001cf834 ebp=001cf900 iopl=0 nv up ei pl zr na pe nc

cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000246

CrackMe!IsRightPassword+0x20:

013f16d0 817d0865010000 cmp dword ptr [ebp+8],165h ss:002b:001cf908=0000021f

0:000> dd [ebp+0x8] L1

001cf908 0000021f

Our password, remember?

And:

0:000> ? 165

Evaluate expression: 357 = 00000165

So our password is being compared against 357. If we run the application again and use 357 we can prove that this is the right password.

Mission accomplished!

Task #2 – Let’s make the application use 0 as password.

OK, now we know the right password and we know where the user’s password is being tested against the application’s password. So we just need to change the right password to 0.

Pay attention to the fact that the same password could be used from different parts of the program, so ideally all parts should be changed.

Let’s restart the application and stop the execution when it asks for the password.

After that let’s disassemble CrackMe!IsRightPassword and get the address of the call just before the cmp call. Depending on the Windows version, the address should be different from last time:

0:001> uf CrackMe!IsRightPassword

*** WARNING: Unable to verify checksum for CrackMe.exe

CrackMe!IsRightPassword [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 54]:

54 00d416b0 55 push ebp

54 00d416b1 8bec mov ebp,esp

54 00d416b3 81ecc0000000 sub esp,0C0h

54 00d416b9 53 push ebx

54 00d416ba 56 push esi

54 00d416bb 57 push edi

54 00d416bc 8dbd40ffffff lea edi,[ebp-0C0h]

54 00d416c2 b930000000 mov ecx,30h

54 00d416c7 b8cccccccc mov eax,0CCCCCCCCh

54 00d416cc f3ab rep stos dword ptr es:[edi]

55 00d416ce 33c0 xor eax,eax

55 00d416d0 817d0865010000 cmp dword ptr [ebp+8],165h

55 00d416d7 0f94c0 sete al

56 00d416da 5f pop edi

56 00d416db 5e pop esi

56 00d416dc 5b pop ebx

56 00d416dd 8be5 mov esp,ebp

56 00d416df 5d pop ebp

56 00d416e0 c3 ret

After hitting the breakpoint at 00d416ce we start the “surgery.”

0:000> a 00d416ce

00d416ce cmp dword ptr [ebp+8], 0

cmp dword ptr [ebp+8], 0

00d416d2

This is the new listing:

0:000> uf CrackMe!IsRightPassword

CrackMe!IsRightPassword [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 54]:

54 00d416b0 55 push ebp

54 00d416b1 8bec mov ebp,esp

54 00d416b3 81ecc0000000 sub esp,0C0h

54 00d416b9 53 push ebx

54 00d416ba 56 push esi

54 00d416bb 57 push edi

54 00d416bc 8dbd40ffffff lea edi,[ebp-0C0h]

54 00d416c2 b930000000 mov ecx,30h

54 00d416c7 b8cccccccc mov eax,0CCCCCCCCh

54 00d416cc f3ab rep stos dword ptr es:[edi]

55 00d416ce 837d0800 cmp dword ptr [ebp+8],0

55 00d416d2 086501 or byte ptr [ebp+1],ah

55 00d416d5 0000 add byte ptr [eax],al

55 00d416d7 0f94c0 sete al

56 00d416da 5f pop edi

56 00d416db 5e pop esi

56 00d416dc 5b pop ebx

56 00d416dd 8be5 mov esp,ebp

56 00d416df 5d pop ebp

56 00d416e0 c3 ret

Note that we messed up the function because the original code:

817d0865010000

Has more bytes than the new code has:

837d0800

What do we need to do? Insert NOP instructions that do not do anything just to fill the remaining bytes. We do that using the NOP instruction. You can calculate the number of instructions or see the disassembly after entering each NOP until you see the right continuation according to the original listing.

In our case we use:

a 00d416d2

And enter NOP instructions then hit Enter after that.

You can also use the approach below that uses the code corresponding to the mnemonic:

eb 00d416d2 90 90 90

If you are using the Disassembly Window or if you use uf again, you will be able to see the modified version.

Ours is:

CrackMe!IsRightPassword:

003716b0 55 push ebp

003716b1 8bec mov ebp,esp

003716b3 81ecc0000000 sub esp,0C0h

003716b9 53 push ebx

003716ba 56 push esi

003716bb 57 push edi

003716bc 8dbd40ffffff lea edi,[ebp-0C0h]

003716c2 b930000000 mov ecx,30h

003716c7 b8cccccccc mov eax,0CCCCCCCCh

003716cc f3ab rep stos dword ptr es:[edi]

003716ce 33c0 xor eax,eax

003716d0 837d0800 cmp dword ptr [ebp+8],0

003716d4 90 nop

003716d5 90 nop

003716d6 90 nop

003716d7 0f94c0 sete al

003716da 5f pop edi

003716db 5e pop esi

003716dc 5b pop ebx

003716dd 8be5 mov esp,ebp

003716df 5d pop ebp

003716e0 c3 ret

The original password is 0. Let’s change the user’s password to 0, too:

0:000> dd [ebp+8] L1

002bfaf0 00000003

In our case we used 3.

Let’s change our password to 0, too:

0:000> ed 002bfaf0 0

You can see it was changed:

0:000> dd [ebp+8] L1

002bfaf0 00000000

At this point you should get the message:

“You have access to the system.”

This happens because the application is comparing the user’s password, which is 0 (we changed it), against the original password, which is also 0 (we changed it too).

Using debuggers like OllyDbg or hexadecimal editors, you can go one step further and save another version of the binary, the patched version. That means the application will consider the number 0 as the correct password whenever you run it! But be warned, it is considered a crime to do that.

Task #3 – Let’s remove the test that validates the password.

After running the application again, we break into the debugger when it asks for the password.

0:000> uf CrackMe!wmain

CrackMe!wmain [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 59]:

59 00ec1c30 55 push ebp

59 00ec1c31 8bec mov ebp,esp

59 00ec1c33 81ecc0000000 sub esp,0C0h

59 00ec1c39 53 push ebx

59 00ec1c3a 56 push esi

59 00ec1c3b 57 push edi

59 00ec1c3c 8dbd40ffffff lea edi,[ebp-0C0h]

59 00ec1c42 b930000000 mov ecx,30h

59 00ec1c47 b8cccccccc mov eax,0CCCCCCCCh

59 00ec1c4c f3ab rep stos dword ptr es:[edi]

60 00ec1c4e e8cbf3ffff call CrackMe!ILT+25(?AskPasswordYAXXZ) (00ec101e)

61 00ec1c53 33c0 xor eax,eax

62 00ec1c55 5f pop edi

62 00ec1c56 5e pop esi

62 00ec1c57 5b pop ebx

62 00ec1c58 81c4c0000000 add esp,0C0h

62 00ec1c5e 3bec cmp ebp,esp

62 00ec1c60 e84ef5ffff call CrackMe!ILT+430(__RTC_CheckEsp) (00ec11b3)

62 00ec1c65 8be5 mov esp,ebp

62 00ec1c67 5d pop ebp

62 00ec1c68 c3 ret

0:000> uf 00ec101e

CrackMe!AskPassword [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 19]:

19 00ec4190 55 push ebp

19 00ec4191 8bec mov ebp,esp

19 00ec4193 81ecd8000000 sub esp,0D8h

19 00ec4199 53 push ebx

19 00ec419a 56 push esi

19 00ec419b 57 push edi

19 00ec419c 8dbd28ffffff lea edi,[ebp-0D8h]

19 00ec41a2 b936000000 mov ecx,36h

19 00ec41a7 b8cccccccc mov eax,0CCCCCCCCh

19 00ec41ac f3ab rep stos dword ptr es:[edi]

21 00ec41ae c745f800000000 mov dword ptr [ebp-8],0

21 00ec41b5 eb09 jmp CrackMe!AskPassword+0x30 (00ec41c0)

CrackMe!AskPassword+0x27 [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 21]:

21 00ec41b7 8b45f8 mov eax,dword ptr [ebp-8]

21 00ec41ba 83c001 add eax,1

21 00ec41bd 8945f8 mov dword ptr [ebp-8],eax

CrackMe!AskPassword+0x30 [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 21]:

21 00ec41c0 837df803 cmp dword ptr [ebp-8],3

21 00ec41c4 0f8dc2000000 jge CrackMe!AskPassword+0xfc (00ec428c)

CrackMe!AskPassword+0x3a [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 23]:

23 00ec41ca 8bf4 mov esi,esp

23 00ec41cc a110a3ec00 mov eax,dword ptr [CrackMe!_imp_?endlstdYAAAV?$basic_ostreamDU?$char_traitsDstd (00eca310)]

23 00ec41d1 50 push eax

23 00ec41d2 683c78ec00 push offset CrackMe!`string' (00ec783c)

23 00ec41d7 8b4df8 mov ecx,dword ptr [ebp-8]

23 00ec41da 83c101 add ecx,1

23 00ec41dd 8bfc mov edi,esp

23 00ec41df 51 push ecx

23 00ec41e0 681478ec00 push offset CrackMe!`string' (00ec7814)

23 00ec41e5 8b150ca3ec00 mov edx,dword ptr [CrackMe!_imp_?coutstd (00eca30c)]

23 00ec41eb 52 push edx

23 00ec41ec e872cfffff call CrackMe!ILT+350(??$?6U?$char_traitsDstdstdYAAAV?$basic_ostreamDU?$char_traitsDstd (00ec1163)

23 00ec41f1 83c408 add esp,8

23 00ec41f4 8bc8 mov ecx,eax

23 00ec41f6 ff1554a3ec00 call dword ptr [CrackMe!_imp_??6?$basic_ostreamDU?$char_traitsDstdstdQAEAAV01HZ (00eca354)]

23 00ec41fc 3bfc cmp edi,esp

23 00ec41fe e8b0cfffff call CrackMe!ILT+430(__RTC_CheckEsp) (00ec11b3)

23 00ec4203 50 push eax

23 00ec4204 e85acfffff call CrackMe!ILT+350(??$?6U?$char_traitsDstdstdYAAAV?$basic_ostreamDU?$char_traitsDstd (00ec1163)

23 00ec4209 83c408 add esp,8

23 00ec420c 8bc8 mov ecx,eax

23 00ec420e ff1508a3ec00 call dword ptr [CrackMe!_imp_??6?$basic_ostreamDU?$char_traitsDstdstdQAEAAV01P6AAAV01AAV01ZZ (00eca308)]

23 00ec4214 3bf4 cmp esi,esp

23 00ec4216 e898cfffff call CrackMe!ILT+430(__RTC_CheckEsp) (00ec11b3)

24 00ec421b 8bf4 mov esi,esp

24 00ec421d a110a3ec00 mov eax,dword ptr [CrackMe!_imp_?endlstdYAAAV?$basic_ostreamDU?$char_traitsDstd (00eca310)]

24 00ec4222 50 push eax

24 00ec4223 68187bec00 push offset CrackMe!`string' (00ec7b18)

24 00ec4228 8b0d0ca3ec00 mov ecx,dword ptr [CrackMe!_imp_?coutstd (00eca30c)]

24 00ec422e 51 push ecx

24 00ec422f e82fcfffff call CrackMe!ILT+350(??$?6U?$char_traitsDstdstdYAAAV?$basic_ostreamDU?$char_traitsDstd (00ec1163)

24 00ec4234 83c408 add esp,8

24 00ec4237 8bc8 mov ecx,eax

24 00ec4239 ff1508a3ec00 call dword ptr [CrackMe!_imp_??6?$basic_ostreamDU?$char_traitsDstdstdQAEAAV01P6AAAV01AAV01ZZ (00eca308)]

24 00ec423f 3bf4 cmp esi,esp

24 00ec4241 e86dcfffff call CrackMe!ILT+430(__RTC_CheckEsp) (00ec11b3)

26 00ec4246 c745ec00000000 mov dword ptr [ebp-14h],0

28 00ec424d 8bf4 mov esi,esp

28 00ec424f 8d45ec lea eax,[ebp-14h]

28 00ec4252 50 push eax

28 00ec4253 8b0d04a3ec00 mov ecx,dword ptr [CrackMe!_imp_?cinstd (00eca304)]

28 00ec4259 ff1500a3ec00 call dword ptr [CrackMe!_imp_??5?$basic_istreamDU?$char_traitsDstdstdQAEAAV01AAHZ (00eca300)]

28 00ec425f 3bf4 cmp esi,esp

28 00ec4261 e84dcfffff call CrackMe!ILT+430(__RTC_CheckEsp) (00ec11b3)

30 00ec4266 8b45ec mov eax,dword ptr [ebp-14h]

30 00ec4269 50 push eax

30 00ec426a e8eecfffff call CrackMe!ILT+600(?IsRightPasswordYA_NHZ) (00ec125d)

30 00ec426f 83c404 add esp,4

30 00ec4272 0fb6c8 movzx ecx,al

30 00ec4275 85c9 test ecx,ecx

30 00ec4277 7409 je CrackMe!AskPassword+0xf2 (00ec4282)

CrackMe!AskPassword+0xe9 [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 32]:

32 00ec4279 e8e9cfffff call CrackMe!ILT+610(?GiveAccessYAXXZ) (00ec1267)

33 00ec427e eb0c jmp CrackMe!AskPassword+0xfc (00ec428c)

CrackMe!AskPassword+0xf2 [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 37]:

37 00ec4282 e8dbcfffff call CrackMe!ILT+605(?DenyAccessYAXXZ) (00ec1262)

38 00ec4287 e92bffffff jmp CrackMe!AskPassword+0x27 (00ec41b7)

CrackMe!AskPassword+0xfc [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 41]:

41 00ec428c 52 push edx

41 00ec428d 8bcd mov ecx,ebp

41 00ec428f 50 push eax

41 00ec4290 8d15b442ec00 lea edx,[CrackMe!AskPassword+0x124 (00ec42b4)]

41 00ec4296 e819ceffff call CrackMe!ILT+175(_RTC_CheckStackVars (00ec10b4)

41 00ec429b 58 pop eax

41 00ec429c 5a pop edx

41 00ec429d 5f pop edi

41 00ec429e 5e pop esi

41 00ec429f 5b pop ebx

41 00ec42a0 81c4d8000000 add esp,0D8h

41 00ec42a6 3bec cmp ebp,esp

41 00ec42a8 e806cfffff call CrackMe!ILT+430(__RTC_CheckEsp) (00ec11b3)

41 00ec42ad 8be5 mov esp,ebp

41 00ec42af 5d pop ebp

41 00ec42b0 c3 ret

CrackMe!ILT+25(?AskPasswordYAXXZ):

00ec101e e96d310000 jmp CrackMe!AskPassword (00ec4190)

From above: [ebp-14] holds the user password and is sent as argument to IsRightPassword().

0:000> uf 00ec125d

CrackMe!IsRightPassword [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 54]:

54 00ec16b0 55 push ebp

54 00ec16b1 8bec mov ebp,esp

54 00ec16b3 81ecc0000000 sub esp,0C0h

54 00ec16b9 53 push ebx

54 00ec16ba 56 push esi

54 00ec16bb 57 push edi

54 00ec16bc 8dbd40ffffff lea edi,[ebp-0C0h]

54 00ec16c2 b930000000 mov ecx,30h

54 00ec16c7 b8cccccccc mov eax,0CCCCCCCCh

54 00ec16cc f3ab rep stos dword ptr es:[edi]

55 00ec16ce 33c0 xor eax,eax

55 00ec16d0 817d0865010000 cmp dword ptr [ebp+8],165h

55 00ec16d7 0f94c0 sete al

56 00ec16da 5f pop edi

56 00ec16db 5e pop esi

56 00ec16dc 5b pop ebx

56 00ec16dd 8be5 mov esp,ebp

56 00ec16df 5d pop ebp

56 00ec16e0 c3 ret

CrackMe!ILT+600(?IsRightPasswordYA_NHZ):

00ec125d e94e040000 jmp CrackMe!IsRightPassword (00ec16b0)

Next step is to replace this:

55 00ec16ce 33c0 xor eax,eax

55 00ec16d0 817d0865010000 cmp dword ptr [ebp+8],165h

For this:

55 00ec16ce mov eax, 1

nop

…

After the changes:

CrackMe!IsRightPassword:

00ec16b0 55 push ebp

00ec16b1 8bec mov ebp,esp

00ec16b3 81ecc0000000 sub esp,0C0h

00ec16b9 53 push ebx

00ec16ba 56 push esi

00ec16bb 57 push edi

00ec16bc 8dbd40ffffff lea edi,[ebp-0C0h]

00ec16c2 b930000000 mov ecx,30h

00ec16c7 b8cccccccc mov eax,0CCCCCCCCh

00ec16cc f3ab rep stos dword ptr es:[edi]

00ec16ce b801000000 mov eax,1

00ec16d3 650100 add dword ptr gs:[eax],eax

00ec16d6 000f add byte ptr [edi],cl

00ec16d8 94 xchg eax,esp

00ec16d9 c05f5e5b rcr byte ptr [edi+5Eh],5Bh

00ec16dd 8be5 mov esp,ebp

00ec16df 5d pop ebp

00ec16e0 c3 ret

However, notice that we changed all other instructions because our instruction used more bytes than the original instruction.

Let’s fix that using NOPs.

0:000> eb 00ec16d3 90 90 90 90 90 90 90

Another approach is to use:

a 00ec16d3

And type 7 NOP pressing Enter after each one.

The first approach has one key benefit: it can be used from scripts or breakpoints. It’s also faster.

Final code:

CrackMe!IsRightPassword:

00ec16b0 55 push ebp

00ec16b1 8bec mov ebp,esp

00ec16b3 81ecc0000000 sub esp,0C0h

00ec16b9 53 push ebx

00ec16ba 56 push esi

00ec16bb 57 push edi

00ec16bc 8dbd40ffffff lea edi,[ebp-0C0h]

00ec16c2 b930000000 mov ecx,30h

00ec16c7 b8cccccccc mov eax,0CCCCCCCCh

00ec16cc f3ab rep stos dword ptr es:[edi]

00ec16ce b801000000 mov eax,1

00ec16d3 90 nop

00ec16d4 90 nop

00ec16d5 90 nop

00ec16d6 90 nop

00ec16d7 90 nop

00ec16d8 90 nop

00ec16d9 90 nop

00ec16da 5f pop edi

00ec16db 5e pop esi

00ec16dc 5b pop ebx

00ec16dd 8be5 mov esp,ebp

00ec16df 5d pop ebp

00ec16e0 c3 ret

So now the IsRightPassword() function is always return 1 (TRUE).

Use g to continue the execution and from the console window enter any number you want as password.

Task #4 – Let’s make the application call the function that is called when typing the right password even when typing the wrong password.

After running the application and breaking it into the debugger when asking for the password, we start what we call the “pre-surgery” phase.

0:000> kL 1000

ChildEBP RetAddr

0046f1ec 75f66f99 kernel32!ReadConsoleInternal+0x15

0046f274 75eeefc6 kernel32!ReadConsoleA+0x40

0046f2bc 67416c3c kernel32!ReadFileImplementation+0x75

0046f350 67416589 MSVCR90D!_read_nolock+0x62c

0046f3a0 673a4453 MSVCR90D!_read+0x219

0046f3c8 673a2748 MSVCR90D!_filbuf+0x113

0046f420 6ce5c8e0 MSVCR90D!fgetc+0x208

0046f430 6ce5c567 MSVCP90D!std::_Fgetc<char>+0x10

0046f518 6ce5c460 MSVCP90D!std::basic_filebuf<char,std::char_traits<char> >::uflow+0xb7

0046f530 6ce58c9a MSVCP90D!std::basic_filebuf<char,std::char_traits<char> >::underflow+0x50

0046f540 6ce59104 MSVCP90D!std::basic_streambuf<char,std::char_traits<char> >::sgetc+0x3a

0046f59c 6ce58c3a MSVCP90D!std::basic_istream<char,std::char_traits<char> >::_Ipfx+0x104

0046f5bc 6ce7eee6 MSVCP90D!std::basic_istream<char,std::char_traits<char> >::sentry::sentry+0x4a

0046f634 0002425f MSVCP90D!std::basic_istream<char,std::char_traits<char> >::operator>>+0x46

0046f724 00021c53 CrackMe!AskPassword+0xcf

0046f7f8 00022598 CrackMe!wmain+0x23

0046f848 000223df CrackMe!__tmainCRTStartup+0x1a8

0046f850 75ec3f39 CrackMe!wmainCRTStartup+0xf

0046f85c 77710409 kernel32!BaseThreadInitThunk+0xe

0046f89c 777103dc ntdll!__RtlUserThreadStart+0x70

0046f8b4 00000000 ntdll!_RtlUserThreadStart+0x1b

AskPassword() is called from main() and looks suspicious. Let’s investigate:

0:000> uf CrackMe!AskPassword

CrackMe!AskPassword [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 19]:

19 00024190 55 push ebp

19 00024191 8bec mov ebp,esp

19 00024193 81ecd8000000 sub esp,0D8h

19 00024199 53 push ebx

19 0002419a 56 push esi

19 0002419b 57 push edi

19 0002419c 8dbd28ffffff lea edi,[ebp-0D8h]

19 000241a2 b936000000 mov ecx,36h

19 000241a7 b8cccccccc mov eax,0CCCCCCCCh

19 000241ac f3ab rep stos dword ptr es:[edi]

21 000241ae c745f800000000 mov dword ptr [ebp-8],0

21 000241b5 eb09 jmp CrackMe!AskPassword+0x30 (000241c0)

CrackMe!AskPassword+0x27 [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 21]:

21 000241b7 8b45f8 mov eax,dword ptr [ebp-8]

21 000241ba 83c001 add eax,1

21 000241bd 8945f8 mov dword ptr [ebp-8],eax

CrackMe!AskPassword+0x30 [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 21]:

21 000241c0 837df803 cmp dword ptr [ebp-8],3

21 000241c4 0f8dc2000000 jge CrackMe!AskPassword+0xfc (0002428c)

CrackMe!AskPassword+0x3a [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 23]:

23 000241ca 8bf4 mov esi,esp

23 000241cc a110a30200 mov eax,dword ptr [CrackMe!_imp_?endlstdYAAAV?$basic_ostreamDU?$char_traitsDstd (0002a310)]

23 000241d1 50 push eax

23 000241d2 683c780200 push offset CrackMe!`string' (0002783c)

23 000241d7 8b4df8 mov ecx,dword ptr [ebp-8]

23 000241da 83c101 add ecx,1

23 000241dd 8bfc mov edi,esp

23 000241df 51 push ecx

23 000241e0 6814780200 push offset CrackMe!`string' (00027814)

23 000241e5 8b150ca30200 mov edx,dword ptr [CrackMe!_imp_?coutstd (0002a30c)]

23 000241eb 52 push edx

23 000241ec e872cfffff call CrackMe!ILT+350(??$?6U?$char_traitsDstdstdYAAAV?$basic_ostreamDU?$char_traitsDstd (00021163)

23 000241f1 83c408 add esp,8

23 000241f4 8bc8 mov ecx,eax

23 000241f6 ff1554a30200 call dword ptr [CrackMe!_imp_??6?$basic_ostreamDU?$char_traitsDstdstdQAEAAV01HZ (0002a354)]

23 000241fc 3bfc cmp edi,esp

23 000241fe e8b0cfffff call CrackMe!ILT+430(__RTC_CheckEsp) (000211b3)

23 00024203 50 push eax

23 00024204 e85acfffff call CrackMe!ILT+350(??$?6U?$char_traitsDstdstdYAAAV?$basic_ostreamDU?$char_traitsDstd (00021163)

23 00024209 83c408 add esp,8

23 0002420c 8bc8 mov ecx,eax

23 0002420e ff1508a30200 call dword ptr [CrackMe!_imp_??6?$basic_ostreamDU?$char_traitsDstdstdQAEAAV01P6AAAV01AAV01ZZ (0002a308)]

23 00024214 3bf4 cmp esi,esp

23 00024216 e898cfffff call CrackMe!ILT+430(__RTC_CheckEsp) (000211b3)

24 0002421b 8bf4 mov esi,esp

24 0002421d a110a30200 mov eax,dword ptr [CrackMe!_imp_?endlstdYAAAV?$basic_ostreamDU?$char_traitsDstd (0002a310)]

24 00024222 50 push eax

24 00024223 68187b0200 push offset CrackMe!`string' (00027b18)

24 00024228 8b0d0ca30200 mov ecx,dword ptr [CrackMe!_imp_?coutstd (0002a30c)]

24 0002422e 51 push ecx

24 0002422f e82fcfffff call CrackMe!ILT+350(??$?6U?$char_traitsDstdstdYAAAV?$basic_ostreamDU?$char_traitsDstd (00021163)

24 00024234 83c408 add esp,8

24 00024237 8bc8 mov ecx,eax

24 00024239 ff1508a30200 call dword ptr [CrackMe!_imp_??6?$basic_ostreamDU?$char_traitsDstdstdQAEAAV01P6AAAV01AAV01ZZ (0002a308)]

24 0002423f 3bf4 cmp esi,esp

24 00024241 e86dcfffff call CrackMe!ILT+430(__RTC_CheckEsp) (000211b3)

26 00024246 c745ec00000000 mov dword ptr [ebp-14h],0

28 0002424d 8bf4 mov esi,esp

28 0002424f 8d45ec lea eax,[ebp-14h]

28 00024252 50 push eax

28 00024253 8b0d04a30200 mov ecx,dword ptr [CrackMe!_imp_?cinstd (0002a304)]

28 00024259 ff1500a30200 call dword ptr [CrackMe!_imp_??5?$basic_istreamDU?$char_traitsDstdstdQAEAAV01AAHZ (0002a300)]

28 0002425f 3bf4 cmp esi,esp

28 00024261 e84dcfffff call CrackMe!ILT+430(__RTC_CheckEsp) (000211b3)

30 00024266 8b45ec mov eax,dword ptr [ebp-14h]

30 00024269 50 push eax

30 0002426a e8eecfffff call CrackMe!ILT+600(?IsRightPasswordYA_NHZ) (0002125d)

30 0002426f 83c404 add esp,4

30 00024272 0fb6c8 movzx ecx,al

30 00024275 85c9 test ecx,ecx

30 00024277 7409 je CrackMe!AskPassword+0xf2 (00024282)

CrackMe!AskPassword+0xe9 [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 32]:

32 00024279 e8e9cfffff call CrackMe!ILT+610(?GiveAccessYAXXZ) (00021267)

33 0002427e eb0c jmp CrackMe!AskPassword+0xfc (0002428c)

CrackMe!AskPassword+0xf2 [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 37]:

37 00024282 e8dbcfffff call CrackMe!ILT+605(?DenyAccessYAXXZ) (00021262)

38 00024287 e92bffffff jmp CrackMe!AskPassword+0x27 (000241b7)

CrackMe!AskPassword+0xfc [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 41]:

41 0002428c 52 push edx

41 0002428d 8bcd mov ecx,ebp

41 0002428f 50 push eax

41 00024290 8d15b4420200 lea edx,[CrackMe!AskPassword+0x124 (000242b4)]

41 00024296 e819ceffff call CrackMe!ILT+175(_RTC_CheckStackVars (000210b4)

41 0002429b 58 pop eax

41 0002429c 5a pop edx

41 0002429d 5f pop edi

41 0002429e 5e pop esi

41 0002429f 5b pop ebx

41 000242a0 81c4d8000000 add esp,0D8h

41 000242a6 3bec cmp ebp,esp

41 000242a8 e806cfffff call CrackMe!ILT+430(__RTC_CheckEsp) (000211b3)

41 000242ad 8be5 mov esp,ebp

41 000242af 5d pop ebp

41 000242b0 c3 ret

At this point it is necessary to analyze what we call “dead listing” in order to get a better understanding of the function and make sure this may be the right function.

In bold above you can see what looks like the routine that checks the password. Remember, we are pretending we don’t know the source code, so to prove the point you should insert breakpoints, debug the application, and see if the DenyAccessYAXXZ above is called when you enter any invalid password.

Ok, so after running the application using breakpoints we prove the point.

However, we don’t know what happens when we enter the right password, but we suppose it is going to call GiveAccessYAXXZ. To prove the point let’s analyze this function.

0:000> uf 00021267

CrackMe!GiveAccess [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 44]:

44 000215b0 55 push ebp

44 000215b1 8bec mov ebp,esp

44 000215b3 81ecc0000000 sub esp,0C0h

44 000215b9 53 push ebx

44 000215ba 56 push esi

44 000215bb 57 push edi

44 000215bc 8dbd40ffffff lea edi,[ebp-0C0h]

44 000215c2 b930000000 mov ecx,30h

44 000215c7 b8cccccccc mov eax,0CCCCCCCCh

44 000215cc f3ab rep stos dword ptr es:[edi]

45 000215ce 8bf4 mov esi,esp

45 000215d0 a110a30200 mov eax,dword ptr [CrackMe!_imp_?endlstdYAAAV?$basic_ostreamDU?$char_traitsDstd (0002a310)]

45 000215d5 50 push eax

45 000215d6 6840780200 push offset CrackMe!`string' (00027840)

45 000215db 8b0d0ca30200 mov ecx,dword ptr [CrackMe!_imp_?coutstd (0002a30c)]

45 000215e1 51 push ecx

45 000215e2 e87cfbffff call CrackMe!ILT+350(??$?6U?$char_traitsDstdstdYAAAV?$basic_ostreamDU?$char_traitsDstd (00021163)

45 000215e7 83c408 add esp,8

45 000215ea 8bc8 mov ecx,eax

45 000215ec ff1508a30200 call dword ptr [CrackMe!_imp_??6?$basic_ostreamDU?$char_traitsDstdstdQAEAAV01P6AAAV01AAV01ZZ (0002a308)]

45 000215f2 3bf4 cmp esi,esp

45 000215f4 e8bafbffff call CrackMe!ILT+430(__RTC_CheckEsp) (000211b3)

46 000215f9 5f pop edi

46 000215fa 5e pop esi

46 000215fb 5b pop ebx

46 000215fc 81c4c0000000 add esp,0C0h

46 00021602 3bec cmp ebp,esp

46 00021604 e8aafbffff call CrackMe!ILT+430(__RTC_CheckEsp) (000211b3)

46 00021609 8be5 mov esp,ebp

46 0002160b 5d pop ebp

46 0002160c c3 ret

CrackMe!ILT+610(?GiveAccessYAXXZ):

00021267 e944030000 jmp CrackMe!GiveAccess (000215b0)

After reviewing the code we located one string. Let’s examine that:

0:000> db 00027840

00027840 59 6f 75 20 68 61 76 65-20 61 63 63 65 73 73 20 You have access

00027850 74 6f 20 74 68 65 20 73-79 73 74 65 6d 2e 00 00 to the system...

00027860 66 00 3a 00 5c 00 64 00-64 00 5c 00 76 00 63 00 f.:.\.d.d.\.v.c.

00027870 74 00 6f 00 6f 00 6c 00-73 00 5c 00 63 00 72 00 t.o.o.l.s.\.c.r.

00027880 74 00 5f 00 62 00 6c 00-64 00 5c 00 73 00 65 00 t._.b.l.d.\.s.e.

00027890 6c 00 66 00 5f 00 78 00-38 00 36 00 5c 00 63 00 l.f._.x.8.6.\.c.

000278a0 72 00 74 00 5c 00 73 00-72 00 63 00 5c 00 63 00 r.t.\.s.r.c.\.c.

000278b0 72 00 74 00 65 00 78 00-65 00 2e 00 63 00 00 00 r.t.e.x.e...c...

Excellent! This is the right function. The string is an ANSI string, not Unicode. A Unicode string should have two bytes for each character.

Here is another way to see that:

0:000> da 00027840

00027840 "You have access to the system."

Let’s see the two calls, for right and wrong passwords:

0:000> uf /c CrackMe!AskPassword

CrackMe!AskPassword (00024190) [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 19]

CrackMe!AskPassword+0x5c (000241ec) [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 23]:

call to CrackMe!ILT+350(??$?6U?$char_traitsDstdstdYAAAV?$basic_ostreamDU?$char_traitsDstd (00021163)

CrackMe!AskPassword+0x66 (000241f6) [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 23]:

call to MSVCP90D!std::basic_ostream<char,std::char_traits<char> >::operator<< (6ce7a520)

CrackMe!AskPassword+0x6e (000241fe) [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 23]:

call to CrackMe!ILT+430(__RTC_CheckEsp) (000211b3)

CrackMe!AskPassword+0x74 (00024204) [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 23]:

call to CrackMe!ILT+350(??$?6U?$char_traitsDstdstdYAAAV?$basic_ostreamDU?$char_traitsDstd (00021163)

CrackMe!AskPassword+0x7e (0002420e) [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 23]:

call to MSVCP90D!std::basic_ostream<char,std::char_traits<char> >::operator<< (6ce79f60)

CrackMe!AskPassword+0x86 (00024216) [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 23]:

call to CrackMe!ILT+430(__RTC_CheckEsp) (000211b3)

CrackMe!AskPassword+0x9f (0002422f) [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 24]:

call to CrackMe!ILT+350(??$?6U?$char_traitsDstdstdYAAAV?$basic_ostreamDU?$char_traitsDstd (00021163)

CrackMe!AskPassword+0xa9 (00024239) [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 24]:

call to MSVCP90D!std::basic_ostream<char,std::char_traits<char> >::operator<< (6ce79f60)

CrackMe!AskPassword+0xb1 (00024241) [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 24]:

call to CrackMe!ILT+430(__RTC_CheckEsp) (000211b3)

CrackMe!AskPassword+0xc9 (00024259) [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 28]:

call to MSVCP90D!std::basic_istream<char,std::char_traits<char> >::operator>> (6ce7eea0)

CrackMe!AskPassword+0xd1 (00024261) [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 28]:

call to CrackMe!ILT+430(__RTC_CheckEsp) (000211b3)

CrackMe!AskPassword+0xda (0002426a) [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 30]:

call to CrackMe!ILT+600(?IsRightPasswordYA_NHZ) (0002125d)

CrackMe!AskPassword+0xe9 (00024279) [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 32]:

call to CrackMe!ILT+610(?GiveAccessYAXXZ) (00021267)

CrackMe!AskPassword+0xf2 (00024282) [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 37]:

call to CrackMe!ILT+605(?DenyAccessYAXXZ) (00021262)

CrackMe!AskPassword+0x106 (00024296) [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 41]:

call to CrackMe!ILT+175(_RTC_CheckStackVars (000210b4)

CrackMe!AskPassword+0x118 (000242a8) [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 41]:

call to CrackMe!ILT+430(__RTC_CheckEsp) (000211b3)

Hopefully the number of bytes is the same:

CrackMe!AskPassword+0xe9 [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 32]:

32 00024279 e8e9cfffff call CrackMe!ILT+610(?GiveAccessYAXXZ) (00021267)

33 0002427e eb0c jmp CrackMe!AskPassword+0xfc (0002428c)

CrackMe!AskPassword+0xf2 [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 37]:

37 00024282 e8dbcfffff call CrackMe!ILT+605(?DenyAccessYAXXZ) (00021262)

We are ready to start the “surgery”:

0:000> u 00021267

CrackMe!ILT+610(?GiveAccessYAXXZ):

00021267 e944030000 jmp CrackMe!GiveAccess (000215b0)

See the indirection? We need 000215b0.

0:000> u 000215b0

CrackMe!GiveAccess [c:\development\my tools\book\crackme\crackme\crackme.cpp @ 44]:

000215b0 55 push ebp

000215b1 8bec mov ebp,esp

000215b3 81ecc0000000 sub esp,0C0h

000215b9 53 push ebx

000215ba 56 push esi

000215bb 57 push edi

000215bc 8dbd40ffffff lea edi,[ebp-0C0h]

000215c2 b930000000 mov ecx,30h

0:000> a 00024282

00024282 call CrackMe!GiveAccess

call CrackMe!GiveAccess

00024287

New listing:

00024277 7409 je CrackMe!AskPassword+0xf2 (00024282)

00024279 e8e9cfffff call CrackMe!ILT+610(?GiveAccessYAXXZ) (00021267)

0002427e eb0c jmp CrackMe!AskPassword+0xfc (0002428c)

00024280 eb05 jmp CrackMe!AskPassword+0xf7 (00024287)

00024282 e829d3ffff call CrackMe!GiveAccess (000215b0)

00024287 e92bffffff jmp CrackMe!AskPassword+0x27 (000241b7)

0002428c 52 push edx

0002428d 8bcd mov ecx,ebp

0002428f 50 push eax

Now if you type “g” and continue the execution entering any invalid password the application is going to show the message that says you have access to the system.

However, the application is going to ask you the password again because you typed the wrong password. In other words, after changing the application according to the instructions above, it will call the function that is supposed to be called only if you type the right password! ;-)

Here is a fun demonstration for Windows XP on 32 bits: http://blogs.msdn.com/debuggingtoolbox/archive/2007/03/28/windbg-script-playing-with-minesweeper.aspx

Another related blog article:

http://blogs.msdn.com/debuggingtoolbox/archive/2007/04/27/windbg-script-tracing-messagebox-calls.aspx

For more WinDbg commands check the Special Commands section in this blog.

[PowerShell Script] PowerDbg v5.3—Using PowerShell to Control WinDbg

rafarah — Mon, 21 Dec 2009 22:05:00 GMT

This version has a fix in Parse-PowerDbgDSO.

Thanks to Igor Dvorkin that found the bug and suggested the fix.

DOWNLOAD POWERDBG

Download PowerDbg

POWERDBG FILES

WinDbg.PSM1 ß Starting with this version this is the only file.

INSTALLATION

WinDbg.PSM1

Goes to %\WindowsPowerShell\Modules\WinDbg

Note: PowerDbg assumes the folder c:\debuggers as the default installation folder. This is true for the default installation of our private debugger version (Microsoft) but not for the public version, so, please, change this variable to reflect your installation:

param($cdbPath = "C:\debuggers\cdb.exe")

In order to know the exact location, use this command from PowerShell:

$profile

REQUIREMENT

PowerShell v2.0

USAGE

First, make sure you can run scripts:

set-executionpolicy remotesigned

From the WinDbg window type:

.server tcp:port=10456,server=ServerName ß ServerName is your server name.

The command above enables a port communication with the WinDbg instance as the server. You can use other port numbers.

From the PowerShell window you must initialize the communication:

Import-module WinDbg ß Imports our module WinDbg.PSM1

Connect-Windbg "tcp:Port=10456,Server=SERVER" ß Connects session to WinDbg instance.

Or:

Connect-Dbg "tcp:Port=10456,Server=SERVER" ß Connects session to WinDbg

Note: Don’t forget to load symbols and your extensions!

At this point you’re ready to use PowerDbg or PowerDbg scripts.

Example:

Analyze-PowerDbgThreads ß Cmdlet.

.\PowerDbgScriptExceptions ß Script.

Example 2:

Send-PowerDbgCommand "~* kpn 1000"

Parse-PowerDbgK

$ht = @{}

$ht = Convert-PowerDbgCSVToHashTable

Let's display the stack for thread 0:

# Replace the internal frame delimiter by new line and displays the stack.

write-host $ht["0"].Replace($global:g_frameDelimiter, "`n")

Example 3:

Send-PowerDbgCommand "!DumpObj 027a4c3c"

Parse-PowerDbgDUMPOBJ

$ht = Convert-PowerDbgCSVToHashTable

$ht["Name:"] <-- Displays the content of the field Name:

$ht["MethodTable:"] <-- Displays the content of the field MethodTable:

NEW FOR POWERDBG v5.3

Parse-PowerDbgDSO is fixed after a small bug was found.

NEW FOR POWERDBG v5.2

PowerDbg now uses just one file that has the content of the previous two files and is not the Profile file anymore.

Easier to install and maintain.

The information from WinDbg to PowerDbg used a text file, now it’s all memory based.

It’s faster now than before and eliminates timing issues that used to happen when the file was being saved and the next PowerDbg command executed.

As a collateral benefit you don’t need to use Start-Sleep in your scripts anymore to avoid timing issues.

Note: The CSV file POWERDBG-PARSED.LOG is still used. It facilitates the troubleshooting and can be used as input for Excel or LogParser.

NEW FOR POWERDBG v5.1

Load-PowerDbgSymbols <$symbolPath>

Load symbols.

Usage:

Load-PowerDbgSymbols “SRV*c:\PUBLICSYMBOLS*http://msdl.microsoft.com/download/symbols"

Parse-PowerDbgASPXPAGES

Maps the output from the !ASPXPages command and saves it into the CSV file POWERDBG-PARSED.LOG

To convert the CSV file to a Hash Table use Convert-PowerDbgCSVToHashTable.

For this version the fields being considered are:

Key: HttpContext

Value: Timeout+Completed+Running+ThreadId+ReturnCode+Verb+RequestPath+QueryString

Parse-PowerDbgCLRSTACK

Maps the output from the !clrstack command or ~* e !clrstack and saves it into the CSV file POWERDBG-PARSED.LOG

To convert the CSV file to a Hash Table use Convert-PowerDbgCSVToHashTable.

Attention! The key is the thread number, and the value is the call stack separated by $global:g_frameDelimiter.

Commas "," are replaced for ";" to avoid confusion with the comma used by the CSV file.

If you use this cmdlet to parse the output from ~* e !clrstack, the threads not running managed code are automatically ignored.

Parse-PowerDbgTHREADS

Maps the output from the !threads command and saves it into the CSV file POWERDBG-PARSED.LOG

To convert the CSV file to a Hash Table use Convert-PowerDbgCSVToHashTable.

The following fields are extracted:

Thread Number - Key

ID+OSID+ThreadOBJ+State+GC+Context+Domain+Count+APT+Exception - Value

Parse-PowerDbgDSO

Maps the output from the !dso or ~* e !dso command and saves it into the CSV file POWERDBG-PARSED.LOG

To convert the CSV file to a Hash Table use Convert-PowerDbgCSVToHashTable.

The Thread Number is the key, and the stack is the value, like the way that Parse-PowerDbgK or Parse-PowerDbgCLRSTACK operates.

Attention! Commas are replaced by ";" and $global:g_FrameDelimiter is used to separate frames.

CMDLETS FROM POWERDBG

Send-PowerDbgCommand <$command>

This was the most complex cmdlet, but now it’s just a wrapper for Invoke-WinDbgCommand.

SendPowerDbgCommand sends commands to WinDbg.

Parse-PowerDbgDT [$useFieldNames]

Parses the output from the dt command and saves it into POWERDBG-PARSED.LOG using a CSV file format.

If $useFieldNames has a value, the cmdlet stores fields from struct/classes and values. Otherwise it stores offsets and values.

To convert the CSV file to a Hash Table use Convert-PowerDbgCSVToHashTable.

Convert-PowerDbgCSVToHashTable

Converts the output from the Parse-PowerDbg* cmdlets to a Hash Table.

Send-PowerDbgDML <$hyperLinkDML> <$commandDML>

Creates a DML command and sends it to WinDbg.

DML stands for Debug Markup Language. Using DML you can create hyperlinks that execute commands.

Parse-PowerDbgNAME2EE

Maps the output from the !name2ee and saves it into the CSV file POWERDBG-PARSED.LOG

Convert-PowerDbgCSVtoHashTable converts the output into a Hash Table.

Parse-PowerDbgDUMPMD

Maps the output from !dumpmd command and saves it into the CSV file POWERDBG-PARSED.LOG.

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Parse-PowerDbgDUMPMODULE

Maps the output from !DumpModule command and saves it into the CSV file POWERDBG-PARSED.LOG

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Parse-PowerDbgDUMPLMI

Maps the output from !lmi command and saves it into the CSV file POWERDBG-PARSED.LOG

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Has-PowerDbgCOMMANDSUCCEEDED

Returns $true if the last command succeeded or $false if not.

Send-PowerDbgComment

Sends a comment, a string in bold, to the WinDbg window.

Parse-PowerDbgVERTARGET

Maps the output from vertarget command, either the Kernel Time or the User Time.

The output is saved into the CSV file POWERDBG-PARSED.LOG.

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Parse-PowerDbgRUNAWAY

Maps the output of !runaway 1 or !runaway 2 and stores the results into the CSV file POWERDBG-PARSED.LOG

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Attention! If you need to know the top threads consuming CPU time, use Convert-PowerDbgRUNAWAYtoArray. The items will be in the same exact order of the original command.

Convert-PowerDbgRUNAWAYtoArray

Returns an array of two dimensions corresponding to the output of !runaway 1 or !runaway 2.

Parse-PowerDbgK

Maps the output of k command and its variations like kv, kbn, kpn, etc.

The output is saved into the CSV file POWERDBG-PARSED.LOG

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Attention! This cmdlet doesn’t work with kPn. Another thing, it replaces “,” with “;” to avoid conflict with the CSV delimiter.

Parse-PowerDbgSymbolsFromK

Maps just the symbols from k command and variations, saving the content into the CSV file POWERDBG-PARSED.LOG

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Attention! This cmdlet doesn’t work with kPn. Another thing, it replaces “,” with “;” to avoid conflict with the CSV delimiter.

Parse-PowerDbgLM1M

Maps just the output from lm1m and stores it into the CSV file POWERDBG-PARSED.LOG

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Classify-PowerDbgThreads

Returns one array where the index is the thread number and the value is one of these values:

0 UNKNOWN_SYMBOL

1 WAITING_FOR_CRITICAL_SECTION

2 DOING_IO

3 WAITING

4 GC_THREAD

5 WAIT_UNTIL_GC_COMPLETE

6 SUSPEND_FOR_GC

7 WAIT_FOR_FINALIZE

8 TRYING_MANAGED_LOCK

9 DATA_FROM_WINSOCK

It’s very easy to add more symbols and constants to get a more granular analysis. Look at the source code for details.

Analyze-PowerDbgThreads

Analyzes and shows what each thread is doing and its corresponding CPU time, sorted by User Mode time.

This cmdlet is very useful for scenarios like hangs, high CPU, and crashes.

Attention! This command requires thread information if debugging a dump file.

Parse-PowerDbgPRINTEXCEPTION

Maps the output from !PrintException command and saves it into the CSV file POWERDBG-PARSED.LOG.

The following fields are considered while others are ignored:

Exception object:

Exception type;

Message:

InnerException:

HRESULT:

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Parse-PowerDbgDD-L1

Maps the output from dd <address> L1 or dd poi(<address>) L1 and saves the results into the CSV file POWERDBG-PARSED.LOG.

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Parse-PowerDbgGCHANDLELEAKS

Maps the output from !GCHandleLeaks command and saves it into the CSV file POWERDBG-PARSED.LOG.

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Parse-PowerDbgDUMPOBJ

Maps the output from !DumpObj command and saves it into the CSV file POWERDBG-PARSED.LOG.

The assembly path and file name are saved using the key name “Assembly:.”

If the object is invalid, the ‘Name:’ field will have the string “Invalid Object.” You may want to check this string to make sure you have valid data.

The keys are the fields or Method Table, and values are the corresponding value.

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Attention! This version maps the fields below “Fields:” using MethodTable as key and Value as value. The problem with this approach is that the same MethodTable may appear more than once. If it happens, the last or most recent MethodTable and value will be considered.

Based on users’ feedback this approach may be changed in the near future.

POWERDBG SCRIPTS

PowerDbgScriptDumpDict.PS1

Extracts the key/value pair from a dictionary.

PowerDbgScriptExceptions.PS1

Displays the call stacks that have inner or hidden exceptions.

PowerDbgScriptGCHandleLeaksChart.PS1

Displays statistics and a chart from the top 20 objects leaking.

PowerDbgScriptHighCPU.PS1

Displays all threads consuming high CPU using a specific time as a threshold.

PowerDbgScriptSaveModule.PS1

Saves all modules that have a specific method. You provide the method name, and it gives you the corresponding modules.

Special Command—Unassembling code with u, ub and uf

rafarah — Thu, 05 Nov 2009 21:27:00 GMT

When debugging sooner or later you will need to disassemble code to get a better understanding of that code.

By disassembling the code, you get the mnemonics translated from the 0s and 1s that constitute the binary code. It is a low level view of the code, but a higher level than seeing just numbers.

The commands syntaxes are:

u[b] [address]

u[b] [range]

u[b]

uf [options] <address>

Where options are:

/c - Displays only the call instructions in a routine.

/D - Creates linked callee names for navigation of the call graph.

/o - Sorts the display by address instead of by function offset.

/O - Creates linked call lines for accessing call information and creating breakpoints.

/i - Displays the number of instructions in a routine.

To demonstrate this command, let’s use this simple Visual C++ application that recursively calculates the Fibonacci from a specific number:

#include "stdafx.h"

using namespace std;

// Recursive function.

unsigned FiboRecursive(unsigned n, int nNum = 0)

{

if(n <= 1)

{

return n;

}

return FiboRecursive(n - 1, 1) + FiboRecursive(n - 2, 2);

}

int _tmain(int argc, _TCHAR* argv[])

{

cout << FiboRecursive(5) << endl;

return 0;

}

Let’s break the execution when the line from main() only is being executed, using a breakpoint for that.

Now let’s disassemble the eip register.

0:000> u @eip

Fibo!wmain+0x1e [c:\development\my tools\fibo\fibo\fibo.cpp @ 21]:

00a7145e 8bf4 mov esi,esp

00a71460 a19882a700 mov eax,dword ptr [Fibo!_imp_?endlstdYAAAV?$basic_ostreamDU?$char_traitsDstd (00a78298)]

00a71465 50 push eax

00a71466 6a00 push 0

00a71468 6a05 push 5

00a7146a e89bfbffff call Fibo!ILT+5(?FiboRecursiveYAIIHZ) (00a7100a)

00a7146f 83c408 add esp,8

00a71472 8bfc mov edi,esp

Using this approach we see the disassembled code starting from eip.

We can see the disassembled code that comes before eip using this approach:

0:000> ub @eip L8

Fibo!wmain+0x3 [c:\development\my tools\fibo\fibo\fibo.cpp @ 20]:

00a71443 81ecc0000000 sub esp,0C0h

00a71449 53 push ebx

00a7144a 56 push esi

00a7144b 57 push edi

00a7144c 8dbd40ffffff lea edi,[ebp-0C0h]

00a71452 b930000000 mov ecx,30h

00a71457 b8cccccccc mov eax,0CCCCCCCCh

00a7145c f3ab rep stos dword ptr es:[edi]

Keep in mind that b is for backward.

Let’s suppose you want to disassemble the entire function without having the work of finding the beginning of the function.

To accomplish that you use uf, and it automatically does that for you:

0:000> uf @eip

Fibo!wmain [c:\development\my tools\fibo\fibo\fibo.cpp @ 20]:

20 00a71440 55 push ebp

20 00a71441 8bec mov ebp,esp

20 00a71443 81ecc0000000 sub esp,0C0h

20 00a71449 53 push ebx

20 00a7144a 56 push esi

20 00a7144b 57 push edi

20 00a7144c 8dbd40ffffff lea edi,[ebp-0C0h]

20 00a71452 b930000000 mov ecx,30h

20 00a71457 b8cccccccc mov eax,0CCCCCCCCh

20 00a7145c f3ab rep stos dword ptr es:[edi]

21 00a7145e 8bf4 mov esi,esp

21 00a71460 a19882a700 mov eax,dword ptr [Fibo!_imp_?endlstdYAAAV?$basic_ostreamDU?$char_traitsDstd (00a78298)]

21 00a71465 50 push eax

21 00a71466 6a00 push 0

21 00a71468 6a05 push 5

21 00a7146a e89bfbffff call Fibo!ILT+5(?FiboRecursiveYAIIHZ) (00a7100a)

21 00a7146f 83c408 add esp,8

21 00a71472 8bfc mov edi,esp

21 00a71474 50 push eax

21 00a71475 8b0d9082a700 mov ecx,dword ptr [Fibo!_imp_?coutstd (00a78290)]

21 00a7147b ff159482a700 call dword ptr [Fibo!_imp_??6?$basic_ostreamDU?$char_traitsDstdstdQAEAAV01IZ (00a78294)]

21 00a71481 3bfc cmp edi,esp

21 00a71483 e8d1fcffff call Fibo!ILT+340(__RTC_CheckEsp) (00a71159)

21 00a71488 8bc8 mov ecx,eax

21 00a7148a ff159c82a700 call dword ptr [Fibo!_imp_??6?$basic_ostreamDU?$char_traitsDstdstdQAEAAV01P6AAAV01AAV01ZZ (00a7829c)]

21 00a71490 3bf4 cmp esi,esp

21 00a71492 e8c2fcffff call Fibo!ILT+340(__RTC_CheckEsp) (00a71159)

23 00a71497 33c0 xor eax,eax

24 00a71499 5f pop edi

24 00a7149a 5e pop esi

24 00a7149b 5b pop ebx

24 00a7149c 81c4c0000000 add esp,0C0h

24 00a714a2 3bec cmp ebp,esp

24 00a714a4 e8b0fcffff call Fibo!ILT+340(__RTC_CheckEsp) (00a71159)

24 00a714a9 8be5 mov esp,ebp

24 00a714ab 5d pop ebp

24 00a714ac c3 ret

Let’s see just the calls made by this function:

0:000> uf /c @eip

Fibo!wmain (00a71440) [c:\development\my tools\fibo\fibo\fibo.cpp @ 20]

Fibo!wmain+0x2a (00a7146a) [c:\development\my tools\fibo\fibo\fibo.cpp @ 21]:

call to Fibo!ILT+5(?FiboRecursiveYAIIHZ) (00a7100a)

Fibo!wmain+0x3b (00a7147b) [c:\development\my tools\fibo\fibo\fibo.cpp @ 21]:

call to MSVCP90D!std::basic_ostream<char,std::char_traits<char> >::operator<< (690fa700) [f:\dd\vctools\crt_bld\self_x86\crt\src\ostream @ 289]

Fibo!wmain+0x43 (00a71483) [c:\development\my tools\fibo\fibo\fibo.cpp @ 21]:

call to Fibo!ILT+340(__RTC_CheckEsp) (00a71159)

Fibo!wmain+0x4a (00a7148a) [c:\development\my tools\fibo\fibo\fibo.cpp @ 21]:

call to MSVCP90D!std::basic_ostream<char,std::char_traits<char> >::operator<< (690f9f60) [f:\dd\vctools\crt_bld\self_x86\crt\src\ostream @ 171]

Fibo!wmain+0x52 (00a71492) [c:\development\my tools\fibo\fibo\fibo.cpp @ 21]:

call to Fibo!ILT+340(__RTC_CheckEsp) (00a71159)

Fibo!wmain+0x64 (00a714a4) [c:\development\my tools\fibo\fibo\fibo.cpp @ 24]:

call to Fibo!ILT+340(__RTC_CheckEsp) (00a71159)

Creates linked callee names from the same function:

0:000> uf /D @eip

Fibo!wmain [c:\development\my tools\fibo\fibo\fibo.cpp @ 20]:

20 00a71440 55 push ebp

20 00a71441 8bec mov ebp,esp

20 00a71443 81ecc0000000 sub esp,0C0h

20 00a71449 53 push ebx

20 00a7144a 56 push esi

20 00a7144b 57 push edi

20 00a7144c 8dbd40ffffff lea edi,[ebp-0C0h]

20 00a71452 b930000000 mov ecx,30h

20 00a71457 b8cccccccc mov eax,0CCCCCCCCh

20 00a7145c f3ab rep stos dword ptr es:[edi]

21 00a7145e 8bf4 mov esi,esp

21 00a71460 a19882a700 mov eax,dword ptr [Fibo!_imp_?endlstdYAAAV?$basic_ostreamDU?$char_traitsDstd (00a78298)]

21 00a71465 50 push eax

21 00a71466 6a00 push 0

21 00a71468 6a05 push 5

21 00a7146a e89bfbffff call Fibo!ILT+5(?FiboRecursiveYAIIHZ) (00a7100a)

21 00a7146f 83c408 add esp,8

21 00a71472 8bfc mov edi,esp

21 00a71474 50 push eax

21 00a71475 8b0d9082a700 mov ecx,dword ptr [Fibo!_imp_?coutstd (00a78290)]

21 00a7147b ff159482a700 call dword ptr [Fibo!_imp_??6?$basic_ostreamDU?$char_traitsDstdstdQAEAAV01IZ (00a78294)]

21 00a71481 3bfc cmp edi,esp

21 00a71483 e8d1fcffff call Fibo!ILT+340(__RTC_CheckEsp) (00a71159)

21 00a71488 8bc8 mov ecx,eax

21 00a7148a ff159c82a700 call dword ptr [Fibo!_imp_??6?$basic_ostreamDU?$char_traitsDstdstdQAEAAV01P6AAAV01AAV01ZZ (00a7829c)]

21 00a71490 3bf4 cmp esi,esp

21 00a71492 e8c2fcffff call Fibo!ILT+340(__RTC_CheckEsp) (00a71159)

23 00a71497 33c0 xor eax,eax

24 00a71499 5f pop edi

24 00a7149a 5e pop esi

24 00a7149b 5b pop ebx

24 00a7149c 81c4c0000000 add esp,0C0h

24 00a714a2 3bec cmp ebp,esp

24 00a714a4 e8b0fcffff call Fibo!ILT+340(__RTC_CheckEsp) (00a71159)

24 00a714a9 8be5 mov esp,ebp

24 00a714ab 5d pop ebp

24 00a714ac c3 ret

Sorts the output by address:

0:000> uf /o @eip

Fibo!wmain [c:\development\my tools\fibo\fibo\fibo.cpp @ 20]:

20 00a71440 55 push ebp

20 00a71441 8bec mov ebp,esp

20 00a71443 81ecc0000000 sub esp,0C0h

20 00a71449 53 push ebx

20 00a7144a 56 push esi

20 00a7144b 57 push edi

20 00a7144c 8dbd40ffffff lea edi,[ebp-0C0h]

20 00a71452 b930000000 mov ecx,30h

20 00a71457 b8cccccccc mov eax,0CCCCCCCCh

20 00a7145c f3ab rep stos dword ptr es:[edi]

21 00a7145e 8bf4 mov esi,esp

21 00a71460 a19882a700 mov eax,dword ptr [Fibo!_imp_?endlstdYAAAV?$basic_ostreamDU?$char_traitsDstd (00a78298)]

21 00a71465 50 push eax

21 00a71466 6a00 push 0

21 00a71468 6a05 push 5

21 00a7146a e89bfbffff call Fibo!ILT+5(?FiboRecursiveYAIIHZ) (00a7100a)

21 00a7146f 83c408 add esp,8

21 00a71472 8bfc mov edi,esp

21 00a71474 50 push eax

21 00a71475 8b0d9082a700 mov ecx,dword ptr [Fibo!_imp_?coutstd (00a78290)]

21 00a7147b ff159482a700 call dword ptr [Fibo!_imp_??6?$basic_ostreamDU?$char_traitsDstdstdQAEAAV01IZ (00a78294)]

21 00a71481 3bfc cmp edi,esp

21 00a71483 e8d1fcffff call Fibo!ILT+340(__RTC_CheckEsp) (00a71159)

21 00a71488 8bc8 mov ecx,eax

21 00a7148a ff159c82a700 call dword ptr [Fibo!_imp_??6?$basic_ostreamDU?$char_traitsDstdstdQAEAAV01P6AAAV01AAV01ZZ (00a7829c)]

21 00a71490 3bf4 cmp esi,esp

21 00a71492 e8c2fcffff call Fibo!ILT+340(__RTC_CheckEsp) (00a71159)

23 00a71497 33c0 xor eax,eax

24 00a71499 5f pop edi

24 00a7149a 5e pop esi

24 00a7149b 5b pop ebx

24 00a7149c 81c4c0000000 add esp,0C0h

24 00a714a2 3bec cmp ebp,esp

24 00a714a4 e8b0fcffff call Fibo!ILT+340(__RTC_CheckEsp) (00a71159)

24 00a714a9 8be5 mov esp,ebp

24 00a714ab 5d pop ebp

24 00a714ac c3 ret

Creates linked call lines for accessing call information and creating breakpoints:

0:000> uf /O @eip

Fibo!wmain [c:\development\my tools\fibo\fibo\fibo.cpp @ 20]:

20 00a71440 55 push ebp

20 00a71441 8bec mov ebp,esp

20 00a71443 81ecc0000000 sub esp,0C0h

20 00a71449 53 push ebx

20 00a7144a 56 push esi

20 00a7144b 57 push edi

20 00a7144c 8dbd40ffffff lea edi,[ebp-0C0h]

20 00a71452 b930000000 mov ecx,30h

20 00a71457 b8cccccccc mov eax,0CCCCCCCCh

20 00a7145c f3ab rep stos dword ptr es:[edi]

21 00a7145e 8bf4 mov esi,esp

21 00a71460 a19882a700 mov eax,dword ptr [Fibo!_imp_?endlstdYAAAV?$basic_ostreamDU?$char_traitsDstd (00a78298)]

21 00a71465 50 push eax

21 00a71466 6a00 push 0

21 00a71468 6a05 push 5

21 00a7146a e89bfbffff call Fibo!ILT+5(?FiboRecursiveYAIIHZ) (00a7100a)

21 00a7146f 83c408 add esp,8

21 00a71472 8bfc mov edi,esp

21 00a71474 50 push eax

21 00a71475 8b0d9082a700 mov ecx,dword ptr [Fibo!_imp_?coutstd (00a78290)]

21 00a7147b ff159482a700 call dword ptr [Fibo!_imp_??6?$basic_ostreamDU?$char_traitsDstdstdQAEAAV01IZ (00a78294)]

21 00a71481 3bfc cmp edi,esp

21 00a71483 e8d1fcffff call Fibo!ILT+340(__RTC_CheckEsp) (00a71159)

21 00a71488 8bc8 mov ecx,eax

21 00a7148a ff159c82a700 call dword ptr [Fibo!_imp_??6?$basic_ostreamDU?$char_traitsDstdstdQAEAAV01P6AAAV01AAV01ZZ (00a7829c)]

21 00a71490 3bf4 cmp esi,esp

21 00a71492 e8c2fcffff call Fibo!ILT+340(__RTC_CheckEsp) (00a71159)

23 00a71497 33c0 xor eax,eax

24 00a71499 5f pop edi

24 00a7149a 5e pop esi

24 00a7149b 5b pop ebx

24 00a7149c 81c4c0000000 add esp,0C0h

24 00a714a2 3bec cmp ebp,esp

24 00a714a4 e8b0fcffff call Fibo!ILT+340(__RTC_CheckEsp) (00a71159)

24 00a714a9 8be5 mov esp,ebp

24 00a714ab 5d pop ebp

24 00a714ac c3 ret

Displays the number of instructions in a routine:

0:000> uf /i @eip

37 instructions scanned

Fibo!wmain [c:\development\my tools\fibo\fibo\fibo.cpp @ 20]:

20 00a71440 55 push ebp

20 00a71441 8bec mov ebp,esp

20 00a71443 81ecc0000000 sub esp,0C0h

20 00a71449 53 push ebx

20 00a7144a 56 push esi

20 00a7144b 57 push edi

20 00a7144c 8dbd40ffffff lea edi,[ebp-0C0h]

20 00a71452 b930000000 mov ecx,30h

20 00a71457 b8cccccccc mov eax,0CCCCCCCCh

20 00a7145c f3ab rep stos dword ptr es:[edi]

21 00a7145e 8bf4 mov esi,esp

21 00a71460 a19882a700 mov eax,dword ptr [Fibo!_imp_?endlstdYAAAV?$basic_ostreamDU?$char_traitsDstd (00a78298)]

21 00a71465 50 push eax

21 00a71466 6a00 push 0

21 00a71468 6a05 push 5

21 00a7146a e89bfbffff call Fibo!ILT+5(?FiboRecursiveYAIIHZ) (00a7100a)

21 00a7146f 83c408 add esp,8

21 00a71472 8bfc mov edi,esp

21 00a71474 50 push eax

21 00a71475 8b0d9082a700 mov ecx,dword ptr [Fibo!_imp_?coutstd (00a78290)]

21 00a7147b ff159482a700 call dword ptr [Fibo!_imp_??6?$basic_ostreamDU?$char_traitsDstdstdQAEAAV01IZ (00a78294)]

21 00a71481 3bfc cmp edi,esp

21 00a71483 e8d1fcffff call Fibo!ILT+340(__RTC_CheckEsp) (00a71159)

21 00a71488 8bc8 mov ecx,eax

21 00a7148a ff159c82a700 call dword ptr [Fibo!_imp_??6?$basic_ostreamDU?$char_traitsDstdstdQAEAAV01P6AAAV01AAV01ZZ (00a7829c)]

21 00a71490 3bf4 cmp esi,esp

21 00a71492 e8c2fcffff call Fibo!ILT+340(__RTC_CheckEsp) (00a71159)

23 00a71497 33c0 xor eax,eax

24 00a71499 5f pop edi

24 00a7149a 5e pop esi

24 00a7149b 5b pop ebx

24 00a7149c 81c4c0000000 add esp,0C0h

24 00a714a2 3bec cmp ebp,esp

24 00a714a4 e8b0fcffff call Fibo!ILT+340(__RTC_CheckEsp) (00a71159)

24 00a714a9 8be5 mov esp,ebp

24 00a714ab 5d pop ebp

24 00a714ac c3 ret

Special Command—Using # to Find Patterns of Assembly Instructions

rafarah — Fri, 23 Oct 2009 14:46:00 GMT

Sometimes you need to look for patterns of disassembled code. You can browse the disassembled code and manually look for a specific pattern, or you can use a command to automate it. The # command does that.

# [Pattern] [Address [L Size ]]

Parameters:

Pattern - Specifies the pattern to search for in the disassembly code. If you have previously used the # command and you omit Pattern, the command reuses the most recently used pattern.

Address - Specifies the address where the search begins.

Size - Specifies the number of instructions to search. If you omit Size, the search continues until the first match occurs.

To demonstrate this command, let’s use this simple Visual C++ application that recursively calculates the Fibonacci from a specific number:

#include "stdafx.h"

using namespace std;

// Recursive function.

unsigned FiboRecursive(unsigned n, int nNum = 0)

{

if(n <= 1)

{

return n;

}

return FiboRecursive(n - 1, 1) + FiboRecursive(n - 2, 2);

}

int _tmain(int argc, _TCHAR* argv[])

{

cout << FiboRecursive(5) << endl;

return 0;

}

Let’s break the execution when the only line from main() is being executed, using a breakpoint for that.

Now let’s disassemble the eip register.

0:000> uf @eip

Fibo!wmain [c:\development\my tools\book\fibo\fibo\fibo.cpp @ 20]:

20 00a71440 55 push ebp

20 00a71441 8bec mov ebp,esp

20 00a71443 81ecc0000000 sub esp,0C0h

20 00a71449 53 push ebx

20 00a7144a 56 push esi

20 00a7144b 57 push edi

20 00a7144c 8dbd40ffffff lea edi,[ebp-0C0h]

20 00a71452 b930000000 mov ecx,30h

20 00a71457 b8cccccccc mov eax,0CCCCCCCCh

20 00a7145c f3ab rep stos dword ptr es:[edi]

21 00a7145e 8bf4 mov esi,esp

21 00a71460 a19882a700 mov eax,dword ptr [Fibo!_imp_?endlstdYAAAV?$basic_ostreamDU?$char_traitsDstd (00a78298)]

21 00a71465 50 push eax

21 00a71466 6a00 push 0

21 00a71468 6a05 push 5

21 00a7146a e89bfbffff call Fibo!ILT+5(?FiboRecursiveYAIIHZ) (00a7100a)

21 00a7146f 83c408 add esp,8

21 00a71472 8bfc mov edi,esp

21 00a71474 50 push eax

21 00a71475 8b0d9082a700 mov ecx,dword ptr [Fibo!_imp_?coutstd (00a78290)]

21 00a7147b ff159482a700 call dword ptr [Fibo!_imp_??6?$basic_ostreamDU?$char_traitsDstdstdQAEAAV01IZ (00a78294)]

21 00a71481 3bfc cmp edi,esp

21 00a71483 e8d1fcffff call Fibo!ILT+340(__RTC_CheckEsp) (00a71159)

21 00a71488 8bc8 mov ecx,eax

21 00a7148a ff159c82a700 call dword ptr [Fibo!_imp_??6?$basic_ostreamDU?$char_traitsDstdstdQAEAAV01P6AAAV01AAV01ZZ (00a7829c)]

21 00a71490 3bf4 cmp esi,esp

21 00a71492 e8c2fcffff call Fibo!ILT+340(__RTC_CheckEsp) (00a71159)

23 00a71497 33c0 xor eax,eax

24 00a71499 5f pop edi

24 00a7149a 5e pop esi

24 00a7149b 5b pop ebx

24 00a7149c 81c4c0000000 add esp,0C0h

24 00a714a2 3bec cmp ebp,esp

24 00a714a4 e8b0fcffff call Fibo!ILT+340(__RTC_CheckEsp) (00a71159)

24 00a714a9 8be5 mov esp,ebp

24 00a714ab 5d pop ebp

24 00a714ac c3 ret

Using the command below we’re going to display the first occurrence of ret.

0:000> # ret 00a71440

Fibo!wmain+0x6c [c:\development\my tools\book\fibo\fibo\fibo.cpp @ 24]:

00a714ac c3 ret

Looking for another pattern:

0:000> # call*Fibo!ILT 00a71440

Fibo!wmain+0x2a [c:\development\my tools\book\fibo\fibo\fibo.cpp @ 21]:

00a7146a e89bfbffff call Fibo!ILT+5(?FiboRecursiveYAIIHZ) (00a7100a)

Now let’s look for push instructions in a specific module/executable:

0:000> lm

start end module name

00a60000 00a7b000 Fibo C (private pdb symbols) C:\development\My Tools\Book\Fibo\Debug\Fibo.pdb

67350000 67473000 MSVCR90D (deferred)

690c0000 69197000 MSVCP90D (private pdb symbols) c:\publicsymbols\msvcp90d.i386.pdb\7B1C9137C0074A0E921BE874ADF944191\msvcp90d.i386.pdb

75e00000 75e44000 KERNELBASE (deferred)

75eb0000 75fb0000 kernel32 (deferred)

776c0000 77840000 ntdll (pdb symbols) c:\publicsymbols\wntdll.pdb\E06BEA155E9748BEA818E2D0DD2FED952\wntdll.pdb

0:000> # push 00a60000

Fibo!__ImageBase+0x40:

00a60040 0e push cs

As you can see, when looking for patterns in disassembled code “#” is the way to go! Here is an script that automates the process.

Tip: Most commands mentioned in the Special Commands section of this blog have a script that uses it and could serve as another example of using the command. Also there’re scripts that use commands from the SOS.DLL extension to debug managed code.

Check out the WinDbg Scripts and PowerDbg Scripts section.

Special Command—Tracing Applications Using wt

rafarah — Mon, 12 Oct 2009 20:27:00 GMT

wt [WatchOptions] [= StartAddress] [EndAddress]

Transcribing the WinDbg documentation, this command runs through the whole function and then displays statistics when executed at the beginning of a function call. Thus, this command can be used just when doing live debugging, not post mortem debugging (dump analysis).

Think about Watch and Trace.

The WinDbg help file describes all options for WatchOptions. Usually we use the simplest form, not these options.

To demonstrate this command, let’s use this simple Visual C++ application that recursively calculates the Fibonacci from a specific number:

#include "stdafx.h"

using namespace std;

// Recursive function.

unsigned FiboRecursive(unsigned n, int nNum = 0)

{

if(n <= 1)

{

return n;

}

return FiboRecursive(n - 1, 1) + FiboRecursive(n - 2, 2);

}

int _tmain(int argc, _TCHAR* argv[])

{

cout << FiboRecursive(5) << endl;

return 0;

}

Now we run the application from WinDbg and insert a breakpoint in the beginning of _tmain().

After hitting the breakpoint, we disassemble the function:

0:000> uf Fibo!wmain

Fibo!wmain [c:\development\my tools\book\fibo\fibo\fibo.cpp @ 20]:

20 001b1440 55 push ebp

20 001b1441 8bec mov ebp,esp

20 001b1443 81ecc0000000 sub esp,0C0h

20 001b1449 53 push ebx

20 001b144a 56 push esi

20 001b144b 57 push edi

20 001b144c 8dbd40ffffff lea edi,[ebp-0C0h]

20 001b1452 b930000000 mov ecx,30h

20 001b1457 b8cccccccc mov eax,0CCCCCCCCh

20 001b145c f3ab rep stos dword ptr es:[edi]

21 001b145e 8bf4 mov esi,esp

21 001b1460 a198821b00 mov eax,dword ptr [Fibo!_imp_?endlstdYAAAV?$basic_ostreamDU?$char_traitsDstd (001b8298)]

21 001b1465 50 push eax

21 001b1466 6a00 push 0

21 001b1468 6a05 push 5

21 001b146a e89bfbffff call Fibo!ILT+5(?FiboRecursiveYAIIHZ) (001b100a)

21 001b146f 83c408 add esp,8

21 001b1472 8bfc mov edi,esp

21 001b1474 50 push eax

21 001b1475 8b0d90821b00 mov ecx,dword ptr [Fibo!_imp_?coutstd (001b8290)]

21 001b147b ff1594821b00 call dword ptr [Fibo!_imp_??6?$basic_ostreamDU?$char_traitsDstdstdQAEAAV01IZ (001b8294)]

21 001b1481 3bfc cmp edi,esp

21 001b1483 e8d1fcffff call Fibo!ILT+340(__RTC_CheckEsp) (001b1159)

21 001b1488 8bc8 mov ecx,eax

21 001b148a ff159c821b00 call dword ptr [Fibo!_imp_??6?$basic_ostreamDU?$char_traitsDstdstdQAEAAV01P6AAAV01AAV01ZZ (001b829c)]

21 001b1490 3bf4 cmp esi,esp

21 001b1492 e8c2fcffff call Fibo!ILT+340(__RTC_CheckEsp) (001b1159)

23 001b1497 33c0 xor eax,eax

24 001b1499 5f pop edi

24 001b149a 5e pop esi

24 001b149b 5b pop ebx

24 001b149c 81c4c0000000 add esp,0C0h

24 001b14a2 3bec cmp ebp,esp

24 001b14a4 e8b0fcffff call Fibo!ILT+340(__RTC_CheckEsp) (001b1159)

24 001b14a9 8be5 mov esp,ebp

24 001b14ab 5d pop ebp

24 001b14ac c3 ret

Let’s use the address right before the recursive function call as the Start address and the address right after it as the End address of wt. Note the “=” right before the first address.

0:000> wt =001b1468 001b146f

2 0 [ 0] Fibo!wmain

1 0 [ 1] Fibo!ILT+5(?FiboRecursiveYAIIHZ)

64 0 [ 1] Fibo!FiboRecursive

1 0 [ 2] Fibo!ILT+5(?FiboRecursiveYAIIHZ)

64 0 [ 2] Fibo!FiboRecursive

1 0 [ 3] Fibo!ILT+5(?FiboRecursiveYAIIHZ)

64 0 [ 3] Fibo!FiboRecursive

1 0 [ 4] Fibo!ILT+5(?FiboRecursiveYAIIHZ)

64 0 [ 4] Fibo!FiboRecursive

1 0 [ 5] Fibo!ILT+5(?FiboRecursiveYAIIHZ)

67 0 [ 5] Fibo!FiboRecursive

1 0 [ 6] Fibo!ILT+340(__RTC_CheckEsp)

2 0 [ 6] Fibo!_RTC_CheckEsp

70 3 [ 5] Fibo!FiboRecursive

71 74 [ 4] Fibo!FiboRecursive

1 0 [ 5] Fibo!ILT+5(?FiboRecursiveYAIIHZ)

67 0 [ 5] Fibo!FiboRecursive

1 0 [ 6] Fibo!ILT+340(__RTC_CheckEsp)

2 0 [ 6] Fibo!_RTC_CheckEsp

70 3 [ 5] Fibo!FiboRecursive

79 148 [ 4] Fibo!FiboRecursive

1 0 [ 5] Fibo!ILT+340(__RTC_CheckEsp)

2 0 [ 5] Fibo!_RTC_CheckEsp

82 151 [ 4] Fibo!FiboRecursive

71 234 [ 3] Fibo!FiboRecursive

1 0 [ 4] Fibo!ILT+5(?FiboRecursiveYAIIHZ)

67 0 [ 4] Fibo!FiboRecursive

1 0 [ 5] Fibo!ILT+340(__RTC_CheckEsp)

2 0 [ 5] Fibo!_RTC_CheckEsp

70 3 [ 4] Fibo!FiboRecursive

79 308 [ 3] Fibo!FiboRecursive

1 0 [ 4] Fibo!ILT+340(__RTC_CheckEsp)

2 0 [ 4] Fibo!_RTC_CheckEsp

82 311 [ 3] Fibo!FiboRecursive

71 394 [ 2] Fibo!FiboRecursive

1 0 [ 3] Fibo!ILT+5(?FiboRecursiveYAIIHZ)

64 0 [ 3] Fibo!FiboRecursive

1 0 [ 4] Fibo!ILT+5(?FiboRecursiveYAIIHZ)

67 0 [ 4] Fibo!FiboRecursive

1 0 [ 5] Fibo!ILT+340(__RTC_CheckEsp)

2 0 [ 5] Fibo!_RTC_CheckEsp

70 3 [ 4] Fibo!FiboRecursive

71 74 [ 3] Fibo!FiboRecursive

1 0 [ 4] Fibo!ILT+5(?FiboRecursiveYAIIHZ)

67 0 [ 4] Fibo!FiboRecursive

1 0 [ 5] Fibo!ILT+340(__RTC_CheckEsp)

2 0 [ 5] Fibo!_RTC_CheckEsp

70 3 [ 4] Fibo!FiboRecursive

79 148 [ 3] Fibo!FiboRecursive

1 0 [ 4] Fibo!ILT+340(__RTC_CheckEsp)

2 0 [ 4] Fibo!_RTC_CheckEsp

82 151 [ 3] Fibo!FiboRecursive

79 628 [ 2] Fibo!FiboRecursive

1 0 [ 3] Fibo!ILT+340(__RTC_CheckEsp)

2 0 [ 3] Fibo!_RTC_CheckEsp

82 631 [ 2] Fibo!FiboRecursive

71 714 [ 1] Fibo!FiboRecursive

1 0 [ 2] Fibo!ILT+5(?FiboRecursiveYAIIHZ)

64 0 [ 2] Fibo!FiboRecursive

1 0 [ 3] Fibo!ILT+5(?FiboRecursiveYAIIHZ)

64 0 [ 3] Fibo!FiboRecursive

1 0 [ 4] Fibo!ILT+5(?FiboRecursiveYAIIHZ)

67 0 [ 4] Fibo!FiboRecursive

1 0 [ 5] Fibo!ILT+340(__RTC_CheckEsp)

2 0 [ 5] Fibo!_RTC_CheckEsp

70 3 [ 4] Fibo!FiboRecursive

71 74 [ 3] Fibo!FiboRecursive

1 0 [ 4] Fibo!ILT+5(?FiboRecursiveYAIIHZ)

67 0 [ 4] Fibo!FiboRecursive

1 0 [ 5] Fibo!ILT+340(__RTC_CheckEsp)

2 0 [ 5] Fibo!_RTC_CheckEsp

70 3 [ 4] Fibo!FiboRecursive

79 148 [ 3] Fibo!FiboRecursive

1 0 [ 4] Fibo!ILT+340(__RTC_CheckEsp)

2 0 [ 4] Fibo!_RTC_CheckEsp

82 151 [ 3] Fibo!FiboRecursive

71 234 [ 2] Fibo!FiboRecursive

1 0 [ 3] Fibo!ILT+5(?FiboRecursiveYAIIHZ)

67 0 [ 3] Fibo!FiboRecursive

1 0 [ 4] Fibo!ILT+340(__RTC_CheckEsp)

2 0 [ 4] Fibo!_RTC_CheckEsp

70 3 [ 3] Fibo!FiboRecursive

79 308 [ 2] Fibo!FiboRecursive

1 0 [ 3] Fibo!ILT+340(__RTC_CheckEsp)

2 0 [ 3] Fibo!_RTC_CheckEsp

82 311 [ 2] Fibo!FiboRecursive

79 1108 [ 1] Fibo!FiboRecursive

1 0 [ 2] Fibo!ILT+340(__RTC_CheckEsp)

2 0 [ 2] Fibo!_RTC_CheckEsp

82 1111 [ 1] Fibo!FiboRecursive

1196 instructions were executed in 1195 events (0 from other threads)

Function Name Invocations MinInst MaxInst AvgInst

Fibo!FiboRecursive 15 70 82 75

Fibo!ILT+340(__RTC_CheckEsp) 15 1 1 1

Fibo!ILT+5(?FiboRecursiveYAIIHZ) 15 1 1 1

Fibo!_RTC_CheckEsp 15 2 2 2

Fibo!wmain 1 2 2 2

0 system calls were executed

eax=00000005 ebx=7efde000 ecx=00000000 edx=00000001 esi=00000000 edi=00000000

eip=001b146f esp=0045f914 ebp=0045f964 iopl=0 nv up ei pl zr na pe nc

cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000246

Fibo!wmain+0x2f:

001b146f 83c408 add esp,8

Notice that the first number right below the command specifies the number of instructions that were executed, the second number specifies the number of instructions executed by child calls of the function, and the third number (in brackets) is the depth of the function in the stack.

MinInst, MaxInst and AvgInst mean the number of instructions by call.

Special Command—Saving Modules Using .writemem

rafarah — Wed, 23 Sep 2009 02:00:00 GMT

This command enables you to save memory into a disk file. The cool thing about it is that you can save modules too; however, it is just the raw memory.

The parameters are:

.writemem <filename> <range>

Here is an example:

0:026> lm

start end module name

00400000 00427000 mtgdi (deferred)

5a700000 5acaf000 mfc90d (deferred)

63df0000 63f13000 MSVCR90D (deferred)

71270000 71283000 dwmapi (deferred)

72cf0000 72d70000 UxTheme (deferred)

73470000 73475000 MSIMG32 (deferred)

73b90000 73b9d000 MFC90ENU (deferred)

74fd0000 75053000 COMCTL32 (deferred)

751d0000 751dc000 CRYPTBASE (deferred)

751e0000 75240000 SspiCli (deferred)

75240000 75259000 sechost (deferred)

75260000 75ea6000 SHELL32 (deferred)

75ee0000 75f8c000 msvcrt (deferred)

75fd0000 76060000 GDI32 (deferred)

76150000 76250000 kernel32 (deferred)

76250000 762ed000 USP10 (deferred)

763b0000 76410000 IMM32 (deferred)

76410000 7649f000 OLEAUT32 (deferred)

764a0000 764e4000 KERNELBASE (deferred)

765c0000 766b0000 RPCRT4 (deferred)

766b0000 76733000 CLBCatQ (deferred)

76a00000 76aa0000 ADVAPI32 (deferred)

76ce0000 76d37000 SHLWAPI (deferred)

76f40000 77040000 USER32 (deferred)

77040000 7710c000 MSCTF (deferred)

77110000 7726b000 ole32 (deferred)

77640000 7764a000 LPK (deferred)

Now let’s save the MFC90ENU.DLL above.

0:026> .writemem c:\downloads\MFC90ENU.dll 73b90000 (73b9d000 - 0x1)

Writing d000 bytes..........................

Note the “- 0x1” above. If you don’t do that, the command will fail because the debugger will try to write from the base address to and including the end address.

Another approach you can use is to get the size of the module and use it as one of the parameters:

0:026> ? 73b9d000 - 73b90000

Evaluate expression: 53248 = 0000d000

0:026> .writemem c:\downloads\MFC90ENU.dll 73b90000 L 0000d000

Writing d000 bytes..........................

Special Command—Using .dump/.dumpcab to Get Dumps and Symbols from Production Servers

rafarah — Wed, 16 Sep 2009 02:55:00 GMT

Using WinDbg you can create a dump file from an application running, for instance, in a production server. After collecting the dump file, you can load it in another machine and debug it. However, to be more effective during your debugging session you need symbols. Thus, thinking about it, here's the trick to get both dump and symbols.

First, to get a dump file with all information use it:

0:000> .dump /mfht c:\Test.dmp

Creating c:\Test.dmp - mini user dump

Dump successfully written

.dump has several arguments, but if you want a mini-dump with all information you just need to use:

/mfht

Where:

/m – This is to use options.

f – Adds full memory data to the minidump. All accessible committed pages owned by the target application will be included.

h – Adds data about the handles associated with the target application to the minidump.

t – Adds additional thread information to the minidump. This includes thread times, which can be displayed by using the !runaway extension or the .ttime command when debugging the minidump. (now you understand why your !runaway command failed that day, huh? :) )

Note: For more arguments check the WinDbg help file.

After collecting the dump file, open it from the production server using WinDbg and reload the symbols again.

Go to the command line and use the .dumpcab command.

The arguments for .dumpcab are –a (force all symbols) and the file name itself.

Here is how to do that:

0:003> .dumpcab -a c:\fulldump

Creating a cab file can take a VERY VERY long time

.Ctrl-C can only interrupt the command after a file has been added to the cab.

Adding C:\test.dmp - added

Adding c:\publicsymbols\wntdll.pdb\E06BEA155E9748BEA818E2D0DD2FED952\wntdll.pdb - added

Wrote c:\downloads\fulldump

After doing that the file FullDump.CAB will have the dump file and all related symbols, so you can transport it to your machine, unpack, load the dumps/symbols, and start debugging it!

Special Command—Using !chksym/!itoldyouso to Check PDB Files Against Modules

rafarah — Fri, 04 Sep 2009 20:46:00 GMT

These are two debugger extensions that are used to see the PDB file that matches a specific module.

Note that !itoldyouso is not documented. The output of both commands is identical.

Usage:

0:025> !chksym ntdll

ntdll.dll

Timestamp: 49EEA706

SizeOfImage: 180000

pdb: wntdll.pdb

pdb sig: E06BEA15-5E97-48BE-A818-E2D0DD2FED95

age: 2

Loaded pdb is c:\publicsymbols\wntdll.pdb\E06BEA155E9748BEA818E2D0DD2FED952\wntdll.pdb

wntdll.pdb

pdb sig: E06BEA15-5E97-48BE-A818-E2D0DD2FED95

age: 2

MATCH: wntdll.pdb and ntdll.dll

0:025> !itoldyouso ntdll

ntdll.dll

Timestamp: 49EEA706

SizeOfImage: 180000

pdb: wntdll.pdb

pdb sig: E06BEA15-5E97-48BE-A818-E2D0DD2FED95

age: 2

Loaded pdb is c:\publicsymbols\wntdll.pdb\E06BEA155E9748BEA818E2D0DD2FED952\wntdll.pdb

wntdll.pdb

pdb sig: E06BEA15-5E97-48BE-A818-E2D0DD2FED95

age: 2

MATCH: wntdll.pdb and ntdll.dll

0:025> !chksym mtgdi

mtgdi.exe

Timestamp: 48785A80

SizeOfImage: 27000

pdb: c:\DOWNLOADS\mtgdi\Debug\mtgdi.pdb

pdb sig: EC1B3DB2-25C1-4337-8676-DFB3C5B1C8C9

age: 3

Loaded pdb is C:\development\My Tools\Book\mtgdi\Debug\mtgdi.pdb

mtgdi.pdb

pdb sig: EC1B3DB2-25C1-4337-8676-DFB3C5B1C8C9

age: 3

MATCH: mtgdi.pdb and mtgdi.exe

0:025> !itoldyouso mtgdi

mtgdi.exe

Timestamp: 48785A80

SizeOfImage: 27000

pdb: c:\DOWNLOADS\mtgdi\Debug\mtgdi.pdb

pdb sig: EC1B3DB2-25C1-4337-8676-DFB3C5B1C8C9

age: 3

Loaded pdb is C:\development\My Tools\Book\mtgdi\Debug\mtgdi.pdb

mtgdi.pdb

pdb sig: EC1B3DB2-25C1-4337-8676-DFB3C5B1C8C9

age: 3

MATCH: mtgdi.pdb and mtgdi.exe

Special Command—Displaying Information From Modules/DLLs with !dlls

rafarah — Thu, 20 Aug 2009 05:10:00 GMT

!dlls extension displays the table entries of all loaded modules. You can also use it to display all modules that a specified thread or process is using.

The WinDbg help file describes all parameters. Here we are going to show the most common usage.

Displays file headers and section headers:

!dlls –a

0:801> !dlls –a

0x00543598: C:\development\My Tools\Book\mtgdi\Debug\MtGdi.exe

Base 0x00400000 EntryPoint 0x00411929 Size 0x00027000

Flags 0x00004000 LoadCount 0x0000ffff TlsIndex 0x00000000

LDRP_ENTRY_PROCESSED

File Type: EXECUTABLE IMAGE

FILE HEADER VALUES

14C machine (i386)

6 number of sections

48785A80 time date stamp Sat Jul 12 00:17:20 2008

0 file pointer to symbol table

0 number of symbols

E0 size of optional header

103 characteristics

Relocations stripped

Executable

32 bit word machine

OPTIONAL HEADER VALUES

10B magic #

9.00 linker version

C400 size of code

7C00 size of initialized data

0 size of uninitialized data

11929 address of entry point

1000 base of code

1000 base of data

----- new -----

00400000 image base

1000 section alignment

200 file alignment

2 subsystem (Windows GUI)

5.00 operating system version

0.00 image version

5.00 subsystem version

27000 size of image

400 size of headers

0 checksum

00100000 size of stack reserve

00001000 size of stack commit

00100000 size of heap reserve

00001000 size of heap commit

00400100 Opt Hdr

0 [ 0] address [size] of Export Directory

23000 [ 8C] address [size] of Import Directory

25000 [ 1E7C] address [size] of Resource Directory

0 [ 0] address [size] of Exception Directory

0 [ 0] address [size] of Security Directory

0 [ 101] address [size] of Base Relocation Directory

1E940 [ 1C] address [size] of Debug Directory

0 [ 0] address [size] of Description Directory

0 [ 0] address [size] of Special Directory

0 [ 0] address [size] of Thread Storage Directory

0 [ 0] address [size] of Load Configuration Directory

0 [ 0] address [size] of Bound Import Directory

23884 [ 7F8] address [size] of Import Address Table Directory

0 [ 0] address [size] of Reserved Directory

SECTION HEADER #1

name

0 virtual size

0 virtual address

0 size of raw data

0 file pointer to raw data

0 file pointer to relocation table

0 file pointer to line numbers

0 number of relocations

0 number of line numbers

0 flags

(no align specified)

SECTION HEADER #2

name

0 virtual size

0 virtual address

0 size of raw data

0 file pointer to raw data

0 file pointer to relocation table

0 file pointer to line numbers

0 number of relocations

0 number of line numbers

0 flags

(no align specified)

SECTION HEADER #3

name

0 virtual size

0 virtual address

0 size of raw data

0 file pointer to raw data

0 file pointer to relocation table

0 file pointer to line numbers

0 number of relocations

0 number of line numbers

0 flags

(no align specified)

SECTION HEADER #4

name

0 virtual size

0 virtual address

0 size of raw data

0 file pointer to raw data

0 file pointer to relocation table

0 file pointer to line numbers

0 number of relocations

0 number of line numbers

0 flags

(no align specified)

SECTION HEADER #5

name

0 virtual size

0 virtual address

0 size of raw data

0 file pointer to raw data

0 file pointer to relocation table

0 file pointer to line numbers

0 number of relocations

0 number of line numbers

0 flags

(no align specified)

SECTION HEADER #6

name

0 virtual size

0 virtual address

0 size of raw data

0 file pointer to raw data

0 file pointer to relocation table

0 file pointer to line numbers

0 number of relocations

0 number of line numbers

0 flags

(no align specified)

Displays version numbers:

!dlls –v

0:801> !dlls -v

0x00543598: C:\development\My Tools\Book\mtgdi\Debug\MtGdi.exe

Base 0x00400000 EntryPoint 0x00411929 Size 0x00027000

Flags 0x00004000 LoadCount 0x0000ffff TlsIndex 0x00000000

LDRP_ENTRY_PROCESSED

Product Name MTGDI Application

Product Version 1, 0, 0, 1

Original Filename MTGDI.EXE

File Description MTGDI MFC Application

File Version 1, 0, 0, 1

0x00543628: C:\Windows\SysWOW64\ntdll.dll

Base 0x77630000 EntryPoint 0x00000000 Size 0x00180000

Flags 0x80004004 LoadCount 0x0000ffff TlsIndex 0x00000000

LDRP_IMAGE_DLL

LDRP_ENTRY_PROCESSED

Company Name Microsoft Corporation

Product Name Microsoft® Windows® Operating System

Product Version 6.1.7100.0

Original Filename ntdll.dll

File Description NT Layer DLL

File Version 6.1.7100.0 (winmain_win7rc.090421-1700)

0x005439a8: C:\Windows\syswow64\kernel32.dll

Base 0x769d0000 EntryPoint 0x769e3e8a Size 0x00100000

Flags 0x80084004 LoadCount 0x0000ffff TlsIndex 0x00000000

LDRP_IMAGE_DLL

LDRP_ENTRY_PROCESSED

LDRP_PROCESS_ATTACH_CALLED

Company Name Microsoft Corporation

Product Name Microsoft® Windows® Operating System

Product Version 6.1.7100.0

Original Filename kernel32

File Description Windows NT BASE API Client DLL

File Version 6.1.7100.0 (winmain_win7rc.090421-1700)

0x00543ac0: C:\Windows\syswow64\KERNELBASE.dll

Base 0x76ad0000 EntryPoint 0x76ad563f Size 0x00044000

Flags 0x80084004 LoadCount 0x0000ffff TlsIndex 0x00000000

LDRP_IMAGE_DLL

LDRP_ENTRY_PROCESSED

LDRP_PROCESS_ATTACH_CALLED

Company Name Microsoft Corporation

Product Name Microsoft® Windows® Operating System

Product Version 6.1.7100.0

Original Filename Kernelbase

File Description Windows NT BASE API Client DLL

File Version 6.1.7100.0 (winmain_win7rc.090421-1700)

Using Module Address to display information from a specific dll:

!dlls –c <moduleAddress>

0:801> !dlls -c 63390000

Dump dll containing 0x63390000:

0x00544998: C:\Windows\WinSxS\x86_microsoft.vc90.debugcrt_1fc8b3b9a1e18e3b_9.0.30729.1_none_bb1f6aa1308c35eb\MSVCR90D.dll

Base 0x63390000 EntryPoint 0x633cc6f0 Size 0x00123000

Flags 0x90084004 LoadCount 0x0000ffff TlsIndex 0x00000000

LDRP_IMAGE_DLL

LDRP_ENTRY_PROCESSED

LDRP_PROCESS_ATTACH_CALLED

LDRP_REDIRECTED

Special Command—Using !for_each_frame to Run Commands

rafarah — Thu, 20 Aug 2009 05:07:00 GMT

!for_each_frame is a favorite among debuggers. It's a very flexible and powerful command that enables you to run commands for each frame of the call stack.

You can use basically any command.

For instance, let’s say you want to see all local variables from each frame of a specific stack. Of course, to see local variables you must have private symbols.

0:025> !for_each_frame dv /i /t /V

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

00 0532fcd0 776f8edc ntdll!NtWaitForSingleObject+0x15

Unable to enumerate locals, HRESULT 0x80004005

Private symbols (symbols.pri) are required for locals.

Type ".hh dbgerr005" for details.

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

01 0532fd34 776f8dc0 ntdll!RtlpWaitOnCriticalSection+0x13e

Unable to enumerate locals, HRESULT 0x80004005

Private symbols (symbols.pri) are required for locals.

Type ".hh dbgerr005" for details.

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

02 0532fd5c 00414ee7 ntdll!RtlEnterCriticalSection+0x150

Unable to enumerate locals, HRESULT 0x80004005

Private symbols (symbols.pri) are required for locals.

Type ".hh dbgerr005" for details.

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

03 0532fdc8 00414834 mtgdi!CBallThread::SingleStep+0x147 [c:\downloads\mtgdi\threads.cpp @ 180]

prv local 0532fdc4 @ebp-0x04 class CBallThread * this = 0x043349f0

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

04 0532fe20 00414d8c mtgdi!CGDIThread::InitInstance+0x44 [c:\downloads\mtgdi\threads.cpp @ 65]

prv local 0532fe1c @ebp-0x04 class CGDIThread * this = 0x043349f0

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

05 0532fe78 5a67d3e3 mtgdi!CBallThread::InitInstance+0x2c [c:\downloads\mtgdi\threads.cpp @ 156]

prv local 0532fe74 @ebp-0x04 class CBallThread * this = 0x043349f0

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

06 0532ff40 6d97dfd3 mfc90d!_AfxThreadEntry+0x303

Unable to enumerate locals, HRESULT 0x80004005

Private symbols (symbols.pri) are required for locals.

Type ".hh dbgerr005" for details.

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

07 0532ff7c 6d97df69 MSVCR90D!_callthreadstartex+0x53

Unable to enumerate locals, HRESULT 0x80004005

Private symbols (symbols.pri) are required for locals.

Type ".hh dbgerr005" for details.

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

08 0532ff88 75ec3f39 MSVCR90D!_threadstartex+0x89

Unable to enumerate locals, HRESULT 0x80004005

Private symbols (symbols.pri) are required for locals.

Type ".hh dbgerr005" for details.

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

09 0532ff94 77710409 kernel32!BaseThreadInitThunk+0xe

Unable to enumerate locals, HRESULT 0x80004005

Private symbols (symbols.pri) are required for locals.

Type ".hh dbgerr005" for details.

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

0a 0532ffd4 777103dc ntdll!__RtlUserThreadStart+0x70

Unable to enumerate locals, HRESULT 0x80004005

Private symbols (symbols.pri) are required for locals.

Type ".hh dbgerr005" for details.

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

0b 0532ffec 00000000 ntdll!_RtlUserThreadStart+0x1b

Unable to enumerate locals, HRESULT 0x80004005

Private symbols (symbols.pri) are required for locals.

Type ".hh dbgerr005" for details.

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

00 0532fcd0 776f8edc ntdll!NtWaitForSingleObject+0x15

Special Command—Displaying More PE Header Information with !dh

rafarah — Thu, 20 Aug 2009 05:05:00 GMT

The !dh extension displays the PE header information from a specified module.

Usage:

!dh [options] <addressOfModule>

Options can be:

-f Displays file headers.

-s Displays section headers.

-a Displays all header information.

Example:

0:532> lm

start end module name

00400000 00427000 mtgdi (deferred)

5a700000 5acaf000 mfc90d (deferred)

692e0000 69403000 MSVCR90D (deferred)

71270000 71283000 dwmapi (deferred)

72cf0000 72d70000 UxTheme (deferred)

73470000 73475000 MSIMG32 (deferred)

73b50000 73b5d000 MFC90ENU (deferred)

74fd0000 75053000 COMCTL32 (deferred)

751d0000 751dc000 CRYPTBASE (deferred)

751e0000 75240000 SspiCli (deferred)

75240000 75259000 sechost (deferred)

75260000 75ea6000 SHELL32 (deferred)

75ee0000 75f8c000 msvcrt (deferred)

75fd0000 76060000 GDI32 (deferred)

76150000 76250000 kernel32 (deferred)

76250000 762ed000 USP10 (deferred)

763b0000 76410000 IMM32 (deferred)

76410000 7649f000 OLEAUT32 (deferred)

764a0000 764e4000 KERNELBASE (deferred)

765c0000 766b0000 RPCRT4 (deferred)

766b0000 76733000 CLBCatQ (deferred)

76a00000 76aa0000 ADVAPI32 (deferred)

76ce0000 76d37000 SHLWAPI (deferred)

76f40000 77040000 USER32 (deferred)

77040000 7710c000 MSCTF (deferred)

77110000 7726b000 ole32 (deferred)

77640000 7764a000 LPK (deferred)

Now we use the start address as argument:

0:532> !dh -a 5a700000

File Type: DLL

FILE HEADER VALUES

14C machine (i386)

4 number of sections

488F15C6 time date stamp Tue Jul 29 06:06:14 2008

0 file pointer to symbol table

0 number of symbols

E0 size of optional header

2102 characteristics

Executable

32 bit word machine

DLL

OPTIONAL HEADER VALUES

10B magic #

9.00 linker version

45B600 size of code

151A00 size of initialized data

0 size of uninitialized data

3F66C0 address of entry point

1000 base of code

----- new -----

5a700000 image base

1000 section alignment

200 file alignment

3 subsystem (Windows CUI)

5.00 operating system version

9.00 image version

5.00 subsystem version

5AF000 size of image

400 size of headers

5B030B checksum

00100000 size of stack reserve

00001000 size of stack commit

00100000 size of heap reserve

00001000 size of heap commit

140 DLL characteristics

Dynamic base

NX compatible

44D0A0 [ F4A5] address [size] of Export Directory

448DB8 [ A0] address [size] of Import Directory

46B000 [ 106C18] address [size] of Resource Directory

0 [ 0] address [size] of Exception Directory

5A7400 [ 23F8] address [size] of Security Directory

572000 [ 38D08] address [size] of Base Relocation Directory

21D0 [ 1C] address [size] of Debug Directory

0 [ 0] address [size] of Description Directory

0 [ 0] address [size] of Special Directory

0 [ 0] address [size] of Thread Storage Directory

59310 [ 40] address [size] of Load Configuration Directory

0 [ 0] address [size] of Bound Import Directory

1000 [ CEC] address [size] of Import Address Table Directory

4471A4 [ 200] address [size] of Delay Import Directory

0 [ 0] address [size] of COR20 Header Directory

0 [ 0] address [size] of Reserved Directory

SECTION HEADER #1

.text name

45B545 virtual size

1000 virtual address

45B600 size of raw data

400 file pointer to raw data

0 file pointer to relocation table

0 file pointer to line numbers

0 number of relocations

0 number of line numbers

60000020 flags

Code

(no align specified)

Execute Read

Debug Directories(1)

Type Size Address Pointer

cv 28 59358 58758 Format: RSDS, guid, 17, mfc90d.i386.pdb

SECTION HEADER #2

.data name

DC3C virtual size

45D000 virtual address

7E00 size of raw data

45BA00 file pointer to raw data

0 file pointer to relocation table

0 file pointer to line numbers

0 number of relocations

0 number of line numbers

C0000040 flags

Initialized Data

(no align specified)

Read Write

SECTION HEADER #3

.rsrc name

106C18 virtual size

46B000 virtual address

106E00 size of raw data

463800 file pointer to raw data

0 file pointer to relocation table

0 file pointer to line numbers

0 number of relocations

0 number of line numbers

40000040 flags

Initialized Data

(no align specified)

Read Only

SECTION HEADER #4

.reloc name

3CCD4 virtual size

572000 virtual address

3CE00 size of raw data

56A600 file pointer to raw data

0 file pointer to relocation table

0 file pointer to line numbers

0 number of relocations

0 number of line numbers

42000040 flags

Initialized Data

Discardable

(no align specified)

Read Only

Special Command—Displaying the PE Header Information with !lmi

rafarah — Thu, 20 Aug 2009 04:59:00 GMT

Like its cousin !dh, the !lmi extension displays the PE header information from a specified module. However, it gives you fewer details than !dh. The output is summarized.

Usage:

!lmi <moduleName>

Examples:

0:532> !lmi mtgdi

Loaded Module Info: [mtgdi]

Module: mtgdi

Base Address: 00400000

Image Name: mtgdi.exe

Machine Type: 332 (I386)

Time Stamp: 48785a80 Sat Jul 12 00:17:20 2008

Size: 27000

CheckSum: 0

Characteristics: 103

Debug Data Dirs: Type Size VA Pointer

CODEVIEW 3b, 200dc, e8dc RSDS - GUID: {EC1B3DB2-25C1-4337-8676-DFB3C5B1C8C9}

Age: 3, Pdb: c:\DOWNLOADS\mtgdi\Debug\mtgdi.pdb

Image Type: FILE - Image read successfully from debugger.

C:\development\My Tools\Book\mtgdi\Debug\MtGdi.exe

Symbol Type: PDB - Symbols loaded successfully from image path.

C:\development\My Tools\Book\mtgdi\Debug\mtgdi.pdb

Compiler: Resource - front end [0.0 bld 0] - back end [9.0 bld 21022]

Load Report: private symbols & lines, not source indexed

C:\development\My Tools\Book\mtgdi\Debug\mtgdi.pdb

0:532> !lmi ole32

Loaded Module Info: [ole32]

Module: ole32

Base Address: 77110000

Image Name: C:\Windows\syswow64\ole32.dll

Machine Type: 332 (I386)

Time Stamp: 49eea66c Tue Apr 21 22:09:00 2009

Size: 15b000

CheckSum: 1607b7

Characteristics: 2102 perf

Debug Data Dirs: Type Size VA Pointer

CODEVIEW 22, 138b1c, 13831c RSDS - GUID: {D66D525C-3DF2-47C7-AB77-594C4E5E2325}

Age: 2, Pdb: ole32.pdb

CLSID 4, 138b18, 138318 [Data not mapped]

Image Type: FILE - Image read successfully from debugger.

C:\Windows\syswow64\ole32.dll

Symbol Type: PDB - Symbols loaded successfully from symbol server.

c:\publicsymbols\ole32.pdb\D66D525C3DF247C7AB77594C4E5E23252\ole32.pdb

Load Report: public symbols , not source indexed

c:\publicsymbols\ole32.pdb\D66D525C3DF247C7AB77594C4E5E23252\ole32.pdb

Note the Base Address above. You can get the base address from a module using different commands like lm or even dd.

dd is supposed to Display DWORDs but it solves module names.

Look:

0:532> dd ole32 L1

77110000 00905a4d

0:532> dd mtgdi L1

00400000 00905a4d

Special Command—CPU Time for Each Thread with !runaway

rafarah — Thu, 20 Aug 2009 04:55:00 GMT

This is one of my favorite commands! !runaway displays information about the CPU time consumed by each thread in User Mode and Kernel Mode.

It is one of those commands you run when you think the application is hung with low or high CPU or has some kind of performance issue.

Parameters:

Bit 0 (0x1)

Causes the debugger to show the amount of user time consumed by each thread.

Bit 1 (0x2)

Causes the debugger to show the amount of kernel time consumed by each thread.

Bit 2 (0x4)

Causes the debugger to show the amount of time that has elapsed since each thread was created.

The output is sorted from the thread consuming the most CPU time to the thread consuming the least CPU time.

Example of displaying just the User Time for each thread (no parameters does that):

0:000> !runaway

User Mode Time

Thread Time

52:33b4 0 days 0:00:01.435

317:2834 0 days 0:00:00.982

93:2d60 0 days 0:00:00.936

479:3858 0 days 0:00:00.842

266:33b0 0 days 0:00:00.842

658:23d8 0 days 0:00:00.826

559:1acc 0 days 0:00:00.826

299:357c 0 days 0:00:00.826

279:bf8 0 days 0:00:00.826

719:39c0 0 days 0:00:00.811

509:38d0 0 days 0:00:00.811

376:19a0 0 days 0:00:00.811

314:3700 0 days 0:00:00.811

84:338c 0 days 0:00:00.811

731:2fa0 0 days 0:00:00.795

691:2070 0 days 0:00:00.795

24:1b88 0 days 0:00:00.795

800:3ca4 0 days 0:00:00.780

761:3c08 0 days 0:00:00.780

720:3968 0 days 0:00:00.780

. . .

[Recommended Books] Advanced .NET Debugging book – Preview

rafarah — Sat, 16 May 2009 06:22:00 GMT

For the second time I have had the privilege of being one of the reviewers for another debugging book. This time I’m referring to Mario Hewardt’s new book: Advanced .NET Debugging.

It was a great learning process. After months of reviewing chapters and source code and giving suggestions, I feel my .NET debugging knowledge is not the same. It is much better now!

Let me tell you why I’m so thrilled with this book.

First of all, this book discusses in depth several tools used by our product teams when isolating internal bugs from .NET application and also the tools used by CSS (Customer Service and Support). In case you ask, they are all public tools!

Also the book gives a deep and clear explanation about the CLR (Common Language Runtime). In my opinion, that is one of the most important parts of the book. If you are starting with .NET Debugging, you may be familiar with some of the SOS commands. That’s the easy part. However, if you want to improve your skills as a debugger you must go deeper and understand the CLR internals, like method table, method descriptor, memory management, etc. The chapter about CLR explains the internals in a superb way, and it’s my favorite!

If you’re new to debugging, you’re going to like the chapter about Basic Debugging Tasks. It’s didactic enough for beginners, demonstrating how to use the most used commands and how to get useful information from basic commands.

Another part of the book talks about more complex scenarios and technologies. It’s a deep explanation! You’re going to learn a lot about Managed Heap, Garbage Collection, Thread Synchronization and Interoperability. Based on my experiences helping customers from different countries, I can tell you the number one application problem from .NET applications: memory issues. I know it’s ironic since GC is supposed to manage memory, so let me explain: GC does the job very well. The problem is mixed code: managed and native. The GC only has control of the managed code. The book explains it very well, and there are lots of sample codes reproducing different issues scattered among the chapters. This is cool because you can use the sample code to apply the knowledge while debugging specific scenarios.

Like Mario’s previous book, this one is also very pragmatic. You won’t see those situations that are unlikely to happen on a production server, actually the opposite: you’ll see issues that we face on a daily basis with the approaches, techniques and tools we use when debugging these issues. Very helpful!

"Fortunately, Mario's book combines just the right amount of explanation of the runtime to help you understand the thought process and the techniques he uses to solve problems with many practical and clever tricks learned from debugging real world applications. So if you want to quickly get up to speed debugging your CLR applications, this is definitely the book to read. It covers many aspects of debugging managed applications—in particular, it gives insights into areas that are hard to diagnose such as thread synchronization problems. The illustration of debugging techniques is largely done with examples which makes it easier to follow." (Excerpted from the Foreword by Patrick Dussud, Technical Fellow, CLR, .NET FX, Microsoft Corporation.)

Here is the index of the book:

Part I – Overview

Part I introduces the reader to the basic concepts required to efficiently debug .NET applications using the Debugging Tools for Windows package.

1. Introduction to the Tools

This chapter contains a brief introduction to the tools used throughout the book, including basic usage scenarios, download locations, and installation instructions. Amongst the tools covered are:

Debugging Tools for Windows

.NET 2.0 redist/SDK

SOS

SOSEX

CLR Profiler

Performance Counters

Reflector for .NET

PowerDBG

Managed Debugging Assistants

2. CLR Fundamentals

This chapter discusses the core fundamentals of the CLR. The chapter begins with a high level overview of the major runtime components and subsequently drills down into the details and covers topics such as:

· CLR and the Windows Loader. Discusses how .NET applications are natively supported using the PE file format.

· Application domains (system, shared, default, custom) as well as loader heaps etc.

· Assembly overview

· Assembly manifest

· Type metadata

· And more…

3. Basic Debugging Tasks

Chapter 3 gives hands on examples of the basic .NET debugging tasks such as:

· Thread management

· Display stack traces

· Display local variables

· Breakpoint management

· Dumping heap based objects

· .NET exception model and how to analyze in the debuggers

· Basics of postmortem debugging

· Remote debugging

Part II – Applied Debugging

Par t II is a practical approach to debugging common .NET problems encountered in the development process. Each chapter is dedicated to a specific CLR component and/or technology and begins by detailing the inner workings of that component. Subsequently, a number of common problems related to the specific technologies are discussed and how the debuggers and tools can be used for root cause analysis.

4. Assembly Loader

The complexity of .NET applications can range from simple command line applications to complex multi-process/multi machine server applications with a large number of assemblies living in harmony. In order to efficiently debug problems in .NET applications one much be careful to understand the dependencies of .NET assemblies. This chapter takes a look at how the CLR assembly loader does its work and common problems surrounding that area.

5. Managed Heap and Garbage Collection

While .NET developers can enjoy the luxury of automatic memory management, care must still be taken to avoid costly mistakes. The highly sophisticated CLR garbage collector is an automatic memory manager that allows developers to focus less on memory management and more on application logic. Even though the CLR manages memory for the developer, care must be taken to avoid pitfalls that can wreak havoc in your applications. In this chapter we look at how the garbage collector works, how to peek into the internals of the garbage collector and some common programming mistakes related to automatic garbage collection (non memory resources, memory fragmentation, finalizer problems etc).

6. Synchronization

A multithreaded environment allows for a great deal of flexibility and efficiency. With this flexibility comes a lot of complexity in the form of thread management. To avoid costly mistakes in your application, care must be taken to ensure that threads perform their work in an orchestrated fashion. This chapter introduces the synchronization primitives available in .NET and discusses how the debuggers and tools can be used to debug common thread synchronization problems. Scenarios such as deadlocks, lock contentions, sync blocks, thin locks and thread pool problems are discussed.

7. Interoperability

.NET relies heavily on underlying Windows components. In order to invoke the native Windows components the CLR exposes two primary methods of interoperability:

1. Platform Invocation

2. COM Interoperability

Since the.NET and Win32 programming models are often very different, idiosyncrasies often lead to hard to track down problems. In this chapter we look at some very common mistakes done when working in the Interoperability layer and how to use the debuggers and tools to troubleshoot the problems.

Part III – Advanced Topics

8. Postmortem Debugging

Quite often it’s not feasible to expect full access to a failing machine so that a problem can be debugged. Bugs that surface on production machines on customer sites are rarely available for debugging. This chapter outlines the mechanisms for debugging a problem without access to the physical machine. Topics discussed include the basics of crash dumps, generating crash dumps, analyzing crash dumps etc.

9. Power Tools

Introduces the reader to power tools such as Debugging via Powershell , IISDiag and more.

10. CLR 4.0

This chapter is reserved for any substantial enhancements coming out while the book is being authored/published.

Index

[PowerShell Script] Statistics from .NET Applications

rafarah — Wed, 15 Apr 2009 08:52:00 GMT

This script is more a template to show you how to use PowerDbg.

I must say the idea is from my teammate Aaron Barth!

This script collects information from all threads running managed code and gives the user statistics by threads like:

- CLR stack.

- Managed objects from the stack.

- ASP.NET page.

- What the thread is doing.

- Exceptions by threads.

- Threads running ASP.NET pages.

Contrary to what you may think, this script is very simple. It’s very easy to customize or improve it. For example, you may want to display the ASP.NET pages or queries/stored procedures by threads. Or you may want to create a report using HTML. The output from this script is text, simple and straight. Again, the idea is to show you how to use the cmdlets from PowerDbg in order to create scripts that automate the debugging session.

Screenshots:

Source code for PowerShellScriptASPXStatistics.ps1:

########################################################################################################

# Script: PowerDbgScriptASPXStatistics

# Parameters: None.

# Purpose: Shows statistics from threads running ASP.NET pages.

# Attention! This script was not tested on Win64.

# Changes History: 04/15/2009 – Fixed bug when there’s just one managed thread.

# Roberto Alexis Farah

# All my functions are provided "AS IS" with no warranties, and confer no rights.

########################################################################################################

set-psdebug -strict

$ErrorActionPreference = "stop"

trap {"Error message: $_"}

write-Host "Scanning all threads and extracting the CLR stack..." -foreground Green -background Black

# First, let's scan all threads and identify those running managed code.

Send-PowerDbgCommand "~* e !clrstack"

Parse-PowerDbgCLRSTACK

# Get all the stacks running managed code.

$clrStack = Convert-PowerDbgCSVToHashTable

write-Host "Done!" -foreground Green -background Black

# Sorts the keys by Thread Number and save them into an array.

$arrayOfThreads = @($clrStack.keys | Sort-Object {[int] $_})

# Let's consider the situation where the dump has no thread running managed code.

if($arrayOfThreads.Count -eq 0)

{

write-Host "There are not threads running managed code!" -foreground Red -background Black

return

}

write-Host "Scanning all threads and extracting the managed objects..." -foreground Green -background Black

Send-PowerDbgCommand "~* e !dso"

Parse-PowerDbgDSO

$dso = Convert-PowerDbgCSVToHashTable

write-Host "Done!" -foreground Green -background Black

write-Host "Collecting information about each thread..." -foreground Green -background Black

Send-PowerDbgCommand "!Threads"

Parse-PowerDbgTHREADS

$threads = Convert-PowerDbgCSVToHashTable

write-Host "Done!" -foreground Green -background Black

write-Host "Collecting information from threads running ASP.NET..." -foreground Green -background Black

Send-PowerDbgCommand "!ASPXPages"

Parse-PowerDbgASPXPAGES

$aspxPages = Convert-PowerDbgCSVToHashTable

write-Host "Done!" -foreground Green -background Black

write-Host "Scanning all threads and preparing statistics..." -foreground Green -background Black

# Scans all threads running managed code.

for($i = 0; $i -lt $arrayOfThreads.Length; $i++)

{

# Make sure the content is not null.

if($arrayOfThreads[$i] -eq "")

{

continue; # Invalid, get next element.

}

write-Progress -activity "Thread Statistics" -status "Thread number $arrayOfThreads[$i]" -percentComplete ($i / $arrayOfThreads.length * 100)

write-Host "==============================================================" -foreground Green -background Black

write-Host "`nThread number: " -foreground Green -background Black -nonewline

write-Host $arrayOfThreads[$i] -foreground Red -background Black

write-Host "`nCLR stack:`n" -foreground Green -background Black

[string] $temp = $clrstack[$arrayOfThreads[$i]]

$temp = $temp.Replace($global:g_frameDelimiter, "`n")

$temp = $temp.Replace(";", ",")

write-Host $temp -foreground Red -background Black

write-Host "`nManaged objects from the stack:`n" -foreground Green -background Black

$temp = $dso[$arrayOfThreads[$i]]

$temp = $temp.Replace($global:g_frameDelimiter, "`n")

$temp = $temp.Replace(";", ",")

write-Host $temp -foreground Red -background Black

write-Host "`nThread Number ID OSID ThreadOBJ State GC Context Domain Count APT Exception`n" -foreground Green -background Black

write-Host " " $arrayOfThreads[$i] " " $threads[$arrayOfThreads[$i]] -foreground Red -background Black

$threadNum = $arrayOfThreads[$i]

# Change context to the current thread being analyzed.

Send-PowerDbgCommand "~ $threadNum s"

# Get exception.

Send-PowerDbgCommand "!PrintException"

Parse-PowerDbgPRINTEXCEPTION

$exception = $null

$exception = Convert-PowerDbgCSVToHashTable

# Makes sure there is an exception coming from that thread.

if($exception["Message:"] -ne $null)

{

write-Host "`nException object:" -foreground Green -background Black

write-Host $exception["Exception object:"] -foreground Red -background Black

write-Host "Exception type:" -foreground Green -background Black

write-Host $exception["Exception type:"] -foreground Red -background Black

write-Host "Message:" -foreground Green -background Black

write-Host $exception["Message:"] -foreground Red -background Black

write-Host "Inner Exception:" -foreground Green -background Black

write-Host $exception["InnerException:"] -foreground Red -background Black

write-Host "HRESULT:" -foreground Green -background Black

write-Host $exception["HResult:"] -foreground Red -background Black

}

# User must press any key to continue after 5 threads were displayed.

if((($i + 1) % 5) -eq 0)

{

write-Host "`n####### Press any key to see 5 more threads... #######"

$keyboard = $host.UI.RawUI.ReadKey("NoEcho,IncludeKeyDown")

}

write-Host "Done!" -foreground Green -background Black

write-Host "`nASP.NET Pages:`n" -foreground Green -background Black

write-Host "HttpContext Timeout Completed Running ThreadId ReturnCode Verb RequestPath QueryString" -foreground Green -background Black

foreach($item in $aspxPages.keys)

{

write-Host $item " " $aspxPages[$item] -foreground Red -background Black

}

[PowerShell Script] PowerDbg v5.2—Using PowerShell to Control WinDbg

rafarah — Wed, 15 Apr 2009 08:43:00 GMT

This version has two improvements and some scripts were changed to be compatible with this new version:

- Starting on version 5.2 the log used to save the command output, enabling the communication between PowerShell and WinDbg is not used anymore. Now all communication uses memory so it’s faster than before.

Note: the file POWERDBG-PARSED.LOG continues to exist since it’s useful for troubleshooting purposes and can be used as input for Excel or LogParser.

- Installation is easier now. Just one file has everything.

I’ve created a simple PowerDbg script that gives ASPX information. It’s going to be blogged after this article and should be used as reference.

Note: It was difficult for me to find some free time and work on PowerDbg. I tested it using all scripts (modified two of them) and using different dumps. However, it may have bugs. If you hunt a bug let me know, please!

DOWNLOAD POWERDBG

Download PowerDbg

POWERDBG FILES

WinDbg.PSM1 ß Starting with this version this is the only file.

INSTALLATION

WinDbg.PSM1

Goes to %\WindowsPowerShell\Modules\WinDbg

param($cdbPath = "C:\debuggers\cdb.exe")

In order to know the exact location, use this command from PowerShell:

$profile

REQUIREMENT

PowerShell v2.0

USAGE

First, make sure you can run scripts:

set-executionpolicy remotesigned

From the WinDbg window type:

.server tcp:port=10456,server=ServerName ß ServerName is your server name.

The command above enables a port communication with the WinDbg instance as the server. You can use other port numbers.

From the PowerShell window you must initialize the communication:

Import-module WinDbg ß Imports our module WinDbg.PSM1

Connect-Windbg "tcp:Port=10456,Server=SERVER" ß Connects session to WinDbg instance.

Or:

Connect-Dbg "tcp:Port=10456,Server=SERVER" ß Connects session to WinDbg

Note: Don’t forget to load symbols and your extensions!

At this point you’re ready to use PowerDbg or PowerDbg scripts.

Example:

Analyze-PowerDbgThreads ß Cmdlet.

.\PowerDbgScriptExceptions ß Script.

Example 2:

Send-PowerDbgCommand "~* kpn 1000"

Parse-PowerDbgK

$ht = @{}

$ht = Convert-PowerDbgCSVToHashTable

Let's display the stack for thread 0:

# Replace the internal frame delimiter by new line and displays the stack.

write-host $ht["0"].Replace($global:g_frameDelimiter, "`n")

Example 3:

Send-PowerDbgCommand "!DumpObj 027a4c3c"

Parse-PowerDbgDUMPOBJ

$ht = Convert-PowerDbgCSVToHashTable

$ht["Name:"] <-- Displays the content of the field Name:

$ht["MethodTable:"] <-- Displays the content of the field MethodTable:

NEW FOR POWERDBG v5.2

PowerDbg now uses just one file that has the content of the previous two files and is not the Profile file anymore.

Easier to install and maintain.

The information from WinDbg to PowerDbg used a text file, now it’s all memory based.

It’s faster now than before and eliminates timing issues that used to happen when the file was being saved and the next PowerDbg command executed.

As a collateral benefit you don’t need to use Start-Sleep in your scripts anymore to avoid timing issues.

Note: The CSV file POWERDBG-PARSED.LOG is still used. It facilitates the troubleshooting and can be used as input for Excel or LogParser.

NEW FOR POWERDBG v5.1

Load-PowerDbgSymbols <$symbolPath>

Load symbols.

Usage:

Load-PowerDbgSymbols “SRV*c:\PUBLICSYMBOLS*http://msdl.microsoft.com/download/symbols"

Parse-PowerDbgASPXPAGES

Maps the output from the !ASPXPages command and saves it into the CSV file POWERDBG-PARSED.LOG

To convert the CSV file to a Hash Table use Convert-PowerDbgCSVToHashTable.

For this version the fields being considered are:

Key: HttpContext

Value: Timeout+Completed+Running+ThreadId+ReturnCode+Verb+RequestPath+QueryString

Parse-PowerDbgCLRSTACK

Maps the output from the !clrstack command or ~* e !clrstack and saves it into the CSV file POWERDBG-PARSED.LOG

To convert the CSV file to a Hash Table use Convert-PowerDbgCSVToHashTable.

Attention! The key is the thread number, and the value is the call stack separated by $global:g_frameDelimiter.

Commas "," are replaced for ";" to avoid confusion with the comma used by the CSV file.

If you use this cmdlet to parse the output from ~* e !clrstack, the threads not running managed code are automatically ignored.

Parse-PowerDbgTHREADS

Maps the output from the !threads command and saves it into the CSV file POWERDBG-PARSED.LOG

To convert the CSV file to a Hash Table use Convert-PowerDbgCSVToHashTable.

The following fields are extracted:

Thread Number - Key

ID+OSID+ThreadOBJ+State+GC+Context+Domain+Count+APT+Exception - Value

Parse-PowerDbgDSO

Maps the output from the !dso or ~* e !dso command and saves it into the CSV file POWERDBG-PARSED.LOG

To convert the CSV file to a Hash Table use Convert-PowerDbgCSVToHashTable.

The Thread Number is the key, and the stack is the value, like the way that Parse-PowerDbgK or Parse-PowerDbgCLRSTACK operates.

Attention! Commas are replaced by ";" and $global:g_FrameDelimiter is used to separate frames.

CMDLETS FROM POWERDBG

Send-PowerDbgCommand <$command>

This was the most complex cmdlet, but now it’s just a wrapper for Invoke-WinDbgCommand.

SendPowerDbgCommand sends commands to WinDbg.

Parse-PowerDbgDT [$useFieldNames]

Parses the output from the dt command and saves it into POWERDBG-PARSED.LOG using a CSV file format.

If $useFieldNames has a value, the cmdlet stores fields from struct/classes and values. Otherwise it stores offsets and values.

To convert the CSV file to a Hash Table use Convert-PowerDbgCSVToHashTable.

Convert-PowerDbgCSVToHashTable

Converts the output from the Parse-PowerDbg* cmdlets to a Hash Table.

Send-PowerDbgDML <$hyperLinkDML> <$commandDML>

Creates a DML command and sends it to WinDbg.

DML stands for Debug Markup Language. Using DML you can create hyperlinks that execute commands.

Parse-PowerDbgNAME2EE

Maps the output from the !name2ee and saves it into the CSV file POWERDBG-PARSED.LOG

Convert-PowerDbgCSVtoHashTable converts the output into a Hash Table.

Parse-PowerDbgDUMPMD

Maps the output from !dumpmd command and saves it into the CSV file POWERDBG-PARSED.LOG.

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Parse-PowerDbgDUMPMODULE

Maps the output from !DumpModule command and saves it into the CSV file POWERDBG-PARSED.LOG

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Parse-PowerDbgDUMPLMI

Maps the output from !lmi command and saves it into the CSV file POWERDBG-PARSED.LOG

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Has-PowerDbgCOMMANDSUCCEEDED

Returns $true if the last command succeeded or $false if not.

Send-PowerDbgComment

Sends a comment, a string in bold, to the WinDbg window.

Parse-PowerDbgVERTARGET

Maps the output from vertarget command, either the Kernel Time or the User Time.

The output is saved into the CSV file POWERDBG-PARSED.LOG.

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Parse-PowerDbgRUNAWAY

Maps the output of !runaway 1 or !runaway 2 and stores the results into the CSV file POWERDBG-PARSED.LOG

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Attention! If you need to know the top threads consuming CPU time, use Convert-PowerDbgRUNAWAYtoArray. The items will be in the same exact order of the original command.

Convert-PowerDbgRUNAWAYtoArray

Returns an array of two dimensions corresponding to the output of !runaway 1 or !runaway 2.

Parse-PowerDbgK

Maps the output of k command and its variations like kv, kbn, kpn, etc.

The output is saved into the CSV file POWERDBG-PARSED.LOG

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Attention! This cmdlet doesn’t work with kPn. Another thing, it replaces “,” with “;” to avoid conflict with the CSV delimiter.

Parse-PowerDbgSymbolsFromK

Maps just the symbols from k command and variations, saving the content into the CSV file POWERDBG-PARSED.LOG

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Attention! This cmdlet doesn’t work with kPn. Another thing, it replaces “,” with “;” to avoid conflict with the CSV delimiter.

Parse-PowerDbgLM1M

Maps just the output from lm1m and stores it into the CSV file POWERDBG-PARSED.LOG

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Classify-PowerDbgThreads

Returns one array where the index is the thread number and the value is one of these values:

0 UNKNOWN_SYMBOL

1 WAITING_FOR_CRITICAL_SECTION

2 DOING_IO

3 WAITING

4 GC_THREAD

5 WAIT_UNTIL_GC_COMPLETE

6 SUSPEND_FOR_GC

7 WAIT_FOR_FINALIZE

8 TRYING_MANAGED_LOCK

9 DATA_FROM_WINSOCK

It’s very easy to add more symbols and constants to get a more granular analysis. Look at the source code for details.

Analyze-PowerDbgThreads

Analyzes and shows what each thread is doing and its corresponding CPU time, sorted by User Mode time.

This cmdlet is very useful for scenarios like hangs, high CPU, and crashes.

Attention! This command requires thread information if debugging a dump file.

Parse-PowerDbgPRINTEXCEPTION

Maps the output from !PrintException command and saves it into the CSV file POWERDBG-PARSED.LOG.

The following fields are considered while others are ignored:

Exception object:

Exception type;

Message:

InnerException:

HRESULT:

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Parse-PowerDbgDD-L1

Maps the output from dd <address> L1 or dd poi(<address>) L1 and saves the results into the CSV file POWERDBG-PARSED.LOG.

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Parse-PowerDbgGCHANDLELEAKS

Maps the output from !GCHandleLeaks command and saves it into the CSV file POWERDBG-PARSED.LOG.

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Parse-PowerDbgDUMPOBJ

Maps the output from !DumpObj command and saves it into the CSV file POWERDBG-PARSED.LOG.

The assembly path and file name are saved using the key name “Assembly:.”

If the object is invalid, the ‘Name:’ field will have the string “Invalid Object.” You may want to check this string to make sure you have valid data.

The keys are the fields or Method Table, and values are the corresponding value.

Convert-PowerDbgCSVToHashTable converts the output into a Hash Table.

Based on users’ feedback this approach may be changed in the near future.

POWERDBG SCRIPTS

PowerDbgScriptDumpDict.PS1

Extracts the key/value pair from a dictionary.

PowerDbgScriptExceptions.PS1

Displays the call stacks that have inner or hidden exceptions.

PowerDbgScriptGCHandleLeaksChart.PS1

Displays statistics and a chart from the top 20 objects leaking.

PowerDbgScriptHighCPU.PS1

Displays all threads consuming high CPU using a specific time as a threshold.

PowerDbgScriptSaveModule.PS1

Saves all modules that have a specific method. You provide the method name, and it gives you the corresponding modules.

[PowerShell Script] PowerDbg v5.1—Using PowerShell to Control WinDbg

rafarah — Thu, 19 Mar 2009 21:28:00 GMT

So, here we go again. This is a minor version with a few new cmdlets. These new cmdlets are those that we use most of the time.

DOWNLOAD POWERDBG

Download PowerDbg

POWERDBG FILES

WinDbg.PSM1 ß Contains cmdlets used to communicate with WinDbg.

Microsoft.PowerShell_Profile.PS1 ß Contains cmdlets that parse command output. Uses WinDbg.PSM1 under the hood.

INSTALLATION

Goes to %\Documents\WindowsPowerShell\Modules\WinDbg

Microsoft.PowerShell_Profile.PS1

Goes to %\Documents\windowspowershell

In order to know the exact location, use this command from PowerShell:

$profile

REQUIREMENT

PowerShell v2.0

USAGE

First, make sure you can run scripts:

set-executionpolicy remotesigned

From the WinDbg window type:

.server tcp:port=10456,server=ServerName ß ServerName is your server name.

The command above enables a port communication with the WinDbg instance as the server. You can use other port numbers.

From the PowerShell window you must initialize the communication:

Import-module WinDbg ß Importing our module WinDbg.PSM1

Connect-Windbg "tcp:Port=10456,Server=SERVER" ß Connects session to WinDbg instance.

Note: Don’t forget to load symbols and your extensions!

At this point you’re ready to use PowerDbg or PowerDbg scripts.

Example:

Analyze-PowerDbgThreads ß Cmdlet.

.\PowerDbgScriptExceptions ß Script.

Example 2:

Send-PowerDbgCommand "~* kpn 1000"

Parse-PowerDbgK

$ht = @{}

$ht = Convert-PowerDbgCSVToHashTable

Let's display the stack for thread 0:

# Replace the internal frame delimiter by new line and displays the stack.

write-host $ht["0"].Replace($global:g_frameDelimiter, "`n")

Example 3:

Send-PowerDbgCommand "!DumpObj 027a4c3c"

Parse-PowerDbgDUMPOBJ

$ht = Convert-PowerDbgCSVToHashTable

$ht["Name:"] <-- Displays the content of the field Name:

$ht["MethodTable:"] <-- Displays the content of the field MethodTable:

For more examples see the source code from PowerDbg scripts.

NEW FOR POWERDBG v5.1

Load-PowerDbgSymbols <$symbolPath>

Load symbols.

Usage:

Load-PowerDbgSymbols “SRV*c:\PUBLICSYMBOLS*http://msdl.microsoft.com/download/symbols"

Parse-PowerDbgASPXPAGES

Maps the output from the !ASPXPages command and saves it into the CSV file POWERDBG-PARSED.LOG

To convert the CSV file to a Hash Table use Convert-PowerDbgCSVToHashTable.

For this version the fields being considered are:

Key: HttpContext

Value: Timeout+Completed+Running+ThreadId+ReturnCode+Verb+RequestPath+QueryString

Parse-PowerDbgCLRSTACK

Maps the output from the !clrstack command or ~* e !clrstack and saves it into the CSV file POWERDBG-PARSED.LOG

To convert the CSV file to a Hash Table use Convert-PowerDbgCSVToHashTable.

Attention! The key is the thread number, and the value is the call stack separated by $global:g_frameDelimiter.

Commas "," are replaced for ";" to avoid confusion with the comma used by the CSV file.

If you use this cmdlet to parse the output from ~* e !clrstack, the threads not running managed code are automatically ignored.

Parse-PowerDbgTHREADS

Maps the output from the !threads command and saves it into the CSV file POWERDBG-PARSED.LOG

To convert the CSV file to a Hash Table use Convert-PowerDbgCSVToHashTable.

The following fields are extracted:

Thread Number - Key

ID+OSID+ThreadOBJ+State+GC+Context+Domain+Count+APT+Exception - Value

Parse-PowerDbgDSO

Maps the output from the !dso or ~* e !dso command and saves it into the CSV file POWERDBG-PARSED.LOG

To convert the CSV file to a Hash Table use Convert-PowerDbgCSVToHashTable.

The Thread Number is the key, and the stack is the value, like the way that Parse-PowerDbgK or Parse-PowerDbgCLRSTACK operates.

Attention! Commas are replaced by ";" and $global:g_FrameDelimiter is used to separate frames.

CMDLETS FROM POWERDBG

Send-PowerDbgCommand <command>

This was the most complex cmdlet, but now it’s just a wrapper for Invoke-WinDbgCommand.

SendPowerDbgCommand sends commands to WinDbg.

Parse-PowerDbgDT [$useFieldNames]

Parses the output from the dt command and saves it into POWERDBG-PARSED.LOG using a CSV file format.

If $useFieldNames has a value, the cmdlet stores fields from struct/classes and values. Otherwise it stores offsets and values.

To convert the CSV file to a Hash Table use Convert-PowerDbgCSVToHashTable.

Convert-PowerDbgCSVToHashTable

Convert the output from the Parse-PowerDbg* cmdlets to a Hash Table.

Send-PowerDbgDML <$hyperLinkDML> <$commandDML>

Creates a DML command and sends it to WinDbg.

DML stands for Debug Markup Language. Using DML you can create hyperlinks that execute commands.

Parse-PowerDbgNAME2EE

Maps the output from the !name2ee and saves it into the CSV file POWERDBG-PARSED.LOG

Convert-PowerDbgCSVtoHashTable converts the output into a Hash Table.

Parse-PowerDbgDUMPMD

Maps the output from !dumpmd command and saves it into the CSV file POWERDBG-PARSED.LOG.

Convert-PowerDbgCSVtoHashTable converts the output into a Hash Table.

Parse-PowerDbgDUMPMODULE

Maps the output from !DumpModule command and saves it into the CSV file POWERDBG-PARSED.LOG

Convert-PowerDbgCSVtoHashTable converts the output into a Hash Table.

Parse-PowerDbgLMI

Maps the output from lmi command and saves it into the CSV file POWERDBG-PARSED.LOG

Convert-PowerDbgCSVtoHashTable converts the output into a Hash Table.

Has-PowerDbgCOMMANDSUCCEEDED

Returns $true if the last command succeeded or $false if not.

Send-PowerDbgComment

Sends a comment, a string in bold, to the WinDbg window.

Parse-PowerDbgVERTARGET

Maps the output from vertarget command, either the Kernel Time or the User Time.

The output is saved into the CSV file POWERDBG-PARSED.LOG.

Convert-PowerDbgCSVtoHashTable converts the output into a Hash Table.

Parse-PowerDbgRUNAWAY

Maps the output of !runaway 1 or !runaway 2 and stores the results into the CSV file POWERDBG-PARSED.LOG

Convert-PowerDbgCSVtoHashTable converts the output into a Hash Table.

Attention! If you need to know the top threads consuming CPU time, use Convert-PowerDbgRUNAWAYtoArray. The items will be in the same exact order of the original command.

Convert-PowerDbgRUNAWAYtoArray

Returns an array of two dimensions corresponding to the output of !runaway 1 or !runaway 2.

Parse-PowerDbgK

Maps the output of k command and its variations like kv, kbn, kpn, etc.

The output is saved into the CSV file POWERDBG-PARSED.LOG

Convert-PowerDbgCSVtoHashTable converts the output into a Hash Table.

Attention! This cmdlet doesn't work with kPn. Another thing, it replaces "," with $global:g_frameDelimiter to avoid conflict with the CSV delimiter.

Parse-PowerDbgSymbolsFromK

Maps just the symbols from k command and variations, saving the content into the CSV file POWERDBG-PARSED.LOG

Convert-PowerDbgCSVtoHashTable converts the output into a Hash Table.

Attention! This cmdlet doesn't work with kPn. Another thing, it replaces "," with $global:g_frameDelimiter to avoid conflict with the CSV delimiter.

Parse-PowerDbgLM1M

Maps just the output from lm1m and stores it into the CSV file POWERDBG-PARSED.LOG

Convert-PowerDbgCSVtoHashTable converts the output into a Hash Table.

Classify-PowerDbgThreads

Returns one array where the index is the thread number and the value is one of these values:

0 UNKNOWN_SYMBOL

1 WAITING_FOR_CRITICAL_SECTION

2 DOING_IO

3 WAITING

4 GC_THREAD

5 WAIT_UNTIL_GC_COMPLETE

6 SUSPEND_FOR_GC

7 WAIT_FOR_FINALIZE

8 TRYING_MANAGED_LOCK

9 DATA_FROM_WINSOCK

It’s very easy to add more symbols and constants to get a more granular analysis. Look at the source code for details.

Analyze-PowerDbgThreads

Analyzes and shows what each thread is doing and its corresponding CPU time, sorted by User Mode time.

This cmdlet is very useful for scenarios like hangs, high CPU, and crashes.

Attention! This command requires thread information if debugging a dump file.

Parse-PowerDbgPRINTEXCEPTION

Maps the output from !PrintException command and saves it into the CSV file POWERDBG-PARSED.LOG.

The following fields are considered while others are ignored:

Exception object:

Exception type;

Message:

InnerException:

HRESULT:

Convert-PowerDbgCSVtoHashTable converts the output into a Hash Table.

Parse-PowerDbgDD-L1

Maps the output from dd <address> L1 or dd poi(<address>) L1 and saves the results into the CSV file POWERDBG-PARSED.LOG.

Convert-PowerDbgCSVtoHashTable converts the output into a Hash Table.

Parse-PowerDbgGCHANDLELEAKS

Maps the output from !GCHandleLeaks command and saves it into the CSV file POWERDBG-PARSED.LOG.

Convert-PowerDbgCSVtoHashTable converts the output into a Hash Table.

Parse-PowerDbgDUMPOBJ

Maps the output from !DumpObj command and saves it indo the CSV file POWERDBG-PARSED.LOG.

The assembly path and file name are saved using the key name ‘Assembly:’.

If the object is invalid the ‘Name:’ field will have the string “Invalid Object.” You may want to check this string to make sure you’ve got valid data.

The keys are the fields or Method Table, and values are the corresponding value.

Convert-PowerDbgCSVtoHashTable converts the output into a Hash Table.

Based on users’ feedback this approach may be changed in the near future.

POWERDBG SCRIPTS

PowerDbgScriptDumpDict.PS1

Extracts the key/value pair from a Dictionary.

PowerDbgScriptExceptions.PS1

Displays the call stacks that have inner or hidden exceptions.

PowerDbgScriptGCHandleLeaksChart.PS1

It displays statistics and a chart from the top 20 objects leaking.

PowerDbgScriptHighCPU.PS1

It displays all threads consuming high CPU using a specific time as a threshold.

PowerDbgScriptSaveModule.PS1

It saves all modules that have a specific method. You provide the method name, and it gives you the corresponding modules.