Gooey Bugs

Getting Started Extending the VS Debugger

2008-09-08T17:39:13Z

The documentation for all the pieces and enhancements made to the VS Debugger automation model over the years is spread throughout the MSDN web-site. Here's a list of start points, for each set of enhancements that were made, starting from Visual Studio 2003 through 2008 SP1:

Original: http://msdn.microsoft.com/en-us/library/aa291845(VS.71).aspx

VS2005: http://msdn.microsoft.com/en-us/library/envdte80.debugger2(VS.80).aspx

VS2008: http://msdn.microsoft.com/en-us/library/envdte90.debugger3.aspx

VS2008 SP1: http://msdn.microsoft.com/en-us/library/envdte90a.debugger4.aspx

I'll leave perusing the docs as an exercise to the reader for right now. But suffice it to say, there is a lot of stuff to play with here, including the ability to setup exception handling, change debugger options, know when processes start or end (that are already being debugged), change symbol settings, and more!

Need to write a new Debugging Engine for your favorite platform?

2008-02-20T22:35:00Z

Jackson, recently posted a new sample for how to write a Visual Studio debug engine. As Joc put it: "in our architecture, the debug engine is the part that does all of the communication with the debugging/diagnostic APIs for a given platform, and communicates those back to the UI through a common set of interfaces. The current debugger ships debug engines for managed code, native code, script, native devices and t-sql in the box. The debugger engine extension point gives people writing new platforms the ability to provide an excellent VS debugging experience."

Please note that writing a debug engine is a complex undertaking. If you have a language that's targetting the .NET runtime and the platforms it supports, then you probably don't need to go this route (a new Expression Evaluator, Language Service and project system are more appropriate targets). But if you have a new platform (CPU, runtime, etc) that needs debugging support, this sample should provide good guidance.

Debugger Tips, Tricks and Tools #12

2007-08-02T18:59:00Z

I've had loads of people ask me the question:

"Is there any way to step into a function without having to bother stepping into all the properties that might have to be executed as parameters before getting to what I'm really interested in?"

The standard answer for Visual Studio 2005 and Visual Studio 2008 has been "well, you probably want to get familiar with the DebuggerStepThrough or DebuggerNonUserCode attributes and use them it in your code." However, if it's not your code that you're debugging through, or you don't have the option or would rather not modify it, then this solution isn't sufficient.

C++ users have had an alternate solution for awhile now. It's called Step Into Specific. The basic idea is that while in the editor the user can right click on a function call to choose from a list of additional function calls that will ultimately be made. The function call chosen becomes the next stepping point.

Why don't we have this for managed code debugging? The short answer is that this type of solution requires support from the .NET runtime. This support is not in .NET 2.0, nor is it in the runtime that will ship with VS 2008 (which is basically mainly an extension of the core 2.0 runtime).

What's a managed programmer to do then? Curse and swear. Or perhaps try to find a workaround. That's where this article comes in. I've created a macro called StepIntoHere(). It analyzes the code on which the editor caret is currently positioned and then attempts to simulate a "step-into" exactly the code that implements that function, bypassing any other function calls or properties that have to be made along the way.

For example, say you have this code:

CoolClass c = new CoolClass();

c.DoSomething(new UnCoolClass().MyProp);

This macro allows you to place the caret on "DoSomething" and then do a step into the "DoSomething" function directly rather than having to first step into the constructor for "UnCoolClass" and then it's "MyProp" property.

Sounds great, and it seems to work quite well for trivial applications I've tried it with. I'm not too sure how well it works for much larger apps, but I'm hoping this blog will help me find out.

(See my Idiot's Guide for creating and using VS Macros for quick and easy steps on how to add this macro to VS. To make it more like the C++ version, add it to the Editor's context menu. For extra credit, add an image for it by copying and pasting the existing step-into icon and modifying it slightly. This can be done while customizing your context menu... hint: right click :) )

Sub StepIntoHere()

If DTE.Debugger Is Nothing Or DTE.Debugger.CurrentMode <> dbgDebugMode.dbgBreakMode Then

Beep()

Else

Dim textSelection As EnvDTE.TextSelection = DTE.ActiveWindow.Selection

If textSelection.IsEmpty Then

textSelection.WordLeft(False)

textSelection.WordRight(True)

End If

textSelection = DTE.ActiveWindow.Selection

Dim bp As EnvDTE.Breakpoint

Dim bps As EnvDTE.Breakpoints

bps = DTE.Debugger.Breakpoints.Add(textSelection.Text)

'If we have at least one bound breakpoint, go, otherwise just do a step-into.

If bps.Count > 0 And bps.Item(1).Children.Count > 0 Then

DTE.Debugger.Go()

Else

DTE.Debugger.StepInto()

End If

' Unselect what we selected.

textSelection = Nothing

' Delete the breakpoints we created.

For Each bp In bps

bp.Delete()

End If

End Sub

Why does Visual Studio require debugger symbol files to exactly match the binary files that they were built with?

2007-07-07T02:41:00Z

Recently a coworker of mine lost the symbol file for one of his binaries. Because he needed to debug that binary, getting those symbols back was of utmost importance since debugging without them is near impossible. He decided to try and use a symbol file from a previous build whose sources exactly matched. Much to his chagrin he found that Visual Studio refused to load them. He asked me:

"Why is it that Visual Studio cannot load a symbol file for a binary that was built with the exact same set of source files?"

Before I answer that, I'll take the question a step further:

"Surely if I build a component twice, consecutively, with no changes to source in-between, I certainly will end up with two identical copies of that component, right?"

Surprisingly, the answer to this question is: "no, that is not necessarily true". The most obvious reason for this is the fact that an internal time-stamp will be different. But even disregarding that, the answer is still the same, the actual layout of the code could be different.

The reason is that compiler writers are far more interested in generating correctly functioning code and generating it quickly than ensuring that whatever is generated is laid out identically on your hard drive. Due to the numerous and varied methods and implementations for optimizing code, it is always possible that one build ended up with a little more time to do something extra or different than another build did. Thus, the final result could be a different set of bits for what is the same functionality.

Here's a very simple example to demonstrate. Imagine that the component you're building consists of a function and a variable.

Does it matter whether the resulting file contents looks like this?

0000: MyFunc()
0020: gGlobalVariable

or this?

0000: gGlobalVariable
0004: MyFunc()

Functionally, it doesn't matter at all, but for the debugger it's HUGELY important. In fact, getting it wrong can wreak havoc on your debugging session. In this example, what if you used the symbol file from build #1 to try and determine the value of the global variable when running with the component built by build #2? The debugger would consult the symbol file and return the value referenced by address 0020. Unfortunately, the global variable isn't at that address in component build #2. Rather some some value that makes up the instruction stream for MyFunc() is there.

A debugger depends on knowing what the internal layout of the component is. So, I can now answer my colleague's original question:

"Because both the Visual Studio and your sanity depends on it."

PS: In practice, you are not likely to see a difference in compiler/linker output from build to build. So in theory, it is possible to make use of mismatched symbols. However, compiler/linker determinism is not guaranteed, our debugger cannot depend on it. Furthermore, we would much rather not take the support cost of allowing some sort of override. Therefore, if you find that you absolutely, positively, have to try and use a symbol file that doesn't match you can use WinDbg. I'll leave getting it to work as an exercise for the reader, since you really, really don't want to do this. :)

Debugger Tips, Tricks and Tools #11

2005-12-12T11:00:00Z

More fun with labelling breakpoints

Back in my first Debugger tips post I provided some macros that allow you to tag a set of breakpoints with a string so that they can easily be enabled or disabled from the command window. Well, just for kicks, I decided to expand on that notion a bit and made a set of new macros that can not only do that, but also support multiple labels (the new name for tags) for breakpoints and tracepoints.

Following this description you'll find the source for these macros. Basically, you can use the instructions provided Idiot's Guide to Creating and Using VS Macros to add these macros to your installation. After you do that, just run the SetupBPLabels macro, and you'll be able to use the following commands in the command window:

labelbps - Allows you to add a new label to the current set of enabled breakpoints or tracepoints. If a label is not provided "label" is used by default.
enablebps - Allows you to enable all breakpoints that are associated with the supplied label. If * is provided as the target label or then all breakpoints are enabled regardless of their label. If no label is provided then any breakpoint without a label will be enabled.
disablebps - Allows you to disable all breakpoints that are associated with the supplied label. If * is provided as the target label then all breakpoints are disabled regardless of their label. If no label is provided then any breakpoint without a label will be disabled.
clearlabel - Allows you to remove a label from your set of breakpoints. If * is provided as the target label, then all labels are cleared. When a label is cleared, it no longer exists, so be careful here.
listbps - Lists the breakpoints associated with the supplied label. If no label is supplied, then all breakpoints are listed along with their labels.

I'm not really sure how useful this set of macros will be for you, but it was fun writing them! Let me know what you think, or if you think there's some improvements that could be made to make them even more useful! Please note that I am not much of a VB programmer, so if you have suggestions as to how to improve the code, I'm all ears!

' -------------------------------------------------------------------------

Sub SetupBPLabels()

DTE.ExecuteCommand("alias enablebps Macros.MyMacros.BPLabels.EnableBPs")

DTE.ExecuteCommand("alias disablebps Macros.MyMacros.BPLabels.DisableBPs")

DTE.ExecuteCommand("alias listbps Macros.MyMacros.BPLabels.ListBPs")

DTE.ExecuteCommand("alias labelbps Macros.MyMacros.BPLabels.LabelEnabledBPs")

DTE.ExecuteCommand("alias clearlabel Macros.MyMacros.BPLabels.ClearBPLabel")

End Sub

' -------------------------------------------------------------------------

Sub LabelEnabledBPs(Optional ByRef strLabel As String = "label")

Dim bps As EnvDTE.Breakpoints = DTE.Debugger.Breakpoints

If (bps.Count > 0) Then

For Each bp As EnvDTE.Breakpoint In bps

Dim strCurLabels As String = GetBPLabels(bp)

If (bp.Enabled = True And Not BPLabeledAs(bp, strLabel)) Then

If (strCurLabels.Length > 0) Then

strCurLabels += ";"

End If

strCurLabels = strCurLabels + strLabel

SetBPLabels(bp, strCurLabels)

End If

Else

System.Windows.Forms.MessageBox.Show("Can't find any breakpoints to label")

End If

End Sub

' -------------------------------------------------------------------------

Sub ClearBPLabel(Optional ByRef strBadLabel As String = "")

Dim bps As EnvDTE.Breakpoints = DTE.Debugger.Breakpoints

If (bps.Count > 0) Then

For Each bp As EnvDTE.Breakpoint In bps

Dim strCurLabels As String = GetBPLabels(bp)

SetBPLabels(bp, "")

If (strBadLabel <> "*") Then

Dim labels() As String = strCurLabels.Split(";")

Dim strNewLabels As String = ""

For Each s As String In labels

If (s.ToLower <> strBadLabel.ToLower) Then

If (strNewLabels.Length > 0) Then

strNewLabels += ";"

End If

strNewLabels = strNewLabels + s

End If

SetBPLabels(bp, strNewLabels)

End If

Else

System.Windows.Forms.MessageBox.Show("Can't find any breakpoints to label")

End If

End Sub

' -------------------------------------------------------------------------

Sub EnableBPs(Optional ByRef strLabel As String = "")

Dim bps As EnvDTE.Breakpoints = DTE.Debugger.Breakpoints

If (bps.Count > 0) Then

For Each bp As EnvDTE.Breakpoint In bps

If (strLabel = "*" Or BPLabeledAs(bp, strLabel) Or (GetBPLabels(bp).Length = 0 And strLabel.Length = 0)) Then

bp.Enabled = True

End If

Else

System.Windows.Forms.MessageBox.Show("Can't find any breakpoints to enable")

End If

End Sub

' -------------------------------------------------------------------------

Sub DisableBPs(Optional ByRef strLabel As String = "")

Dim bps As EnvDTE.Breakpoints = DTE.Debugger.Breakpoints

If (bps.Count > 0) Then

For Each bp As EnvDTE.Breakpoint In bps

If (strLabel = "*" Or BPLabeledAs(bp, strLabel) Or (GetBPLabels(bp).Length = 0 And strLabel.Length = 0)) Then

bp.Enabled = False

End If

Else

System.Windows.Forms.MessageBox.Show("Can't find any breakpoints to enable")

End If

End Sub

' -------------------------------------------------------------------------

Sub ListBPs(Optional ByRef strLabel As String = "")

Dim bps As EnvDTE.Breakpoints = DTE.Debugger.Breakpoints

Dim outpane As EnvDTE.CommandWindow = DTE.Windows.Item(EnvDTE.Constants.vsWindowKindCommandWindow).Object

Dim i As Integer = 1

For Each bp As EnvDTE.Breakpoint In bps

If (strLabel.Length = 0 Or BPLabeledAs(bp, strLabel)) Then

Dim strEnabled As String = ""

If bp.Enabled Then

strEnabled = "x"

End If

outpane.OutputString("BP #" + i.ToString() + " " + strEnabled + vbTab + "Name: " + bp.Name + vbCrLf)

If (strLabel.Length = 0) Then

outpane.OutputString(vbTab + vbTab + "Labels: " + GetBPLabels(bp) + vbCrLf)

End If

i = i + 1

End Sub

' -------------------------------------------------------------------------

Private Function GetBPLabels(ByRef bp As EnvDTE.Breakpoint) As String

Dim strStartTag As String = "<labels>"

Dim strEndTag As String = "</labels>"

Dim ndxStart As Integer = bp.Tag.IndexOf(strStartTag) + strStartTag.Length

Dim ndxEnd As Integer = bp.Tag.IndexOf(strEndTag)

If (ndxStart > 0 And ndxEnd > ndxStart) Then

GetBPLabels = bp.Tag.Substring(ndxStart, ndxEnd - ndxStart)

Else

GetBPLabels = ""

End If

End Function

' -------------------------------------------------------------------------

Private Sub SetBPLabels(ByRef bp As EnvDTE.Breakpoint, ByRef labels As String)

bp.Tag = "<labels>" + labels + "</labels>"

End Sub

' -------------------------------------------------------------------------

Private Function BPLabeledAs(ByRef bp As EnvDTE.Breakpoint, ByRef strLabel As String) As Boolean

Dim bLabelFound As Boolean = False

If (strLabel.Length > 0) Then

Dim strLabels = GetBPLabels(bp)

Dim labels() As String = strLabels.Split(";")

For i As Integer = 0 To labels.Length - 1 And bLabelFound = False

If (labels(i).ToLower = strLabel.ToLower) Then

bLabelFound = True

End If

Return bLabelFound

End Function

Debugger Tips, Tricks and Tools #10

2005-11-30T11:42:00Z

Create a one-step operation to attach to a process

If you find your debugging session typically involves attaching to a specific process you can save yourself some time in VS 2005 by creating a macro to do it for you.

Choose Tools->Macros->Record Temporary Macro
Choose Tools->Attach to Process and choose the process you want to attach to. You can choose any combination of process, transport, qualifier and code-type.
Click the Stop Recording macro toolbar button, or press Ctrl-Shift-R.
Choose Tools->Macros->Save Temporary Macro. The caret will be moved to the Macro Explorer and you'll be prompted to change the name of TemporaryMacro to the name of your choosing.

Here's an example macro that was generated when I attached to calc.exe with the native engine:

Sub AttachToCalc()
Try

Dim dbg2 As EnvDTE80.Debugger2 = DTE.Debugger

Dim trans As EnvDTE80.Transport = dbg2.Transports.Item("Default")

Dim dbgeng(1) As EnvDTE80.Engine

dbgeng(0) = trans.Engines.Item("Native")

Dim proc2 As EnvDTE80.Process2 = dbg2.GetProcesses(trans, "JIMSMACHINE").Item("calc.exe")

proc2.Attach2(dbgeng)

Catch ex As System.Exception

MsgBox(ex.Message)

End Try

End Sub

Now, invoking this macro through the Macro Explorer will cause the exact same attach to whatever program you originally chose! If you want to make your life even easier you could add this macro to a toolbar by:

Choose Tools->Customize
Activate the Commands Tab.
Scroll down to "Macros" in the Categories list.
Drag Macros.MyMacros.RecordingModule.AttachToCalc to a toolbar

You could also create an alias for the macro that you can invoke from the command window as follows:

alias attcalc Macros.MyMacros.RecordingModule.AttachToCalc

Now to attach to calc, all you have to do is type "attcalc" in the command window.

Of course, the macro that is created for you is a very simple skeleton. If you'd like you could modify it to do all sorts of other things, like attach to additional processes, or set a breakpoint or whatever!

You might also want to peruse my Idiot's Guide to Creating and Using VS Macros for more helpful hints.

Debugger Tips, Tricks and Tools #9

2005-11-28T23:53:00Z

Making native array viewing easier with Enhanced DataTips

I'm back from vacation, and have a new found respect for people (mostly US wives/women, I presume) that slave all day over preparing a Thanksgiving meal and then cleaning it all up. Yep, I did the whole turkey, potatoes, yams, corn, etc. thing this year for me and my kids. By the end of the day I was completely exhausted. But it all turned out great!

Anyway onto a new debugging trick. This one is geared directly at users of native C/C++ debugger in Visual Studio 2005. It revolves around a little registry tweak you can do to make viewing arrays of objects a little easier using Enhanced DataTips if all you have is pointer to one of the items. This is not a supported behavior and cannot be adjusted outside of doing this registry change. As is per usual with any registry tweaking, please be careful -- if you mess up (significantly) you can destroy your entire OS installation.

First, shut down Visual Studio. Since VS writes a new set of option values to the registry everytime you shut it down, any changes you make to the registry prior to doing so will be lost.

Now open the registry and navigate to:

HKCU\Software\Microsoft\VisualStudio\8.0\Debugger

You'll see a boatload of debugger options, most of which are addressable under the Tools->Options functionality of VS. However there is one, OfferArrayExpansion, that is not exposed. Change it from 0 to 1 and reopen VS.

Now when you activate a DataTip over any pointer to any type and you expand it once, you'll see a message allowing you to expand it as an array. If you do so, you'll see the first 100 items as if the pointer was really pointing to an array of 100.

Debugger Tips, Tricks and Tools (On vacation)

2005-11-21T11:00:00Z

Spending the week on vacation. I'll return with more tips next week! Happy Thanksgiving!

Debugger Tips, Tricks and Tools #8

2005-11-18T16:00:00Z

Noisy breakpoints!

Ok this feature has been around for a long time, but it's really almost impossible to discover on your own. Scott Nonnenberg asked me to remind him out to do this yesterday and I thought HEY good idea for a tip! Here's what you do:

Open the Windows Control Panel
If you are in Category View choose "Sounds, Speech, and Audio Devices", then "Change the sound scheme"
If you are in Classic View choose "Sounds and Audio Devices", then choose the "Sounds" tab.
In the "Program events" listbox scroll down to "Microsoft Development Environment"
Change the "Breakpoint Hit" event to the sound of your choice.

Debugger Tips, Tricks and Tools #7

2005-11-17T18:57:00Z

Moving tracepoints and breakpoints around

I have found the addition of tracepoints in Visual Studio 2005 extremely useful, allowing me to debug issues that have normally been pretty difficult to deal with. For example, UI debugging often involves breaking into code that was invoked due to the receipt of a WM_PAINT message. In the pre-tracepoint world, I would often end up resorting to printf-style debugging. Either that, or I'd need to make sure that my debugger is on a separate monitor, or in no way covering up my window, otherwise I end up in a difficult to manage breakpoint-hit cycle. With tracepoints I can do printf-style debugging without needing to muck with my code.

However, I often find myself wanting to move a tracepoint that I carefully setup to different place in my code. Breakpoints didn't suffer from that need nearly as much since not much effort usually goes into creating them.

Well, fortunately tracepoints (and breakpoints) can be moved. It's a little more difficult than it should be, but usually not as much effort as removing then redoing it in a different place.

Here's how to do it.

Determine the line to which you want to move the tracepoint.
Right click on the tracepoint and choose Location...
Change the line number to the line you'd like the tracepoint to fire.
Click OK

Pretty straightforward and easy. I fully intend on making sure future versions of Visual Studio will allow you to drag the glyph, but for now this will do. :)

Debugger Tips, Tricks and Tools #6

2005-11-16T19:04:00Z

Create an Object ID to keep track of an object while debugging

In yesterday's tip I hinted at another new feature of the debugger specially designed for C# and J# programmers. This is the ability to create an Object ID for any particular object during your debugging session, no matter what your current context is. In other words, it's quite similar to being able to get the address of an object in C or C++ so that you can refer to that particular object at any time during debugging.

For example, given the following short program:

using System;

using System.Collections.Generic;

using System.Text;

namespace ConsoleApplication1

{

class Program

{

static void Main(string[] args)

{

List<string> stringList = new List<string>();

stringList.Add("Hello");

stringList.Add("World");

foreach (string str in stringList)

{

Console.WriteLine(ReverseString(str));

}

static string ReverseString(string str)

{

if (str.Length > 1)

{

return ReverseString(str.Substring(1)) + str.Substring(0, 1);

}

return str;

}

Put a breakpoint on the first line of ReverseString and run the program under the debugger. When the breakpoint is hit, hover your mouse over the 'str' variable, and on the DataTip that appears right click to bring up the context menu.

You will see the following:

Note the highlighted menu item called "Make Object ID". Select that and your DataTip will change to show the value of 'str' to be "Hello" {1#}". What's that "1#" doing on the end of the value? That's the Object ID. You can use that expression to refer to that particular instance of the object anywhere you can evaluate expressions in the debugger.

To prove it, first press F5 again. You will break into ReverseString again, but this time the value of 'str' is "ello". Now add "1#" to the watch window. You'll see:

When you're done with that object you can right click on it in the watch window and choose "Delete Object ID".

A welcome tool for improving Windows color management!

2005-11-16T09:50:00Z

It's been a very long time since I blogged about digital photography and yet I noticed that my recent debugger blogs have been appearing on the Digital Photography Community. So, rather than fight the fight I would need to do to get filtering of my debugger specific blogs from there, I thought it might be easier to actually blog about digital photography.

Truthfully, due to some interesting personal issues I haven’t been able to focus on my hobby/passion for well over a year. So please don’t consider me an expert on anything at this point. I certainly have more questions these days than answers.

On to the topic at hand though... Something in the Windows digital imaging arena recently struck me as intriguing. A new, free control panel applet was recently released that claims to simplify some of the headaches of configuring color on Windows systems. You can find it here, if you haven't already.

As I read what it is intended to do, I thought to myself “Hallelujah! I wish this existed years ago!” So I installed it and gave it a whirl.

Well, after playing with it I can't say it answered all my questions or appears to make color management a breeze, but I STILL wish it existed years ago just for the simple feature of being able to associate humanly readable names with some of the worst named files I’ve ever come across. Example? The standard color profile for my Epson 2200 is called “ee231__1.icm”. Go figure.

The 3D Gamut sub-display is also pretty nifty. At first, I didn’t have a clue as to why there was the ability to view the gamut as a wireframe or mesh, or to change the opacity. However, checking the “Compare To:” checkbox and choosing another gamut, it becomes clear what these adjustments can be used for when comparing two gamuts:

Note that you can see my Epson’s Premium Glossy gamut (pictured as a gray blob) has the ability to display hues of greenish blue that aren’t part of the Adobe RGB gamut (assuming I’m understanding this view correctly).

In any event, if you've been struggling with getting your prints to accurately match what you're seeing on your display, this tool is a necessity for clarifying some of the mysteries surrounding color matching and management on Windows systems. Other things you can do to make sense of it all? Maybe I'm preaching to the choir, but you could probably fill a small book about the intricacies color profiling, matching and management, let alone the science behind it all. Maybe somebody out there has a pointer to some good sites. Here's one I made use of a long time ago -- looks like it's even better now!

Debugger Tips, Tricks and Tools #5

2005-11-15T18:52:00Z

Enhanced DataTips have a context menu

Because of the transient nature of DataTips inside Visual Studio 2005, you may not be aware that there useful operations you can perform on selected item within a datatip by opening a context menu on it.

In the above example, the cursor has been hovered to the m_strTitle member of pDoc->m_strTitle and the right mouse button was clicked. This invokes the context menu for the item and your presented with a set of operations you can perform on it.

Copy - Copies both the name and the value of the item.
Copy Expression - Copies just the expression portion of the item.
Copy Value - Copies just the value portion of the item.
Edit Value - If the item is not read-only you can edit the value. The same can be done by left-clicking the mouse once, or pressing F2.
Add Watch - Adds the item as a top-level watch item.
Hexadecimal Display - Toggles the debugger's numeric base for displaying integer values.
When debugging C# or J# you're presented with another operation -- "Make Object Id". In a future tip, I will discuss that particular feature.

My biggest complaint about this feature? I wish that I had had time to add the "Copy Expression" and "Copy Value" commands to the watch window. :)

Debugger Tips, Tricks and Tools #4

2005-11-14T18:50:00Z

Keyboard navigation and transparency with Enhanced DataTips

Steve Steiner recently blogged about how to make DataTips temporarily transparent in case you need to see what they might be covering up. This was going to be one of my upcoming tips, but he beat me to it! :) Anyway, with that post he touched on another aspect of DataTips that you might not know about -- they have some limited keyboard support. To get access to it, you first must expand a top-most tip one level via the mouse.

From there you can:

Use the [down] and [up] arrows to move through the items in the expanded tip.
Use [pgup] and [pgdn] to scroll a page at a time in tips that are scrollable
Use the [right] arrow to expand an expandable item into a new child tip.
Use the [left] arrow to close the current tip and go back a level.
Use [F2] to edit a value (uhoh, did you know about this feature? Or is there another debugger tip looming?)
Use [ctrl] to make all the tips transparent (as Steve pointed out).

There is certainly room for further improvement. In particular, I would have like to have implemented the ability to invoke a DataTip via the keyboard and to support [home] and [end] for scrollable tips, but time constraints cut that effort short. Hopefully, you'll see more in a future release!

Debugger Tips, Tricks and Tools #3

2005-11-11T19:14:00Z

Use tracepoints to log execution flow and even modify it!

You may or may not know about tracepoints one of the cool new features of Visual Studio 2005. Basically a tracepoint is just like a breakpoint, except it doesn't break execution of your program. What use is it then? Well, rather than break, they allow you to print a message when they are "hit". This functionality is fairly obvious in its usefullness, I think (hope).

For you newbies out there, to create a tracepoint, you:

Create any type of breakpoint.
Right click on the breakpoint glyph in the channel to the left of your editor and choose "When Hit..."

A dialog will appear that allows you to choose what message you'd like to print when the tracepoint is hit. If you clear the "Continue execution" checkbox at the bottom of the dialog, the tracepoint will turn into a breakpoint.

This is all great, but did you know you could use a tracepoint to modify the flow of your program's execution while under a live debugging session? I've used this feature a couple times for real-life bug investigation. Say you have the following code:

int j = 1000;

for (int i = 0; i < 1000; i++)

{

j--;

printf("j = %d\n", j);

}

Let's say you determine for whatever reason, that the j-- should NOT occur inside this loop (making the loop useless, but go with me on this) and you don't really want to stop debugging to rebuild and start the process again. First, switch to view disassembly for this block of code to see something like this (the order here is somewhat important as will be explained below):

int j = 1000;

004113DC mov dword ptr [j],3E8h

for (int i = 0; i < 1000; i++)

004113E3 mov dword ptr [i],0

004113EA jmp wmain+55h (4113F5h)

004113EC mov eax,dword ptr [i]

004113EF add eax,1

004113F2 mov dword ptr [i],eax

004113F5 cmp dword ptr [i],3E8h

004113FC jge wmain+84h (411424h)

{

j--;

004113FE mov eax,dword ptr [j]

00411401 sub eax,1

00411404 mov dword ptr [j],eax

printf("j = %d\n", j);

00411407 mov esi,esp

00411409 mov eax,dword ptr [j]

0041140C push eax

0041140D push offset string "j = %d\n" (41563Ch)

00411412 call dword ptr [__imp__printf (4182C4h)]

00411418 add esp,8

0041141B cmp esi,esp

0041141D call @ILT+315(__RTC_CheckEsp) (411140h)

}

Set a breakpoint on address 0x004113FE. Right click on the bp glyph and choose "When Hit...". Rather than print a message we want to change the current IP to be somewhere other than this line. To do this we will supply a message that actually evaluates a special expression and has the side-effect of changing the current instruction pointer. The message looks like the following:

Skipping j-- line! {eip=0x00411407}

When this message is "printed" it basically changes the value of the eip register, thus making the next instruction the beginning of the printf line. Setting other properties of the tracepoint, such as hit count and condition allows you to get even fancier.

There are some big caveats here though. First, the more the tracepoint is hit the slower your process will run. Second, tracepoints are NOT hit when you step to or over them. This is actually because the execution break is occuring due to the ending of the step operation as opposed to being the result of "hitting" the tracepoint (Yes, this behavior is considered a bug that we plan to address in a future release). But probably most importantly is that using this technique can wreak havoc on a debugging session if you're not aware of it's use. In addition to the obvious problems of specifying invalid or problematic places to jump, if you change and rebuild your application, your addresses will also very likely change! Obviously, if you have a tracepoint that sets the current instruction to the same place an particular piece of code USED to be, your application will likely get into a very, very bad state. It is for this reason that I asked you to switch to disassembly mode BEFORE setting your breakpoint. When a bp is set in disassembly, an address breakpoint is created. Address bp's are always automatically disabled when a debugging session is restarted. However, there's no reason why you couldn't have set a source bp BEFORE switching to disassembly and made it a tracepoint that had the same functionality. In this case though, the tracepoint will NOT be automatically disabled.

Nevertheless, when the need arises, this can be a powerful little trick!

Gooey Bugs

Getting Started Extending the VS Debugger

Need to write a new Debugging Engine for your favorite platform?

Debugger Tips, Tricks and Tools #12

Why does Visual Studio require debugger symbol files to *exactly* match the binary files that they were built with?

Debugger Tips, Tricks and Tools #11

Debugger Tips, Tricks and Tools #10

Debugger Tips, Tricks and Tools #9

Debugger Tips, Tricks and Tools (On vacation)

Debugger Tips, Tricks and Tools #8

Debugger Tips, Tricks and Tools #7

Debugger Tips, Tricks and Tools #6

A welcome tool for improving Windows color management!

Debugger Tips, Tricks and Tools #5

Debugger Tips, Tricks and Tools #4

Debugger Tips, Tricks and Tools #3

Why does Visual Studio require debugger symbol files to exactly match the binary files that they were built with?