Bug Babble

Simple VS Addin (updated)

SteveJS — Wed, 21 Dec 2005 22:41:00 GMT

Back during the PDC for Whidbey I wrote a post about Creating a Simple Addin. Recently I was looking to update it and found a horrible bug in it. First, here is the corrected code:

public void OnConnection(object application, ext_ConnectMode connectMode, object addInInst, ref Array custom)
{
    _applicationObject = (DTE2)application;
    _addInInstance = (AddIn)addInInst;
    _debuggerEvents = _applicationObject.Events.DebuggerEvents;
    _debuggerEvents.OnEnterBreakMode += new _dispDebuggerEvents_OnEnterBreakModeEventHandler(DebuggerEvents_OnEnterBreakMode);

}

private DebuggerEvents _debuggerEvents;
private System.Collections.Generic.Queue<EditPoint> epQ = new System.Collections.Generic.Queue<EditPoint>();
void DebuggerEvents_OnEnterBreakMode(dbgEventReason Reason, ref dbgExecutionAction ExecutionAction)
{
    TextDocument td = (TextDocument) _applicationObject.ActiveDocument.Object("TextDocument");
    EditPoint ep = td.Selection.ActivePoint.CreateEditPoint();
    ep.SetBookmark();
    epQ.Enqueue(ep);
    if (epQ.Count > 5)
    {
        epQ.Dequeue().ClearBookmark();
    }

}

I've changed a couple of things. I now create the Editpoint first and use it to set the bookmark rather than setting the bookmark on the selection then creating an editpoint. This removes that possibility that I save a different place then where I set the bookmark.

The more important issue is I keep a member variable for _applicationObject.Events.DebuggerEvents. Why? Because it keeps that event provider from getting garbage collected. I had made the assumption that the lifetime of _applicationObject.Events.DebuggerEvents would be strictly tied to the DTE object, and managed by automation. That is not the case. The way this bug manifests is pretty ugly since the Addin works as expected for a while, until the Event Provider (_applicationObject.Events.DebuggerEvents) is collected and Finalized, at which point the events just stop coming.

I wasn't sure if this was by design or not. However, I found this KB article: Visual Studio .Net events being disconnected from add-in, so I am going with it being by design.

Naming threads in Win32 And .Net

SteveJS — Tue, 20 Dec 2005 08:25:00 GMT

When you are debugging an application with multiple threads it can be handy to have a better name than just the thread id. This is simple to do in managed code. There is a property on the Thread object that you can set. It is also possible to do this for native code. However, there is simply no way you will ever discover it unless someone points you to the difficult to find docs. I knew about the feature and still missed the docs the first time I looked for them!

For completness, here is the managed version: Setting a Thread Name (Managed).

The more interesting one is the native version: Setting a Thread Name (Unmanaged) (There is a typo in the documentation's code; 'except' should be '__except'. I've included a corrected copy below.)

The native method of setting the thread name is implemented by raising an SEH exception that is continued. If you go to the docs on RaiseException you'll see part of the reason for this strange mechanism. An attached native debugger will get a 'first chance' notification of the exception. Raising an exception is precisely what you need to do to get the native debugger's attention. The one raised here (0x406D1388) is recognized by VS (and WinDbg).

Note: If anything in your process installs a vectored exception handler, that handler will not be called on this exception code when the VS debugger is attached. The VS debugger always handles this exception code and continues execution, preventing any further handlers. In WinDbg you can go to Debug/Event Filters and set the "Visual C++ exception" to be disabled. With that setting, your vectored exception handler will get a shot at 0x406D1388 while WinDbg is debugging it.

//
// Usage: SetThreadName (-1, "MainThread");
//
typedef struct tagTHREADNAME_INFO
{
   DWORD dwType; // must be 0x1000
   LPCSTR szName; // pointer to name (in user addr space)
   DWORD dwThreadID; // thread ID (-1=caller thread)
   DWORD dwFlags; // reserved for future use, must be zero
} THREADNAME_INFO;

void SetThreadName( DWORD dwThreadID, LPCSTR szThreadName)
{
   THREADNAME_INFO info;
   info.dwType = 0x1000;
   info.szName = szThreadName;
   info.dwThreadID = dwThreadID;
   info.dwFlags = 0;

   __try
   {
      RaiseException( 0x406D1388, 0, sizeof(info)/sizeof(DWORD), (DWORD*)&info );
   }
   __except(EXCEPTION_CONTINUE_EXECUTION)
   {
   }
}

FuncEvil, the Clipboard, and Deadlocks

SteveJS — Fri, 09 Dec 2005 10:22:00 GMT

Mike Stall has a great post that, for a debugger dev, is the moral equivalent of sticking a fork in a toaster. The basic problem he addresses is that once you are stopped in the debugger the UI for your application will no longer paint. What if you need to see something on that UI? If you like to live on the edge, Mike’s Evil trick to render UI when stopped at a breakpoint is the answer for you. Unfortunately, you need to be stopped on the UI thread for it to work. Pumping messages on another thread will not unblock the UI to paint. Even more unfortunate is that switching threads in the debugger won’t help. Function evaluation when debugging managed code requires you to be at a reasonable point on the thread. It is very unlikely you will randomly be at such a point, unless you stopped on the thread in question. You can see the result I got when I stop on another thread, then switched to the UI thread:

?System.Windows.Forms.MessageBox.Show("pump")

Cannot evaluate expression because a thread is stopped at a point where garbage collection is impossible, possibly because the code is optimized.”

Now don’t let this dissuade you from how useful this very, very, very dangerous trick can be. Sometimes playing with fire is useful.

Not rendering the UI is only one problem that can happen due to the debugger stopping a process. A really bad problem that lack of message pumping can cause is clipboard deadlock. The real problem is due to delayed rendering, which requires the app that copied to the clipboard to be live. You can read about delayed rendering on the MSDN Clipboard Operations page. The basic problem is that the application you are debugging owns the clipboard, and other apps need it to respond to messages to get the contents of the clipboard. You can see this by creating a trivial winform app with a button and a RichTextbox:

(Note you need to use a RichTextBox here … a normal TextBox will actually copy the contents to the clipboard rather than using delayed rendering.)

First set a breakpoint in the button handler. Then select the “Copy me at your own Risk!” text and copy it to the clipboard with Ctrl-C. Now hit the button. Nothing is wrong yet. However if you go to any application and hit paste that application is now hung, and will remain so until you continue from the breakpoint. If you hit paste in the debugger you’ll deadlock that too without any way to continue from the bp. Clipboard deadlock has hit me few a times composing mail while stopped at a breakpoint. Using the trick that Mike outlines does free the applications waiting for your winform app to provide the clipboard data.

If you do not understand why func eval is evil I must end by pointing you to Mike's older post Func-eval is evil, and GreggM's more recent post Func eval and System.Windows.Forms.Form. Func Eval is powerful, and with great power comes a bunch of power to shoot yourself in the foot. Be careful using func eval.

Change Debugger behavior with Attributes

SteveJS — Sun, 04 Dec 2005 00:54:00 GMT

Certain metadata attributes modify the stepping, breakpoint, and callstack behavior of the VS managed Debugger. This is useful if you are creating an interpreter, language runtime, or a tool that modifies the code in a managed assembly. There are three flavors of these attributes that affect execution control. Each behaves slightly differently, and I always forget which does what, so I'm recording it here for my own reference as well.

The code below shows examples using all three attributes: DebuggerStepThrough, DebuggerHidden, and DebuggerNonUserCode. The example also shows the hidden line directive. I've inlined the explanation as comments in the code so it can travel with the example. This general area of writing tools that interact with the debugger has been well explored on Mike Stall's blog:

#line hidden and 0xFeeFee sequence points

How can I debug Just My Code?

Debug support for arbitrary state-machines

using System;

// This example shows the use of the three debugger attributes
// that modify the runtime behavior of the VS debugger.

// The motivating example is part of an AOP (Aspect Oriented Programming)
// implemenatation.
// The concepts around AOP are pretty interesting, but for this
// example it is sufficient to know that anything called a 'Thunk'
// is something added by the tool that is of no interest to the user.

// The example code is what a weaver that uses C# as a backend could
// send to the compiler to accomplish the hiding of
// JoinPointThunks, and the calls to those thunks.

// Try stepping through this with this option on:
// Tools/Options/Debugging/General/Enable Just My Code
// Then try the same with the option off.

// You'll see that each attribute creates a slightly different end user
// behavior and slightly different ease of debugging for yourself
// when you need to debug the glue code you are hiding from the user.

class DynamicallyAddedUserCode
{
    public static void bar()
    {
        Console.WriteLine("Hello advice\n");
    }
};
class Weaver
{
    [System.Diagnostics.DebuggerStepThrough]
    public static void ExampleJoinPointThunk_withDebuggerStepThrough()
    {
        //_________________________________________________________________
        // Tools/Options/Debugging/Enable Just My Code ON:
        //CALLSTACK view: Frame is viewable
        //BP: Will hit bp.
        //STEPPING: Will not stop inside due to a step.
        // (Even from bp hit! Any step is a step out.)
        // Step in will land inside called methods.
        //_________________________________________________________________
        // Tools/Options/Debugging/Enable Just My Code OFF:
        //CALLSTACK view: [External Code] frame.
        //   Real frame is viewable by selecting 'Show External Code'
        //   in Callstack window.
        //BP: No bp will set or hit. A warning is provided,
        //STEPPING: Will not stop inside method (step in or step out)
        // A step in will land in called methods.
        //_________________________________________________________________
        DynamicallyAddedUserCode.bar();
        int thisGoesTo = 11;
    }
    [System.Diagnostics.DebuggerHidden]
    public static void ExampleJoinPointThunk_withDebuggerHidden()
    {
        //_________________________________________________________________
        // Tools/Options/Debugging/Enable Just My Code ON:
        //CALLSTACK view: Frame is completely hidden.
        //BP: A Bp will set with no error, but will not hit!
        //stepping: A Step in will land in called methods.
        //_________________________________________________________________
        // Tools/Options/Debugging/Enable Just My Code OFF:
        //CALLSTACK view: Frame is completely hidden. (i.e. No [External Code])
        //BP:Warning provided, no bp will set or hit.
        //STEPPING: Will not stop inside method (step in or step out)
        // A step in will land in called methods.
        //_________________________________________________________________
        DynamicallyAddedUserCode.bar();
        int thisGoesTo = 11;
    }
    [System.Diagnostics.DebuggerNonUserCode]
    public static void ExampleJointPointThunk_withDebuggerNonUserCode()
    {
        //_________________________________________________________________
        // [Same as normal code.]
        // Tools/Options/Debugging/Enable Just My Code ON:
        //CALLSTACK view: Normal view, callstack is viewable.
        //BP: A bp will set and hit.
        //STEPPING: Step as normal.
        //_________________________________________________________________
        // [Same as Debugger StepThrough]
        // Tools/Options/Debugging/Enable Just My Code OFF
        //CALLSTACK view:[ External Code] frame.
        //   Real frame is viewable by selecting 'Show External Code'
        //   in Callstack window.
        //BP: Warning provided, no bp will set or hit.
        //STEPPING: Will not stop inside method (step in or step out)
        // A step in will land in called methods.
        //_________________________________________________________________
        DynamicallyAddedUserCode.bar();
        int thisGoesTo = 11;
    }
};
class Program
{
    static void foo()
    {
#line hidden //this emits a Sequence Point with line = 0xFeeFEE
        //Imagine these hidden areas were added dynamically
        Weaver.ExampleJoinPointThunk_withDebuggerStepThrough();
#line default
        int theone = 1;
#line hidden
        Weaver.ExampleJoinPointThunk_withDebuggerHidden();
#line default
        int volume = 11;
#line hidden
        Weaver.ExampleJointPointThunk_withDebuggerNonUserCode();
#line default
        int answer = 42;
    }
    static void Main(string[] args)
    {
        foo();
    }
}

// The #line hidden directive places 0xFeeFee sequence point in the line
// information. The 0xFeeFee sequence point tells the debugger to continue
// stepping. It is not necessary unless a step can end at the location.
// If you are writing a compiler and emitting your own debug information,
// then it is often possible to simply omit the line info for code that would
// be marked hidden. If you decompile the above code with ILDasm /Linenum you
// will see an example of that strategy.
// However there are cases where it is important to use the a hidden sequence
// point. Any place where a branch instruction lands must use a hidden sequence
// point if you do not want the debugger to stop there.
//
// The motivating example for this is having all exit points from a Try block
// branch to a single leave instruction. If you produce that type of codegen
// you will need to use 0xFeeFee to create reasonable stepping behavior.

Netmodules: Sort of like a lib

SteveJS — Mon, 28 Nov 2005 06:19:00 GMT

I first ran into netmodules before VS 2002 was shipped. The netmodule gave me precisely what I asked for, without actually solving my problem. I wanted to use multiple languages (C#, and C++ /clr) in a single assembly. It turns out a single assembly can be multi-module. Multi-module means a single .Net assembly is spread across multiple win32 dlls. What I really wanted was a single module assembly written in multiple languages. Netmodules couldn't solve my problem back then, but with the VS 2005 C++ linker they can now.

Check out JuFeng's articles for some more in depth coverage, and an explanation why you might want a multi-module assembly:

MultiModule Assemblies

Netmodule vs. Assembly

Since JuFeng's examples didn't show how to create debug info I thought I'd make an updated example here.

Here's a quick example with a multi-module assembly:

From the Visual Studio 2005 Command Prompt:

C:\blog\linkexample>vbc /target:module /debug+ b.vb

C:\blog\linkexample>csc /debug+ /addmodule:b.netmodule a.cs

C:\blog\linkexample>devenv /debugexe a.exe

Check out the modules window:

And the callstack window:

To get a single file assembly I'll need to recompile the cs file into a netmodule then use the c++ linker.

C:\blog\linkexample>csc /addmodule:b.netmodule /target:module /debug+ a.cs

C:\blog\linkexample>link /entry:fooCS.Main /out:theone.exe /subsystem:console /debug a.netmodule b.netmodule

C:\blog\linkexample>devenv /debugexe theone.exe

Now check out the modules window:

And the callstack window:

XSLT debugging in Visual Studio

SteveJS — Thu, 17 Nov 2005 10:40:00 GMT

Visual Studio is a big tool. It's easy to miss some useful features. Hopefully the XSLT debugger won't get missed. If you want to learn how to use XSLT debugging take a look at Neetu Rajpal's msdn article Introducing Xml tools in VS 2005. Below you can see the debugger in action as an xslt transform is being applied to some XML.

(Please excuse the nonsense XSLT I use in the example. I haven't had a chance to play around with XSLT for quite some time.)

XSLT debugging is interesting to me because much of it is the managed debugger. XSLT is compiled to msil, and the managed debugger is used to provide things like breakpoints and stepping. You can see this by opening the modules window. You'll see in the last column that the process is a managed process called the Microsoft.XSLDebugProxy.exe, and there will be some modules loaded with names like System.Xml.Xsl.CompiledQuery.1. All of the debugger windows work as expected: callstack, locals, immediate, threads, modules. You can see below the callstack for my nonsense example:

However! Go to the callstack window and right click to get the context menu. There will be an item called Show External Code. Pick that and you get to see much in the callstack window that was hidden before. The grey colored frames are because there is no debug info (pdb file) for that assembly, which is why those frames were marked as 'External Code' before. The CompiledQuery frames were hidden because the XSLT compiler placed an attribute on them to tell the debugger they were not of interest. With the expanded view you can now see the compiled query and the code that dispatched it.

However, while that is interesting it is unlikely to help you get at a bug in your XSLT. The Locals window however appears to be one of the more useful windows, that can help you see what is happening as the transforms are applied.

Edit and Continue on 64 bit Windows

SteveJS — Tue, 15 Nov 2005 21:10:00 GMT

Pascal writes about the joy of x64 Windows XP, and some drawbacks. One drawback is no Edit and Continue using the 64 bit CLR. You can, however, use E&C against the 32 bit CLR on a x64 machine. Here's how you do it:

You need to compile your managed assembly with a target CPU of x86. This will cause the 32 bit CLR to be used rather than the 64 bit CLR.

For a VB Project, right click on the project and go to Properties/Compile/Advanced Compile Options/Target CPU and set it to "x86".

For a C# Project, right click on the project and go to Properites/Build/Platform Target and set it to "x86".

If you want to (or must) use the 64 bit CLR, we unfortunately cannot provide E&C. In C# if you try to edit there is a dialog that tells you edits are not allowed . To prevent the dialog, disable E&C: Tools/Options/Debugging/Edit and Continue, and uncheck 'Enable Edit and Continue'.

Make DataTips Transparent

SteveJS — Sat, 12 Nov 2005 20:08:00 GMT

Enhanced datatips are my favorite new debugger feature. However, sometimes you want to see underneath the tip without losing your spot. Fortunately, JimGries thought of that. You can make the DataTip transparent by pressing the ctrl key or middle mouse button.

Here is an Enhanced DataTip In Visual Studio 2005:

Here is the same DataTip with the Crtl key pressed:

Update: JimGries pointed out that you can also make the DataTip transparent by pressing the middle mouse button (or wheel). If you have the wheel (and who doesn't today) you can also scroll with it, just as you'd expect.

Post Mortems

SteveJS — Thu, 10 Nov 2005 20:18:00 GMT

I won the guess the murderer contest. Shai and I compete to guess who the real murderer is on Law and Order Criminal Intent. Last night we watched Sunday's two hour show. It's a toss up who wins, but one of us always gets it by two thirds of the way through. That's the point where the amount of time left means the current suspect can't be the real murderer, and typically there was only one other possible suspect. The more unlikely the suspect, the higher our certainty. That just makes for better TV.

This mechanism of diagnosis is not very helpful for figuring out real world problems. There is no way to use a Tivo to tell you everything will be resolved in the next 20 minutes. It does however have something in common with Post Mortems. A Post Mortem is where the team shines the bright light of 20/20 hindsight on all the decisions that were hard. This perspective is much like having the extra knowledge that everything will be wrapped up in 20 minutes. Things that were unknowns are now knowns. Those tough calls that could have caused problems, yet no problems appeared: obviously good decisions. Those tough calls that did cause problems: obviously we shouldn't do that again … duh!

Unfortunately these fantastic insights are entirely informed via knowledge not present at the time the decision was made. I'm certainly not saying that Post mortems are useless! It's just that after you've categorized the bad decisions and the good decisions you still need to synthesize some useful advice that does not require precognition to use properly.

On the other hand, making decisions based on what makes for good TV sounds like it could be fun.

Perf Awareness is not Premature

SteveJS — Mon, 07 Nov 2005 02:38:00 GMT

I've found myself adding a timer to each of the unit test frameworks I use so I can see how long my tests take. I am not trying to do anything complex. I just want to be aware of the perf. The same way pass and fail are visible, 1ms vs 90ms is visible.

Premature optimization rightly has a bad rep. Creating complex code where performance doesn't matter is a bad tradeoff. That's not what I'm doing here. I am not trying to code for perf. I'm trying to be perf aware. If the test is fast I'm certainly not going to use that as an excuse to start changing code. If it's slow, I know about it and I won't check in something that kills perf. Once you have a number, the real question is: How do you judge if it is fast or slow? The answer is a perf budget.

Rico Mariani has a great Designing for Performance post where he talks about using Perf budgets.

In the absence of a perf budget, spend your time getting a perf budget rather than mucking about with code. Once you have perf budgets, being aware of resource consumption makes it easier to see problems earlier. A perf problem is a bug. Fixing a bug before it gets into the source tree is always a good idea.