Bug Babble

Simple VS Addin (updated)

I've changed a couple of things.  I now create the Editpoint first and use it to set the bookmark rather than setting the bookmark on the selection then creating an editpoint.  This removes that possibility that I save a different place then where I set the bookmark. 

The more important issue is I keep a member variable for _applicationObject.Events.DebuggerEvents.  Why?  Because it keeps that event provider from getting garbage collected.  I had made the assumption that the lifetime of _applicationObject.Events.DebuggerEvents would be strictly tied to the DTE object, and managed by automation.  That is not the case.  The way this bug manifests is pretty ugly since the Addin works as expected for a while, until the Event Provider (_applicationObject.Events.DebuggerEvents) is collected and Finalized, at which point the events just stop coming. 

I wasn't sure if this was by design or not.  However, I found this KB article: Visual Studio .Net events being disconnected from add-in, so I am going with it being by design.

Naming threads in Win32 And .Net

When you are debugging an application with multiple threads it can be handy to have a better name than just the thread id.  This is simple to do in managed code.  There is a property on the Thread object that you can set.  It is also possible to do this for native code. However, there is simply no way you will ever discover it unless someone points you to the difficult to find docs.  I knew about the feature and still missed the docs the first time I looked for them!

For completness, here is the managed version: Setting a Thread Name (Managed).

The more interesting one is the native version: Setting a Thread Name (Unmanaged) (There is a typo in the documentation's code; 'except' should be '__except'. I've included a corrected copy below.)

The native method of setting the thread name is implemented by raising an SEH exception that is continued.  If you go to the docs on RaiseException you'll see part of the reason for this strange mechanism.  An attached native debugger will get a 'first chance' notification of the exception.  Raising an exception is precisely what you need to do to get the native debugger's attention. The one raised here (0x406D1388) is recognized by VS (and WinDbg).

Note: If anything in your process installs a vectored exception handler, that handler will not be called on this exception code when the VS debugger is attached.  The VS debugger always handles this exception code and continues execution, preventing any further handlers.  In WinDbg you can go to Debug/Event Filters and set the "Visual C++ exception" to be disabled.  With that setting, your vectored exception handler will get a shot at 0x406D1388 while WinDbg is debugging it.

//
// Usage: SetThreadName (-1, "MainThread");
//
typedef struct tagTHREADNAME_INFO
{
   DWORD dwType; // must be 0x1000
   LPCSTR szName; // pointer to name (in user addr space)
   DWORD dwThreadID; // thread ID (-1=caller thread)
   DWORD dwFlags; // reserved for future use, must be zero
} THREADNAME_INFO;

void SetThreadName( DWORD dwThreadID, LPCSTR szThreadName)
{
   THREADNAME_INFO info;
   info.dwType = 0x1000;
   info.szName = szThreadName;
   info.dwThreadID = dwThreadID;
   info.dwFlags = 0;

   __try
   {
      RaiseException( 0x406D1388, 0, sizeof(info)/sizeof(DWORD), (DWORD*)&info );
   }
   __except(EXCEPTION_CONTINUE_EXECUTION)
   {
   }
}

FuncEvil, the Clipboard, and Deadlocks

Mike Stall has a great post that, for a debugger dev, is the moral equivalent of sticking a fork in a toaster.  The basic problem he addresses is that once you are stopped in the debugger the UI for your application will no longer paint.  What if you need to see something on that UI?  If you like to live on the edge, Mike’s Evil trick to render UI when stopped at a breakpoint is the answer for you.  Unfortunately, you need to be stopped on the UI thread for it to work.  Pumping messages on another thread will not unblock the UI to paint.  Even more unfortunate is that switching threads in the debugger won’t help.  Function evaluation when debugging managed code requires you to be at a reasonable point on the thread. It is very unlikely you will randomly be at such a point, unless you stopped on the thread in question.  You can see the result I got when I stop on another thread, then switched to the UI thread:

?System.Windows.Forms.MessageBox.Show("pump")

Cannot evaluate expression because a thread is stopped at a point where garbage collection is impossible, possibly because the code is optimized.”

Now don’t let this dissuade you from how useful this very, very, very dangerous trick can be.  Sometimes playing with fire is useful.

Not rendering the UI is only one problem that can happen due to the debugger stopping a process.  A really bad problem that lack of message pumping can cause is clipboard deadlock.  The real problem is due to delayed rendering, which requires the app that copied to the clipboard to be live.  You can read about delayed rendering on the MSDN Clipboard Operations page.  The basic problem is that the application you are debugging owns the clipboard, and other apps need it to respond to messages to get the contents of the clipboard. You can see this by creating a trivial winform app with a button and a RichTextbox:

  (Note you need to use a RichTextBox here … a normal TextBox will actually copy the contents to the clipboard rather than using delayed rendering.)

First set a breakpoint in the button handler.  Then select the “Copy me at your own Risk!” text and copy it to the clipboard with Ctrl-C.  Now hit the button.  Nothing is wrong yet.  However if you go to any application and hit paste that application is now hung, and will remain so until you continue from the breakpoint.  If you hit paste in the debugger you’ll deadlock that too without any way to continue from the bp. Clipboard deadlock has hit me few a times composing mail while stopped at a breakpoint.   Using the trick that Mike outlines does free the applications waiting for your winform app to provide the clipboard data.

If you do not understand why func eval is evil I must end by pointing you to Mike's older post Func-eval is evil, and GreggM's more recent post Func eval and System.Windows.Forms.Form.  Func Eval is powerful, and with great power comes a bunch of power to shoot yourself in the foot.  Be careful using func eval.

Change Debugger behavior with Attributes

Netmodules: Sort of like a lib

I first ran into netmodules before VS 2002 was shipped. The netmodule gave me precisely what I asked for, without actually solving my problem.  I wanted to use multiple languages (C#, and C++ /clr) in a single assembly.  It turns out a single assembly can be multi-module.  Multi-module means a single .Net assembly is spread across multiple win32 dlls.  What I really wanted was a single module assembly written in multiple languages. Netmodules couldn't solve my problem back then, but with the VS 2005 C++ linker they can now.

Check out JuFeng's articles for some more in depth coverage, and an explanation why you might want a multi-module assembly:

MultiModule Assemblies

Netmodule vs. Assembly

Since JuFeng's examples didn't show how to create debug info I thought I'd make an updated example here.

Here's a quick example with a multi-module assembly:

From the Visual Studio  2005 Command Prompt:

C:\blog\linkexample>vbc /target:module /debug+ b.vb

C:\blog\linkexample>csc /debug+ /addmodule:b.netmodule a.cs

C:\blog\linkexample>devenv /debugexe a.exe

Check out the modules window:

And the callstack window:

To get a single file assembly I'll need to recompile the cs file into a netmodule then use the c++ linker.

C:\blog\linkexample>csc /addmodule:b.netmodule /target:module /debug+ a.cs

C:\blog\linkexample>link /entry:fooCS.Main /out:theone.exe /subsystem:console /debug a.netmodule b.netmodule

C:\blog\linkexample>devenv /debugexe theone.exe

Now check out the modules window:

And the callstack window:

XSLT debugging in Visual Studio

Visual Studio is a big tool.  It's easy to miss some useful features.  Hopefully the XSLT debugger won't get missed.   If you want to learn how to use XSLT debugging take a look at Neetu Rajpal's msdn article Introducing Xml tools in VS 2005.  Below you can see the debugger in action as an xslt transform is being applied to some XML.

(Please excuse the nonsense XSLT I use in the example.  I haven't had a chance to play around with XSLT for quite some time.)

XSLT debugging is interesting to me because much of it is the managed debugger.  XSLT is compiled to msil, and the managed debugger is used to provide things like breakpoints and stepping. You can see this by opening the modules window.  You'll see in the last column that the process is a managed process called the Microsoft.XSLDebugProxy.exe, and there will be some modules loaded with names like System.Xml.Xsl.CompiledQuery.1.   All of the debugger windows work as expected: callstack, locals, immediate, threads, modules.  You can see below the callstack for my nonsense example:

However! Go to the callstack window and right click to get the context menu.  There will be an item called Show External Code.  Pick that and you get to see much in the callstack window that was hidden before.  The grey colored frames are because there is no debug info (pdb file) for that assembly, which is why those frames were marked as 'External Code' before.  The CompiledQuery frames were hidden because the XSLT compiler placed an attribute on them to tell the debugger they were not of interest.   With the expanded view you can now see the compiled query and the code that dispatched it.

However, while that is interesting it is unlikely to help you get at a bug in your XSLT.  The Locals window however appears to be one of the more useful windows, that can help you see what is happening as the transforms are applied.

Edit and Continue on 64 bit Windows

Pascal writes about the joy of x64 Windows XP, and some drawbacks.  One drawback is no Edit and Continue using the 64 bit CLR.  You can, however, use E&C against the 32 bit CLR on a x64 machine.  Here's how you do it:

You need to compile your managed assembly with a target CPU of x86.  This will cause the 32 bit CLR to be used rather than the 64 bit CLR. 

For a VB Project, right click on the project and go to Properties/Compile/Advanced Compile Options/Target CPU and set it to "x86".

For a C# Project, right click on the project and go to Properites/Build/Platform Target and set it to "x86".

If you want to (or must) use the 64 bit CLR, we unfortunately cannot provide E&C.  In C# if you try to edit there is a dialog that tells you edits are not allowed .  To prevent the dialog, disable E&C: Tools/Options/Debugging/Edit and Continue, and uncheck 'Enable Edit and Continue'.

Make DataTips Transparent

Enhanced datatips are my favorite new debugger feature. However, sometimes you want to see underneath the tip without losing your spot. Fortunately, JimGries thought of that. You can make the DataTip transparent by pressing the ctrl key or middle mouse button.

Here is an Enhanced DataTip In Visual Studio 2005:

Here is the same DataTip with the Crtl key pressed:

Update: JimGries pointed out that you can also make the DataTip transparent by pressing the middle mouse button (or wheel).  If you have the wheel (and who doesn't today) you can also scroll with it, just as you'd expect.

Post Mortems

I won the guess the murderer contest.   Shai and I compete to guess who the real murderer is on Law and Order Criminal Intent.  Last night we watched Sunday's two hour show.  It's a toss up who wins, but one of us always gets it by two thirds of the way through.  That's the point where the amount of time left means the current suspect can't be the real murderer, and typically there was only one other possible suspect.  The more unlikely the suspect, the higher our certainty.  That just makes for better TV. 

This mechanism of diagnosis is not very helpful for figuring out real world problems.  There is no way to use a Tivo to tell you everything will be resolved in the next 20 minutes.  It does however have something in common with Post Mortems.   A Post Mortem is where the team shines the bright light of 20/20 hindsight on all the decisions that were hard.  This perspective is much like having the extra knowledge that everything will be wrapped up in 20 minutes.  Things that were unknowns are now knowns.  Those tough calls that could have caused problems, yet no problems appeared: obviously good decisions.  Those tough calls that did cause problems: obviously we shouldn't do that again … duh! 

Unfortunately these fantastic insights are entirely informed via knowledge not present at the time the decision was made.  I'm certainly not saying that Post mortems are useless!  It's just that after you've categorized the bad decisions and the good decisions you still need to synthesize some useful advice that does not require precognition to use properly.

On the other hand, making decisions based on what makes for good TV sounds like it could be fun.

Perf Awareness is not Premature

I've found myself adding a timer to each of the unit test frameworks I use so I can see how long my tests take.  I am not trying to do anything complex.  I just want to be aware of the perf.  The same way pass and fail are visible, 1ms vs 90ms is visible.

Premature optimization rightly has a bad rep.  Creating complex code where performance doesn't matter is a bad tradeoff.   That's not what I'm doing here.  I am not trying to code for perf.  I'm trying to be perf aware.  If the test is fast I'm certainly not going to use that as an excuse to start changing code.  If it's slow, I know about it and I won't check in something that kills perf.  Once you have a number, the real question is: How do you judge if it is fast or slow?  The answer is a perf budget.

Rico Mariani has a great Designing for Performance post where he talks about using Perf budgets.

In the absence of a perf budget, spend your time getting a perf budget rather than mucking about with code.  Once you have perf budgets, being aware of resource consumption makes it easier to see problems earlier.  A perf problem is a bug.  Fixing a bug before it gets into the source tree is always a good idea.

Checking the result of new is a bug in C++

At least, it is a bug in VC8.  That check won't happen.  Reading Larry Osterman's recent posts "What's wrong with this code, part 15" and the answers, reminded me this behavior changed in VC8.  If you check the result of new in code compiled with VC8, your code is wrong.  The call to new will throw.  Yes, your code worked in 5.0, 6.0, (maybe) 7.0 and 7.1, but it doesn't now.

You can get the old behavior if you link with nothrownew.obj.  The linker will then make every call to new in your dll (or exe) non-throwing.   This is incompatible with STL, or any other code that relies on new throwing.

The only other option is to use std::nothrow placement new.  Don't do that. 

If you are writing code that uses no throw semantics, you probably need those same semantics from the libs you link as well.   If you are linking a lib that was coded expecting new to return NULL rather than throw, the linker doesn't know that.  In VC8 it will link the throwing version.  Linking with nothrownew.obj tells the linker that everything needs the old, non-standard version of new that returns NULL.

What if you are using 2 libs?  One that needs the non-standard new, and one that needs the standard new?  If that is the case, package one in it's own dll, or change it so you have consistency on the definition of new.  Don't bother trying to get the linker to do something magical.  It won't.  The linker makes a decision on what new means and that's it for the entire dll.

History of new through the VC's:

In VC5.0, and VC6.0 you basically got nothrow new and had to jump through hoops to get a throwing version.  MFC did use a throwing version, but it doesn't throw std::bad_alloc.  Also new(std::nothrow), had bugs which caused it to throw in some circumstances.  For users of non throwing new, life was good.  For everyone else, not so much. See this msdn article "Don't Let Memory Allocation Failures crash your lagacy STL Application" for the details.

In 7.0 and 7.1 the VC++ team attempted to do something clever.  The version of new you get is controlled by the first significant header in the first obj linked.  This means a large number of examples do exactly what you'd expect.  It also means there are corner cases where it is almost impossible to understand what's happening.  You could force a standard complying new by linking thrownew.obj.

In 8.0 the VC++ team decided to clean up many standard compliance issues.  Now the C++ standard is used.  Full Stop.  If you want the old behavior, you must link nothrownew.obj.

Finally, I must thank Martyn Lovell for explaining the background on this issue, and clarifying some of the details.  If there are any errors here they are due to me alone.

Tabs vs Spaces

Of course, there is only one answer to whether tabs should be allowed in a source file.  The utility of one is just so obvious I'll simply avoid commenting on it.   :-) 

If you must wander from sources written with one convention to those written in another, you should definitely set Visual Studio (or whatever editor you use) to show whitespace as visible.  You can change this in VS by going to Edit/Advanced/View White Space.

I also change the default color for visible whitespace to silver.  Silver on a white background is unobtrusive, yet noticeable when there is inconsistency.

(Next time, I'll go into whether Big Endian or Little Endian byte ordering is better. :-)

Data Breakpoints

The VS debugger allows two types of breakpoints. There are location breakpoints and data breakpoints.  Each has lots of bells and whistles.  However, sometimes people confuse a Location bp with a condition, for a data bp.  That's unfortunate because a data bp solves a different problem.

A data breakpoint involves telling the debugger to stop when the value at an address changes.  Use a data bp when you know some data is getting changed and you want to know where in the instruction stream that is happening.

A location breakpoint with a condition triggers a stop when the instruction pointer reaches the proper address.  After the stop we then evaluate the condition to determine if we should continue automatically or notify you that the breakpoint hit.  Use a condition when the underlying breakpoint gets hit too often and you want to constrain it to certain circumstances.

Data bps are implemented with a hardware register on the cpu.  In VS Whidbey, we allow Data bps for native code.  We restrict the number of them to the number of available hardware registers that support data bps.  Though the hardware supports read based data bps, in VS we only implement stopping when the value is written.   Managed code does not have Data Bps.  Mike Stall has a blog topic asking for votes on potential feature requests.  Data breakpoints are on that list.

In Whidbey, to set a data bp, open the breakpoint window (Debug/Windows/Breakpoints).  Hit the 'New' menu button, and pick 'New Data Breakpoint'.  The dialog has specific suggestions on what to enter.  I suggest entering it as a hex address.  Regardless of what you enter, the line in the Breakpoints window will show a hex address. Make sure it points to the data you are interested in.  If you try to create more than four data bps you will get a dialog saying the maximum number of data breakpoints have already been set.

This UI metaphor is primitive compared to what we had in the past.  Why did it change to something that seems inferior?  In past versions you could enter an arbitrary expression.  We would figure out what needed to be watched and stop when the value of that expression changed.  This is awesome except in the vast majority of cases where it would trigger something called 'Emulation'.  Someone using data bps would find it difficult to figure out when emulation would get triggered, and believe me it was something to avoid.  Emulation is where we cannot provide the Data bp using a register.  Instead we single step and check the value at the end of each step.  This is slow.  Not just slow, incredibly slow.  Go walk the dog, get a cup of coffee, read the newspaper slow.   There are a few tricks we could have done to keep the old UI and avoid emulation, but they fall down in certain cases.  We didn't want to have data bps, and 'sort of data bps'.  Instead, we made a decision to expose what really works in a simpler manner.  That avoids the trap of accidentally falling into 'emulation'.

At the beginning I mentioned that people sometimes confuse the bells and whistles with the type of breakpoint.  The bells and whistles apply to both kinds of underlying bps.  Each of those bells and whistles is (10 cent word warning) Orthogonal.  You can add a condition to a data bp, or add a hit count, or make a data bp into a data bp tracepoint with 'When hit'.  All of these are accessible via a right click in the breakpoint window.  Composing those bells and whistles can provide powerful debugging techniques.  Just make sure you are using the right kind of underlying bp.

Stop Mid Func Eval, Nested Break States

A coworker mentioned Func Eval sounds like "Funky-val".  Stopping at a breakpoint in the middle of a function evaluation could be considered funky.  It is also useful.

VS Whidbey allows stopping at a BP or Exception during a function evaluation in C# or VB code.  To use it, you need to do the function evaluation from the 'Immediate Window', rather than the 'Watch', or 'Quick Watch' windows. 

Why is nesting restricted to the Immediate Window?  Despite the name, the Immediate window is not immediate.  Watch windows do synchronous Function evaluation.  If the evaluation takes too long, it will timeout and give an error.  That doesn't happen in the immediate window, because it is infinitely patient.  The immediate window is implementing a REPL (Read Eval Print Loop).  However, it doesn't bother to wait for the Eval & Print to finish, instead it executes an asynchronous func eval. The 'Print' is going to happen whenever the Eval gets done, which allows us to do the Nested Break States.  Here's an example:

static bool Foo()

{

return false;

}

In the immediate window evaluate Foo 3 times:

?Foo()

?Foo()

?Foo()

Here is the Callstack:

When I unrolled the set of Evaluations by hitting F5 three times, here is what I got in the immediate Window:

?Foo()

?Foo()

?Foo()

false

false

true

You might ask … why is the last one true?  It's because I used Edit and Continue to change the return value to true before hitting F5 the third time.  Nested Func Eval allows you to literally "push a debug task" on the stack.  Once you've figured out why Foo was doing something wrong you can go back to debugging what you were looking at before.
 

More hours in the day

In the summer of '94 I sketched a design for my ideal 'mp3' player.  I didn't spec the compression format.  If I had, it would have been called a 'bit' player back then.  I wanted a walkman replacement.  However, the feature I most craved was understandable fast play, and rewind.  My motive was to record all my lectures for a day, then listen to them later in fast play, thus creating more hours in my day.

In 'Digital Death Lab' class, my lab partner, Gio, and I had implemented fast play as our final project.  Maintaining pitch is the hard part, you can toss pauses between words, but you can't just toss samples without making it sound like 'A Very Special Chipmunks Physics Lecture'.  I see that Windows media player 10 has fast play.  Does anyone know if mobile products expose it? 

Do any mp3 players solve my 'more hours in the day scenario' for a mobile device?