Several people have asked how to write something that runs some executable under a harness and then dumps all the exceptions that are thrown.
Back in November, I wrote  a similar harness to dump load module events using MDbg.  You can easily modify that to dump exceptions. See that blog entry for an explanation of how the code below works.

Here's sample code for a harness to print exceptions.

// Simple harness to dump exceptions.
// Be sure to reference MdbgCore.dll (which ships in the Whidbey beta 2 SDK)
using System;
using Microsoft.Samples.Debugging.MdbgEngine;

class Program
{
    static void Main(string[] args)
    {
        if (args == null || args.Length != 1)
        {
            Console.WriteLine("Usage: PrintEx <filename>");
            Console.WriteLine("   Will run <filename> and print all exceptions.");
            return;
        }
        Console.WriteLine("Run '{0}' and print all exceptions.", args[0]);

        MDbgEngine debugger = new MDbgEngine();
        debugger.Options.CreateProcessWithNewConsole = true;

        // Specify which debug events we want to receive.
        // The underlying ICorDebug API will stop on all debug events.
        // The MDbgProcess object implements all of these callbacks, but only stops on a set of them
        // based off the Options settings.
        // See CorProcess.DispatchEvent and MDbgProcess.InitDebuggerCallbacks for more details.
        debugger.Options.StopOnException = true;
        //Do 'debugger.Options.StopOnExceptionEnhanced = true;' to get additional exception notifications.
       
        // Launch the debuggee.
        MDbgProcess proc = debugger.CreateProcess(args[0], "", DebugModeFlag.Default, null);
       
        while(proc.IsAlive)
        {           
            // Let the debuggee run and wait until it hits a debug event.
            proc.Go().WaitOne();
            object o = proc.StopReason;

            // Process is now stopped. proc.StopReason tells us why we stopped.
            // The process is also safe for inspection.           
            ExceptionThrownStopReason m = o as ExceptionThrownStopReason;
            if (m != null)
            {
                try
                {
                    MDbgThread t = proc.Threads.Active;
                    MDbgValue ex = t.CurrentException;
                    MDbgFrame f = t.CurrentFrame;
                    Console.WriteLine("Exception is thrown:" + ex.TypeName + "(" + m.EventType +
                        ") at function " + f.Function.FullName + " in source file:"
                         + t.CurrentSourcePosition.Path + ":" + t.CurrentSourcePosition.Line);
                }
                catch(Exception e   )
                {
                    Console.WriteLine("Exception is thrown, but can't inspect it.");
                }
            }       
        }

        Console.WriteLine("Done!");
    }
} // end class

So compile the above (making sure you include the reference to mdbgcore.dll) and then run it on an app like:

using System;

class Foo
{
static void Main()
{
	System.Console.WriteLine("Hi!");
	try {
		throw new Exception("Bong!"); // line 11
	}
	catch(System.Exception)
	{
	}
	
	throw new Exception("Bong#2"); // line 15

}
}

And it prints:
Run 'c:\temp\t.exe' and print all exceptions.
Exception is thrown:System.Exception(DEBUG_EXCEPTION_FIRST_CHANCE) at function Foo.Main in source file:c:\temp\t.cs:11
Exception is thrown:System.Exception(DEBUG_EXCEPTION_FIRST_CHANCE) at function Foo.Main in source file:c:\temp\t.cs:15
Exception is thrown:System.Exception(DEBUG_EXCEPTION_UNHANDLED) at function Foo.Main in source file:c:\temp\t.cs:15
Done!
 

You could certainly make additional tweaks like:
1) printing the whole callstack instead of just the leaf-most method.
2) printing more of the exception (such as the message string, nested exceptions, or other fields)
3) printing other exception notifications (such as catch-handler-found or unwind notifications)
4) or support for attaching to an existing app.