VS (and mdbg) expose the $exception pseudo-variable which shows you the most recent exception. (kudos to Shaykatc for mentioning this a year ago).

Debugger authors can implement this with:
    HRESULT ICorDebugThread::GetCurrentException([out] ICorDebugValue **ppExceptionObject)

This will return an ICorDebugValue representing the current Exception object in the debuggee. Current exception means from the time the exception is thrown until the end of the catch block (so it will include filters and finallys). If there is no exception, it yields S_FALSE with a null out-parameter.

Some cases where this is useful:
1. You're inside an anonymous catch block, so there's no local variable corresponding to your exception.
2. You've stopped at a first chance exception, and haven't hit the catch-block yet. Or perhaps you're at a filter or finally inbetween the first-chance exception and catch block.
3. You're at an unhandled exception so there is no catch block
4. During a predicate, such as a conditional breakpoint.  

See the .idl file for additional caveats. Some key caveats that I want to repeat here:
1. Exceptions can be nested. GetCurrentException() just returns the most recent one.
2. If you func-eval at an exception, GetCurrentException() is cleared for the duration of the func-eval, and restored once the func-eval is completed. That gives you the most natural behavior for a nested break state.

Is this just a convenience function?
It's fair to ask if this is technically just a convenience function. Afterall, couldn't the debugger know the current exception object by tracking the various Exception notifications and Funceval Complete callbacks?
It turns out that this is not redundant with the rest of ICD because there is no "Catch Handler Finished" notification.  The debugger would get an Exception notification and thus know the starting lifetime for $exception, but it would never know when it was finished.