Just FYI,  we don't support interop-debugging on 64-bit CLR (or on win9x).
You can still do 32-bit interop-debugging within the WOW on 64-bit.
(You also can't do managed-debugging locally across the WOW boundary.  You can set up a remote debugging channel to yourself across the WOW boundary.)

The reason for the interop-debugging restrictions is that Interop-debugging's current architecture is extremely OS dependent, and we do lots of strange things and use native debugging API in a very aggressive and unusual way. Just making managed debugging work on 64-bit was an accomplishment. 

Attempting to do interop-debugging on these platforms will generate a CORDBG_E_INTEROP_NOT_SUPPORTED hr (see corerror.h) from ICorDebugProcess::CanLaunchOrAttach, ICorDebug::CreateProcess, and ICorDebug::DebugActiveProcess.

One workaround is to take the VB4 approach for debugging COM objects (This is not an officially supported tactic!!!):
1) Attach a managed-only debugger like normal.
2) attach a native-debugger (like windbg) that gives you full control over how to route exceptions ("gn" vs "gh"). Then be sure to "gn" all first-chance exceptions that come up to make sure they get passed onto the managed debugger. (This will especially be single-step and breakpoint exceptions).  (See here and here for more technical details).
The managed-only debugger will likely be frozen anytime the native debugger is stopped, so you'll only be able to use one at a time.
I repeat, this is not an officially supported tactic, but it may be enough to solve your problem. Proceed at your own risk. If you get the "gn" / "gh" wrong in windbg, everything will blow up and your debuggee and managed debugger may crash or hang. If you don't know what gn / gh means or you don't understand how win32 exceptions works; please don't try this. This is not guaranteed to work in all scenarios. (Do I have enough disclaimers? I can add more ...)