I need to make an embarrassing public service announcement (PSA): Pinvokes are 100x slower under the debugger in VS2005 / Whidbey.  For example, this is the issue that came up here. It may also manifest as your app "hanging" under the debugger w/ 100% CPU usage.

How did this happen?
I'll explain why, but I first want to make it clear this explanation is not an attempt to justify this horribly broken behavior.

1. In Whidbey, we introduced Managed Debugging Assistants, which are runtime checks designed to catch a certain class of end-user bugs that corrupt the runtime. These catch things like stack-imbalance on pinvokes.

2. A certain set of MDAs are on by-default under the debugger. Debuggers aren't supposed to change behavior, so we tried to be very rigorous about scrubbing this list. Any MDA with notable side-effects (eg, side-effects larger than the overhead of the manage-debugging channel) was cut from the list. For example, we cut the QueryInterface MDA (which made additional QI calls to ensure that an object was complying with the QI calls) because making additional QI calls was too big of a change. One of those extra QIs would crash, and then we'd get "this app only crashes under the debugger" problems.

3. We didn't identify performance side-effects in our MDA scrub. One of the pinvoke MDAs (PInvokeStackImbalance - for checking stack balance) changes the code-gen of the pinvoke stubs to go down a slow path.  We didn't explicitly check for performance-side effects.

This slipped through because several individual teams did the right thing in isolation, but didn't really step back and look at the whole system:
On one hand: This pinvoke check is empirically extremely valuable. Stack imbalance pinvokes corrupt the runtime and are near impossible for an end-user to diagnose. This single error check may save you days of debugging. Thus, the pinvoke team had a strong motivation to push for this to be on by default under a debugger. Furthermore, pinvoke team advises that pinvokes should only be used very rarely (relative to your overall managed code). Thus we didn't explicitly stress high volume pinvokes under the debugger because we didn't believe it was a core scenario (this frees our limited resources up to test other core scenarios).
On the other hand: The MC++ team uses pinvokes extremely liberally. (In a worst case, every function call in MC++ could be a pinvoke).

What's the workaround?
If you get hit with this slowdown, disable the PInvokeStackImbalance MDA.  See here for steps on how to disable MDAs (there are a variety of options).