You may have noticed that debugging optimized builds (eg, what you commonly get when you attach to a end-user instance of your app that wasn't launched from your debugging environment) is usually a degraded experience.
At an architectural level, there's a fundamental tension between being debuggable and optimizing. The debugger generally requires the code to behave to a certain contract established between the compiler and the debugger (and recorded in the PDB). That contract mainly requires the code to maintain a close relationship to the original source code. The Optimizer makes aggressive transforms to the code for the sole purpose of making it execute more efficiently, with no regards to the original source code.
Here are some of the things that break and why. This is not a complete list. This is general, so isn't managed or native code specific:
In theory, the you can compensate for many of these problems by making the PDB contracts richer so that the optimizer can describe to the debugger what it's actually doing.