Are you profiling your application and now you can't debug anymore?

This is an interesting case I got very recently and that made me scratch my head a lot, wondering what was going wrong on customer's machine for a while... until I called Doug on the rescue and we (he) drove the call to an happy ending

I got this case from a colleague on the Visual Studio team; the customer reported a weird problem with the debugging of their ASP.NET 1.1 application (with Visual Studio 2003): when trying to step into the code line by line with F11, the debugger actually stopped only on the signature of every method skipping completely the body of the method, and to add something more, this happened only when they were using a specific namespace (e.g. MyCompany.MyNamespace.Group); if they used the same exact code but in a different namespace, everything worked fine . Moreover, this was happening on that very specific web server only, if they moved the code to another machine the debugger worked like charm (but moving to a different machine was not an option for them).
My first thought went to the .pdb files, some kind of mismatch between the between the code esecuted in memory and the symbols used for debugging.

At this regard I also have to say that the customer had an unusual configuration in his development environment: they had a Win2003 machine with Visual Studio installed, the local IIS had a virtual directory to host the application but the actual files were on a network disk; the files where residing on a shared folder on a different Win2003 server, which was the real web server running the ASP.NET application. Basically the customer used his local IIS (on the DEV machine) just to open the project, edited the code files (saved and compiled on disk Y:, which pointed to a different machine), and when they needed to debug the application they first set a breakpoint in the code, then browsed the target page on their machine (to load the application in the remote worker process), then attached manually the remote w3wp.exe. It worth also to say that even if in my opinion this is not exactly the best configuration they could use, this worked for quite a long time for them.

We first tryed to empty all the temporary folders involved (IE, ASP.NET etc...) but with no luck; we then captured some logs (network traces, Process Monitor, a memory dump of both Visual Studio and the affected w3wp.exe, to see what was happening during the network communication between the two machines, and where the .pdb files where from etc...), but again we got nowhere. The dump showed they had quite a few strong named assemblies in the /bin folders of their applications, and this is not good (as Tess explained here).

We first installed the strong named assemblies in the GAC and ensured that the symbol files where in the correct folder, Visual Studio module list showed the correct symbol file is loaded; we compiled the assembly without the strong name and deploy to the /bin folder rather than the GAC then stepping through the functions worked fine. Sn.exe reported that the checked assemblies where ok, and also testing with a new test key pair the problem was still there.

Doug then had an EasyAssist session with the customer (EasyAssist is a tool based on LiveMeeting, cut out to fit the needs of a remote assistance session) and found that also debugging a local application failed with the same symptoms... From examining environment variables it was noticed that CLR Profiling services were enabled and this pointed to a component of a 3^rd party diagnostic tool. Further debugging showed this tool was intercepting function calls during the attempted debug. Disabling this tool allowed debugging with CorDBG to now work and at this point the remote debugging problem was solved.

So the lesson learnt here is: if you need profiling tools use them, but pay attention that they don't interfere with Visual Studio and the debugger, and possibly have those services/tools disabled and enable them only when really needed.

Doug, if you want to add something more on this, you are very welcome!

Carlo