Because this is my first post on this blog, let me introduce myself.  My name is John Allen.  I’ve been on the PFE team for 5 years and been with Microsoft for 9 years.  All of those years I’ve been debugging and troubleshooting all kinds of customer applications.   I focus on all developer technologies and products.

One of the main things that myself and the others on the PFE - Developer team do is capture and analyze memory dump files.  A majority of the time we are debugging .NET applications and have found that sometimes the debugger will hang while trying to analyze a dump.  There are a couple of key characteristics:

1. This is a 100% CPU Hang and the debugger does not recover - If your machine has more than one CPU the debugger will only be fully utilizing one of them.
2. The Hang occurs when loading symbols for a managed assembly
3. The Application that was dumped is .NET Framework 2.0 or greater.

We will talk more about symbol loading in a subsequent post but here is a workaround for this problem:

1. Create an empty SymSrv.ini file in your Debuggers Directory (Default - C:\Program Files\Debugging Tools for Windows).  For more information check out the Symsrv.ini documentation on MSDN - http://msdn2.microsoft.com/en-us/library/ms681416.aspx
2. Then open that file and add the following:

[exclusions] 

System.Windows.Forms.pdb     
System.pdb     
System.Web.pdb

After you have done this you can try loading the dump file again and see if the hang occurs again.  If this does not resolve your problem in run the following command:

0:020> lm m *_ni start end module name 

637a0000 63d02000 System_Xml_ni (deferred) 
64890000 6498a000 System_Configuration_ni (deferred) 
65140000 657a6000 System_Data_ni (deferred) 
65f20000 66ac6000 System_Web_ni (deferred) 
698f0000 69ad0000 System_Web_Services_ni (deferred) 
790c0000 79b90000 mscorlib_ni (deferred) 
7a440000 7ac06000 System_ni (deferred) 

This will dump all of the native image assemblies - these are the assemblies that have been nGened and are loaded from the nGen cache.  Then for each assembly that is listed add them to the [exclusions] list in the SymSrv.ini file.  Be sure to remove the _ni, replace it with .dll and replace the remaining underscores with a dot.  For instance in the above output the [exclusion] list would look like:

[exclusions] 
System.Xml.pdb 
System.Configuration.pdb 
System.Data.pdb
System.Web.pdb System.Web.Services.pdb
mscorlib.pdb
System.pdb

Finally, if you are looking for additional information on managed debugging you should check out Tess's blog - http://blogs.msdn.com/tess/

Thanks,
John Allen

This one kept me up late the other night.  We had an issue where ASP.NET was recompiling a page and we were getting an exception:

System.Exception:

Error loading VJSharpCodeProvider, Version=2.0.0.0, Culture=neutral,PublicKeyToken=b03f5f7f11d50a3a

This was interesting for two reasons:

1. The application does not use J#.
2. This is more subtle but nagged the back of my mind - Why is this thrown as a System.Exception and not a System.IO.FileNotFoundException or something more explicit since the framework typically throws specific exceptions.

Some additional background - this problem was only occurring during acceptance testing and did not occur in dev.  They were able to load up the page and successfully run through the application.  Then after stressing the application they would eventually start getting these J# errors and other Dynamic compilation errors from ASP.NET.  This continued until they restarted IIS and cleared the Temporary ASP.NET folder.

They did not have the J# Runtime installed on the system in dev or test and this was completely excepted because they were not using J#.  I got a look at the application and verified this.

Side Note - I have been doing product support for all 6 years that I have been working at Microsoft.  For those of you out there doing similar stuff I am sure you know this and if not take heed - always verify things when working to resolve issues like this.  It is not that anyone out there is intentionally hiding things but people frequently are unaware and when you ask, "Is such and such the case?" they may very well answer incorrectly.  So verify and double check.  Part of having someone from support come in is to bring a fresh perspective that analyzes the problem from the outside and that can frequently be invaluable.

So there is no J#.....Then why is this even trying to load?

I am glad you would ask that because I have an answer.  Deep in the bowels of the ASP.NET compile engine and the Code DOM we enumerate all of the "Code Providers" and see if they are valid.  Part of this test is done by trying to load the assemblies.  So the default code providers are:

• C#
• VB.NET
• JavaScript
• J#
• MC++

You can actually see these defaults documented in the machine.config.comments file in your %windir%\Microsoft.net\framework\v2.0.50727\config folder:

<compilers>
    <compiler language="c#;cs;csharp" extension=".cs" type="Microsoft.CSharp.CSharpCodeProvider, System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" />
    <compiler language="vb;vbs;visualbasic;vbscript" extension=".vb" type="Microsoft.VisualBasic.VBCodeProvider, System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" />
    <compiler language="js;jscript;javascript" extension=".js" type="Microsoft.JScript.JScriptCodeProvider, Microsoft.JScript, Version=8.0.1100.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
    <compiler language="vj#;vjs;vjsharp" extension=".jsl;.java" type="Microsoft.VJSharp.VJSharpCodeProvider, VJSharpCodeProvider, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
    <compiler language="c++;mc;cpp" extension=".h" type="Microsoft.VisualC.CppCodeProvider, CppCodeProvider, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
</compilers>

Those are the 5 that will get searched by default and you can add additional code providers in your config file (you cannot remove from these 5 defaults ones as far as I can tell).

All of this said it would seem that every application that does a dynamic compile must be looking for this J# provider.  So I went to do a quick test.  I created a simple ASPX page that did a dynamic compile.  Before I ran the page I when to the Fusion Log Viewer (http://msdn2.microsoft.com/en-us/library/e74a18c4(VS.80).aspx).  This tool will allow me to see what happens.  I clicked "Settings..." and checked "Log Binding Failures to Disk" and exited the application.  Then I did an IIS Reset to make sure that the setting was picked up and then I browsed to my page.  After the page came up I reopen the Fusion Log Viewer and sure enough there was a bind failure for the VJSharpCodeProvider.  If you do these same steps assuming you do not have the J# runtime installed you should see the same. 

I am seeing the same failure to load the VJSharpCodeProvider but I did not any errors and the dynamic compilation did not fail.  This however is similar to what they see - It works at first and then later breaks...Reviewing briefly - We know that the VJSharpCodeProvider is not on their system but the failure to load this file is normal so why does it start becoming a problem?

Well this is where their ResolveEventHandler (http://msdn2.microsoft.com/en-us/library/system.resolveeventhandler.aspx) comes in.  We found in one event log entry where they log the callstack that a ResolveEventHanlder was on top where they subscribed to the AssemblyResolve Event - http://msdn2.microsoft.com/en-us/library/system.appdomain.assemblyresolve.aspx.  From their we began looking into the code. 

First off for some background - The AssemblyResolve Event is part of the Assembly loading that .NET does.  After it has searched for and failed to find an assembly it will raise this event.  The application can then explicitly load (typically using LoadFrom or something) the assembly and return a reference to the assembly.  This is essentially a way to redirect the application to an assembly.  Once we go the code we found that they filtered out a couple of specific exceptions and then on any that they did not have a location for they just threw a System.Exception.  Ahhhhh, that explains the nagging question about why the exception type was System.Exception and not something else.

We took a look back at the AssemblyResolve Event and the docs have a glaring oversight - What do you do when you receive a request of an assembly that you do not know where it is?  The answer turns out to be - Return Null.  I verified this internally and some in the community has been nice enough to add this note in the "Community Content" section of the docs -http://msdn2.microsoft.com/en-us/library/system.appdomain.assemblyresolve.aspx

In conclusion, when the application is first launched their code has not executed yet therefore the ResolveEventHandler has not been hooked up to the AssemblyResolve event.  However later in the run of the application when a recompile occurs and we reprobe for the VJSharpCodeProvider the exception is thrown and the ASP.NET Compile code is not expecting and exception do this leads to other errors.

Once we fixed up the handler to return NULL the application worked.  It sailed through testing and I got to go to bed!!  I hope that this helps someone else out.

Thanks,
Zach

I was onsite the other day and the customer wanted to use Visual Studio 2005 to auto generate the Unit Test stubs for their ASP.NET application.  They have a lot of rules tied up in the ASP.NET application project.  When we tried to generate the Unit tests we kept getting the following error:

Source Code cannot be sufficiently parsed for code generation.  Please try to compile your project and fix any issues.

Well, needless to say the project compiled fine (and had been for a while) and ran fine.  We could not find any issues to fix.  My first thought was that this was a limitation because of the way they created their ASP.NET project.  So we got some details:

1. Originally Created in VS.NET 2003
2. Migrated to VS 2005
3. They did not convert to the new Page Model with App_Code and such.

After doing some digging I found out that the Unit Test Generation for some reason expects that the project has at least an empty "Settings."  So we opened the properties on the project and clicked Settings.  Sure enough there was nothing there.  So we clicked the link to create the "Settings" and got the grid that would enable us to add settings.  We just left it blank and rebuilt.  At that point we retried the generation of the Unit Tests and everything went smoothly.

Thanks,
Zach

Just a quick note about the team that is doing the blogging!

We are a group of engineers that do onsite support of Microsoft customers that are developing applications (internal to their company and ones that are shipped externally) that are based on Microsoft technologies.  This means a lot of debugging and troubleshooting. We are actually the team that is responsible that owns the Advanced Win32 Debugging and Advanced .NET Debugging courses that are taught to Microsoft's Premier Customers.

In posts on this blog we hope to share some of those experiences with the community to hopefully make things easier for others out there.  This will be a team effort so we will be introducing bloggers as they make their debut!

Since I am kicking off this post I will do my obligatory introduction.  My name is Zach Kramer.  I have been at Microsoft for about 6 years and on the PFE team for the past 2 years.  I focus on C++, ASP.NET, Visual Studio and other developer products.  I spend a lot of time debugging issues and I am an instructor both of the debug courses.  I have worked with customers all over the US and even some in Europe.  I will be following this up with a couple of technical posts so it is not all just fluff. ;-)

Thanks,
Zach