If broken it is, fix it you should

Using the powers of the debugger to solve the problems of the world - and a bag of chips    by Tess Ferrandez, ASP.NET Escalation Engineer (Microsoft)

ASP.NET Memory: If your application is in production… then why is debug=true

ASP.NET Memory: If your application is in production… then why is debug=true

Rate This
  • Comments 86

Statement

 

“Ensure that the debug="false" on the <compilation> element in the web.config file of each and every ASP.NET application on the server. The default during development is "true" and it is a common mistake to allow this development time setting to find its way onto production servers during deployment. You don't need it set to true in production and it often leads to memory overhead and inefficiencies.”

 

What problems does leaving debug=true cause?

 

There are three main differences between debug=true and debug=false:

 

  1. ASP.NET Timeouts
  2. Batch compilation
  3. Code optimization

 

ASP.NET Timeouts

 

When debug is set to true, asp.net requests will not time out. This is to allow you to debug with visual studio at your own pace without having to worry about the requests suddenly disappearing. Of course in a production environment timeouts are crucial to avoid for requests to be stuck indefinitely, so this is reason #1 to make sure debug is set to false when the application is deployed into production.

 

Batch compilation

 

In short, when debug=true, we don’t batch compile, when debug=false we do…

 

What does this mean? 

 

When an aspx, asax, or ascx page is first requested it gets compiled into an assembly. This assembly has a name like 3ks0rnwz.dll or similar (8 characters) and stores the class for the actual ascx, asax, or aspx page (not the code behind).  The assembly goes into a folder in the C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\Temporary ASP.NET Files with the same name as the application.

 

The code behind class gets compiled into the main dll for the assembly, and it along with all the other dlls in the applications bin directory get shadow copied to the Temporary ASP.NET files.

 

Back to the 3ks0rnwz.dll… If we have debug=true, we create one dll per aspx, asax, or ascx page and this dll is compiled in debug mode, so if you have 100 web pages, you will have 100 assemblies, and they are generated as the pages are requested.

 

If we instead have debug=false, we batch compile, which means that the first time you request any page in your application, we compile the whole batch into one big assembly. This is a truth with some modification.  The user controls (ascx pages) are compiled into a separate assembly from the aspx pages and the aspx pages are compiled in groups based on what other files (read usercontrols) they include. The global.asax is also compiled separately.  And batch compilation occurs on a directory bases, meaning that if your application has subdirectories, the subdirectories are compiled separately to avoid for example name clashes, as it is valid to have two aspx pages with the same name in different directories. But all in all, instead of 100 dlls, you might end up with 3 or 4.

 

Ok, big deal? It’s the same code so the size of the combined assemblies shouldn’t much differ from the size of the individual assemblies right?  Truth is, there probably isn’t an enormous difference. But… and this is a big but… there is overhead for each dll, and if the dll is compiled in debug mode there is overhead for items needed for debugging, and … last but not least (in fact probably most important), the assemblies won’t be laid exactly side by side, so with a large number of assemblies you start fragmenting the virtual address space making it harder and harder to find large enough spaces to store the managed heaps, potentially causing out of memory exceptions. 

 

One caution even if you have debug=false, is that if you go in and change something in one of your aspx pages, this page will have to be recompiled, but this doesn’t cause an appdomain reload so the whole application is not batch compiled again.  This has the effect that the page will now get recompiled separately and get its own dll, so don’t change your aspx pages on a live server too often. 

 

There is a setting in machine.config determining how many recompiles are allowed before the app domain restarts, by default it is set to 15, so after 15 recompilations the app domain will restart, just as it would if you touched the web.config or touched the bin directory.  

 

Code optimization

 

In order to be able to step through code line by line the JITter can’t really optimize the code which means that your debug dlls will be less performant than if they were compiled in release mode.

 

So as you can probably figure, there is a large benefit to having debug=false in production…

 

How can you identify it in a memory dump?

 

To find out if any of the applications on your server run with debug=true you can run a nifty command in sos.dll called !finddebugtrue which will list out all applications where debug=true in the web.config, now how easy is thatJ

 

0:016> !finddebugtrue

Debug set to true for Runtime: 61b48dc, AppDomain: /MyDebugApplication

Debug set to true for Runtime: 1f50e6d8, AppDomain: /MemoryIssues

Total 16 HttpRuntime objects

 

And to find out if you forgot to compile some of your assemblies in release mode run !finddebugmodules

 

0:016> !finddebugmodules

Loading all modules.

Searching for modules built in debug mode...

 

MyDebugApplication.dll built debug

MemoryIssues.dll built debug

fl4sq-9i.dll built debug

wepr1t3k.dll built debug

r9ocxl4m.dll built debug

zmaozzfb.dll built debug

 

Done Seaching

 

 

The dlls above with weird 8 character names are the dlls generate when JITing the aspx pages, so they will go away when debug=false.

 

 

Oh, before I forget… when you change from debug=true to debug=false it is a good idea to clean out your Temporary ASP.NET files for this application so you don’t have some old junk in there causing it to still not batch compile.

 

 

In ASP.NET 2.0 there is a switch that can be turned on in machine.config that turns off all debug=true, so in 2.0 applications you can do this directly without worrying about finding out which applications do and don’t have it.

 

<configuration>

    <system.web>

          <deployment retail=”true”/>

    </system.web>

</configuration>

 

If you want some more goodies about debug=true, read ScottGu’s blog post about it http://weblogs.asp.net/scottgu/archive/2006/04/11/442448.aspx

 

 

 

 





  • Only thing I can think of is that disabling debugging generated some type of exception, perhaps because of duplicate class names or similar. I would attach a debugger and try to log in to see if something like that is happening.
  • We set debug=false in our production environment, and this caused the site to crash due to CS1595 [user control] is defined in multiple places. IISReset + wiping the Temp ASP.NET files clears up this error temporarily, but it reoccurs again later if a change is made to the web.config. I've seen all the threads about multiple DLLs, Src vs. CodeBehind, compiler flags, but this is not the cause in our case. We never had this issue until setting debug=false.
  • Hi Jeremy,

    This can also happen if you have multiple virtual directories pointing to the same physical path, in which case there can be a problem like this during re-compilation of the page, because the old dlls don't get removed.

    It only happens if batch compilation is set to true, which is why you only see it when debug=true.

    You can use this to turn batch compilation off while debug is still set to false, but then you get one dll per page

    <compilation defaultLanguage="c#" debug="false" batch="false" />

    HTH
  • PingBack from http://spicycode.com/articles/2006/08/08/how-to-win-friends-and-improve-app-performance
  • PingBack from http://spicycode.com/articles/2006/05/15/how-to-win-friends-and-improve-app-performance
  • Question regarding debug settings in VS 2005... There's a new setting for debug info, pdb-only.  How does this compare to full debug info?
  • Just to avoid confusion (in case there was any:)) setting the app to debug mode in visual studio is different from debug=true

    http://msdn2.microsoft.com/en-us/library/8cw0bt21.aspx

    With PdbOnly, the dll is still compiled in release mode, but a symbol file is also generated.  With debug the dll is not optimized as much as in release mode to allow for injection points for the debugger. However, when you start an application under a managed debugger, the debugger will set attributes on the dlls allowing tracking and less optimization even if the dll is built in release mode, so you will still be able to debug. In an unmanaged debugger this does not happen, unless you do it explicitly with an ini file, but in 98% of the cases that is not important since you dont usually do source stepping anyways with an unmanaged debugger.  I would definitely recommend no debug or pdbonly in production.

    Having said this, for ASP.NET 2.0 this is only relevant for components used by your Asp.Net pages (not the asp.net pages themselves or the codebehind pages).  The compilation model for ASP.NET changed radically from 1.1 to 2.0, where in 1.1 the code behind classes were compiled at design time in visual studio into a dll (stored in the bin directory usually), and in 2.0 it moved over to a different model with several types of compilation strategies and partial classes (see: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnvs05/html/Internals.asp). I believe (but don't quote me on this) that if you build from visual studio (visual web developer) you don't really compile into executable dlls anymore, it is merely a pre-check for compilation errors, which means that effectively the release/debug option in visual studio has no effect on the web applications dlls.

    The actual compilation occurrs at runtime.

    If you have the visual studio help installet you can check out this topic for more info about the new compilation/deployment model
    ms-help://MS.VSCC.v80/MS.MSDN.v80/MS.VisualStudio.v80.en/dv_vwdcon/html/3ef36871-2cb9-452a-8c96-2068fccead18.htm
  • Continuing from my previous post on common causes for memory leaks, remote debugging is another ASP.NET...
  • It sounds like you should always run a production site in Debug=false in the web.config what about running that same site using Dlls that where compiled in debug mode.  Do you still run into issues with Debug compiled Dlls?

  • Hi Kevin,

    The timeout and non-batchcompilation is specific to web apps running with debug=true, but you do run into some of the other things i.e. the code is not optimized and there is an overhead on the loader heap for debug true.

  • PingBack from http://dotnetdebug.net/2006/04/15/tips-for-a-healthy-aspnet-application-in-production/

  • Very good informative article.

  • Hi,

    I am setting <compilation debug="false"> in Web.config to control my compilation.  I am using aspnet_compiler to build my site.  Now, my question sounds very simple but I can't seem to find a clear answer to it: does it matter whether I set <compilation debug> to "false" BEFORE or AFTER running aspnet_compiler?  I thought that I had figured out that it was necessary to set debug="false" _before_ compiling (makes sense?), but now I am just confused.

    If it is important to set debug="false" BEFORE running aspnet_compiler, then I have a follow-up question: why do Visual Studio Web Deployment Projects set it AFTER?!  (Have a look at the AspNetCompiler target in Microsoft.WebDeployment.targets).

    Thanks v. much.

  • You may have noticed high CPU usage and sluggish performance on your server before. When you fire up

Page 3 of 6 (86 items) 12345»
Leave a Comment
  • Please add 7 and 2 and type the answer here:
  • Post