This post shares case studies of high CPU utilization of ASP.NET web sites. High CPU utilization was caused by lack of batch compilation, multiple folders, and use of XmlSerializer. In all cases the result was high CPU and poor performance; the symptom was .NET CLR Loading\Current Assemblies counter showing “unusual” number of loaded assemblies.

Case Study #1 – Lack Of Batch Compilation

Customer reported a high CPU utilization during load/stress testing. After quick investigation we identified that debug attribute was configured to “true” in web.config. Here is what ASP.NET authorities have to say about configuring debug to “true” for ASP.NET web sites:

• ScottGu in his Don’t run production ASP.NET Applications with debug=”true” enabled says: “Doing so causes a number of non-optimal things to happen including:

  1) The compilation of ASP.NET pages takes longer (since some batch optimizations are disabled)
  2) Code can execute slower (since some additional debug paths are enabled)
  3) Much more memory is used within the application at runtime
  4) Scripts and images downloaded from the WebResources.axd handler are not cached”

• Thomas Marquardt in his ASP.NET Performance Monitoring, and When to Alert Administrators (adopted by PAG) says: ”…excessive memory consumption is caused by an unusually large number of loaded assemblies… Note that if the <%@ page debug=%> attribute is set to true, or if <compilation debug=/> is set to true, batch compilation is disabled.”
• Tess in her ASP.NET Memory: If your application is in production… then why is debug=true says: “

  There are three main differences between debug=true and debug=false:

  1. ASP.NET Timeouts
  2. Batch compilation
  3. Code optimization”.

To prevent further deployment mistakes like this one we created simple performance deployment checklist that included checking debug=”false” configuration among other checks. Following are others checks you might consider when deploying newer version of ASP.NET web site to production:

1. Delete Temporary ASP.NET files.
2. Short-circuit the HTTP pipeline
3. Thread Pool Is Tuned By Using The formula To Reduce Contention

Case Study #2 – Multiple Folders

Customer reported slow response of the ASP.NET web site. After quick investigation we identified that the application’s deployment assumes multiple folders – high tens of folders. This approach took place due to team’s decision to develop every functional module as a separate folder.

ASP.NET batch compilation is the process of compiling ASP.NET markup (content of aspx files) into temporary dll’s. Compilation requires invoking compiler (csc.exe for C#) – that is pretty heavy activity. Process Explorer shows it clearly:

ASP.NET batch compilation occurs on per folder basis. Said that, if your application divided into multiple sub-folders that contain ASP.NET pages each time any of the folders accessed for the first time the batch compilation is invoked.

To overcome the issue we created simple solution we called ASP.NET Warmer. Complete description of the solution can be found here - Use FREE Tools From IIS Resource Kit To Warm Up Your ASP.NET 1.1 Application By Batch Compilation. The solution included three simple steps:

1. Identify ASPX pages to “touch” thus invoking batch compilation. We used LogParser to accomplish this task.
2. Build a program that will actually “touch” these ASPX pages identified in step 1. We used free command line utility called TinyGet from IIS Resource Kit to accomplish this task.
3. Schedule periodic invocation of the utility.

Case Study #3 – XmlSerializer

Customer reported high memory consumption and high CPU utilization that usually required process recycling. After quick code inspection we identified that the customer’s code uses XmlSerializer to serialize and deserialize types. The serializalbe type was built up of 250 properties. Each request generated 450 such types.

Tess has published complete lab with the source code and analysis for such case, including perfom snapshot (Tess, what a great resource!!). Here Process Explorer shows clearly activation of CSC.exe - C# compiler - on the fly during each request. Green – new instance, Red – disposed instance:

Conclusion

• Case #1 shows the importance of deploying of ASP.NET web application with performance in mind.
• Case #2 shows the importance of planning (architecture and design) of ASP.NET web application with performance in mind.
• Case #3 shows the importance of coding of ASP.NET web application with performance in mind.
• All cases share simple techniques of monitoring ASP.NET web application in production (maintenance)
• All the case studies could be easily prevented by proper education of the dev and IT team for Performance Development Lifecycle (PDL).

My Related Posts

• Best ASP.NET Performance Winner For Data Binding - Hands Up To Response.Write()
• ASP.NET Data Binding Performance – Collection Is More Expensive Than Datatable

My name is Alik Levin and I am focusing on Security and Performance in .Net applications.
When I do not blog about Security and Performance I blog about personal development.

This post was made with PracticeThis.com blog post template plugin for Windows Live Writer