ASP.NET Debugging : High CPU

XPerf: A CPU Sampler

Tom — Fri, 10 Oct 2008 13:00:00 GMT

Seema just had a great about using XPerf to troubleshoot CPU issues when using Silverlight. This can also be used in the same way to troubleshoot ASP.NET or IIS.

What can XPerf tell you? Seema answers that question, it can find out:

Is my app asking Silverlight to constantly spin on CPU cycles.
Whether one UI layout or design is more expensive than the other.
Whether the time is spent in drawing (agcore.dll), the plug-in’s interactions with (npctrl.dll), or in compilation/JIT (coreclr.dll)
How much is stretching/blending/rotating that video going to cost in terms of CPU cycles? How about if you encode it differently?

You can read more about XPerf and how to download and use it from Seema post here.

Note: The tools only work on Vista and Windows Server 2008.

How do you test the performance of your web site?

Tom — Wed, 10 Sep 2008 13:00:00 GMT

I wanted to get a feel for what tools people are using to test the performance of their website. My questions are:

Do you check page-load times?
Do you dig in to see which files are taking the time
How do you track down a page that uses a lot of memory
How do you track down a high CPU consuming page

I don’t want to list any possible tools as I’d like to hear what people are using without me putting any suggestions out.

If I don’t see some being mentioned, I’ll be sure to bring it up so we have a holistic view of the tools.

Look forward to seeing what people use.

ASP.NET Tips: Using Process Explorer to help with high CPU

Tom — Tue, 24 Jun 2008 13:00:00 GMT

I am not sure how many people are familiar with the great tools that Mark and the Sysinternals folks have created, but this is one that is a must for monitoring the performance of your machine. It is called Process Explorer. With it, you can see what is taking up your CPU time in real time as well as which process has what handles or DLL’s open.

This tool can be used with ASP.NET to keep a quick eye on the performance of your web server. You can add it to your startup folder so it always starts and you can watch things progress in your server and make sure things don’t get out of control. For a great video on using this tool, check out: The case of the Unexplained. You can also check out Mark’s blog for many examples that use Process Explorer. This tool has saved me a lot of time in tracking down performance issues and a lot of the time, you can use it instead of needed to attach a debugger or look at a dump.

I would highly suggest watching the video or checking out Mark’s blog for more details and to see if you can apply the same things to your environment.

ASP.NET and Performance

Tom — Mon, 23 Jun 2008 21:29:34 GMT

I wanted to talk about monitoring performance in ASP.NET for a bit. I have already posted a few postings that talk around this issue, namely:

But there are some other things to keep in mind. The best place to look for information on how to watch the performance of an ASP.NET Application is in the “Improving .NET Application Performance” guide on MSDN. Specifically Chapter 15 – Measuring .NET Application Performance. There are a lot of other very useful pieces of information in this. So I would suggest you review any Chapter that applies to your application.

If you want to get a description of all the counters, check out: Performance Counters for ASP.NET and Memory Performance Counters

As for how to capture the perfmon data (perfmon is the tool that we use to capture the performance information), take a look at my various posts around the different situations. This goes through what counters are important in each situation:

ASP.NET Tips- What to gather to troubleshoot

The main ones to keep an eye on are the memory counters (to watch if memory is staying under control or becoming unbounded) and the ones around the ThreadPool (Queuing, Request Execution Time, etc) to make sure things aren’t getting queued up or blocked in some manner.

ASP.NET Tips: What to gather to troubleshoot

Tom — Wed, 21 May 2008 18:38:59 GMT

So now that I am done with all the individual posts, I thought I would wrap them all together. So here are all of the posts that I created on gathering information when you are having a problem.

Please keep in mind that these are geared towards the type of information you would be sending to Microsoft Support. So if you are troubleshooting something yourself, you have have some better methods, such as Tracing or Logging.

So without further ado, here is the complete list:

Hope that these are useful to everyone and hope that you can put these to use. If you think there is a situation that I haven’t covered, please let me know and I will add it also.

ASP.NET Tips: What to gather to troubleshoot – part 1 – High CPU or Hang

Tom — Fri, 02 May 2008 19:13:18 GMT

On machines running IIS5 and IIS5.1, these are the Inetinfo.exe, DLLHost.exe and/or ASPNET_WP.exe processes. On machines running IIS 6 or 7, this is the w3wp.exe process.

The following steps will allow us to get log files of the processes hosting the IIS or ASP.NET applications, as well as obtain the thread ID that is using the CPU that will directly correspond to the log file. Please let us know if you have any questions about using this process.

Before the issue occurs

A. Download DebugDiag:
======================

Install the Debug Diagnostic tool
The default installation location for the package is C:\Program Files\IIS Resources\DebugDiag. It can be changed.
Make sure any other monitoring and debugging tools are not running.

B. Create a Performance Monitor log:
====================================

In Performance Monitor, expand Performance Logs and Alerts
Right Click on Counter Logs
Choose New Log Settings...
Enter a descriptive name
Note the log file location for later (or go to the Log Files tab and change the location)
Click the Add Counters button
Click the All Counters radio button. Click the Select instances from list radio button, and select the process hosting the IIS or ASP.net application. On Windows 2000 and Windows XP, this will be inetinfo, dllhost, or aspnet_wp . On Windows 2003 and Windows 2008 this will be w3wp. NOTE: It is possible you will see multiple instances of dllhost on IIS5/5.1 or w3wp on IIS6/IIS7 in the list of available processes. If you do, or if you are unsure of which process to monitor, please cltrl-click to multiselect all of the available IIS/ASP.net choices.
If the application is not an ASP.net application, select the following from the Performance Object dropdown, being sure to Add each one as you select it:
- Process
- Thread
If the application is an ASP.net application, select the following from the Performance Object dropdown, being sure to Add each one as you select it:
- Process
- Thread
- .NET CLR Data
- .NET CLR Exceptions
- .NET CLR Interop
- .NET CLR Jit
- .NET CLR Loading
- .NET CLR LocksAndThreads
- .NET CLR Memory
- .NET CLR Networking
- .NET CLR Remoting
- .NET CLR Security
- ASP.NET
- ASP.NET Applications
Click Close
Click OK

For the ASP.NET counters, select the version that you are wanting to monitor. Ex. For 1.1 framework, select ASP.NET v1.1.4322 and ASP.NET Applications v1.1.4322.

IMPORTANT: The Data Sampling Interval and time to start monitoring is subjective to when the CPU spike reproduces. Due to the log size, monitor the server to gather the needed data while not overwhelming the server.

Note: For more information on Performance monitor, see http://support.microsoft.com/?id=248345

After the issue occurs

A. Let the process run
======================
Now let the server run for a few minutes to document the environment in which the high CPU/hang is occurring in.

B. Capture the memory Dumps
===========================

Launch DebugDiag
- Start, Program Files, IIS Diagnostics, Debug Diagnostics, Debug Diagnostic Tool
From the Tools menu, select Create IIS Hang Dump

Repeat Step 2 every 1 - 2 minutes until you have 3 sets of memory dumps. Make sure the previous memory dumps completed before continuing to capture the next set of memory dumps. The multiple memory dumps will allow us to see if the process is progressing during the hang/high CPU symptoms or whether the process is actually performing work.

Files will be created in the following path by default:
C:\Program Files\IIS Resources\DebugDiag\Logs\Misc

C. Stop the Performance Monitor log
===================================

In Performance Monitor:

Right click on your log that is now listed under Counter Logs
Choose Stop Log