We have a chapter in our IMPROVING .NET App Perf & Scale guide called Troubleshooting Performance and Scalability Issues.

What specific troubleshooting Steps/tips/techniques/tools would you like us to address in this chapter, to help you effectively troubleshoot perf Issues in your .NET apps?

We are yet to finalize the list, to give you an idea of some of areas that we are planning to cover....

  • How To: Logging Performance Monitor Counters
  • ASP.NET Performance Troubleshooting
    • ASP.NET: Troubleshooting Slow Pages
    • ASP.NET: Troubleshooting High-CPU
    • ASP.NET: Troubleshooting Memory Leak
  • CLR Performance Troubleshooting
    • Troubleshooting Too Many Allocations
    • Troubleshooting Too Many Survivors
    • Troubleshooting Leaking
    • How To: Isolating Managed vs. Un-Managed memory leak issue?
  • Data Access
    • Troubleshooting slow running queries
  • Mini-HowTo's for these tools - CLRProfiler, System Monitor, Snippets of instrumentation code (QueryPerformanceCounter to measure perf of your managed code upto nanosecond), EIF, ACT, SQL Profiler.
  • ...<your wish list>

To give you an idea of what each section might look like...I have pasted below an unedited version (not yet reviewed, just an initial dump) of Troubleshooting Slow ASP.NET section.

<BELOW JUST A SAMPLE, UNREVIEWED>
ASP.NET: Troubleshooting Slow Pages
You have an ASP.NET page is that is loading slowly; there could be multitude of symptoms that you see when your pages are rendered slow,  why this is happening as shown below:

  • Pages slow only on first request
  • Pages with server controls loading slowly
  • High System Resources
  • Low CPU and Pages still blocked/slow

The following steps can help you isolate and find the cause of issue:

  • Pages slow only on first request. If this is the reason then your pages are getting compiled on the first request and subsequent requests are severed form the compiled version. You can try these two things:
    • From a design standpoint move as much code as possible to code behind such that they are compiled ahead of time
    • For no code behind pages, you can write a 'command line utility script' or application that can scan and walk thru all pages in the virtual directly and call them so it gets precompiled
  • Pages with server controls loading slowly. If your pages are using server controls then turn on trace as shown below and check the Control Tree section and ViewState Size Byte column to see which column is consuming large amount of memory:

 

<%@Page Language="C#" Trace=true %>

 

Note: For applications on production your might not be able to turn ON Tracing, in such cases you can do a view source on the rendered page (in the browser), copy the contents of the hidden variable __VIEWSTATE to a notepad, save the file and check the overall size.

 

Disable ViewState if you have the following conditions:

    • If you are not using the postback data
    • Replace server controls with simple rendering if code manageability is not a huge overhead for performance critical pages
    • Use data caching to fetch from the cache, if your control renders stale data instead of fetching from database per request

Note: some of DataGrid’s functions such as Paging, Sorting etc may not work properly if Viewstate is disabled for the DataGrid. The same may apply for other server controls. So, care should be taken before disabling the viewstate.

Check the following link for more info:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnaspnet/html/asp11222001.asp  

 

  • High System Resources. Examine your server resources such as CPU and Memory to see if they are at optimal levels (see, Measuring chapter for system counter thresholds) and are not pegged.
    • If your CPU is pegged, then see section ASP.NET: Troubleshooting High-CPU
    • If your system memory is pegged, then see section ASP.NET: Troubleshooting Memory Leak 
  • Low CPU and Pages still blocked/slow. If your CPU is not pegged, then your request is either blocked or queued. Again there could be several reasons, the common ones are you might have made an out-bound call to another machine accessing some shared resource that is blocked (your are waiting to read a table locked exclusively by another user)
    • The limiting factor could also because the ASP.NET runtime has reached a threshold, ran out threads in ThreadPool.
    • You can dump the process and load the dump file in windbg to analyze the thread state, this is the closest you can possibly get to find the cause of the thread starvation. For more information, see <link>
    • As alternatives, you can
      §         Revisit your design, to see if you can use caching or queuing to reduce/ avoid the blocking call
      §         Using async delegates might not help if you are facing threadpool starvation, because under the hood it fetchs thread from the threadpool to do the operation
  • Other reasons. There could be several other specific reasons; however, we have highlighted a few common scenarios as shown above. To wrap up the following topics can also cause your pages to slow down:
    • Heavy use of graphics, particularly if you are using SSL
    • Script file downloaded to client
    • Large Complex page with several hundred of lines of code. See if you can simplify the page by breaking it down, use page caching or use fragment caching.
    • Storing large amount of data in the session state (managed by Out of process state service or SQL Server) might slow down the response due to serialization and de-serialization of large amount of data for each request
    • Look at the event logs to see if the process is recycling due to deadlocks, hit memory thresholds etc.