Richard Florance's Solution Performance blog

Easy wins tuning the performance of a web site

2008-06-17T12:44:00Z

I've been on several ASP.Net web site projects over the last couple of yeras doing performance analysis and I've found that every time there is a common series of quick wins I've been able to make that can have a substantial effect on the responsiveness and scalability of web sites. It's now got to the point where I've written a list of things that I want the solution development team to check before I'll come and do any deeper analysis on the project. Here is the core list of work items I've come up with that you should look at first when tuning ASP.Net web sites:

· Turn on IIS Compression (this is off by default on IIS6 and IIS7). See here on how to do this:

http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/d52ff289-94d3-4085-bc4e-24eb4f312e0e.mspx?mfr=true

· Turn on IIS content Expiry (again this is off by default). See here:

http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/0fc16fe7-be45-4033-a5aa-d7fda3c993ff.mspx?mfr=true

· Merge static content into as few files as possible (e.g. only have one .js file and .css file). It is faster to download one big file that several smaller ones over the internet.

· Enable ASP.Net output caching where possible. You need to look at this on a page by page basis but can give huge wins for pages that contain mostly static content. See here: http://support.microsoft.com/kb/323290

· Enable Web service output caching where possible. If you have any public web services that only return a limited or predictable range of values based on their input parameters you should look at this. See Here: http://support.microsoft.com/kb/318299

· If your Site makes web service calls, increase MaxConnections in the Machine.config to handle the level of concurrency you want to support. See: "Threading Explained" here: http://msdn.microsoft.com/en-us/library/ms998549.aspx#scalenetchapt06_topic8. However don't take this as an absolute rule. For MaxConnections it is OK to go higher if your middle tier makes lots of long running Web service calls.

· Change the IIS threading configuration in machine.config to support better scaling. See: "Threading Explained" here: http://msdn.microsoft.com/en-us/library/ms998549.aspx#scalenetchapt06_topic8

· Ensure ViewState is turned off where possible, and then only enable it for the controls that really need it. Viewstate can be huge and is on by default. turning off ViewState can radically reduce the size of pages you send over the wire. See here: http://msdn.microsoft.com/en-us/library/ms972976.aspx#viewstate_topic9

· If you are using Ajax, ensure <compilation debug=”false”/> is set in your web config. This avoids very costly parameter validation on both the client and server. See "Significantly Better Debugging Support" in ScottGu'sblog here: http://weblogs.asp.net/scottgu/archive/2006/10/20/ASP.NET-AJAX-Beta-1-Released.aspx

· Run SQL Profiler against your database whilst doing normal activity, identify all SQL access that has high duration or CPU values and optimise them.

Tracing gotchas

2005-08-22T12:03:00Z

When I'm profiling solutions I work on, many times one of the first bottlenecks I come across is tracing code - even if tracing is currently turned off in the solution! The reason for this is how the solution checks to determine whether to generate tracing output.

Everyone knows that tracing is expensive and should only be generated if you're currently trying to debug the solution, so the classic (very simple) pattern for implementing tracing is something like this:

if (TracingFlag == True)

{

GenerateTracingOutput("It's all going wrong here");

}

The problem is where you go to get the value of "TracingFlag" and how often you get it. Invariably the value is stored as part of the solutions configuration somewhere, either in the registry or an XML config file, which is fine. What isn't fine is to reread this value from the registry or file every single time you need to determine whether or not to trace. The classic reasons for doing this are: Laziness, registry access is 'free' (yeah right) or I want to dynamically control tracing so I need to keep re-reading the value.

Taking these in turn:

Laziness - how hard is it to create a static variable to cache the tracing flag?

Registry access is 'free' - Registry access isn't that fast. It requires a transition to the kernel and is protected by a global lock which means if all your threads are constantly reading the tracing flag they all end up getting serialised. You really need to read it once and cache that value.

Dynamic tracing - From my experience being able to dynamically control tracing on a solution is a nice to have that is actually rarely used in the real world. However if you really need think you need it then don't reread it from source each time but read it once, cache it and then setup a ChangeNotificationRequest on the file or registry key so you can re-read the flag if it ever gets changed.

Another classic performance hit with tracing is to implement a tracing class and then instantiate an instance of the class everytime you want to check for tracing because the trace flag is a member of the class... Don't do it! Either make your tracing class a static class or instantiate an instance at startup and cache it.

The relative speed of I/O

2005-08-12T14:47:00Z

It still amazes me how often I come across dev's who have no appreciation of just how slow hard disks and networks actually are, especially when you compare their latency to the speed of your average Pentium processor. As a result they often break all my golden rules and cause severe pain to their solutions because they have expectations that going to disk or network is relatively inexpensive.

To try and give the person I was talking to a better perspective on just how slow I/O is, I looked up the relative latencies of current state of the art technologies and put together timelines based on the idea of 'if one CPU clock takes one second how long does X take'. Here is what I came up with:

Relative access frequencies of current state of the art:

CPU 3 GHz (0.3ns - Current mainstream Intel Pentium IV clock speed)

Memory 800 MHz (1.25ns access time – the speed of fast DDR2 memory)

LAN 500 Hz (2ms - typical latency on a 1GHz LAN - I'm guessing a bit on this one but it seems reasonable)

Disk 200 Hz (5ms - disk seek time for an average random disk access)

WAN 25 Hz (40ms- typical latency on US domestic Internet WAN link)

Translating these frequencies into numbers of CPU clock cycles gives:

1 Memory access = 4 CPU cycles

1 LAN access = 6,000,000 CPU cycles

1 Disk access = 15,000,000 CPU cycles

1 WAN access = 120,000,000 CPU cycles

Therefore if one clock cycle on a 3GHz processor takes 1 second then…

A memory access takes 3.75 seconds

A network round trip takes 69.4 days

A disk access takes 173.5 days

Accessing an internet site takes 3.8 years!

This gives you a pretty good feeling for why caching just about everything is good for performance!

Golden rules for high performance solutions

2005-07-27T18:33:00Z

Over the last three years I've analysed the performance of at least a dozen completely different solutions. However the performance bottlenecks I find mostly tend to follow similar themes. Having analysed the types of issues that these bottlenecks are I believe I have distilled out three rules, that if you follow them religiously, you are well on your way to having a solution that performs as well as possible in the environment that it is running. So without further ado here they are with a couple of examples of common issues that match each rule:

Thou shalt never do anything more than once.

Examples

Cache everything you are likely to use more than once. It is a cardinal sin to retrieve the same data from a SQL Server or a Web Service call twice.

Nothing is free – Never assume that anything you do doesn’t cost any time or resources. A classic example of this is accessing registry keys - especially tracing flags! When you're getting hardcore about performance tuning even memory allocation falls into this rule.

Thou shalt not do anything you don’t need to.

Examples

Don’t try and reuse an existing stored procedure which returns tons of data when you only want one field that it returns – create a new stored procedure which just gets the data you need - you're unnecessarily burning CPU cycles and disk access on the SQL server and hitting network latency sucking the redundant data back to the calling tier.

Use stored procedures. Dynamic SQL generally causes SQL Server to repeatedly do a lot of unnecessary work with compilation and Execution plan generation.

Thou shalt get everything in one go.

Examples

Round trips, especially over a network or from a disk are expensive in latency. Don’t use lots of fine grained web service calls when one chunky call can be used instead. Aim for one Web page = at most one web service call.

Don’t use several stored procedures to get a set of data. Create a new one which does everything and returns the complete result set. Aim for one Web Page\Web Service call = at most one SQL Server call.

These may sound obvious but you will be astounded at how often I see examples of these rules being broken and performance suffering (often radically!) as a result, usually because the developer is unaware of the impact of what he\she is calling, especially when you have a 100 users all calling the code concurrently...

Hello, good afternoon and welcome...

2005-07-26T14:43:00Z

Seeing as this is my first blog I guess a good place to start is to tell you who I am, what I do and what pearls of wisdom I hope to impart in my blog.

My name is Richard Florance, but a lot of people end up calling me Flo. I've been at Microsoft in the UK for longer than I care to remember and spent a good while as a third line support engineer for Windows NT\2000 before moving over to become a Consultant. However I've managed to create an interesting niche for myself as a performance specialist and I work mostly on customer solutions we develop in house based on all our (usually) latest and greatest technology. Every now and again I'll go out and help customers on site when they're running into performance problems with their own solutions.

I'll look at everything on a solution, from the scalability of the architecture to refining web page response times to tuning and optimising the business logic and to identifying poorly performing SQL queries. In this blog I'll talk about some of the interesting and common performance issues I've found along the way and discuss some of the tools I use. As we get close to shipping Visual Studio 2005 I hope to post some good stuff on the load testing tool and code profilers we are shipping as part of the product which are rapidly becoimg the main tools of my trade as they stabilise.