As most of you know, the ASP.NET runtime in IIS6.0 uses threads from the CLR ThreadPool in order to process requests. As requests come into IIS, they are dispatched via the ThreadPool.QueueUserWorkItem static method which basically queues the unit of work until a thread is available. If there are no threads available, then the ThreadPool will spin up a new thread, as long as it has not reached its maximum thread limit. This maximum thread limit is normally set by the ThreadPool.SetMaxThreads method. There is also a SetMinThreads method to configure the minimum number of threads to be spun up when the ThreadPool is first accessed.
Note that the default for the maximum threads is 25 X Number of processors. The default minimum number of threads is the Number of processors. There is only one ThreadPool per process; every AppDomain within the process will share the same ThreadPool.
So where am I going with all of this? Well this works very efficiently until you start throwing a lot of load on the server. At this point you will notice a couple of things. First off, if you observe the w3wp.exe process in Task Manager, specifically the Threads column, you will notice the number of threads increasing, which is to be expected, but the rate of this increase will be very slow. Secondly, you may also notice that at some point you will reach a ceiling and the threads will not increase any more although there are plenty of requests still waiting to be processed. Let's see how we can address these two problems.
The second problem is an easier one to deal with - as discussed above, there is a SetMaxThreads setting on the ThreadPool which can define the maximum number of threads in the ThreadPool. So the first thought would be to call this method someone in the application initialization code and set the max threads to some higher number. While you can do this in a console or Windows Forms application, you will get a server error 500 if you try to do this in your ASP.NET code behind! Remember that IIS is hosting many application domains within the instance of the w3wp.exe process and the ThreadPool is shared across all AppDomains so it cannot allow a single application to control the ThreadPool.
The correct way to set the maximum number of threads in the ThreadPool is to use the <processModel> section of the machine.config. This section has a maxWorkerThreads attribute which internally sets the MaxThreads for the ThreadPool. See here for more details.
Now this may seem like a funny thing to do – why bother setting the minimum number of threads in the ThreadPool? Isn't it wastefull to keep a bunch of threads hanging around when you may never need them? As you noticed above, the rate of increase in the number of threads was very slow. This is because of an optimization in the ThreadPool implementation that enforces a delay of 500ms between spinning up of new threads. This is done so that when there is a burst of activity, the CPU is not overwhelmed with spinning up the new threads at the expense of processing the threads that are already running. So how do we resolve this issue? This is where the minimum number of threads comes in. By setting the minWorkerThreads in the machine.config to some reasonable number, which should be at least as high as the average load on the server, we tell the ThreadPool to spin all these threads without delay when the ThreadPool is first accessed. This way we are guaranteed that we can handle the typical load on the server without any further delays.
Scalability concerns
Not setting the minWorkerThreads to some more reasonable number has been a limiting factor when the server tries to scale up. For some reason (that I haven't quite figured out) IIS will not reach the maxWorkerThreads in some situations (I noticed it in in a WF application) if the minWorkerThreads is too low. Typically I set it to half the maxWorkerThreads number which seems to work for me.