The Tragedy of Thread Happiness Disease

The Tragedy of Thread Happiness Disease

Rate This
  • Comments 13

A JOS reader interested in developing server software asked recently

Is it possible to determine the number of concurrent threads a server can support from the server's specification?

Now, as I've said before, I'm no expert on performance tuning multi-threaded applications, but I have picked up a thing or two hanging around the real experts on the IIS team.  In fact, the perf teams get questions in this vein all the time from customers, both internal and external.

The interesting thing about the question is that it is symptomatic of Thread Happiness, a peculiar disease which usually strikes programmers who are relatively new to large-scale multi-threaded software development. 

How do I know that the questioner is getting Thread Happy?  Because if they were writing an application where there were two threads or five threads or ten threads then they wouldn't be asking that question.  They'd be asking "I want to write a server app with two/five/ten/whatever threads -- how buff does the server have to be?"

But they're asking the question "how many threads can I create on a given server?" so I can only assume that they are thinking of writing some application that is going to create a large and variable number of threads, as many as possible.

That's Thread Happiness right there.  Don't do that!  It's almost certainly a bad design.  Massively multi-threaded applications cause more problems than they solve; they are very difficult to get correct, and even harder to get performant.

To actually answer the question, no, there's no way of telling that from the server spec. Why?  Because there is nowhere near enough information in the "static" facts about the hardware.  Server application threading performance depends on "dynamic" facts like:

  • What other processes are going to be running? 
  • What are those threads going to be doing? 
  • How much memory do you have free? 
  • How much contention, locking, busy waiting, context switching, blah blah blah, is there going to be?
  • What are the performance metrics that will determine when things have gotten too bad.

So, in short, don't even go there.  Come up with a design that uses a small number of threads and tune that.

Now, you might be wondering "What about real-world server applications which do spin up a variable number of threads then?  How do they decide how many is too many?"

It's quite hard.  These things used to be configurable in IIS, but a few years back the IIS team had the realization that (a) you can't expect a human being to figure out what the ideal thread limit is, and (b) the ideal thread limit changes dynamically because of all the factors I mentioned above.

IIS therefore keeps a relatively small thread pool, and continually monitors its own performance, tweaking the thread pool count, thread priorities, etc, as conditions change.  It tries to keep the server well-tuned dynamically. 

This was non-trivial code to write.  Though I've done quite a bit of optimization of software that runs in-process with IIS, the details of the IIS thread timing algorithms are way, way beyond my skills.  I wouldn't recommend such an approach unless you've got a lot of experience in the field.  A huge performance lab with a wide range of server hardware and a dedicated staff of experienced performance testers doesn't hurt either!

  • > It's quite hard. These things used to be configurable in IIS, but a few years back the IIS team had the realization that (a) you can't expect a human being to figure out what the ideal thread limit is, and (b) the ideal thread limit changes dynamically because of all the factors I mentioned above.

    It's unfortunate that the ASP.NET team hasn't figured this out as well:

    http://support.microsoft.com/default.aspx?scid=kb;en-us;821268

    Most users should never even care about implementation details like maxWorkerThreads, maxIoThreads, minFreeThreads, minLocalRequestFreeThreads etc.

  • Nothing new under the Sun:

    http://groups.google.com/groups?selm=01bbca48%24df50ef60%244a2fcbc7%40dns.brm.co.il&oe=UTF-8&output=gplain

    "If you are only targeting the new NT" ;-)

    OTOH consider that there are different environments that do espouse this sort of design, Erlang for example. They do have a very different implementation of threads, however.
  • The New NT? In 1999–2000 we first heard of Windows 2000 “based on New Technology Technology”, but it turns out that in 1996 you were already talking of New New Technology? :)
  • The complexity involved with synchronization isn't dependent on the number of threads; if it works for two threads, it sure will work with n. However, I agree that performance isn't directly proportional to the number of threads alone. Perhaps, a configurable pool is the best idea and if somebody is a bit above average (we call those coders "Rambo(es)" in our company), he should try writing a self-monitoring-and-adjusting server.
  • It's always struck me as a little bit odd that Active Server Pages, a web server, encourages developers to use VBScript and JScript to write server-side scripts. I mean, the whole point of a web server is that it produces complex strings as blindingly fast as possible, on demand.
  • Well, the situation has changed with .NET and it was never the "scripting" way in J2EE.

    I find scripting most beneficial when it comes to doing something quick and dirty with no compiler available.
  • There are many, many programming situations that call for a multithreaded approach, and not just when an application is deployed on a "super" server, either. Any program that must satisfy many concurrent requests, but for which somewhere in the processing chain blocking will occur (such as running a largish query against a database), will experience both performance and ease-of-maintenance benefits from an explicitly multithreaded approach (as opposed to asynchronous delegates, an abominable approach for anything non-trivial).

    Of course, ASP.NET takes care of most of this for you; even in a remoting service or the like, it's just plain simpler to let the system handle most threading if you can. If you need more than the default 25 threads per processor, you can just deploy your remoting host on IIS and configure it as you would a website.

    For an application that does a lot of blocking like this, even a simple ASP.NET site or web service that queries a database, you naturally want it to accept as many requests as possible. However, you may need to introduce an artificial bottleneck, to avoid (for instance) swamping your database server with work, perhaps slowing down other applications. For a large-scale ASP.NET reporting application, the two easiest ways to do this are to set the max number of IIS worker threads in the config file, and to appropriately configure the maximum size of your database connection pool.

    My main point is that it doesn't mean someone is automatically "thread happy" if they're seeking knowledge about the appropriate thread settings for a particular server, and it doesn't even mean they're writing explicit threading code. They're just rank newbies, is all-- otherwise, they'd build up their own experience by running tests and monitoring performance.

    Also, "performant" is not a word. :O) Nice post.
  • It's always struck me as a little bit odd that Active Server Pages, a web server, encourages developers to use VBScript and JScript to write server-side scripts. I mean, the whole point of a web server is that it produces complex strings as blindingly

  • PingBack from http://blogs.msdn.com/ericlippert/archive/2003/12/01/speeding-can-slow-you-down.aspx

  • Well, I intended to spend the last three weeks blogging about C# design process in anticipation of our

  • Given 1) that dual, quad-core servers are common place now, 2) the Task Parallel Library portions of .NET 4, and 3) the time since the original post, does your advice of "come up with a design that uses a small number of threads and tune that" change at all?

Page 1 of 1 (13 items)