Vance Morrison's Weblog : Concurrency

Analysis of Reader-Writer lock

vancem — Thu, 30 Mar 2006 04:11:00 GMT

In my last post I posted readerWriterDemo.cs which is an implementation of a Reader-Writer lock. I held it up as an example of good design of a concurrent data structure. I want to now show you a bit of what my thinking was when it was designed and what the important properties it has. Before you read the rest of this entry, you need to review the code carefully (It is not a lot of code).

Point 1: Follow the strict locking protocol whenever possible.

The first point, is that once I created the spin lock, I could use it to follow the locking protocol described in my concurrency article. In particular I commented the exact set of memory that the spin lock protects, and I very methodically entered the lock before reading or writing that memory. In general I tried to follow the 'Monitor' protocol and Enter the lock on entry to a public method and Exit before leaving the method.

I did not do this completely uniformly, however. In the Release methods, I exit the lock before setting the event that wakes up any threads who went to sleep (see ExitAndWakeUpAppropriateWaiters). Why did I do this? It turns out I did not need to for correctness. In fact if I had set the event while holding the lock the code would have been cleaner and easier to analyze. The problem is performance on multi-processor machines. If I had set the event while holding the spin lock, it would have immediately caused a thread to wake up, and this thread would immediately try to enter the spin lock (See the code in WaitOnEvent). Since the thread that set the event still owns the spin lock, the waking thread would have to spin for a while. This is unfortunate, and can be avoided by setting the event after leaving the spin lock.

Point 2: Scrutinize all work done outside a lock!

ALWAYS be suspicious of work done outside of locks! In this example, we do waiting on events, lazy initialization of events, and the setting of events all outside the protection of the spin lock and thus we need to be EXTRA careful reasoning about relationships between state that is not protected by a common lock (in this case the relationship betwen the reader-writer lock state (eg. owner, numWriteWaiters ...), and the state of the events (whether writeEvent is set or reset).

How to we be careful? Effectively we need a proof that the program has the properties we want (eg threads are waiting if and only if the reader-writer lock has been taken in a conflicting way by another thread). The way these proofs work is by induction. For sequential programs find an interesting property that is invariant over all state transitions (eg. methods calls). Stated another way, we need to find a property that holds when the object is constructed, and given that it holds before all possible method calls, we can show that it holds after that method call. If this is true, then by induction, the property will hold for all time.

For concurrent programs we have to modify the proof. Whenever the spin lock is released, other threads can cause state transitions. Thus we need the property to hold whenever the spin lock is not held. Because we release the locks in some places in the middle of methods, the property we choose must hold at these places too.

So the trick is to find these invariant properties, and sometimes it is not easy (which is why automated program validation is not a reality). The good news, however is that once you find the invariant, validating it is usually a straightforward (although tedious and error prone) process. In the case of our reader-writer lock one property we need is something to insure that if a locks is waiting, it has a reason to. Here is our first attempt (incorrect) at this invariant for the write event:

If writeEvent is reset then the reader-writer lock is in a state that would cause writers to block (namely the lock has been taken by some other thread).

Why did we pick this property? Because we know that threads only wait on an event if it is reset, this statement is pretty close to the property that we really want: that threads are not blocked unless they have reason to be). Unfortunately, the property above is not true. When the lock is first initialized we don't even have an event, but if we did it would probably be reset, which violates the condition above. Thinking about this, we realize that we don't care about the event in the non-contention case. The purpose of the 'numWriters' variable is to indicate when the events matter (need to be set), so we modify the invariant to be

If numWriters is nonzero and the writeEvent is reset, then the reader-writer lock is in a state that would cause writers to block (namely the lock has been taken by some other thread).

Let's see if we can prove it. Initially, numWriters is zero, so the property is trivially true. There is only one place we increment numWriters and reset the event, which is in the WaitOnEvent method, and right before we did this we validated that the reader-writer locks was in a condition that it needed to block writers. Thus our desired property is true when the spin lock is released (when the thread is about to call WaitOne). Once the spin lock is released, other threads could race to do proceed with any work that can happen outside a lock, while our original thread proceeds to call 'WaitOne'. We need to go through all the cases of what could happen after a myLock.Exit(). This includes calling any API method as well as continuing from any place that myLock.Exit() was called in the middle of a method. We need to go through these

Methods that could be called from other threads

AcquireReadLock could be called. Since we get to assume our invariant is true, we know that numWriters is nonzero. This means that AcquireReadLock will block (readers wait for writers), and thus no state transition will occur at all
AquireWriteLock could be called. By the same rationale, there is no state transition in this case either.
ReleaseReaderLock is called. This is legal only previously some thread aquired the read lock (owners > 0). This does cause a state transition, if owners is still nonzero after the update, then writers should still wait (and thus our condition holds). If owners == 0 after the transition, then the logic in ExitAndWakeUpAppropriateWaiters releases the spin lock and sets the writeEvent.

Opps! At the instant the spin lock is released, the reader-writer lock is in a state where writers could enter, and the writeEvent is in the reset state! Our condition does not hold! We need to modify our invariant and try again. Let’s try

If and numWriters is nonzero and the reader-writer lock is in a state that would writers would not block (that is the reader-writer lock is free), then writeEvent is set or will eventually be set.

This is a weaker condition, but still gets us what we want (writers will not block indefinitely when the reader-writer lock is free), and IS true at the end of ExitAndWakeUpAppropriateWaiters when the spin lock is released (since eventually the writeEvent.Set() will be called. Thus this change seems to fix our problem. However since we changed our invariant, we have to go back and check that all of our proofs that we have done to date still work with this weaker invariant (since you don't get to assume as much on entry). It is left as an exercise to confirm this (:-).

OK we are not quite done yet, we still need to check the case of when ReleaseWriteLock() is called (it is effectively the same as the ReleaseReadLock() case), and that all the other places after we call myLock.Exit() are OK (in particular, before and after calling WaitOne, and before and after calling Set()). These all check out (please think about them).

Note that while we now have a nice invariant, it does not quite get us the property we want which is

Writers never wait indefinitely when the lock is free.

The reason is that we weakened the original invariant to only say something when 'numWaiters' is nonzero. Thus we need another invariant

numWaiters is always greater than the number of threads that are calling 'writeEvent.WaitOne'.

This invariant is pretty easy to show because we always increment this count right before waiting and decrement it after waiting. There is a time when it is an overestimation (we incremented it but did not call WaitOne yet), but it is never an underestimation.

At this point we can do our proof. We do this 'by contradition', by assuming that the the negation and showing a contradition. We start by assuming that there is a thread that is waiting forever but the lock is free. That means that it was calling WaitOne, which means that numWaiters is nonzero (numWaiters is always bigger than the number of threads in WaitOne), which means that eventually Set() will be called (our invariant we showed earlier), which means that the thread should not be waiting (contradition since we assumed the thread was waiting forever). Thus we have our proof.

Phew! Quite a lot of reasoning for such a small problem! Now you can see why it pays to keep things simple. Some other interesting observations

The proof above is not quite correct. Since writeEvent was an AutoResetEvent, it does not follow that because Set() was eventually called, that the original thread should not be waiting. AutoResetEvents only release one thread, not all threads waiting. In fact the statement we are trying to prove is not true only a weaker statement is true

If the reader-writer lock is free, at least one writer (a thread which called AquireWriterLock but not ReleaseWriterLock), is unblocked.

This property is good enough, since only one writer can make progress anyway, but it shows that you have to be careful..

In ExitAndWakeUpAppropriateWaiters it would be tempting to exit the spin lock at the beginning of the method and then do the checking to see if the events needed to be set. This would break the code however. The problem is that the instant the spin lock is released, the reader writer variables can change and thus you might not set the event when you need to. It is a good exercise to find where the proof breaks down in this case.
We only handled the case for the write event. We need to repeat the process for the read event. Luckily the logic is almost identical.

Recap

I apologize for this blog getting so technical. To do it justice you really need to work though the proof yourself looking over the code carefully. This will take hours to do properly. I realize that most of you will not do this. This is OK. However those who are not willing to do this kind of analysis should NOT be writing code like MyReaderWriterLock. It is too easy to get this stuff wrong.

Finally, we have not even talked about fairness issues and reentrancy. Stay tuned for more on that.

Low-Lock Techniques in action: Implementing a Reader-Writer lock

vancem — Tue, 28 Mar 2006 18:40:00 GMT

In my article What Every Dev Must Know About Multithreaded Apps I discuss the fundamental principles of using locks correctly. In that article I strongly encourage the use of reader-writer locks because these locks create the protection you need (insuring that readers and writers don't conflict), while potentially allowing significantly more concurrency to take place (by allowing multiple readers to enter the lock simultaneously). I did not have space in that article to discuss reader-writer locks in detail however. I would like to start correcting that here.

The .NET Runtime implementation of a reader-writer lock is System.Threading.ReaderWriterLock. It is really a pretty simple API at its heart (ignore stuff about LockCookies, it is there to support nesting of locks, which you should probably avoid anyway). Here is a sample use

MyReaderWriterLock myLock = new MyReaderWriterLock();

myLock.AcquireReaderLock(Timeout.Infinite);
// No writers can be in here (AquireWriterLock will block)
myLock.ReleaseReaderLock();

myLock.AcquireWriterLock(Timeout.Infinite);
// No readers or other writers can be in here (Aquire*Lock blocks)
myLock.ReleaseWriterLock();

It is pretty straightforward to convert a program from using ordinary locks into using reader-writer locks (for those blocks that only read use AcquireReaderLock, otherwise use AquireWriterLock).

This is all very nice. Unfortunately, there is a problem. The current .NET Runtime implementation of ReaderWriterLock is about 8 times slower than System.Monitor (normal locks), in the common case where there is no lock contention. This is quite unfortunate, as it acts as a disincentive for doing the right thing (and using reader-writer locks). Microsoft is likely to fix this in the next release, however what do you do in the mean time?

Implementing a Reader-Writer Lock

The obvious answer is to implement an efficient 'drop in' version of ReaderWriterLock that you can use until Microsoft provides an better one. This is what this article is all about. Sadly, if our goal is to be as efficient as System.Montor, we can't use the standard locking protocol to implement a Reader-Writer Lock since every operation would have to lock and unlock, making it at least twice as slow as System.Monitor. Thus we are forced to look into 'Low Lock Techniques', for an answer.

In my article Understand the Impact of Low-Lock Techniques in Multithreaded Apps I list the techniques that if used very carefully, can sometimes be applied to do very efficient multi-threaded synchronization. I also mention that these techniques should only be used when the use of standard locks are problematic (usually because of performance issues). Reader-Writer locks fall into this problematic cataegory.

Possible Low lock techniques to use

In the article the techniques are listed in order of decreasing utility, and are

Avoid locks on reads
Spin Locks
Raw Interlocked operations
Lazyinit

Because the important APIs for a reader-writer lock always do an update, technique 1 is not applicable. Thus our best bet is Spin locks (note the later two techniques are not NEARLY as useful as the first two).

Implementing a spin locks

It turns out that spin locks have some very nice properties. Namely they ARE locks, so all the nice properties of locks apply (once you take the lock you have mutual exclusion and reasoning about the program gets MUCH easier). They are also efficient (one interlocked operation per Enter-Exit pair). Thus we should be able to implement a Reader-writer lock with only one interlocked operation per method call, which is the same as System.Monitor (which also has one interlocked operation per method call).

Please refer to the code attached to this post readerWriterDemo.cs. The first class is the spin lock. It has an 'Enter' and 'Exit' API. For more information on spin locks see my low-lock article or Jeff Richter's article Concurrent Affairs. The implementation in readerWriterDemo.cs deals with all the subtle details you need to worry about. It might need some tuning on how long it spins, but otherwise it is a very respectable implementation and you should feel free to use it if you need a spin lock yourself. The only caveat I curretly have is that I have commented out calls to Thread.BeginCriticalRegion and Thread.EndCriticalRegion because they are relatively expensive (they make my perf numbers look bad), and is only needed in environments like SQL Server where threads may be aborted at any instant. If a thread is aborted while the spin lock is held, the SQL Server host wants to know so that it can kill the entire appdomain instead of just the thread. If you are not in SQL Server you can get away without this.

Before using the spin lock, we need to confirm that our read-writer lock is a good candidate for its use. To be a good candidate is must be the case that the time the lock is held is very short (measured in at most dozens of instructions). We believe that this is the case since all of our API for the reader-writer lock do little and are perf critical, so the code paths NEED to be short. However after we get our implementation we need to confirm that we don't have rare paths that might hold the lock for a long time. It turns out with a bit of work we can make certain this is the case.

Implementing a reader-writer lock with the spin lock

With a spin lock in hand, writing the reader-writer lock is not too much code (~ 150 lines, with comments), and hopefully is easy to understand. Every API takes a lock, does work, and leaves. To deal with the case where you must wait, we need OS structures to wait on (eg ManualResetEvent, AutoResetEvent). When a ManualResetEvent is 'Set' all threads waiting will wake up. This is appropriate for readers. For writers, it makes more sense to only wake up the first thread, and this is what AutoResetEvent does. The code has a lot of comments, so please take a look. There are actually a lot of subtleties to think about (what happens in various error conditions, what APIs can fail, how efficient the lock is in various scenarios, how long we are holding the spin lock ...). I point out some of these subtleties in comments, however the really important point is that the code itself is short and straightforward. This is CRITICAL because the more code you have, the more difficult it is to confirm the code works in all the subtle cases. It pays to keep things simple.

Performance Results of the reader-writer lock

If you run the program, it will measure the performance of an Enter/Exit pair. Here is what I got.

10Meg Enter/Exit 371.6953 msec = 59.61 CPU clocks per Enter/Exit
10Meg RW Locks (.NET library) 3175.8594 msec = 8.5X monitor time
10Meg New RW Locks 428.8234 msec = 1.2X monitor time

As you can see, on my machine, the .NET library is 8.5X slower than System.Monitor, but the new RW locks are only 20% slower. With some more tuning I suspect I can get even closer, however it will make the code more complicated, so I don't want to do that in this published code (cleanliness is worth something).

As written, the lock has the property that if a writer is waiting for the locks, new readers can't come in and block it indefinitely. Instead the readers wait until all the other readers drain, and the writer gets its turn. The lock is unfair in the sense that when a lock is released it is not guaranteed that the first waiter gets the lock. Thus starvation is possible (however unlikely). It turns out that fairness causes performance issues (I will discuss in a later blog), so the current unfair protocol has better performance characteristics in almost all cases. It is not too hard to change the implementation to be fair, and I might take that up in a later blog (please request it if you are interested).

The lock as written is also not reentrant. It is relatively easy and efficient to add this (although it is not clear this feature is a good idea), and I show how to do that too (please request it if you are interested).

The code is really worth looking at carefully and I encourage you to do so. It is small enough that you can understand the intent quickly, but there are many scenarios to think about.

Recap

What are the take-aways?

Use reader-writer locks as suggested in my first article. If lock perf is a problem, take the implementation here as a 'drop in' replacement for the .NET System.ReaderWriterLock. Feel free to use this implementation until Microsoft fixes the version in its library.
Spin-locks are a really valuable low lock technique. There were used here to make a clean, simple but efficient reader-writer lock written competely in managed code. This lock is significantly simpler than most implementations I have seen, yet is clost to optimal. The reason for this is that the use of spin locks GREATLY simplifies the design and analysis of the lock, without sacrificing performance. Feel free to use the Spin lock implementation show here to make other high performance concurrent constructs.
Even something as simple as a Reader-Writer lock has subtleties about its behavior (especially when you factor in performance concerns). Keeping it simple really pays off here.

Two articles on concurrent programming I wrote

vancem — Tue, 21 Feb 2006 01:24:00 GMT

This is just a quick plug for two articles that I wrote for MSDN magazine a few months back. They are

The first article, as its title suggests, is what I think every programmer should know about Multithreaded programming. I know from my own ramp up on multi-threading issues that while the principles of concurrent programming are actually relatively simple, the ramifications of those principles are VERY subtle. The upshot is that if you write multi-threaded apps (and it turns out that EVERY managed app that uses finalizers is multi-threaded), you should take a look at the first article (either you will already know it (which is good), or you will be glad you learned it.

The second article is really trying to warn you way from what is often called ‘lock free’ programming. If concurrent programming is subtle, concurrent lock free programming is 10 times as subtle. The article is mostly an exposition of SOME of the subtleties’, and hopefully that will be enough to scare you away from using lock free techniques in all but the most constrained circumstances.

Enjoy the articles.