Fabulous Adventures In Coding : Immutability

What is this thing you call "thread safe"?

Eric Lippert — Mon, 19 Oct 2009 13:50:00 GMT

Caveat: I am not an expert on multi-threading programming. In fact, I wouldn't even say that I am competent at it. My whole career, I've needed to write code to spin up a secondary worker thread probably less than half a dozen times. So take everything I say on the subject with some skepticism.

A question I'm frequently asked: "is this code thread safe?" To answer the question, clearly we need to know what "thread safe" means.

But before we get into that, there's something I want to clear up first. A question I am less frequently asked is "Eric, why does Michelle Pfeiffer always look so good in photographs?" To help answer this pressing question, I consulted Wikipedia:

"A photogenic subject is a subject that usually appears physically attractive or striking in photographs."

Why does Michelle Pfeiffer always look so good in photographs? Because she's photogenic. Obviously.

Well, I'm glad we've cleared up that mystery, but I seem to have wandered somehwat from the subject at hand. Wikipedia is just as helpful in defining thread safety:

"A piece of code is thread-safe if it functions correctly during simultaneous execution by multiple threads."

As with photogenicity, this is obvious question-begging. When we ask "is this code thread safe?" all we are really asking is "is this code correct when called in a particular manner?" So how do we determine if the code is correct? We haven't actually explained anything here.

Wikipedia goes on:

"In particular, it must satisfy the need for multiple threads to access the same shared data, ..."

This seems fair; this scenario is almost always what people mean when they talk about thread safety. But then:

"...and the need for a shared piece of data to be accessed by only one thread at any given time."

Now we're talking about techniques for creating thread safety, not defining what thread safety means. Locking data so that it can only be accessed by one thread at a time is just one possible technique for creating thread safety; it is not itself the definition of thread safety.

My point is not that the definition is wrong; as informal definitions of thread safety go, this one is not terrible. Rather, my point is that the definition indicates that the concept itself is completely vague and essentially means nothing more than "behaves correctly in some situations". Therefore, when I'm asked "is this code thread safe?" I always have to push back and ask "what are the exact threading scenarios you are concerned about?" and "exactly what is correct behaviour of the object in every one of those scenarios?"

Communication problems arise when people with different answers to those questions try to communicate about thread safety. For example, suppose I told you that I have a "threadsafe mutable queue" that you can use in your program. You then cheerfully write the following code that runs on one thread while another thread is busy adding and removing items from the mutable queue:

if (!queue.IsEmpty) Console.WriteLine(queue.Peek());

Your code then crashes when the Peek throws a QueueEmptyException. What is going on here? I said this thing was thread safe, and yet your code is crashing in a multi-threaded scenario.

When I said "the queue is threadsafe" I meant that the queue maintains its internal state consistently no matter what sequence of individual operations are happening on other threads. But I did not mean that you can use my queue in any scenario that requires logical consistency maintained across multiple operations in a sequence. In short, my opinion of "correct behaviour" and your opinion of the same differed because what we thought of as the relevant scenario was completely different. I care only about not crashing, but you care about being able to reason logically about the information returned from each method call.

In this example, you and I are probably talking about different kinds of thread safety. Thread safety of mutable data structures is usually all about ensuring that the operations on the shared data always operate on the most up-to-date state of the shared data as it mutates, even if that means that a particular combination of operations appears to be logically inconsistent, as in our example above. Thread safety of immutable data structures is all about ensuring that use of the data across all operations is logically consistent, at the expense of the fact that you're looking at an immutable snapshot that might be out-of-date.

The problem here is that the choice about whether to access the first element or not is based on "stale" data. Designing a truly thread-safe mutable data structure in a world where nothing is allowed to be stale can be very difficult. Consider what you'd have to do in order to make the "Peek" operation above actually threadsafe. You'd need a new method:

if (!queue.Peek(out first)) Console.WriteLine(first);

Is this "thread safe"? It certainly seems better. But what if after the Peek, a different thread dequeues the queue? Now you're not crashing, but you've changed the behaviour of the previous program considerably. In the previous program, if, after the test there was a dequeue on another thread that changed what the first element was, then you'd either crash or print out the up-to-date first element in the queue. Now you're printing out a stale first element. Is that correct? Not if we always want to operate on up-to-date data!

But wait a moment -- actually, the previous version of the code had this problem as well. What if the dequeue on the other thread happened after the call to Peek succeeded but before the Console.WriteLine call executed? Again, you could be printing out stale data.

What if you want to ensure that you are always printing out up-to-date data? What you really need to make this threadsafe is:

queue.DoSomethingToHead(first=>{Console.WriteLine(first);});

Now the queue author and the queue user agree on what the relevant scenarios are, so this is truly threadsafe. Right?

Except... there could be something super-complicated in that delegate. What if whatever is in the delegate happens to cause an event that triggers code to run on another thread, which in turn causes some queue operation to run, which in turn blocks in such a manner that we've produced a deadlock? Is a deadlock "correct behaviour"? And if not, is this method truly "safe"?

Yuck.

By now you take my point I'm sure. As I pointed out earlier, it is unhelpful to say that a building or a hunk of code is "secure" without somehow communicating which threats the utilized security mechanism are and are not proof against. Similarly, it is unhelpful to say that code is "thread safe" without somehow communicating what undesirable behaviors the utilized thread safety mechanisms do and do not prevent. "Thread safety" is nothing more nor less than a code contract, like any other code contract. You agree to talk to an object in a particular manner, and it agrees to give you correct results if you do so; working out exactly what that manner is, and what the correct responses are, is a potentially tough problem.

************

(*) Yes, I'm aware that if I think something on Wikipedia is wrong, I can change it. There are two reasons why I should not do so. First, as I've already stated I'm not an expert in this area; I leave it to the experts to sort out amongst themselves what the right thing to say here is. And second, my point is not that the Wikipedia page is wrong, but rather that it illustrates that the term itself is vague by nature.

Events and Races

Eric Lippert — Wed, 29 Apr 2009 18:00:00 GMT

Here’s a question similar to one I saw on stackoverflow (*) the other day. Suppose you have an event:

public event Action Foo;

The standard pattern for firing this event is:

Action temp = Foo;
if (temp != null)
temp();

What the heck is up with that? Why not just call “Foo()” ?

First off, this pattern ensures that the thing that is invoked is not the null delegate reference, which would cause a null reference exception to be thrown. But if that’s what we want, then surely we could have skipped the temporary variable – just “if (Foo != null) Foo();” would do. Why the temp?

The temporary variable ensures that this is “thread safe”. If the event’s handlers are being modified on another thread it is possible for Foo to be non-null at the point of the check, then set to null on a different thread, and then the invocation throws.

Using the temporary variable effectively makes a copy of the current set of event handlers. Remember, multi-cast delegates are immutable; when you add or remove a handler, you replace the existing multi-cast delegate object with a different delegate object that has different behaviour. You do not modify an existing object, you modify the variable that stores the event handler. Therefore, stashing away the current reference stored in that variable into a temporary variable effectively makes a copy of the current state.

Is that clear? Make sure it is, because now things start to get really confusing.

A common criticism of this pattern is that it trades one race condition for another. Let’s consider that scenario again more carefully:

The event handler contains a single handler, H. Thread Alpha makes a copy of the delegate to H in temp and determines that it is not null. Thread Beta decides that H must no longer be called when the event is fired, so it sets the handler to null. Thread Beta then assumes that H will never be called, and destroys a bunch of state that H needs to execute successfully. Thread Alpha then regains control and executes H, which behaves crazily since its necessary state has been destroyed. Thread Beta’s attempt to ensure that H is not called has been defeated by the race condition.

A frequently-stated principle of good software design is that code which calls attention to bugs by throwing exceptions is better than code which hides bugs by muddling on through and doing the wrong thing. This code has a race condition that results in incorrect behaviour. Perhaps an application of that principle is to stop using a temporary and instead crash if it races. That is to say, this code has a failure mode; surely it is better to highlight that failure with a crisp exception than to turn it into crazy wrong behaviour.

That seems plausibly argued, I agree, but the conclusion is wrong.

Suppose we remove the temporary variable but keep the null check. Does that solve the problem? No! We still have the same race conditions. Suppose the event handler delegate contains a reference to H. Thread Alpha checks to see whether it is null; it is not. Thread Alpha pushes the object to invoke on the runtime stack. Between the push of the delegate value and the call to invoke it, thread Beta sets the event handler to null. And once again, H will be invoked after thread Beta has removed the handler.

Suppose we remove the temporary and the null check and just invoke the delegate directly. Does that help? No, we still have the same race condition. The contents of the event handling variable can still be changed between the push of the delegate object onto the stack and the invocation of the delegate.

Removing the code around the call site does not decrease the number of race conditions in the code, and it does increase the number of crashes. Worse, doing so makes the race condition harder to detect by shrinking the window in which the race can occur without eliminating it.

Removing the null check and/or the temporary is a bad idea; the already-bad situation is only made worse.

Essentially there are two problems here that are being conflated. The two problems are:

1) The event handler delegate can be null at any time.
2) A “stale” handler can be invoked even after it has been removed.

These two problems are actually orthogonal and have different solutions. The onus for solving the first problem is laid upon the code which does the invocation; it is required to ensure that it does not dereference null. The store-in-temporary-and-test pattern ensures that null is never dereferenced. (There are other ways to solve this problem; for example, initializing the handler to have an empty action that is never removed. But doing a null check is the standard pattern.)

The onus for solving the second problem is laid upon the code being invoked; event handlers are required to be robust in the face of being called even after the event has been unsubscribed. In the scenario I described, the bug is actually in H. It needs to be robust enough to check to see whether the state it needs is still there, and bail out cleanly if it is not. (Or, alternatively, some additional locking mechanism needs to be implemented which ensures that the code which fires the event cooperates with the code that changes the event handlers to ensure the desired behaviour.)

The point though is that the "null ref problem" and the "stale handler problem" are two separate problems that are easily confused because the symptoms of both arise at the exact same call site code. You've got to solve both problems if you want to do threadsafe events. How you do so is up to you; just don't confuse the solution of one problem for a solution of the other.

(Thanks to Microsofties Levi Broderick, Chris Burrows, Curt Hagenlocher and Wolf Logan; this article is based on a conversation about their analysis of this pattern.)

*****************

(*) A number of people have asked me why I do not post stuff like this to stackoverflow.

First off, I want to say that stackoverflow is awesome, that incenting good behaviour through a reputation point system is genius, and that I applaud the collaborative problem solving culture exemplified by Jeff and Joel. Also, it is a valuable resource for me to see what people find confusing about the C# language.

The thing is: everything that I write at work is the property of Microsoft Corporation. When I post Microsoft-owned content to my own blog on the MSDN site, I know that I can correct mistakes, there are no copyright issues, and any problems associated with, say, unfair use of the material are a problem for our lawyers. Also, my readers can be 100% clear that what I post here was written by me; there are no attribution issues.

None of that is true when I post to third-party sites. I am not a lawyer; I do not know what the legal consequences are of posting Microsoft-owned material to a third party site, I don’t know that I’ll always be able to find and fix mistakes, and so on.

I am sure that Jeff and Joel have a generous fair-use policy and mechanisms in place for posters to correct mistakes, I know they use a reasonable identity verification scheme, and so on. But notwithstanding all that, I’ve decided recently that it is my personal policy to not take on the burden of vetting third-party sites, even awesome ones.

If the content I would have posted on stackoverflow is valuable then odds are good that it will reach the right people when I post it here.

Arrays considered somewhat harmful

Eric Lippert — Mon, 22 Sep 2008 21:04:00 GMT

I got a moral question from an author of programming language textbooks the other day requesting my opinions on whether or not beginner programmers should be taught how to use arrays.

Rather than actually answer that question, I gave him a long list of my opinions about arrays, how I use arrays, how we expect arrays to be used in the future, and so on. This gets a bit long, but like Pascal, I didn't have time to make it shorter.

Let me start by saying when you definitely should not use arrays, and then wax more philosophical about the future of modern programming and the role of the array in the coming world.

You probably should not return an array as the value of a public method or property, particularly when the information content of the array is logically immutable. Let me give you an example of where we got that horridly wrong in a very visible way in the framework. If you take a look at the documentation for System.Type, you'll find that just looking at the method descriptions gives one a sense of existential dread. One sees a whole lot of sentences like "Returns an array of Type objects that represent the constraints on the current generic type parameter." Almost every method on System.Type returns an array it seems.

Now think about how that must be implemented. When you call, say, GetConstructors() on typeof(string), the implementation cannot possibly do this, as sensible as it seems.

public class Type {
   private ConstructorInfo[] ctorInfos;
   public ConstructorInfo[] GetConstructors()
   {
     if (ctorInfos == null) ctorInfos = GoGetConstructorInfosFromMetadata();
     return ctorInfos;
   }

Why? Because now the caller can take that array and replace the contents of it with whatever they please. Returning an array means that you have to make a fresh copy of the array every time you return it. You get called a hundred times, you’d better make a hundred array instances, no matter how large they are. It’s a performance nightmare – particularly if, like me, you are considering using reflection to build a compiler. Do you have any idea how many times a second I try to get type information out of reflection? Not nearly as many times as I could; every time I do it’s another freakin’ array allocation!

The frameworks designers were not foolish people; unfortunately, we did not have generic types in .NET 1.0. clearly the sensible thing now for GetConstructors() to return is IList<ConstructorInfo>. You can build yourself a nice read-only collection object once, and then just pass out references to it as much as you want.

What is the root cause of this malaise? It is simple to state: The caller is requesting values. The callee fulfills the request by handing back variables.

An array is a collection of variables. The caller doesn’t want variables, but it’ll take them if that’s the only way to get the values. But in this case, as in most cases, neither the callee nor the caller wants those variables to ever vary. Why on earth is the callee passing back variables then? Variables vary. Therefore, a fresh, different variable must be passed back every time, so that if it does vary, nothing bad happens to anyone else who has requested the same values.

If you are writing such an API, wrap the array in a ReadOnlyCollection<T> and return an IEnumerable<T> or an IList<T> or something, but not an array. (And of course, do not simply cast the array to IEnumerable<T> and think you’re done! That is still passing out variables; the caller can simply cast back to array! Only pass out an array if it is wrapped up by a read-only object.)

That’s the situation at present. What are the implications of array characteristics for the future of programming and programming languages?

Parallelism Problems

The physics aspects of Moore’s so-called “Law” are failing, as they eventually must. Clock speeds have stopped increasing, transistor density has stopped increasing. The laws of thermodynamics and the Uncertainty Principle are seeing to that. But manufacturing costs per chip are still falling, which means that our only hope of Moore’s "Law" continuing to hold over the coming decades is to cram more and more processors into each box.

We’re going to need programming languages that allow mere mortals to write code that is parallelizable to multiple cores.

Side-effecting change is the enemy of parallelization. Parallelizing in a world with observable side effects means locks, and locks means choosing between implementing lock ordering and dealing with random crashes or deadlocks. Lock ordering requires global knowledge of the program. Programs are becoming increasingly complex, to the point where one person cannot reasonably and confidently have global knowledge. Indeed, we prefer programming languages to have the property that programs in them can be understood by understanding one part at a time, not having to swallow the whole thing in one gulp.

Therefore we tools providers need to create ways for people to program effectively without causing observable side effects.

Of all the sort of “basic” types, arrays most strongly work against this goal. An array’s whole purpose is to be a mass of mutable state. Mutable state is hard for both humans and compilers to reason about. It will be hard for us to write compilers in the future that generate performant multi-core programs if developers use a lot of arrays.

Now, one might reasonably point out that List<T> is a mass of mutable state too. But at least one could create a threadsafe list class, or an immutable list class, or a list class that has transactional integrity, or uses some form of isolation or whatever. We have an extensibility model for lists because lists are classes. We have no ability to make an “immutable array”. Arrays are what they are and they’re never going to change.

Conceptual Problems

We want C# to be a language in which one can draw a line between code that implements a mechanism and code that implements a policy.

The “C” programming language is all about mechanisms. It lays bare almost exactly what the processor is actually doing, providing only the thinnest abstraction over the memory model. And though we want you to be able to write programs like that in C#, most of the time people should be writing code in the “policy” realm. That is, code that emphasizes what the code is supposed to do, not how it does it.

Coding which is more declarative than imperative, coding which avoids side effects, coding which emphasizes algorithms and purposes over mechanisms, that kind of coding is the future in a world of parallelism. (And you’ll note that LINQ is designed to be declarative, strongly abstract away from mechanisms, and be free of side effects.)

Arrays work against all of these factors. Arrays demand imperative code, arrays are all about side effects, arrays make you write code which emphasizes how the code works, not what the code is doing or why it is doing it. Arrays make optimizing for things like “swapping two values” easy, but destroy the larger ability to optimize for parallelism.

Practical Problems

And finally, given that arrays are mutable by design, the way an array restricts that mutability is deeply weird. All the contents of the collection are mutable, but the size is fixed. What is up with that? Does that solve a problem anyone actually has?

For this reason alone I do almost no programming with arrays anymore. Arrays simply do not model any problem that I have at all well – I rarely need a collection which has the rather contradictory properties of being completely mutable, and at the same time, fixed in size. If I want to mutate a collection it is almost always to add something to it or remove something from it, not to change what value an index maps to.

We have a class or interface for everything I need. If I need a sequence I’ll use IEnumerable<T>, if I need a mapping from contiguous numbers to data I’ll use a List<T>, if I need a mapping across arbitrary data I’ll use a Dictionary<K,V>, if I need a set I’ll use a HashSet<T>. I simply don’t need arrays for anything, so I almost never use them. They don’t solve a problem I have better than the other tools at my disposal.

Pedagogic Problems

It is important that beginning programmers understand arrays; it is an important and widely used concept. But it is also important to me that they understand the weaknesses and shortcomings of arrays. In almost every case, there is a better tool to use than an array.

The difficulty is, pedagogically, that it is hard to discuss the merits of those tools without already having down concepts like classes, interfaces, generics, asymptotic performance, query expressions, and so on. It’s a hard problem for the writer and for the teacher. Fortunately, for me, it's not a problem that I personally have to solve.

Mutating Readonly Structs

Eric Lippert — Wed, 14 May 2008 17:10:00 GMT

Consider this program which attempts to mutate a readonly mutable struct. What happens?

struct Mutable {
    private int x;
    public int Mutate() {
        this.x = this.x + 1;
        return this.x;
    }
}

class Test {
    public readonly Mutable m = new Mutable();
    static void Main(string[] args) {
        Test t = new Test();
        System.Console.WriteLine(t.m.Mutate());
        System.Console.WriteLine(t.m.Mutate());
        System.Console.WriteLine(t.m.Mutate());
    }
}

There are a number of things this program could do. Does it:

1) Print 1, 2, 3 -- because m is readonly, but the "readonly" only applies to m, not to its contents.
2) Print 0, 0, 0 -- because m is readonly, x cannot be changed. It always has its default value of zero.
3) Throw an exception at runtime, when the attempt is made to mutate the contents of a readonly field.
4) Do something else

People are frequently surprised to learn that the answer is (4). In fact, this prints 1, 1, 1.

Why?

Because, remember, accessing a value type gives you a COPY of the value. When you say t.m, you get a copy of whatever is presently stored in m. m is immutable, but the copy is not. The copy is then mutated, and the value of x in the copy is returned. But m remains untouched.

The relevant section of the specification is 7.5.4, which states that when resolving "E.I" where E is an object and I is a field...

...if the field is readonly and the reference occurs outside an instance constructor of the class in which the field is declared, then the result is a value, namely the value of the field I in the object referenced by E.

The important word here is that the result is the value of the field, not the variable associated with the field. Readonly fields are not variables outside of the constructor. (The initializer here is considered to be inside the constructor; see my earlier post on that subject.)

Great. What about that second dot, as in ".Mutate()"? We look at section 7.4.4 to find out how to invoke E.M():

If E is not classified as a variable, then a temporary local variable of E's type is created and the value of E is assigned to that variable. E is then reclassified as a reference to that temporary local variable. The temporary variable is accessible as this within M, but not in any other way. Thus, only when E is a true variable is it possible for the caller to observe the changes that M makes to this.

And there you go. Value semantics are tricky!

This is yet another reason why mutable value types are evil. Try to always make value types immutable.

Trivial Projections Are (Usually) Optimized Away

Eric Lippert — Mon, 12 May 2008 16:57:00 GMT

OK, computers aren't entirely dumb when it comes to LINQ. Here's an example of a place where we're a bit smarter.

Consider the following query:

IEnumerable<int> query = from n in number_array orderby n select n;

Does this get transformed by the compiler into

IEnumerable<int> query = number_array.OrderBy(n => n);

IEnumerable<int> query = number_array.OrderBy(n => n).Select( n => n );

This question caused tremendous debate amongst the members of the C# design and implementation teams. I am quite happy with the compromise we achieved.

The sections of the C# specification which cover this topic are sections 7.15.2.3, and 7.15.2.5:

***

A query expression of the form “from x in e select v” is translated into “( e ) . Select ( x => v )” except when v is the identifier x, the translation is simply “( e )”

A query expression of the form “from x in e select x” is translated into “( e ) . Select ( x => x )”.

A degenerate query expression is one that trivially selects the elements of the source. A later phase of the translation removes degenerate queries introduced by other translation steps by replacing them with their source. It is important however to ensure that the result of a query expression is never the source object itself, as that would reveal the type and identity of the source to the client of the query. Therefore this step protects degenerate queries written directly in source code by explicitly calling Select on the source. It is then up to the implementers of Select and other query operators to ensure that these methods never return the source object itself.

***

It might sound like the second statement contradicts the first. The first statement says that the “.Select(x=>v)” is omitted when “v” is “x”, and the second statement says that no, the “.Select(x=>x)” is in fact generated.

The spec is not particularly clear on this point, I agree, though the explanatory text about “degenerate query expressions” does help clear it up somewhat. Basically what we are saying here is that the optimization to remove degenerate selects is only performed on the result of an earlier query translation.

Some examples will help.

If you have from n in number_array orderby n select n then this is first translated into from n in (number_array).OrderBy(n=>n) select n and then, since the selector is the same as the range variable, this is translated into ((number_array).OrderBy(n=>n)).

We optimize away the Select(n=>n) because from n in (number_array).OrderBy(n=>n) select n is the intermediate result of an earlier query translation.

If you had from n in number_array select n then this would be considered a degenerate query expression requiring a Select. We cannot simply translate it into (number_array) because then you could assign that to a variable and have referential identity with the collection. The result of the query should not “leak” information about the collection out; it should just be an enumerable object of the appropriate type, and not have an identity relationship. Therefore, we take the performance hit and protect the identity of the collection by emitting a degenerate query translation (number_array).Select(n=>n).

This is important because the collection might be a mutable collection. One does not expect that the result of a query over that collection could be used to mutate the collection! The query results should be read-only.

We do not need to append the degenerate Select(n=>n) when you have an “orderby” in there because the OrderBy(n=>n) call already protects the identity of the original collection. This saves on the performance cost of the no-op Select.

I think we’ve made a reasonable compromise in the case of degenerate queries. If you are crazy enough to write “from x in xs select x” when you mean “(xs)”, then you’ll have to live with the decreased performance. But if you are introducing a degenerate select at the end of a meaningful query, then it will be optimized away.

Protected Semantics, Part Five: More on immutability

Eric Lippert — Mon, 05 May 2008 19:52:47 GMT

I asked a second follow-up question back in Part Two:

Suppose you wanted to make this hierarchy an immutable collection, where "Add" and "Remove" returned new collections rather than mutating the existing collection. How would you represent the parenting relationship?

The short answer is "I wouldn't". First off, it sounds an awful lot like what is intended to be represented here is some sort of highly mutable system, where objects are being added and removed from containers. It might be more sensible to represent that thing using mutable data structures rather than immutable data structures.

If you did want to represent the thing with immutable data structures, then I would think twice before attempting to represent parenting at all. Here's why: when you have an immutable tree structure without parents, and you want to make a new tree structure that is based on an old one, typically the only stuff you have to change is from the edited node on up to the root node; usually hierarchies are shallow, so this is not very many nodes.

But this logically implies that the root node changes on every edit. If the root node changes, then all of its children have a new parent, and all of their children then have a new parent... and you've just rewritten the entire tree structure, not just a few nodes.

Now, it is often highly convenient to know what the parent of a given node is. To achieve that, what I'd probably do is try to not need to query the parent of any node until I was at a point where I wanted to ask about a lot of parents, and the tree was likely to remain unedited while I was asking. A quick visitor pass that builds up a hash table mapping every node to its parent would be easy enough to write, and then the parent table could be discarded and its memory reclaimed once I was done.

Why Do Initializers Run In The Opposite Order As Constructors? Part Two

Eric Lippert — Mon, 18 Feb 2008 18:25:12 GMT

As you might have figured out, the answer to last week's puzzle is "if the constructors and initializers run in their actual order then an initialized readonly field of reference type is guaranteed to be non null in any possible call. That guarantee cannot be met if the initializers run in the expected order."

Suppose counterfactually that initializers ran in the expected order, that is, derived class initializers run after the base class constructor body. Consider the following pathological cases:

class Base
{
    public static ThreadsafeCollection t = new ThreadsafeCollection();
    public Base()
    {
        Console.WriteLine("Base constructor");
        if (this is Derived) (this as Derived).DoIt();
        // would deref null if we are constructing an instance of Derived
        Blah();
        // would deref null if we are constructing an instance of MoreDerived
        t.Add(this);
        // would deref null if another thread calls Base.t.GetLatest().Blah();
        // before derived constructor runs
    }
    public virtual void Blah() { }
}
class Derived : Base
{
    readonly Foo derivedFoo = new Foo("Derived initializer");
    public DoIt()
    {
        derivedFoo.Bar();
    }
}
class MoreDerived : Derived
{
    public override void Blah() { DoIt(); }
}

Calling methods on derived types from constructors is dirty pool, but it is not illegal. And stuffing not-quite-constructed objects into global state is risky, but not illegal. I'm not recommending that you do any of these things -- please, do not, for the good of us all. I'm saying that it would be really nice if we could give you an ironclad guarantee that an initialized readonly field is always observed in its initialized state, and we cannot make that guarantee unless we run all the initializers first, and then all of the constructor bodies.

Note that of course, if you initialize your readonly fields in the constructor, then all bets are off. We make no guarantees as to the fields not being accessed before the constructor bodies run.

Next time on FAIC: how to get a question not answered.

Why Do Initializers Run In The Opposite Order As Constructors? Part One

Eric Lippert — Sat, 16 Feb 2008 02:01:00 GMT

Pop quiz!

What do you expect the output of this program to be?

using System;

class Foo
{
    public Foo(string s)
    {
        Console.WriteLine("Foo constructor: {0}", s);
    }
    public void Bar() { }
}
class Base
{
    readonly Foo baseFoo = new Foo("Base initializer");
    public Base()
    {
        Console.WriteLine("Base constructor");
    }
}
class Derived : Base
{
    readonly Foo derivedFoo = new Foo("Derived initializer");
    public Derived()
    {
        Console.WriteLine("Derived constructor");
    }
}
static class Program
{
    static void Main()
    {
        new Derived();
    }
}

I got a question from a user recently noting that the order was not as he expected. One naively expects that the order will go "base initializers, base constructor body, derived initializers, derived constructor body". In fact the order actually is that first all the initializers run in order from derived to base, and then all the constructor bodies run in order from base to derived.

The latter bit makes perfect sense; the more derived constructors may rely upon state initialized by the less derived constructors, so the constructors should run in order from base to derived. But most people assume that the call sequence of the code above is equivalent to this pseudocode:

// Expected
BaseConstructor()
{
    ObjectConstructor();
    baseFoo = new Foo("Base initializer");
    Console.WriteLine("Base constructor");
}
DerivedConstructor()
{
    BaseConstructor();
    derivedFoo = new Foo("Derived initializer");
    Console.WriteLine("Derived constructor");
}

When in point of fact it is equivalent to this:

// Actual
BaseConstructor()
{
    baseFoo = new Foo("Base initializer");
    ObjectConstructor();
    Console.WriteLine("Base constructor");
}
DerivedConstructor()
{
     derivedFoo = new Foo("Derived initializer");
    BaseConstructor();
    Console.WriteLine("Derived constructor");
}

That explains the mechanism whereby the initializers run in order from derived to base and the constructor bodies run in the opposite order, but why did we choose to implement that mechanism instead of the more intuitively obvious former way?

Puzzle that one over for a bit, and then read on for a hint.

...

The "readonly" modifiers in there were no accident. The code gives the appearance that any call to derivedFoo.Bar() and baseFoo.Bar() should never fail with a null dereference exception because both are readonly fields initialized to non-null values.

Is that appearance accurate, or misleading?
Now suppose initializers ran in the "expected" order and answer question (1) again.

I'll post the answers and analysis next week. Have a fabulous weekend!

Immutability in C# Part Eleven: A working double-ended queue

Eric Lippert — Wed, 13 Feb 2008 02:40:00 GMT

At long last, let's get to that double-ended queue. Sorry for leaving you in suspense!

Last time we came up with a non-solution -- as commenters immediately discovered, the given algorithm and data structure are deeply inefficient. However, the basic idea is sound. There are a number of ways to solve this problem, but the way I'm going to talk about today is a much more efficient variation on the theme of our previous unsuccessful attempt. That is, have cheap access to the endpoints, and have some kind of more expensive "cold storage" in the middle.

In order to solve this problem I'm first going to introduce some helper classes which solve a much simpler problem. I want to implement a very limited kind of deque, namely one which can only store one, two, three or four items. A "dequelette", if you'll allow the silly name.

Note that there is no "empty" dequelette. Since this class does not actually fulfill the contract for a deque, I'm not going to have it implement IDeque<T>. Rather, I'll just have a base class that these four dequelette types derive from:

public class Deque<T> : IDeque<T>
{
private abstract class Dequelette
{
    public abstract int Size { get; }
    public virtual bool Full { get { return false; } }
    public abstract T PeekLeft();
    public abstract T PeekRight();
    public abstract Dequelette EnqueueLeft(T t);
    public abstract Dequelette EnqueueRight(T t);
    public abstract Dequelette DequeueLeft();
    public abstract Dequelette DequeueRight();
}

The implementations of dequelette classes One, Two, Three and Four are obvious; rather than take up a lot of space here, I'll put them on a separate page along with the rest of the implementation.

OK. What does this trivial little class hierarchy buy us? We have an immutable deque which can contain one, two, three or four elements, and that's all. Useful? Yes.

Here's my new recursive definition of a deque: a deque of T's is:

empty, or
a single element of type T, or
a left dequelette of T, followed by a middle deque of dequelettes of T, followed by a right dequelette of T.

That bit in boldface is the key. The cold storage in the middle is NOT another deque of items, it is a deque of dequelettes of items. (In particular in this implementation it will furthermore be a deque of dequelettes where every dequelette is of size three. Each item in the cold storage holds three items of the outer deque.)

This use of the type system may be a bit confusing. Let's draw it out. Without loss of generality, let's suppose that we're enqueuing 1, 2, 3, 4, ... onto the right hand side of a deque, starting with the empty deque. The structures we built up go like this:

Deque<int>.Empty
Deque<int>.SingleDeque (1)
Deque<int>( One(1) -- Deque<Deque<int>.Dequelette>.Empty -- One(2) )
Deque<int>( One(1) -- Deque<Deque<int>.Dequelette>.Empty -- Two(2, 3) )
Deque<int>( One(1) -- Deque<Deque<int>.Dequelette>.Empty -- Three(2, 3, 4) )
Deque<int>( One(1) -- Deque<Deque<int>.Dequelette>.Empty -- Four(2, 3, 4, 5) )

Now at this point we are about to enqueue 6, but there is no more room on the cheap storage at the right endpoint. We need to make more room, so let's take three of those out of there, make a dequelette out of them, and stuff them into the right hand side of the cold storage. Notice that the thing in the middle is a single-element deque which contains a dequelette. It is not a three-element deque of integers:

Deque<int>( One(1) -- Deque<Deque<int>.Dequelette>.Single (Three(2, 3, 4) ) -- Two(5, 6))
Deque<int>( One(1) -- Deque<Deque<int>.Dequelette>.Single (Three(2, 3, 4) ) -- Three(5, 6, 7))
Deque<int>( One(1) -- Deque<Deque<int>.Dequelette>.Single (Three(2, 3, 4) ) -- Four(5, 6, 7, 8))

Again, as we are about to enqueue 9, we're out of space in the cheap seats. Move stuff into cold storage. That causes the cold storage to go from a single-element deque to a two-element deque. Note the type of the innermost middle empty element is a deque of dequelettes, where each dequelette contains three dequelettes, each of which contains three ints. So the innermost cold storage will be storing nine items per node when eventually we start pushing stuff into it.

Deque<int>(
    One(1)
    Deque<Deque<int>.Dequelette>(
        One(Three(2, 3, 4))
        Deque<Deque<Deque<int>.Dequelette>.Dequelette>.Empty
        One(Three(5, 6, 7)))
    Two(8, 9))

Our last example was inefficient. Is this efficient?

When enqueueing, most of the time the operation is cheap, just creating a new dequelette. Every third enqueue is expensive; it requires us to do two operations: first, to turn a four-dequelette into a two-dequelette, and second, to create a three dequelette and enqueue it into the cold storage. Every ninth enqueue causes the cold storage to itself have to do an expensive operation on its cold storage. And every twenty-seventh operation requires... and so on. If you sum up that series, you find that the most expensive possible operation costs O(log n), but those are so few and far between that the average cost of enqueueing each element is O(1). Which is awesome.

The other objection that my psychic powers are picking up on is "holy goodness, what horrors must that be wrecking upon the type system?" Relax. The generic type system is plenty buff enough to handle this. Remember, it's a logarithmic thing. A deque with a million integers in it will have a Deque<Deque<Deque<... <int>...> no more than, say, twenty deep. The generic type system is plenty buff enough to handle a recursively defined generic type a mere twenty levels deep.

Also, remember, this is a generic type, not a C++ template. All the code for that crazy generic type is generated once by the jitter, and then the same code is shared for every reference constructed type where the type argument is a reference type -- which in this case, is all of them. That's the great thing about generic types; because they guarantee at compile time that everything in the implementation will work out at runtime, the code can be reused.

What about dequeueing? Same deal. When the cheap storage on the end points gets down to a single element dequelette, just dequeue a three-dequelette from the code storage and use that as the new cheap storage. Again, at least two thirds of all operations are cheap and the price of the expensive ones grows slowly, so the amortized cost of each dequeue is constant.

As a more advanced topic, which I may come back to at some point in the future, we can also implement cheap concatenation of two deques if we allow the cold storage to hold both twos and threes, instead of just threes. I have an implementation but the code for that gets really quite tedious, so I won't show it here. Using twos or threes makes it potentially slightly more expensive than just using threes because you go from a guarantee of two thirds of the operations being cheap to only half being cheap. But on average the amortized cost is still constant per operation whether it is twos or threes.

If you're still having trouble grasping the details of this data structure, try drawing out all the references on a whiteboard. You get a very odd tree-like structure. With most trees, the nodes "in the middle" are the "obvious" ones because they are near the root. In this tree structure (and it is a tree structure in the final analysis!) its the leaf notes near the edges which are the most "shallow" nodes. The middle is where the tree gets "deeper". This is exactly what you want for a structure where the stuff on the edges needs to be cheap to access compared to the stuff in the middle.

Incidentally, this kind of tree is an example of a "finger tree". The "fingers" are the special references to the various special "edge" nodes.

We could keep on going and talking about more and more advanced analysis of immutable structures, but I think I'll stop here. This gives the flavour of this style of programming pretty well I think, so no need to labour the point further.

Acknowledgements:

One of the reasons I write about this stuff is because it forces me to learn about it. I learned a lot, and I hope you did too. For this series, I am indebted to Chris Okasaki's excellent book "Purely Functional Data Structures"; the thesis it was based upon is here. Also to Ralf Hinze and Ross Paterson's excellent paper on finger trees.

Coming up next: I'm not exactly sure. I have a pretty kick-ass implementation of some very fun algorithms which demonstrate the practical advantages of programming in an immutable style. Or perhaps I'll change gears a bit and talk about something else entirely. Either way, it'll be a fabulous adventure I'm sure.

Immutability in C# Part 10: A double-ended queue

Eric Lippert — Tue, 22 Jan 2008 18:15:53 GMT

Based on the comments, the implementation of a single-ended queue as two stacks was somewhat mind-blowing for a number of readers. People, you ain't seen nothing yet.

Before we get into the actual bits and bytes of the solution, think for a bit about how you might implement an immutable queue which could act like both a stack or a queue at any time. You can think of a stack as "it goes on the left end, it comes off the left end", and a queue as "it goes on the left end, it comes off the right end". Now we want "it goes on and comes off either end". For short, we'll call a double ended queue a "deque" (pronounced "deck"), and give our immutable deque this interface:

    public interface IDeque<T>
    {
        T PeekLeft();
        T PeekRight();
        IDeque<T> EnqueueLeft(T value);
        IDeque<T> EnqueueRight(T value);
        IDeque<T> DequeueLeft();
        IDeque<T> DequeueRight();
        bool IsEmpty { get; }
    }

Attempt #1

We built a single-ended queue out of two stacks. Can we pull a similar trick here? How about we have the "left stack" and the "right stack". Enqueuing on the left pushes on the left stack, enqueuing on the right pushes on the right stack, and so on.

Unfortunately, this has some problems. What if you are dequeuing on the right and you run out of items on the right-hand stack? Well, no problem, we'll pull the same trick as before -- reverse the left stack and swap it with the right stack.

The trouble with that is, suppose the left stack is { 1000, 999, ..., 3, 2, 1 } and the right stack is empty. Someone dequeues the deque on the right. We reverse the stack, swap them and pop the new right stack. Now we have an empty left-hand stack and { 2, 3, 4, .... 1000 } on the right hand stack. It took 1000 steps to do this. Now someone tries to dequeue on the left. We reverse the right queue, swap, and pop, and now we have { 999, 998, ... 3, 2 }. That took 999 steps. If we keep on dequeuing alternating on the right and left we end up doing on average five hundred pushes per step. That's terrible performance. Clearly this is an O(n²) algorithm.

Attempt #2

Our attempt to model this as a pair of stacks seems to be failing. Let's take a step back and see if we can come up with a recursively defined data structure which makes it more apparent that there is cheap access to each end.

The standard recursive definition of a stack is "a stack is either empty, or an item (the head) followed by a stack (the tail)". It seems like we ought to be able to say "a deque is either empty, or an item (the left) followed by a deque (the middle) followed by an item (the right)".

Perhaps you have already seen the problem with this definition; a deque by this definition always has an even number of elements! But we can fix that easily enough. A deque is:

1) empty, or
2) a single item, or
3) a left item followed by a middle deque followed by a right item.

Awesome. Let's implement it.

    // WARNING: THIS IMPLEMENTATION IS AWFUL. DO NOT USE THIS CODE.
    public sealed class Deque<T> : IDeque<T>
    {
        private sealed class EmptyDeque : IDeque<T>
        {
            public bool IsEmpty { get { return true; } }
            public IDeque<T> EnqueueLeft(T value) { return new SingleDeque(value); }
            public IDeque<T> EnqueueRight(T value) { return new SingleDeque(value); }
            public IDeque<T> DequeueLeft() { throw new Exception("empty deque"); }
            public IDeque<T> DequeueRight() { throw new Exception("empty deque"); }
            public T PeekLeft () { throw new Exception("empty deque"); }
            public T PeekRight () { throw new Exception("empty deque"); }
        }
        private sealed class SingleDeque : IDeque<T>
        {
            public SingleDeque(T t) { item = t; }
            private readonly T item;
            public bool IsEmpty { get { return false; } }
            public IDeque<T> EnqueueLeft(T value) { return new Deque<T>(value, Empty, item); }
            public IDeque<T> EnqueueRight(T value) { return new Deque<T>(item, Empty, value); }
            public IDeque<T> DequeueLeft() { return Empty; }
            public IDeque<T> DequeueRight() { return Empty; }
            public T PeekLeft () { return item; }
            public T PeekRight () { return item; }
        }
        private static readonly IDeque<T> empty = new EmptyDeque();
        public static IDeque<T> Empty { get { return empty; } }
        public bool IsEmpty { get { return false; } }
        private Deque(T left, IDeque<T> middle, T right)
        {
            this.left = left;
            this.middle = middle;
            this.right = right;
        }
        private readonly T left;
        private readonly IDeque<T> middle;
        private readonly T right;
        public IDeque<T> EnqueueLeft(T value)
        {
            return new Deque<T>(value, middle.EnqueueLeft(left), right);
        }
        public IDeque<T> EnqueueRight(T value)
        {
            return new Deque<T>(left, middle.EnqueueRight(right), value);
        }
        public IDeque<T> DequeueLeft()
        {
            if (middle.IsEmpty) return new SingleDeque(right);
            return new Deque<T>(middle.PeekLeft(), middle.DequeueLeft(), right);
        }
        public IDeque<T> DequeueRight()
        {
            if (middle.IsEmpty) return new SingleDeque(left);
            return new Deque<T>(left, middle.DequeueRight(), middle.PeekRight());
        }
        public T PeekLeft () { return left; }
        public T PeekRight () { return right; }
    }

I seem to have somewhat anticipated my denouement, but this is coding, not mystery novel writing. What is so awful about this implementation? It seems like a perfectly straightforward implementation of the abstract data type. But it turns out to be actually worse than the two-stack implementation we first considered. What are your thoughts on the matter?

Next time, what's wrong with this code and some groundwork for fixing it.

Immutability in C# Part Nine: Academic? Plus my AVL tree implementation

Eric Lippert — Tue, 22 Jan 2008 03:13:00 GMT

Good Monday morning all. I have received a lot of good comments on this series so far that I thought I would speak to a bit.

Academic?

First and foremost, a number of people have asked questions which could be summed up as "isn't this just an academic exercise? I have a job to do here!"

No, I do not believe that it is just an academic exercise, not at all. When I compare the practical, solving-real-problems code I write using immutable or ("mostly" immutable objects) to that which uses highly mutable objects, I find that the solutions which use immutable objects are easier to reason about. Knowing that an object isn't going to be "edited" by some other bit of code later on means that anything I deduce about it now is going to continue to be true forever.

I also find that programming in a more "functional" style using immutable objects somehow encourages me to write very small, clear methods that each do one thing very well. This is certainly possible when writing code with mutable objects, but I find that something about writing code for immutable objects encourages that style more. I haven't managed to quite figure out why that is yet. I like that style of programming a lot. Take a look at the short little methods below, for, say, tree rotation. Each of them is about four lines long and obviously correct if you read it carefully. That gives you confidence that those rotation helpers can then be composed to make a correct balancer, which gives you confidence that the tree is balanced.

Another reason why immutable objects tend to be easier to reason about is that this style encourages non-nullable programming. Take a look at the code from the previous entries in this series. The keyword "null" does not appear in any of them! And why should it? Why should we represent an empty stack the same way we represent an empty queue, the same way we represent an empty tree? By ensuring that everything which points to something else is never null, all that boring, slow, hard to read, hard to reason about code which checks to see if things are null just goes away. You need never worry whether that reference is null again if you design your objects so that there's never a null reference in the first place.

And as we'll see in later episodes of FAIC, immutable objects also make it easier to implement performance improvements such as memoization of complex calculations. If an object never changes then you can always cache it for reuse later. An object which someone can edit is less useful that way. Immutable objects stored in a stack make undo-redo operations trivial. And so on.

Learning how to use this style in your own code is therefore potentially quite handy. I believe it is also the case that more and more objects produced by other people will be immutable, so knowing how to deal with them will be increasingly important. For example, the objects which represent expression trees in C# 3.0 are all immutable. If you want to take an expression tree and transform it, you're simply not going to be able to transform it "in place". You're going to have to build a new expression tree out of the old one; knowing how to do so efficiently will help.

Thread safe?

The second question I've gotten a lot is "are immutable objects really more thread safe?" Well, it depends on what you mean by "thread safe". Immutable objects are not necessarily "more" thread safe, but they're a different kind of thread safe.

Let's explore this a bit more. What exactly do we mean by "thread safe"? We can talk about lock order and deadlocks and all of those aspects of thread safety, but let's put those aside for a moment and just consider basic race conditions. Suppose you have two threads each enqueuing jobs onto a queue object, and a third thread dequeuing jobs. Normally we think of the queue as being "thread safe" if no matter what the timing of those three threads happens to be, there is no way that a job is ever "lost". Two enqueues which happen "at the same time" result in a queue with those two things on it, and at some point, both will be dequeued.

If the queue is immutable then implementing this scenario just moves the problem around; now it becomes about serializing access to the mutable variable which is being shared by all three threads. Immutable objects make this kind of scenario harder to reason about and implement; better to just write a threadsafe mutable queue in the first place if that's the problem you're trying to solve.

But let's think about this a different way. Having a threadsafe mutable queue means that you never have any accurate information whatsoever about the state of the queue! You ask the queue "are you empty?" and it says "no", and you dequeue it and hey, you get an "empty queue" exception. Why? Because some other thread dequeued it in the time between when you asked and when you issued your dequeue request. You end up living in a world where enqueuing a queue can possibly make it shorter and dequeuing can make it longer depending on what is going on with other threads. It becomes impossible to reason locally about operations on an object; your consciousness has to expand to encompass all possible operations on the object. It's this necessity for global understanding that makes multithreaded programming so error-prone and difficult.

Immutable queues, on the other hand, give you complete thread safety in this regard. You enqueue an element onto a particular immutable queue and the result you get back is always the same no matter what is happening to that queue on any other thread. You ask a queue if it is empty, if it says no, then you have a 100% iron-clad guarantee that you can safely dequeue it. You can share data around threads as much as you like without worrying that some other thread is going to make an edit which messes up your logic. You can take a chunk of data and run ten different analyzers over it on ten different threads, each one making "changes" to the data during its analysis, and never interfering with each other.

AVL Tree implementation

As promised last time, my supremely unexciting implementation of a self-balancing height-balanced immutable AVL tree in C#. Note that I have made the choice to cache the height in every node rather than recalculating it. That trades memory usage for a bit of extra speed, which is probably worth it in this case. (Though of course to truly answer the question we'd want to set goals, try it both ways and measure the results.)

This is all pretty straightforward. Next time I want to start to wrap up this series by looking at a tricky data structure that solves the problem of building an immutable double-ended queue.

     public sealed class AVLTree<K, V> : IBinarySearchTree<K, V> where K : IComparable<K>
    {
        private sealed class EmptyAVLTree : IBinarySearchTree<K, V>
        {
            // IBinaryTree
            public bool IsEmpty { get { return true; } }
            public V Value { get { throw new Exception("empty tree"); } }
            IBinaryTree<V> IBinaryTree<V>.Left { get { throw new Exception("empty tree"); } }
            IBinaryTree<V> IBinaryTree<V>.Right { get { throw new Exception("empty tree"); } }
            // IBinarySearchTree
            public IBinarySearchTree<K, V> Left { get { throw new Exception("empty tree"); } }
            public IBinarySearchTree<K, V> Right { get { throw new Exception("empty tree"); } }
            public IBinarySearchTree<K, V> Search(K key) { return this; }
            public K Key { get { throw new Exception("empty tree"); } }
            public IBinarySearchTree<K, V> Add(K key, V value) { return new AVLTree<K, V>(key, value, this, this); }
            public IBinarySearchTree<K, V> Remove(K key) { throw new Exception("Cannot remove item that is not in tree."); }
            // IMap
            public bool Contains(K key) { return false; }
            public V Lookup(K key) { throw new Exception("not found"); }
            IMap<K, V> IMap<K, V>.Add(K key, V value) { return this.Add(key, value); }
            IMap<K, V> IMap<K, V>.Remove(K key) { return this.Remove(key); }
            public IEnumerable<K> Keys { get { yield break; } }
            public IEnumerable<V> Values { get { yield break; } }
            public IEnumerable<KeyValuePair<K,V>> Pairs { get { yield break; } }
        }
        private static readonly EmptyAVLTree empty = new EmptyAVLTree();
        public static IBinarySearchTree<K, V> Empty { get { return empty; } }
        private readonly K key;
        private readonly V value;
        private readonly IBinarySearchTree<K, V> left;
        private readonly IBinarySearchTree<K, V> right;
        private readonly int height;
        private AVLTree(K key, V value, IBinarySearchTree<K, V> left, IBinarySearchTree<K, V> right)
        {
            this.key = key;
            this.value = value;
            this.left = left;
            this.right = right;
            this.height = 1 + Math.Max(Height(left), Height(right));
        }
        // IBinaryTree
        public bool IsEmpty { get { return false; } }
        public V Value { get { return value; } }
        IBinaryTree<V> IBinaryTree<V>.Left { get { return left; } }
        IBinaryTree<V> IBinaryTree<V>.Right { get { return right; } }
        // IBinarySearchTree
        public IBinarySearchTree<K, V> Left { get { return left; } }
        public IBinarySearchTree<K, V> Right { get { return right; } }
        public IBinarySearchTree<K, V> Search(K key)
        {
            int compare = key.CompareTo(Key);
            if (compare == 0)
                return this;
            else if (compare > 0)
                return Right.Search(key);
            else
                return Left.Search(key);
        }
        public K Key { get { return key; } }
        public IBinarySearchTree<K, V> Add(K key, V value)
        {
            AVLTree<K, V> result;
            if (key.CompareTo(Key) > 0)
                result = new AVLTree<K, V>(Key, Value, Left, Right.Add(key, value));
            else
                result = new AVLTree<K, V>(Key, Value, Left.Add(key, value), Right);
            return MakeBalanced(result);
        }
        public IBinarySearchTree<K, V> Remove(K key)
        {
            IBinarySearchTree<K, V> result;
            int compare = key.CompareTo(Key);
            if (compare == 0)
            {
                // We have a match. If this is a leaf, just remove it
                // by returning Empty. If we have only one child,
                // replace the node with the child.
                if (Right.IsEmpty && Left.IsEmpty)
                    result = Empty;
                else if (Right.IsEmpty && !Left.IsEmpty)
                    result = Left;
                else if (!Right.IsEmpty && Left.IsEmpty)
                    result = Right;
                else
                {
                    // We have two children. Remove the next-highest node and replace
                    // this node with it.
                    IBinarySearchTree<K, V> successor = Right;
                    while (!successor.Left.IsEmpty)
                        successor = successor.Left;
                    result = new AVLTree<K, V>(successor.Key, successor.Value, Left, Right.Remove(successor.Key));
                }
            }
            else if (compare < 0)
                result = new AVLTree<K, V>(Key, Value, Left.Remove(key), Right);
            else
                result = new AVLTree<K, V>(Key, Value, Left, Right.Remove(key));
            return MakeBalanced(result);
        }
        // IMap
        public bool Contains(K key) { return !Search(key).IsEmpty; }
        IMap<K, V> IMap<K, V>.Add(K key, V value) { return this.Add(key, value); }
        IMap<K, V> IMap<K, V>.Remove(K key) { return this.Remove(key); }
        public V Lookup(K key)
        {
            IBinarySearchTree<K, V> tree = Search(key);
            if (tree.IsEmpty)
                throw new Exception("not found");
            return tree.Value;
        }
        public IEnumerable<K> Keys { get { return from t in Enumerate() select t.Key; } }
        public IEnumerable<V> Values { get { return from t in Enumerate() select t.Value; }
        }
        public IEnumerable<KeyValuePair<K,V>> Pairs
        {
            get
            {
                return from t in Enumerate() select new KeyValuePair<K, V>(t.Key, t.Value);
            }
        }
        private IEnumerable<IBinarySearchTree<K, V>> Enumerate()
        {
            var stack = Stack<IBinarySearchTree<K,V>>.Empty;
            for (IBinarySearchTree<K,V> current = this; !current.IsEmpty || !stack.IsEmpty; current = current.Right)
            {
                while (!current.IsEmpty)
                {
                    stack = stack.Push(current);
                    current = current.Left;
                }
                current = stack.Peek();
                stack = stack.Pop();
                yield return current;
            }
        }
        // Static helpers for tree balancing
        private static int Height(IBinarySearchTree<K, V> tree)
        {
            if (tree.IsEmpty)
                return 0;
            return ((AVLTree<K,V>)tree).height;
        }
        private static IBinarySearchTree<K, V> RotateLeft(IBinarySearchTree<K, V> tree)
        {
            if (tree.Right.IsEmpty)
                return tree;
            return new AVLTree<K, V>( tree.Right.Key, tree.Right.Value,
                new AVLTree<K, V>(tree.Key, tree.Value, tree.Left, tree.Right.Left),
                tree.Right.Right);
        }
        private static IBinarySearchTree<K, V> RotateRight(IBinarySearchTree<K, V> tree)
        {
            if (tree.Left.IsEmpty)
                return tree;
            return new AVLTree<K, V>( tree.Left.Key, tree.Left.Value, tree.Left.Left,
                new AVLTree<K, V>(tree.Key, tree.Value, tree.Left.Right, tree.Right));
        }
        private static IBinarySearchTree<K, V> DoubleLeft(IBinarySearchTree<K, V> tree)
        {
            if (tree.Right.IsEmpty)
                return tree;
            AVLTree<K, V> rotatedRightChild = new AVLTree<K, V>(tree.Key, tree.Value, tree.Left, RotateRight(tree.Right));
            return RotateLeft(rotatedRightChild);
        }
        private static IBinarySearchTree<K, V> DoubleRight(IBinarySearchTree<K, V> tree)
        {
            if (tree.Left.IsEmpty)
                return tree;
            AVLTree<K, V> rotatedLeftChild = new AVLTree<K, V>(tree.Key, tree.Value, RotateLeft(tree.Left), tree.Right);
            return RotateRight(rotatedLeftChild);
        }
        private static int Balance(IBinarySearchTree<K, V> tree)
        {
            if (tree.IsEmpty)
                return 0;
            return Height(tree.Right) - Height(tree.Left);
        }
        private static bool IsRightHeavy(IBinarySearchTree<K, V> tree) { return Balance(tree) >= 2; }
        private static bool IsLeftHeavy(IBinarySearchTree<K, V> tree) { return Balance(tree) <= -2; }
        private static IBinarySearchTree<K, V> MakeBalanced(IBinarySearchTree<K, V> tree)
        {
            IBinarySearchTree<K, V> result;
            if (IsRightHeavy(tree))
            {
                if (IsLeftHeavy(tree.Right))
                    result = DoubleLeft(tree);
                else
                    result = RotateLeft(tree);
            }
            else if (IsLeftHeavy(tree))
            {
                if (IsRightHeavy(tree.Left))
                    result = DoubleRight(tree);
                else
                    result = RotateRight(tree);
            }
            else
                result = tree;
            return result;
        }
    }
}

Immutability in C# Part Eight: Even More On Binary Trees

Eric Lippert — Fri, 18 Jan 2008 22:02:13 GMT

Last year we declared a relatively simple interface to represent an immutable binary tree. We noticed that it was different from every other interface that we've declared so far, in that it really said nothing at all about the immutability of the tree. One normally thinks of immutable data types not in terms of their shape, but rather in terms of what operations may be performed on the data type. Operations are usually either to query the object somehow, or to "modify" it by producing new modified versions of the old immutable object.

This binary tree interface is also unsatisfying because it does not address a problem to be solved (other than "organize this data into a binary tree", I suppose.) It seems to be a solution in search of a problem. So let's find a problem and build a better binary tree to solve that problem.

The problem I want to consider today is the implementation of immutable "dictionaries" or "maps". That is, objects which allow you to store a list of "keys" and their associated "values", and then rapidly look up the value associated with the key. Here's a reasonable definition for an immutable dictionary interface:

public interface IMap<K, V> where K : IComparable<K>
{
    bool Contains(K key);
    V Lookup(K key);
    IMap<K, V> Add(K key, V value);
    IMap<K, V> Remove(K key);
    IEnumerable<K> Keys { get; }
    IEnumerable<V> Values { get; }
    IEnumerable<KeyValuePair<K,V>> Pairs { get; }
}

Now, we could easily implement this interface by, say, inventing a class which holds on to an immutable stack of pairs. When you want to add a pair, just push it on to the stack. To find a pair, just enumerate the stack, comparing each key as you go. To remove an item, pop every item off the old stack and onto a new stack, skipping the item to remove.

That would fulfill the contract, but the obvious problem with this approach is that adding is cheap, but removal and searching become O(n) for a dictionary containing n items. A stack-based dictionary with ten thousand items takes between 1 and 10000 comparisons to find an item (and always takes the full 10000 if the item is not found.)

You could optimize this somewhat by keeping the stack sorted... except, oh, that then makes insertion O(n) as well, and lookups really only get faster in the case where you are looking up something that isn't there that happens to come early in the list. When you get to something bigger than it, you can fail then. But lookups for things at the end of the list are still potentially slow.

We can do a lot better than that if we organize the pairs into a binary tree sorted so that every keys in the left subtree is not greater than every key in the right subtree. Such a tree is (for obvious reasons) called a binary search tree. We can define a new contract for it which says that this has the shape of a binary tree of values, but also may be treated as a map:

public interface IBinarySearchTree<K, V> :
    IBinaryTree<V>,
    IMap<K, V>
    where K : IComparable<K>
{
    K Key { get; }
    new IBinarySearchTree<K, V> Left { get; }
    new IBinarySearchTree<K, V> Right { get; }
    new IBinarySearchTree<K, V> Add(K key, V value);
    new IBinarySearchTree<K, V> Remove(K key);
    IBinarySearchTree<K, V> Search(K key);
}

But how on earth are we going to insert a node into a binary search tree? Since an immutable binary tree has got to be built "from the leaves", aren't we going to have to rebuild the entire tree every time we insert a node? (Without loss of generality, assume that the item is not in the tree already. We'll discuss what to do in that situation later.)

Let's think about it. If the tree is empty, then insertion is trivial -- you just return a new single-node tree.

If the tree is not empty, then either the new element fits in the right subtree or the left subtree. Suppose it's the right. Then you do not have to rebuild any of the left subtree, only the root node and the right subtree. That eliminates half of the possible work right there! But that then is true of the right subtree as well -- either it is trivial, or half of its work will be eliminated immediately. We keep on eliminating half the work every time, and 1/2 + 1/4 + 1/8 + ... gets pretty darn close to "all the work".

Awesome. Or, perhaps not.

Remember, it is perfectly legal to have a binary search tree where every left subtree is empty. You just have a long, long train of trees down the right hand side. Essentially what you have is the sorted stack that we had before. Inserting a new item at the bottom would require building the whole tree again. Eliminating the left hand side only eliminates half the work if half the nodes are on the left hand side!

But suppose we had a way to guarantee that these sorts of pathological situations never happened. What then would the cost of inserting an item into a binary tree be? Clearly the cost of inserting the node is the cost of generating O(h) new tree nodes, where h is the depth at which the new node is inserted. Therefore, all we need to do in order to keep insertion relatively cheap is to keep the tree height as small as possible everywhere. Such a tree is called a balanced binary tree, and there are lots of ways to balance a binary tree. You can balance a tree by guaranteeing that the left side has some fraction of the total nodes (a "weight balanced tree") or that the height of the left side is less than some multiplier of the height of the right side, or that the left and right heights differ by less than some constant (a height balanced tree).

I don't want this to turn into a tutorial on tree balancing; there are plenty of web sites out there that can discuss the intricacies of tree rotations for height balancing. So tell you what, anyone who is interested, read the wikipedia page on AVL tree balancing, and next time on FAIC I'll post an implementation of an immutable height-balanced AVL tree which implements the contract above.

(This is pretty straightforward stuff; the immutable queue was slightly mind-blowing to a number of readers, but immutable balanced binary trees are just straightforward busywork. After we get through that I'll show you guys how to make a combination-stack-and-queue that can be efficiently read from both ends, and is immutable. It uses the generic type system in an interesting way to make it all work efficiently.)

Immutability in C# Part Seven: More on Binary Trees

Eric Lippert — Wed, 19 Dec 2007 23:01:00 GMT

Lots of good comments on my previous post. To briefly follow up:

One of the downsides of immutable tree implementations is that usually the tree must be built from the leaves up, which is not always convenient. We'll look at implementations which hide this fact from the user in future posts.

One smartypants pointed out that sure, this tree can have cycles -- if you have another binary tree implementation that has cycles and then paste such a cyclic tree in. True; what I meant was that using only the stated implementation, it's impossible to create a tree with cycles. If you are hell bent on making a bad tree, you can surely do that. Again, in a future post we'll have a tree which is built up by calling tree "mutating" operations, and those really will be guaranteed acyclic even in a world where there are other bad implementations hanging around.

And finally, one reader correctly identified the problem with my recursive implementation of in-order iteration. A tree of maximum height h ends up allocating O(h) iterator objects. The recursive calls get O(h) deep. If h is very large, this could blow the call stack. And if the tree has n nodes, each with an average height of O(h), then iterating each node will require O(h) recursive calls apiece. Therefore the total time cost in calls for iterating the entire tree is O(n h).

In a binary tree with n nodes, h is always between log n and n, so that means that this iterator has best case asymptotic performance of O(n log n) and worst case of O(n²) in time, and uses between O(log n) and O(n) in both call stack and heap space. That's pretty lousy.

A better implementation would be to write a tree traversal algorithm which does not use recursion. Use an explicit stack rather than the call stack:

public static IEnumerable<T> InOrder<T>(this IBinaryTree<T> tree)
{
    IStack<IBinaryTree<T>> stack = Stack<IBinaryTree<T>>.Empty;
    for (IBinaryTree<T> current = tree; !current.IsEmpty || !stack.IsEmpty; current = current.Right)
    {
        while (!current.IsEmpty)
        {
            stack = stack.Push(current);
            current = current.Left;
        }
        current = stack.Peek();
        stack = stack.Pop();
        yield return current.Value;
    }
}

This consumes O(n) time, O(h) heap space, and O(1) call stack space. It is also painful to read and analyze compared to the slow naive implementation.

I would very much like to add new syntax to a hypothetical future version of C# which would be a syntactic sugar for the code above. If, hypothetically, we were prioritizing features for unannounced future versions of C#, unfortunately that one would not be our highest priority, so I wouldn't expect it any time soon. My colleague Wes has written a great article that goes into more details about the problem above and ways we might solve it in the future, so check that out if you want the details.

I have the code for the next few blog posts written, but odds are good that I'm not going to have time to write the surrounding text until after the festive holiday season is over. I hope you have a wonderful time during the remainder of 2007 and we'll see you bright and early in 2008 for more fabulous adventures!

Immutability in C# Part Six: A Simple Binary Tree

Eric Lippert — Tue, 18 Dec 2007 22:30:00 GMT

OK, we've gotten pretty good at this by now. A straightforward implementation of an immutable generic binary tree requires little comment on the basics. A binary tree is either empty, or a value, left tree and right tree:

public interface IBinaryTree<V>
{
    bool IsEmpty { get; }
    V Value { get; }
    IBinaryTree<V> Left { get; }
    IBinaryTree<V> Right { get; }
}

And our implementation has, as always, a singleton empty tree. One minor point is that unlike previous data structures, where the structure itself knew how to build more of the same, this one does not. There's no obvious "insert" operation in a binary tree, so we'll just leave it up to the implementer. (A binary tree is essentially defined by its shape, not by the operations one can perform on it.) In this case, we'll just make a constructor:

public sealed class BinaryTree<V> : IBinaryTree<V>
{
    private sealed class EmptyBinaryTree : IBinaryTree<V>
    {
        public bool IsEmpty { get { return true; } }
        public IBinaryTree<V> Left { get { throw new Exception("Empty tree"); } }
        public IBinaryTree<V> Right { get { throw new Exception("Empty tree"); } }
        public V Value { get { throw new Exception("Empty tree"); } }
    }
    private static readonly EmptyBinaryTree empty = new EmptyBinaryTree();
    public static IBinaryTree<V> Empty { get { return empty; } }

    private readonly V value;
    private readonly IBinaryTree<V> left;
    private readonly IBinaryTree<V> right;

    public bool IsEmpty { get { return false; } }
    public V Value { get { return value; } }
    public IBinaryTree<V> Left { get { return left; } }
    public IBinaryTree<V> Right { get { return right; } }
    public BinaryTree(V value, IBinaryTree<V> left, IBinaryTree<V> right)
    {
        this.value = value;
        this.left = left ?? Empty;
        this.right = right ?? Empty;
    }
}

Note that another nice feature of immutable data structures is that it is impossible to accidentally (or deliberately!) create a tree which contains a cycle. In a mutable tree you could do something stupid like making the root a child of one of the leaves (or, for that matter, a child of itself!) Then you have to either live with the possibility that a user will create a cyclic "tree" and hope for the best (or, in other words, crash and/or hang) or you have to build yourself a cycle detector. Cycle detectors have potentially serious performance implications. It may be best to simply prevent the situation from arising in the first place.

A feature conspicuous by its absence from the above implementation is enumeration. Unlike a stack or a queue, there's no obvious best order in which to enumerate the members a binary tree. First off, there's the question of whether to enumerate in depth-first or breadth-first order, and then the matter of whether the parent comes before, after or between the children.

We could create extension methods which did various different enumerations on a binary tree. Here is a bad implementation of an "in order" traversal (depth first, parent comes between the children -- so-called because in a binary search tree, this enumerates the nodes in sorted order.)

public static IEnumerable<V> InOrder<V>(this IBinaryTree<V> tree)
{
    // BAD IMPLEMENTATION, DO NOT DO THIS
    if (tree.IsEmpty) yield break;
    foreach(V v in tree.Left.InOrder()) yield return v;
    yield return tree.Value;
    foreach(V v in tree.Right.InOrder()) yield return v;
}

This seems perfectly straightforward and sensible, but this implementation is actually quite flawed. We can do a lot better. What is wrong with this implementation of enumeration, and how would you fix it? (Hint: start your analysis with the assumption that the tree has maximum height h and total number of nodes n.)

Next time: a better implementation of traversal, and then after that, some more complex kinds of immutable binary tree.

Immutability in C# Part Five: LOLZ!

Eric Lippert — Thu, 13 Dec 2007 22:19:39 GMT

My sadly soon-to-be-erstwhile coworker Cyrus made me a lolgeek shirt to go with this series of blog articles:

Cyrus, needless to say, is a big goof. Thanks, dude!