Which memory model?

pedramr — Fri, 28 Dec 2007 18:24:00 GMT

In his blog, Eric Eilebrecht explains why when writing multithreaded applications today we should stick to the weak ECMA memory model instead of CLR’s much stronger memory model.

In principal, I have no issue with using a weaker model than the CLR memory model but my main concern is that “at what cost are we prepared to endorse a weaker model?”

Writing correct lock-free or low-lock data structures can be challenging to say the least, even when relying on a strong memory model such as CLR’s. The complexity could increase a few fold when targeting a weaker memory model.

Consciously or more often instinctively we write code based on a set of assumptions (requirements, APIs, memory models and hardware architecture). These assumptions however do change. Not always these assumptions are documented or easily transferable to other developers.

Imagine the following simplified implementation of a lock-free stack:

public class LockFreeStack<T> where T : class

{

private LinkedListNode<T> _head;

public void Push(T item)

{

var newNode = new LinkedListNode<T>();

newNode.Item = item;

{

newNode.NextNode = _head;

//_head = newNode;

} while (!SyncHelper.CompaneAndSwape<LinkedListNode<T>>(

ref _head, newNode.NextNode, newNode));

}

public T Pop()

{

LinkedListNode<T> node;

{

node = _head;

if (node == null)

return default(T);

//_head = node.NextNode;

} while (!SyncHelper.CompaneAndSwape<LinkedListNode<T>>(

ref _head, node, node.NextNode));

return node.Item;

}

It makes a number of assumptions. For instance, it assumes that you are not interested in knowing the length of the stack. Adding a Count property to this class would require a massive rethink of its lock-free algorithm (this may add a possible second write to a shared memory at the time of Pop and Push). It might even require a full rewrite of the class. Therefore when assumptions change, it is not always trivial to refactor and repurpose the code. Utmost level of care must be taken to ensure that issues and bugs are avoided.

In my experience, it is often simpler to rewrite the code and use as much testing as possible on multiprocessor machines.

The second reason that writing applications for a weaker memory model today is not advisable is because we don’t have a runtime to test our applications. Many concurrency issues and bugs are discovered at the time of testing.

CLR 2.0 memory model

pedramr — Fri, 28 Dec 2007 16:10:00 GMT

Memory is usually a shared resource on multithreaded systems therefore access to it must be regulated and fully specified. This specification is often called a “Memory Model”.

Optimisations performed by compilers and the emergence of multi-core processors are some of the factors testing the agility of today’s memory models. The simplest such model is the Sequential Consistency model in which all reads and writes from all threads are effectively queued and sequentially performed. Although it is simple but it hampers scalability and performance. Read this great MSDN magazine article on memory models.

I don’t want to bore you by revisiting some of the basic concepts of multiprocessor machines. However let me also remind you that processor buffers and caches heavily restrict memory models. Caches effectively can move reads and writes. For instance the value for a memory location present in a processor cache can be a copy of an earlier value for that location in the main memory, and reading that value effectively brings the read earlier in time.

Here I have tried to summarise the memory model rules for CLR 2.0 (.NET 2.0, 3.0 and 3.5) as I know them (Joe also covers the same topic here):

1. Reads and writes cannot be introduced

2. Reads cannot move before entering a lock and writes cannot move after exiting a lock

3. No reads or writes can move before a volatile read or after a volatile write

4. All writes have the effect of volatile write

5. Duplicate adjacent reads/writes from/to the same location from the same thread can be reduced to one read/write

6. Reads cannot move before a write to the same location from the same thread

7. Reads and writes cannot move after a full-barrier (such as Thread.MemoryBarrier).

So what happened to the following rules?

- “Writes cannot move past other writes from the same thread” is prevented by rule 4

- “Data dependence among reads and writes is never violated” is enforced by rules 4 and 6

- “All writes have release semantics” is enforced by rule 4

- “Reads or writes from a given thread to a given location cannot pass a write from the same thread to the same location” is enforced by rules 4

Now that we know the rules, wouldn’t it be nice to have a tool that could automatically check our .NET code to ensure that any valid reordering still keeps the semantic intact and to suggest possible additions of barriers and volatile operations? Well there is such a tool but as far as I understand, this one heavily relies on the weaker memory model defined by EMMA CLI spec and it is not complete. Also if you are interested in knowing more about memory model verification, take a look at this.

The other interesting aspect of CLR 2.0 is that it guarantees that: read and write access to properly aligned (default behaviour) memory location no larger than the native Integer (32 bit processor = 4-byte aligned, 64 bit processor = 8-byte aligned) is atomic. This is defined in section 12.6 of the ECMA CLI Spec. Therefore you can be certain that at least 4-byte assignments in CLR 2.0 are atomic.

Why do you need to know this? Well, writing good concurrent code particularly lock-free or low-lock data structures rely heavily on developers having a deep understanding of the memory model. Also, memory models can change so if you stick to a weaker memory model today such as the ECMA memory model it is more likely that your application stays forward-compatible.

Pedram Rezaei's Ramblings : Memory Model

Which memory model?

CLR 2.0 memory model