Pedram Rezaei's Ramblings : CLR

Which memory model?

pedramr — Fri, 28 Dec 2007 18:24:00 GMT

In his blog, Eric Eilebrecht explains why when writing multithreaded applications today we should stick to the weak ECMA memory model instead of CLR’s much stronger memory model.

In principal, I have no issue with using a weaker model than the CLR memory model but my main concern is that “at what cost are we prepared to endorse a weaker model?”

Writing correct lock-free or low-lock data structures can be challenging to say the least, even when relying on a strong memory model such as CLR’s. The complexity could increase a few fold when targeting a weaker memory model.

Consciously or more often instinctively we write code based on a set of assumptions (requirements, APIs, memory models and hardware architecture). These assumptions however do change. Not always these assumptions are documented or easily transferable to other developers.

Imagine the following simplified implementation of a lock-free stack:

public class LockFreeStack<T> where T : class

{

private LinkedListNode<T> _head;

public void Push(T item)

{

var newNode = new LinkedListNode<T>();

newNode.Item = item;

{

newNode.NextNode = _head;

//_head = newNode;

} while (!SyncHelper.CompaneAndSwape<LinkedListNode<T>>(

ref _head, newNode.NextNode, newNode));

}

public T Pop()

{

LinkedListNode<T> node;

{

node = _head;

if (node == null)

return default(T);

//_head = node.NextNode;

} while (!SyncHelper.CompaneAndSwape<LinkedListNode<T>>(

ref _head, node, node.NextNode));

return node.Item;

}

It makes a number of assumptions. For instance, it assumes that you are not interested in knowing the length of the stack. Adding a Count property to this class would require a massive rethink of its lock-free algorithm (this may add a possible second write to a shared memory at the time of Pop and Push). It might even require a full rewrite of the class. Therefore when assumptions change, it is not always trivial to refactor and repurpose the code. Utmost level of care must be taken to ensure that issues and bugs are avoided.

In my experience, it is often simpler to rewrite the code and use as much testing as possible on multiprocessor machines.

The second reason that writing applications for a weaker memory model today is not advisable is because we don’t have a runtime to test our applications. Many concurrency issues and bugs are discovered at the time of testing.

CLR 2.0 memory model

pedramr — Fri, 28 Dec 2007 16:10:00 GMT

Memory is usually a shared resource on multithreaded systems therefore access to it must be regulated and fully specified. This specification is often called a “Memory Model”.

Optimisations performed by compilers and the emergence of multi-core processors are some of the factors testing the agility of today’s memory models. The simplest such model is the Sequential Consistency model in which all reads and writes from all threads are effectively queued and sequentially performed. Although it is simple but it hampers scalability and performance. Read this great MSDN magazine article on memory models.

I don’t want to bore you by revisiting some of the basic concepts of multiprocessor machines. However let me also remind you that processor buffers and caches heavily restrict memory models. Caches effectively can move reads and writes. For instance the value for a memory location present in a processor cache can be a copy of an earlier value for that location in the main memory, and reading that value effectively brings the read earlier in time.

Here I have tried to summarise the memory model rules for CLR 2.0 (.NET 2.0, 3.0 and 3.5) as I know them (Joe also covers the same topic here):

1. Reads and writes cannot be introduced

2. Reads cannot move before entering a lock and writes cannot move after exiting a lock

3. No reads or writes can move before a volatile read or after a volatile write

4. All writes have the effect of volatile write

5. Duplicate adjacent reads/writes from/to the same location from the same thread can be reduced to one read/write

6. Reads cannot move before a write to the same location from the same thread

7. Reads and writes cannot move after a full-barrier (such as Thread.MemoryBarrier).

So what happened to the following rules?

- “Writes cannot move past other writes from the same thread” is prevented by rule 4

- “Data dependence among reads and writes is never violated” is enforced by rules 4 and 6

- “All writes have release semantics” is enforced by rule 4

- “Reads or writes from a given thread to a given location cannot pass a write from the same thread to the same location” is enforced by rules 4

Now that we know the rules, wouldn’t it be nice to have a tool that could automatically check our .NET code to ensure that any valid reordering still keeps the semantic intact and to suggest possible additions of barriers and volatile operations? Well there is such a tool but as far as I understand, this one heavily relies on the weaker memory model defined by EMMA CLI spec and it is not complete. Also if you are interested in knowing more about memory model verification, take a look at this.

The other interesting aspect of CLR 2.0 is that it guarantees that: read and write access to properly aligned (default behaviour) memory location no larger than the native Integer (32 bit processor = 4-byte aligned, 64 bit processor = 8-byte aligned) is atomic. This is defined in section 12.6 of the ECMA CLI Spec. Therefore you can be certain that at least 4-byte assignments in CLR 2.0 are atomic.

Why do you need to know this? Well, writing good concurrent code particularly lock-free or low-lock data structures rely heavily on developers having a deep understanding of the memory model. Also, memory models can change so if you stick to a weaker memory model today such as the ECMA memory model it is more likely that your application stays forward-compatible.

Dedicated thread or a Threadpool thread?

pedramr — Sun, 05 Aug 2007 17:48:00 GMT

Each .NET CLR based process is given a pool of threads. The size of the thread pool is configurable however by default the maximum number of threads created by the thread pool is set to 1000 IO threads in total, and 25 worker threads per logical or physical processor. In other words, when using a dual core processor, you will have the maximum value set to 50 worker threads.

These threads are managed in a very efficient and intelligent manner by the runtime. For instance, the actual size of the thread pool varies based on how the application is consuming those threads. It is possible to set the minimum number of threads for the thread pool. Once that many threads are created, they are kept alive for the lifetime of the process. However if more threads are required, the thread pool creates new threads. Once those threads finish executing their activities, they are then returned to the thread pool. It is important to emphasise that those threads are not destroyed. They will self destroy after approximately 2 minutes of inactivity. Therefore, the usage of system resources is constantly being adjusted automatically by the thread pool itself.

Once the minimum number of threads is reached, the thread pool aims to limit the number of threads being created to one per 500 milliseconds. This is an intelligent mechanism, avoiding the expensive cost of creating a new thread when multiple thread pool threads may be released within that time period.

The combination of the above approaches makes .NET’s thread pool a very intelligent thread pool which uses as little resources as possible to give maximum performance. Therefore in most scenarios it is recommended to use this thread pool for performing asynchronous operations.

.NET’s thread pool has some shortcomings which can affect the choice between using a dedicated thread instead of a thread from the thread pool. It is usually said that a dedicated thread is favourable in the following scenarios:

- When a foreground thread is required: All thread pool threads are initialised as background threads.

- When it is required to have a thread with a particular priority.

- When a thread is required to be aborted prematurely

- When a thread must be placed in a single-threaded apartment (STA): All thread pool threads are set in the MTA apartment by default

- For long running tasks when the thread pool thread is often blocked for long periods (This may starve other parts of the application which rely on threads from the thread pool)

The thread pool is shared by all AppDomains in a process. Consider this if you have more than one AppDomain in your process.

Non-CLS Exceptions

pedramr — Sun, 07 Jan 2007 16:56:00 GMT

Many developers are not aware that according to CLR it is perfectly OK to throw exceptions which are NOT derived from System.Exception: Exceptions such as DateTime, Int64 and String. Although languages such as C# do not allows programmers to throw these types of exceptions it should still be possible to catch non-CLS compliant exceptions using a catch block.

Pre to CLR version 2.0, the only way to catch non-CLS compliant exceptions was to introduce a catch block with no Exception section:

try

{

// Some code here which might

// throw non-CLS compilant exception

}

catch (Exception ex)

{

// Here we will be catching any

// CLS compliant type of exceptions

}

catch

{

// Here we will be catching any

// non-CLS compliant exceptions...

// Note that, this block on its own

// is capable of catching both compliant

// and non-compinat exceptions

}

In version 2.0 of CLR, any non-CLS compliant exception is automatically wrapped up by System.Runtime.CompilerServices.RuntimeWrappedException class which inherits from the Exception class. This class has a property which is set to the actual object which was thrown.

It is still possible to tell CLR not to wrap up the non-CLS compliant exceptions with the RuntimeWrappedException class. This is done using an attribute which affects the whole assembly:

[assembly: RuntimeCompatibility(WrapNonExceptionThrows = false)]