Pedram Rezaei's Ramblings : Concurrency

Introducing Parallel Extensions to the .NET Framework

pedramr — Wed, 06 Aug 2008 23:35:00 GMT

There is no escaping from concurrency challenges... or is there?

(A slightly modified version of this article was published in the August 2008 edition of the MSDN Flash newsletter)

Dual, quad, and eight-core processors are becoming the norm. Is your application capable of utilising all available processors? In order to achieve this level of utilisation on an n-processor machine, an application ideally needs at least n threads concurrently performing operations.

Although writing multi-threaded applications has become simpler over the years, many still find it challenging. Also many of the constructs such as the .NET ThreadPool have deficiencies that make them less suitable as the manycore shift begins to accelerate.

Over the past few years, Microsoft has been hard at work developing a new set of parallel extensions for .NET, aptly named Parallel Extensions to the .NET Framework. One of the additions is a set of lightweight and scalable thread-safe data structures and synchronization primitives such as a concurrent dictionary and a spin lock.

It also includes an implementation of LINQ-to-Objects that automatically executes queries in parallel, scaling to utilise most or all of the available processors without developers explicitly managing the distribution of work across all processors. This technology is called Parallel LINQ or PLINQ and the query syntax is almost identical to that of LINQ-to-Objects.

var q = from c in customers.AsParallel()

join r in regions on c.RegionID equals r.RegionID

where c.City == "London" && r.Name == "Kingston"

select c;

When a repetitive computation can be parallelised, the Parallel Extensions to the .NET Framework offers a few interesting data-oriented operations such as For and ForEach that execute a loop in which iterations may run in parallel on parallel hardware. For example the following classic foreach loop can easily be converted to a parallel foreach:

Sequential execution

foreach (var c in formulae)

Calculate(f);

Possible parallel execution

Parallel.ForEach(formulae, f => Calculate(f));

The actual parallelisation of actions is managed by the Task Parallel Library (TPL) that provides an abstraction layer on top of raw threads. TPL uses the notion of Task that is the smallest unit of work and could potentially be executed by any thread owned by TPL. Therefore correctly identifying tasks that could be executed in parallel is crucial. It is then the job of the scheduler component of TPL to decide whether those tasks should execute sequentially or in parallel. This decision is usually made based on available system resources and possible user preferences.

// Create a task to call Calculate(object o) and schedule it for execution

Task t = Task.Create(Calculate);

// Other activities...

// Wait for this task to complete

t.Wait();

Developing concurrent applications requires a shift in mindset. For example, only a single exception can be thrown at any one time when sequentially executing a for loop. However executing the same loop in parallel may warrant the need for dealing with multiple concurrent exceptions. There are also a few parallelism blockers that should be avoided if you are planning to benefit from Parallel Extensions to the .NET Framework when it is released.

There is a lot of promise in Parallel Extensions. If you want to find out more, I’d encourage you to download the June 2008 Community Technology Preview build.

Coordination Data Structures – WriteOnce

pedramr — Tue, 03 Jun 2008 01:29:00 GMT

This is an article in a series of blog entries describing a set of new Coordination Data Structures (CDS) introduced in the June 2008 CTP of the Parallel Extensions for .NET Framework.

In C#, when a field declaration includes a readonly modifier, assignments to the fields introduced by the declaration can only occur as part of the declaration or in a constructor in the same class:

readonly Company _company = new Company();

The assignment window is limited to the constructor or the declaration. In many applications however, it is desirable to have the readonly behaviour (set once and read many times) without this limitation. As you might have guessed, the Parallel Extensions now provides a type (System.Threading.WriteOnce<T>) for this exact purpose.

WriteOnce<T> exposes a Value property that can only be set once. This is obviously done in a thread-safe manner:

// this is a struct, so it will not require an explicit construction

WriteOnce<Guid> wo;

wo.Value = Guid.NewGuid();

As you would expect, attempting to reassign the Value property will throw an exception (“The variable is already initialized”):

WriteOnce<T> is a thread-safe type so if trying to set the Value property concurrently, one of the setters will throw the above exception. In order to avoid this, use the TrySetValue method instead:

WriteOnce<Guid> wo;

if (wo.TrySetValue(Guid.Empty))

{

// the value was set successfuly

}

Attempting to access the Value of a WriteOnce<T> before it has been set invalidates the WriteOnce<T> instance for all future access. The instance will be corrupt and can no longer be accessed for future gets and set.

It corrupts the instance because there is a race condition that leads to a read before the write. Proper synchronization should be used to ensure that this race cannot happen. Therefore always ensure that the value is set before accessing it. When you are not certain if the Value is set, you can access it through the TryGetValue. This will return default(T) if the value is not set and will not corrupt the state of the instance for future use:

Guid val;

if (wo.TryGetValue(out val))

{

// the value has been set previously

}

Please note that WriteOnce<T> is a value type (struct), and as such, you need to be careful about access patterns. If you accidentally make a copy of the struct, you’ll be copying by value meaning that you will be using a replica rather than the original. For example in the code below, both assignments to what might appear as the same WriteOnce instance are allowed. The reality is however that wo1 and wo2 are two different instances:

WriteOnce<Guid> wo1;

Task.Create(

s =>

{

WriteOnce<Guid> wo2 = (WriteOnce<Guid>)s;

// assignment (1)

wo2.Value = Guid.NewGuid();

Console.WriteLine(wo2.Value);

}

, wo1);

// assignment (2)

wo1.Value = Guid.NewGuid();

In the example below however, it will not be possible to assign to wo2.Value (assignment 2 will fail). Although wo2 is a replica copy of wo1, internally it is referencing the same value object assigned in (1) which is not cloned:

WriteOnce<Guid> wo1;

// assignment (1)

wo1.Value = Guid.NewGuid();

Task.Create(

s =>

{

WriteOnce<Guid> wo2 = (WriteOnce<Guid>)s;

// assignment (2)

wo2.Value = Guid.NewGuid();

}

, wo1);

(Thanks to Joe Duffy, Ed Essey and Stephen Toub for their inputs and support)

Coordination Data Structures – SpinLock

pedramr — Mon, 02 Jun 2008 20:36:00 GMT

This is an article in a series of blog entries describing a set of new Coordination Data Structures (CDS) introduced in the June 2008 CTP of the Parallel Extensions for .NET Framework.

Waiting on locks usually result in a thread context switch and associated kernel transition which at times can be considered costly. On a multi-processor machine, often it might be more efficient to busy wait for a short period of time instead of paying the cost of performing an expensive context switch and a possible transition to kernel mode.

One approach to busy waiting is to wait in a loop repeatedly checking until the lock becomes available. These types of locks are called spin-locks.

System.Threading.SpinLock provides a mutual exclusion lock primitive where a thread trying to acquire the lock waits in a loop repeatedly checking until the lock becomes available.

When acquiring a spin-lock, it uses the CompareExchange interlocked operation to ensure that the owner of the lock is assigned in a thread-safe manner (this implementation detail is subject to change):

if(Interlocked.CompareExchange(ref _owner, newOwner, owner) == owner && ...)

Failing to become the owner of the lock, other threads will wait using a SpinWait until the lock is released (pseudo code):

public void Enter()

{
SpinWait sw = new SpinWait();
...

while (true)

{

if (!IsOwnerSet())

{

// set the owner - atomic

if (SetOwner())

{

// this thread is the owner so return

return;

}

// spin wait once

sw.SpinOnce();

}

...

}

One point to mention here is the effect of a spin-lock on a single-processor machine. On these systems, the spin-lock will hamper the performance by adding an unnecessary delay into the whole process. It will not result in a deadlock because after executing its quantum, the spinning thread is context-switched or pre-empted by another thread, however the time spent looping has not been productive.

Due to this behaviour, the System.Threading.SpinLock primitive ensures we always yield (switch threads) on single-processor machines instead of using busy waits.

Unlike Monitor, SpinLock publicly offers a set of reliable enter methods for acquiring a lock (i.e. ReliableEnter and TryReliableEnter). You should probably never use the unreliable enter methods because in the case of any asynchronous exceptions such as ThreadAbortException and OutOfMemoryException, you may end up with an orphaned lock that is never released. This is particularly true in ASP.NET since it uses aborts very aggressively.

As a side-note, you may wonder how we achieve this reliability. As it happens, the default CLR host postpones async exceptions inside a finally block until the end of the block. Therefore any part of the code that could result in an orphaned lock can be included inside a finally block:

try

{

}

finally

{

// take the lock in here

// ...

}
Please note that another host could choose to escalate thread aborts to rude thread aborts, which can interrupt finally blocks.

Closer investigating of the SpinLock.Exit method reveals an overload that takes a Boolean. If this Boolean is set to true, SpinLock flushes the write buffers associated with the lock in order to ensure that all processors are immediately made aware that the lock is now available. This is more expensive but will prevent a situation where one processor is given an unfair advantage to reacquire the lock:

SpinLock sharedLock = new SpinLock();

while (true)

{

sharedLock.Enter();

sharedLock.Exit();

// very little between Exit and retry Enter;

// possibly no other processor would see the lock available!

}

There are a few more things to remember when using the SpinLock:

1- When using the default constructor, the thread that owns the lock can enter the same lock and effectively cause a deadlock:

SpinLock l = new SpinLock();
l.Enter(); // non-blocking call

l.Enter(); // blocking call

2- When using the default constructor, any thread can release the lock (This is not a recommended practice and should be avoided in normal circumstances):

SpinLock l = new SpinLock();

l.Enter();

ThreadPool.QueueUserWorkItem(delegate

{

l.Exit();

});

3- In the current implementation, you can change the above behaviour. Simply set the isThreadOwnerTrackingEnabled parameter to true when creating a new instance of SpinLock:

Setting isThreadOwnerTrackingEnabled to true changes the behaviour of the lock by:

- Prohibiting the owner thread from trying to enter the same lock – throws a LockRecursionException; and,

- Ensuring the Exit method is only called by the owner thread – a SynchronizationLockException is thrown otherwise

Something else that’s worth mentioning is that SpinLock is a struct, and as such, you need to be careful about access patterns. If you accidentally make a copy of the struct, you’ll be copying by value meaning that you will be using a replica rather than the original lock. For example in the code below, both the main thread and the ThreadPool thread would successfully enter the lock. The morale of the story is that ideally these are not passed around, and instead they are used as local variables or member fields:

SpinLock sl = new SpinLock();

ThreadPool.QueueUserWorkItem(state =>

{

SpinLock theLock = (SpinLock)state;

theLock.Enter();

}, sl);

sl.Enter();

(Thanks to Stephen Toub, Joe Duffy and Ed Essey for their input and support)

Coordination Data Structures – LazyInit

pedramr — Mon, 02 Jun 2008 20:34:00 GMT

This is an article in a series of blog entries describing a set of new Coordination Data Structures (CDS) introduced in the June 2008 CTP of the Parallel Extensions for .NET Framework.

LazyInit<T> provides support for several common patterns of lazy initialization. In here we will explore some of those patterns, but first a point or two regarding lazy initialization:

- Lazy initialization is often used when the initialization process is an expensive activity which is not always required. With this approach, the execution of the initialize procedure is deferred to immediately before when it is required.

- In order to improve the start-up performance of an application, one may decide to defer some of the initialization activities.

The first pattern covered by LazyInit<T> is the optimistic lazy instantiation pattern. In this pattern, multiple instances of T may be created but only one of those instances is published for all threads to access. In pseudo code, the implementation of such structure may look similar to this:

public class Singleton

{

static volatile Singleton _instance;

private Singleton()

{

// stuff ...

}

public static Singleton Instance

{

get

{

if (_instance == null)

{

Singleton local = new Singleton();

if (Interlocked.CompareExchange<Singleton>(

ref _instance, local, null) != null)

{

IDisposable disposable = local as IDisposable;

if (disposable != null)

{

disposable.Dispose();

}

return _instance;

}

As you can see, unwanted instances are disposed automatically. This optimistic approach to concurrency removes the need for a full lock, but has the potential of multiple instantiations.

The LazyInit<T> struct offers a similar functionality. Simply create an instance of the LazyInit<T> using its default constructor:

LazyInit<Company> s = new LazyInit<Company>();

// do some stuff here

// ...

Company c = s.Value;

The above example uses reflection to create an instance of Company when the Value property is first accessed. Obviously Company must have a public parameter-less constructor for this to succeed.

It is worth emphasising that the default behaviour of LazyInit does not prevent the instantiation of more than one Company object by multiple threads concurrently accessing the Value property when the value is not yet set. However it guarantees that only one of those instances is published for all threads to access.

Another pattern offered by LazyInit<T> is the pessimistic lazy initialization pattern. This is very similar to the well-known double-check locking pattern and in pseudo code looks like this:

public class Singleton

{

static volatile Singleton _instance;

static object s_lock = new object();

private Singleton()

{

// stuff ...

}

public static Singleton Instance

{

get

{

// double-check locking pattern

if (_instance == null)

{

lock (s_lock)

{

if (_instance == null)

{

_instance = new Singleton();

return _instance;

}

return _instance;

}

As you can see, no more than a single instance of Singleton is ever instantiated. This is achieved at the cost of acquiring a lock.

LazyInit<T> also provides a similar functionally. Here are some variations of this pattern implemented using LazyInit<T>:

Pessimistic Lazy Instantiation:

LazyInit<Company> s = new LazyInit<Company>(

() => new Company(), LazyInitMode.EnsureSingleExecution);

// do some stuff here

// ...

Company c = s.Value;

Pessimistic Lazy Initialization:

In the example below, a database connection is opened only when needed. The connection is then kept open and no other connection is created for this instance of DataAccess:

class DataAccess

{

LazyInit<SqlConnection> cnn;

public DataAccess()

{

cnn = new LazyInit<SqlConnection>(

delegate

{

var c = new SqlConnection(

"Server=.;Database=Northwind;Integrated Security=true");

c.Open();

return c;

}, LazyInitMode.EnsureSingleExecution);

}

public int GetProductCount()

{

// perform a db query: this is when we actually

// create and open a connection to the database

using (var cmd = new SqlCommand(

"select count(*) from products", cnn.Value))

{

return (int)cmd.ExecuteScalar();

}

Another pattern offered by LazyInit<T> allows an initialization per thread such that each thread will get its own value. In this pattern, the value from the initialization is stored in the thread local storage (TLS):

LazyInit<Company> l = new LazyInit<Company>(

() => new Company { ID = Guid.NewGuid() },

LazyInitMode.ThreadLocal);

// l.Value access from the main thread

Console.WriteLine(l.Value.ID);

ThreadPool.QueueUserWorkItem(delegate

{

// l.Value access from a ThreadPool thread

Console.WriteLine(l.Value.ID);

});

In the example above, the main thread and the ThreadPool thread each create a different instance of Company with different IDs:

Now that you have seen some different lazy initialization patterns, it is time to fully introduce the LazyInitMode enum that can be specified as a parameter to the LazyInit<T> constructor:

LazyInitMode Value	Description
AllowMultipleExecution	The initialization function may be executed multiple times if multiple threads race to initialize the value, but only one value will be published for all threads to access.
EnsureSingleExecution	The initialization function will only be executed once, even if multiple threads race to initialize the value. This value will be published for all threads to access.
ThreadLocal	The initialization function will be invoked once per thread such that each thread will get its own published value.

One last point, LazyInit<T> is a value type and does not have the overhead of being a class. However you need to be careful about access patterns. If you accidentally make a copy of the struct, you’ll be copying by value meaning that you will be using a replica rather than the original.

(Thanks to Stephen Toub, Joe Duffy and Ed Essey for their input and support)

More on self-replicating tasks

pedramr — Thu, 03 Apr 2008 13:22:00 GMT

Some more stuff to remember when dealing with self-replicating tasks. (See my earlier post for an introduction to Parallel FX and self-replicating tasks):

- Self-replicating tasks should have an inter-replica communication mechanism for communicating the progress/details of the activity. This depends on what the activity is trying to achieve. See here for an example.

- Self-replicating tasks should have an inter-replica communication mechanism for communicating the completion of the overall activity.

- Only use when the cost of this communication and the management of partitions is considerably less than the potential benefit gained from parallelism

- Do not assume that the task is always replicated. It is only replicated if there are available resources. For the same reason also, do not assume that there will be a specific number of replicas.

- In some instances, the number of replicas could far exceed the number of cores in the machine.

- You may choose to use optimistic concurrency when it is possible to correctly deal with multiple executions of the same step.

- In general replicating tasks are an advances feature that can be very useful in specific scenarios. Use with caution.

(All based on the first Parallel Extensions CTP)

Digging deeper into PLINQ’s internal implementation

pedramr — Fri, 11 Jan 2008 20:21:00 GMT

PLINQ is built on top of the Task Parallel Library (TPL) and promises to revolutionise the way we write programs that can benefit from the multi-core processor era. But how does it work internally?

This article assumes that you are familiar with the basics of LINQ and have an understanding PLINQ and TPL.

In this short article, I will concentrate on the techniques used by the first CTP of PLINQ to partition work streams and associate different partitions to different threads. It is worth mentioning that by default, TPL provides one thread per processor core.

The deferred execution and lazy evaluation characteristics of LINQ allow for creation of infinite size lists as the source for a LINQ query. For instance it is possible to define an enumerator that represents the set of Natural Numbers:

class NaturalNumbersEnumerable : IEnumerable<uint>

{

public IEnumerator<uint> GetEnumerator()

{

uint i = 1;

while (true)

yield return i++;

}

IEnumerator IEnumerable.GetEnumerator()

{

return GetEnumerator();

}

(I know that it is not really a full set of natural numbers and it can only go up to uint.MaxValue before either throwing an OverflowException or starting back from 0, but it fits the purpose of this article)

The fact of the matter is that the above list never ends because the more you call MoveNext on the enumerator the more natural numbers are generated. So somehow we need to limit the execution by using functions that limit the size of the returned list. As shown below, Take can be one of those functions:

var linq =

new NaturalNumbersEnumerable()

.Take(100);

foreach (uint i in linq)

Console.WriteLine(i);

Executing the above code prints numbers from 1 to 100 in a sequential manner. With PLINQ however, it is possible to execute the same query using more than one thread. The work distribution algorithms used here are the focus of the remainder of this article.

An enumerator (stream) is partitioned into multiple smaller enumerators of type IQueryOperatorEnumerator<T>. The partitioned stream is represented by PartitionedStream<T> which exposes an array of IQueryOperatorEnumerator<T> that can be iterated concurrently.

One way of building a partitioned stream is through manual creation of partitions, effectively allowing for the introduction of custom partitioning algorithms. However the PartitionedStream<T> derived type responsible for this (ManuallyPartitionedStream<T>) is an internal class at present. If you believe that such extensibility can be useful then let the product team know through here. I personally think that it would be a valuable feature as the users often have a deep understanding of the source stream and can better decide on the size and the content of each partition resulting in an improved experience.

Internally there are at least 2 different algorithms used to create partitioned streams. One algorithm partitions the stream based on a range and the other based on a hashcode.

Range based partitioning is sensitive to the type of the source stream. If the source is an indexible data type (i.e. an array or an implementer of IList<T>) then the partitioning is done based on a unique range given to each partition. Also, the source stream is shared by all partitions and there are no range overlaps between partitions. Obviously the access to the source does not need to be synchronised, therefore the indexer of the source may need to be thread-safe.

Note that range partitioning assumes that the work required to process each element will take roughly the same time and will be largely homogenous. This might not always be true and a custom partitioning solution might perform better.

However if the source is not an IList<T>, we can no longer assume that we are dealing with a finite set of items. This in itself takes away the ability to index into the source stream. Nonetheless we still need to find a way to partition the source.

As the source can have an infinite range, we can no longer decide on the actual size of each partition. This means that each partition can grow substantially during the execution. As part of the growth phase a subset of elements from the source stream are read and stored locally in memory. When a MoveNext is called, the next item is returned from this internal buffer and if needed a new growth operation is performed synchronously.

Due to the nature of these source streams, requests for a move of the reader and the read of the current value must be synchronised using a synchronization primitive such as a lock. Therefore amazingly, you can safely assume that MoveNext() and the Current property of the source stream are not called by more than one thread at a time. Hence, in our Natural Number example, we can rely on ‘i++’ instead of Interlocked.Increment of ‘i’:

class NaturalNumbersEnumerable : IEnumerable<uint>

{

public IEnumerator<uint> GetEnumerator()

{

uint i = 1;

while (true)

{

yield return i;

Interlocked.Increment(ref i);

}

...

This chunk partitioning with self-expanding partitions has a number of implications:

- High cost of synchronization

- Memory cost of caching sections of the source stream per partition

- Reading data elements from the source stream that could never be used:

In the example below, it is realistic to expect the generation of more than 100 natural numbers even though Take(100) can only consume the first 100 numbers. In fact, on my dual core processor, it generated around 130 numbers

var linq =

new NaturalNumbersEnumerable().AsParallel()

.Take(100);

foreach (uint i in linq)

Console.Write(i);

Read the next paragraph with a pinch of salt as I cannot find signs of its use by the current CTP!

The hash based partitioning algorithm aims to divide the source stream into smaller enumerators. The content of each partition is decided based on the hashcode provided by or for each source element. If the source stream is an IList<T> then based on the hashcode value of the items in the list, they are assigned to partitions. If however the source stream is an ICollection<T> or any other IEnumerable<T>, it is first converted into an IList<T> and then divided between partitions. Please note that it would not be possible to convert NaturalNumbersEnumerable above to an IList<T>.

How to cancel a task in Parallel FX?

pedramr — Wed, 02 Jan 2008 19:52:00 GMT

Task Parallel Library (TPL) allows you to easily cancel tasks. Effectively you need to call the Cancel method on the task in question. Imagine the simple sample below:

Task task1 = Task.Create(Foo, 10000);

static void Foo(object o)

{

for (int i = 0; i < (int)o; i++)

{

// some code here

}

A task is created and Foo is automatically called. At some point if you decide to cancel the task while it is running, you can call the Cancel method on the task as shown below:

Task task = Task.Create(Foo, 10000);

task.Cancel();

When you call cancel, all you are doing is to set a Boolean field to true. You can query its value through the IsCanceled property:

task.IsCanceled

When Cancel is called, if the task has not yet prepared for execution, it will not run. However if the task has already been prepared and assigned to a thread, it will start executing. The execution is not automatically interrupted and requires manual intervention in order to cancel. Effectively you need to build the cancelation logic into your application:

static void Foo(object o)

{

for (int i = 0; i < (int)o; i++)

{

// Could it have been cancelled?

ThrowIfCurrentTaskCanceled();

// some code here

}

internal static void ThrowIfCurrentTaskCanceled()

{

Task current = Task.Current;

if (current != null && current.IsCanceled)

{

throw new TaskCanceledException(current);

}

As you can see, it is possible to access the current Task and check to see if it has already been cancelled. One thing to remember regarding the Boolean field used by the IsCanceled property is that it is a volatile field meaning that all reads are volatile reads and have acquire semantics which can be expensive.

On a slightly different note, if you are unsure about the spelling of “Cancelled” then take a look at this post.

Which memory model?

pedramr — Fri, 28 Dec 2007 18:24:00 GMT

In his blog, Eric Eilebrecht explains why when writing multithreaded applications today we should stick to the weak ECMA memory model instead of CLR’s much stronger memory model.

In principal, I have no issue with using a weaker model than the CLR memory model but my main concern is that “at what cost are we prepared to endorse a weaker model?”

Writing correct lock-free or low-lock data structures can be challenging to say the least, even when relying on a strong memory model such as CLR’s. The complexity could increase a few fold when targeting a weaker memory model.

Consciously or more often instinctively we write code based on a set of assumptions (requirements, APIs, memory models and hardware architecture). These assumptions however do change. Not always these assumptions are documented or easily transferable to other developers.

Imagine the following simplified implementation of a lock-free stack:

public class LockFreeStack<T> where T : class

{

private LinkedListNode<T> _head;

public void Push(T item)

{

var newNode = new LinkedListNode<T>();

newNode.Item = item;

{

newNode.NextNode = _head;

//_head = newNode;

} while (!SyncHelper.CompaneAndSwape<LinkedListNode<T>>(

ref _head, newNode.NextNode, newNode));

}

public T Pop()

{

LinkedListNode<T> node;

{

node = _head;

if (node == null)

return default(T);

//_head = node.NextNode;

} while (!SyncHelper.CompaneAndSwape<LinkedListNode<T>>(

ref _head, node, node.NextNode));

return node.Item;

}

It makes a number of assumptions. For instance, it assumes that you are not interested in knowing the length of the stack. Adding a Count property to this class would require a massive rethink of its lock-free algorithm (this may add a possible second write to a shared memory at the time of Pop and Push). It might even require a full rewrite of the class. Therefore when assumptions change, it is not always trivial to refactor and repurpose the code. Utmost level of care must be taken to ensure that issues and bugs are avoided.

In my experience, it is often simpler to rewrite the code and use as much testing as possible on multiprocessor machines.

The second reason that writing applications for a weaker memory model today is not advisable is because we don’t have a runtime to test our applications. Many concurrency issues and bugs are discovered at the time of testing.

CLR 2.0 memory model

pedramr — Fri, 28 Dec 2007 16:10:00 GMT

Memory is usually a shared resource on multithreaded systems therefore access to it must be regulated and fully specified. This specification is often called a “Memory Model”.

Optimisations performed by compilers and the emergence of multi-core processors are some of the factors testing the agility of today’s memory models. The simplest such model is the Sequential Consistency model in which all reads and writes from all threads are effectively queued and sequentially performed. Although it is simple but it hampers scalability and performance. Read this great MSDN magazine article on memory models.

I don’t want to bore you by revisiting some of the basic concepts of multiprocessor machines. However let me also remind you that processor buffers and caches heavily restrict memory models. Caches effectively can move reads and writes. For instance the value for a memory location present in a processor cache can be a copy of an earlier value for that location in the main memory, and reading that value effectively brings the read earlier in time.

Here I have tried to summarise the memory model rules for CLR 2.0 (.NET 2.0, 3.0 and 3.5) as I know them (Joe also covers the same topic here):

1. Reads and writes cannot be introduced

2. Reads cannot move before entering a lock and writes cannot move after exiting a lock

3. No reads or writes can move before a volatile read or after a volatile write

4. All writes have the effect of volatile write

5. Duplicate adjacent reads/writes from/to the same location from the same thread can be reduced to one read/write

6. Reads cannot move before a write to the same location from the same thread

7. Reads and writes cannot move after a full-barrier (such as Thread.MemoryBarrier).

So what happened to the following rules?

- “Writes cannot move past other writes from the same thread” is prevented by rule 4

- “Data dependence among reads and writes is never violated” is enforced by rules 4 and 6

- “All writes have release semantics” is enforced by rule 4

- “Reads or writes from a given thread to a given location cannot pass a write from the same thread to the same location” is enforced by rules 4

Now that we know the rules, wouldn’t it be nice to have a tool that could automatically check our .NET code to ensure that any valid reordering still keeps the semantic intact and to suggest possible additions of barriers and volatile operations? Well there is such a tool but as far as I understand, this one heavily relies on the weaker memory model defined by EMMA CLI spec and it is not complete. Also if you are interested in knowing more about memory model verification, take a look at this.

The other interesting aspect of CLR 2.0 is that it guarantees that: read and write access to properly aligned (default behaviour) memory location no larger than the native Integer (32 bit processor = 4-byte aligned, 64 bit processor = 8-byte aligned) is atomic. This is defined in section 12.6 of the ECMA CLI Spec. Therefore you can be certain that at least 4-byte assignments in CLR 2.0 are atomic.

Why do you need to know this? Well, writing good concurrent code particularly lock-free or low-lock data structures rely heavily on developers having a deep understanding of the memory model. Also, memory models can change so if you stick to a weaker memory model today such as the ECMA memory model it is more likely that your application stays forward-compatible.

First look at Parallel FX and self-replicating tasks

pedramr — Wed, 05 Dec 2007 00:54:00 GMT

The Parallel Computing Platform team at Microsoft has recently launched the Parallel Computing Development Centre along with our first CTP of Parallel FX. In here, I will explore some aspects of the framework. If you feel comfortable with the basics of concurrency then read on.

There is obviously PLINQ and the methods defined in the static System.Threading.Parallel class. It is implemented on top of the lower-level System.Threading.Tasks.Task class, which can be used directly to solve parallel problems with greater flexibility and control over the way work is partitioned and distributed.

Let me start by introducing the notion of replicable or self-replicating tasks. A replicable task is a task that its execution can be shared between multiple threads whilst abstracting from the dynamics of work distribution. In other words, you can potentially execute the same action using multiple threads where all executions stride towards achieving a common goal – getting closer to the result of the operation). Follow the following rules when using self-replicating tasks:

- One thread must be able to complete the operation

- It should be possible to complete the operation using multiple threads all executing the same operation at once

- The same set of activities should not be executed multiple times by different threads

Due to its nature, replicable tasks are very powerful and often very difficult to correctly construct. Parallel FX internally benefits from self replicating tasks. For instance Parallel.For and Parallel.ForEach are implemented using replicable tasks.

For example, we could create a simple parallel For in the following manner:

static void SimpleFor(int from, int to, int step, Action<int> body)

{

if (body == null) return;

if (to > from)

{

int sharedIndex = from;

Task task = Task.Create(delegate

{

int i;

while ((i = Interlocked.Add(ref sharedIndex, step)) < to)

{

body(i);

}

TaskCreationOptions.SelfReplicating);

task.Wait();

}

There are a few things to mention here:

- The shared integer and the Interlocked operation are effectively enforcing isolation between threads hence allowing all idle processor cores to participate in the operation

- Task.Create is called with TaskCreationOptions.SelfReplicating option

- If the work carried out by the action is small, cache contention, and performance overhead of calling an Interlocked operation may become an issue

- Depending on the value of step, you could use other Interlocked operations such as Increment, Decrement and CompareExchange to improve efficiency

To avoid possible performance overheads, it would be advisable to process a stride of indices at once:

static void SimpleFor(int from, int to, int step, Action<int> body)

{

if (body == null) return;

if (to > from)

{

const int stride = 10; // take a stride of size 10

int sharedIndex = from;

int increment = stride * step;

Task task = Task.Create(delegate

{

int startIndex;

{

startIndex =

Interlocked.Add(ref sharedIndex, increment) - increment;

for (int i = 0; i < stride && i + startIndex < to; i += step)

{

body(startIndex + i);

}

} while (startIndex + stride < to);

TaskCreationOptions.SelfReplicating);

task.Wait();

}

If you have not already done so, download the December CTP of Parallel Extensions to .NET Framework and let the team know your thoughts through here.

A Performance Comparison of ReaderWriterLockSlim with ReaderWriterLock

pedramr — Sun, 07 Oct 2007 21:11:00 GMT

If you have ever used the System.Threading.ReaderWriterLock to achieve synchronised access to shared resources which are frequently read but infrequently updated then you probably know why we have created a new Reader/Write lock. In this MSDN Magazine’s concurrent affairs column, Jeffery Richter explains some of the short comings of the existing ReaderWriterLock in .NET Framework.

One of the hidden gems of the new .NET Framework 3.5 is a fast Reader/Writer lock called “ReaderWriterLockSlim”. If you are not familiar with this new lock then read this blog entry.

Before recommending this new lock to my customers, I wanted to see some comparisons with Monitor and the existing ReaderWriterLock. So, I devised the following comparison tests:

- Acquiring a Read lock in comparison with taking a lock using Monitor

- Acquiring a Write lock in comparison with taking a lock using Monitor

- Acquiring a Read/Write lock in comparison with taking a Read/Write lock using ReaderWriterLock

Read lock comparison with Monitor

After executing the test for 1000 times, it became apparent that ReaderWriterLockSlim with no recursion support is 1.67 times slower than Monitor and it is 1.74 times slower when recursion is supported:

Write lock comparison with Monitor

ReaderWriterLockSlim, in average, is 1.71 times slower than Monitor, and when recursion is supported it is 1.77 times slower.

It also turns out that acquiring a Read lock using the new slim lock class is slightly quicker than taking a Write lock.

ReaderWriterLockSlim in comparison with ReaderWriterLock

ReaderWriterLock is significantly slower than its successor:

- Taking a ReaderWriterLockSlim Read lock is almost 3 times quicker than ReaderWriterLock

- Taking a ReaderWriterLockSlim Write lock is 2.8 times quicker than ReaderWriterLock

Results at a glance:

The table below compares both Reader/Writer locks with Monitor:

	Monitor	ReaderWriterLock	ReaderWriterLockSlim (No recursion support)	ReaderWriterLockSlim (recursion support)
Read	1	5	1.67	1.74
Write	1	4.71	1.71	1.77

The above comparison was performed on a Toshiba M400 with a Centrino Duo processor using Visual Studio 2008 beta 2.

.NET Thread Apartment and COM Interop

pedramr — Sun, 05 Aug 2007 17:51:00 GMT

Before a thread can call into a COM object it has to define its apartment by declaring whether it will enter a single-threaded apartment (STA) or a multi-threaded apartment (MTA). STA client threads call CoInitialize(NULL) or CoInitializeEx(0, COINIT_APARTMENTTHREADED) to enter an STA apartment and MTA threads call CoInitializeEx(0, COINIT_MULTITHREADED) to enter an MTA.

When calling a COM component, it is important to ensure that the threading models of the component and the client are compatible. For instance VB6 components insist on using the STA threading model. With an STA component, a message queue is associated with a hidden window and requests are queued and a single thread in STA apartment processes the messages synchronously of the queue. This ensures that access to the component is synchronised. However, if the client uses an MTA threading model to instantiate this STA component, not only a thread switch between the MTA and the STA thread needs to occur but also an apartment marshalling is required which can be an expensive process. Therefore it is recommended to use compatible threading models to ensure that apartment threaded component instance is created in the client's apartment.

Similarly, in the .NET managed world, you have the option of allowing the calling thread in the managed space declare its apartment affinity. For example, it is possible to associate a thread in .NET to the single-threaded apartment. This is usually the recommended approach when calling legacy VB6 COM.

Threads inside .NET’s thread pool are associated to the multi-threaded apartment (MTA) therefore requiring a thread switch and an apartment marshalling when calling VB6 components. It is recommended to use dedicated threads with ApartmentState property set to STA when calling into an STA component (use SetApartmentState method).

Dedicated thread or a Threadpool thread?

pedramr — Sun, 05 Aug 2007 17:48:00 GMT

Each .NET CLR based process is given a pool of threads. The size of the thread pool is configurable however by default the maximum number of threads created by the thread pool is set to 1000 IO threads in total, and 25 worker threads per logical or physical processor. In other words, when using a dual core processor, you will have the maximum value set to 50 worker threads.

These threads are managed in a very efficient and intelligent manner by the runtime. For instance, the actual size of the thread pool varies based on how the application is consuming those threads. It is possible to set the minimum number of threads for the thread pool. Once that many threads are created, they are kept alive for the lifetime of the process. However if more threads are required, the thread pool creates new threads. Once those threads finish executing their activities, they are then returned to the thread pool. It is important to emphasise that those threads are not destroyed. They will self destroy after approximately 2 minutes of inactivity. Therefore, the usage of system resources is constantly being adjusted automatically by the thread pool itself.

Once the minimum number of threads is reached, the thread pool aims to limit the number of threads being created to one per 500 milliseconds. This is an intelligent mechanism, avoiding the expensive cost of creating a new thread when multiple thread pool threads may be released within that time period.

The combination of the above approaches makes .NET’s thread pool a very intelligent thread pool which uses as little resources as possible to give maximum performance. Therefore in most scenarios it is recommended to use this thread pool for performing asynchronous operations.

.NET’s thread pool has some shortcomings which can affect the choice between using a dedicated thread instead of a thread from the thread pool. It is usually said that a dedicated thread is favourable in the following scenarios:

- When a foreground thread is required: All thread pool threads are initialised as background threads.

- When it is required to have a thread with a particular priority.

- When a thread is required to be aborted prematurely

- When a thread must be placed in a single-threaded apartment (STA): All thread pool threads are set in the MTA apartment by default

- For long running tasks when the thread pool thread is often blocked for long periods (This may starve other parts of the application which rely on threads from the thread pool)

The thread pool is shared by all AppDomains in a process. Consider this if you have more than one AppDomain in your process.