Pedram Rezaei's Ramblings : Parallel Extensions

Introducing Parallel Extensions to the .NET Framework

pedramr — Wed, 06 Aug 2008 23:35:00 GMT

There is no escaping from concurrency challenges... or is there?

(A slightly modified version of this article was published in the August 2008 edition of the MSDN Flash newsletter)

Dual, quad, and eight-core processors are becoming the norm. Is your application capable of utilising all available processors? In order to achieve this level of utilisation on an n-processor machine, an application ideally needs at least n threads concurrently performing operations.

Although writing multi-threaded applications has become simpler over the years, many still find it challenging. Also many of the constructs such as the .NET ThreadPool have deficiencies that make them less suitable as the manycore shift begins to accelerate.

Over the past few years, Microsoft has been hard at work developing a new set of parallel extensions for .NET, aptly named Parallel Extensions to the .NET Framework. One of the additions is a set of lightweight and scalable thread-safe data structures and synchronization primitives such as a concurrent dictionary and a spin lock.

It also includes an implementation of LINQ-to-Objects that automatically executes queries in parallel, scaling to utilise most or all of the available processors without developers explicitly managing the distribution of work across all processors. This technology is called Parallel LINQ or PLINQ and the query syntax is almost identical to that of LINQ-to-Objects.

var q = from c in customers.AsParallel()

join r in regions on c.RegionID equals r.RegionID

where c.City == "London" && r.Name == "Kingston"

select c;

When a repetitive computation can be parallelised, the Parallel Extensions to the .NET Framework offers a few interesting data-oriented operations such as For and ForEach that execute a loop in which iterations may run in parallel on parallel hardware. For example the following classic foreach loop can easily be converted to a parallel foreach:

Sequential execution

foreach (var c in formulae)

Calculate(f);

Possible parallel execution

Parallel.ForEach(formulae, f => Calculate(f));

The actual parallelisation of actions is managed by the Task Parallel Library (TPL) that provides an abstraction layer on top of raw threads. TPL uses the notion of Task that is the smallest unit of work and could potentially be executed by any thread owned by TPL. Therefore correctly identifying tasks that could be executed in parallel is crucial. It is then the job of the scheduler component of TPL to decide whether those tasks should execute sequentially or in parallel. This decision is usually made based on available system resources and possible user preferences.

// Create a task to call Calculate(object o) and schedule it for execution

Task t = Task.Create(Calculate);

// Other activities...

// Wait for this task to complete

t.Wait();

Developing concurrent applications requires a shift in mindset. For example, only a single exception can be thrown at any one time when sequentially executing a for loop. However executing the same loop in parallel may warrant the need for dealing with multiple concurrent exceptions. There are also a few parallelism blockers that should be avoided if you are planning to benefit from Parallel Extensions to the .NET Framework when it is released.

There is a lot of promise in Parallel Extensions. If you want to find out more, I’d encourage you to download the June 2008 Community Technology Preview build.

Coordination Data Structures – WriteOnce

pedramr — Tue, 03 Jun 2008 01:29:00 GMT

This is an article in a series of blog entries describing a set of new Coordination Data Structures (CDS) introduced in the June 2008 CTP of the Parallel Extensions for .NET Framework.

In C#, when a field declaration includes a readonly modifier, assignments to the fields introduced by the declaration can only occur as part of the declaration or in a constructor in the same class:

readonly Company _company = new Company();

The assignment window is limited to the constructor or the declaration. In many applications however, it is desirable to have the readonly behaviour (set once and read many times) without this limitation. As you might have guessed, the Parallel Extensions now provides a type (System.Threading.WriteOnce<T>) for this exact purpose.

WriteOnce<T> exposes a Value property that can only be set once. This is obviously done in a thread-safe manner:

// this is a struct, so it will not require an explicit construction

WriteOnce<Guid> wo;

wo.Value = Guid.NewGuid();

As you would expect, attempting to reassign the Value property will throw an exception (“The variable is already initialized”):

WriteOnce<T> is a thread-safe type so if trying to set the Value property concurrently, one of the setters will throw the above exception. In order to avoid this, use the TrySetValue method instead:

WriteOnce<Guid> wo;

if (wo.TrySetValue(Guid.Empty))

{

// the value was set successfuly

}

Attempting to access the Value of a WriteOnce<T> before it has been set invalidates the WriteOnce<T> instance for all future access. The instance will be corrupt and can no longer be accessed for future gets and set.

It corrupts the instance because there is a race condition that leads to a read before the write. Proper synchronization should be used to ensure that this race cannot happen. Therefore always ensure that the value is set before accessing it. When you are not certain if the Value is set, you can access it through the TryGetValue. This will return default(T) if the value is not set and will not corrupt the state of the instance for future use:

Guid val;

if (wo.TryGetValue(out val))

{

// the value has been set previously

}

Please note that WriteOnce<T> is a value type (struct), and as such, you need to be careful about access patterns. If you accidentally make a copy of the struct, you’ll be copying by value meaning that you will be using a replica rather than the original. For example in the code below, both assignments to what might appear as the same WriteOnce instance are allowed. The reality is however that wo1 and wo2 are two different instances:

WriteOnce<Guid> wo1;

Task.Create(

s =>

{

WriteOnce<Guid> wo2 = (WriteOnce<Guid>)s;

// assignment (1)

wo2.Value = Guid.NewGuid();

Console.WriteLine(wo2.Value);

}

, wo1);

// assignment (2)

wo1.Value = Guid.NewGuid();

In the example below however, it will not be possible to assign to wo2.Value (assignment 2 will fail). Although wo2 is a replica copy of wo1, internally it is referencing the same value object assigned in (1) which is not cloned:

WriteOnce<Guid> wo1;

// assignment (1)

wo1.Value = Guid.NewGuid();

Task.Create(

s =>

{

WriteOnce<Guid> wo2 = (WriteOnce<Guid>)s;

// assignment (2)

wo2.Value = Guid.NewGuid();

}

, wo1);

(Thanks to Joe Duffy, Ed Essey and Stephen Toub for their inputs and support)

Coordination Data Structures – SpinLock

pedramr — Mon, 02 Jun 2008 20:36:00 GMT

This is an article in a series of blog entries describing a set of new Coordination Data Structures (CDS) introduced in the June 2008 CTP of the Parallel Extensions for .NET Framework.

Waiting on locks usually result in a thread context switch and associated kernel transition which at times can be considered costly. On a multi-processor machine, often it might be more efficient to busy wait for a short period of time instead of paying the cost of performing an expensive context switch and a possible transition to kernel mode.

One approach to busy waiting is to wait in a loop repeatedly checking until the lock becomes available. These types of locks are called spin-locks.

System.Threading.SpinLock provides a mutual exclusion lock primitive where a thread trying to acquire the lock waits in a loop repeatedly checking until the lock becomes available.

When acquiring a spin-lock, it uses the CompareExchange interlocked operation to ensure that the owner of the lock is assigned in a thread-safe manner (this implementation detail is subject to change):

if(Interlocked.CompareExchange(ref _owner, newOwner, owner) == owner && ...)

Failing to become the owner of the lock, other threads will wait using a SpinWait until the lock is released (pseudo code):

public void Enter()

{
SpinWait sw = new SpinWait();
...

while (true)

{

if (!IsOwnerSet())

{

// set the owner - atomic

if (SetOwner())

{

// this thread is the owner so return

return;

}

// spin wait once

sw.SpinOnce();

}

...

}

One point to mention here is the effect of a spin-lock on a single-processor machine. On these systems, the spin-lock will hamper the performance by adding an unnecessary delay into the whole process. It will not result in a deadlock because after executing its quantum, the spinning thread is context-switched or pre-empted by another thread, however the time spent looping has not been productive.

Due to this behaviour, the System.Threading.SpinLock primitive ensures we always yield (switch threads) on single-processor machines instead of using busy waits.

Unlike Monitor, SpinLock publicly offers a set of reliable enter methods for acquiring a lock (i.e. ReliableEnter and TryReliableEnter). You should probably never use the unreliable enter methods because in the case of any asynchronous exceptions such as ThreadAbortException and OutOfMemoryException, you may end up with an orphaned lock that is never released. This is particularly true in ASP.NET since it uses aborts very aggressively.

As a side-note, you may wonder how we achieve this reliability. As it happens, the default CLR host postpones async exceptions inside a finally block until the end of the block. Therefore any part of the code that could result in an orphaned lock can be included inside a finally block:

try

{

}

finally

{

// take the lock in here

// ...

}
Please note that another host could choose to escalate thread aborts to rude thread aborts, which can interrupt finally blocks.

Closer investigating of the SpinLock.Exit method reveals an overload that takes a Boolean. If this Boolean is set to true, SpinLock flushes the write buffers associated with the lock in order to ensure that all processors are immediately made aware that the lock is now available. This is more expensive but will prevent a situation where one processor is given an unfair advantage to reacquire the lock:

SpinLock sharedLock = new SpinLock();

while (true)

{

sharedLock.Enter();

sharedLock.Exit();

// very little between Exit and retry Enter;

// possibly no other processor would see the lock available!

}

There are a few more things to remember when using the SpinLock:

1- When using the default constructor, the thread that owns the lock can enter the same lock and effectively cause a deadlock:

SpinLock l = new SpinLock();
l.Enter(); // non-blocking call

l.Enter(); // blocking call

2- When using the default constructor, any thread can release the lock (This is not a recommended practice and should be avoided in normal circumstances):

SpinLock l = new SpinLock();

l.Enter();

ThreadPool.QueueUserWorkItem(delegate

{

l.Exit();

});

3- In the current implementation, you can change the above behaviour. Simply set the isThreadOwnerTrackingEnabled parameter to true when creating a new instance of SpinLock:

Setting isThreadOwnerTrackingEnabled to true changes the behaviour of the lock by:

- Prohibiting the owner thread from trying to enter the same lock – throws a LockRecursionException; and,

- Ensuring the Exit method is only called by the owner thread – a SynchronizationLockException is thrown otherwise

Something else that’s worth mentioning is that SpinLock is a struct, and as such, you need to be careful about access patterns. If you accidentally make a copy of the struct, you’ll be copying by value meaning that you will be using a replica rather than the original lock. For example in the code below, both the main thread and the ThreadPool thread would successfully enter the lock. The morale of the story is that ideally these are not passed around, and instead they are used as local variables or member fields:

SpinLock sl = new SpinLock();

ThreadPool.QueueUserWorkItem(state =>

{

SpinLock theLock = (SpinLock)state;

theLock.Enter();

}, sl);

sl.Enter();

(Thanks to Stephen Toub, Joe Duffy and Ed Essey for their input and support)

Coordination Data Structures – LazyInit

pedramr — Mon, 02 Jun 2008 20:34:00 GMT

This is an article in a series of blog entries describing a set of new Coordination Data Structures (CDS) introduced in the June 2008 CTP of the Parallel Extensions for .NET Framework.

LazyInit<T> provides support for several common patterns of lazy initialization. In here we will explore some of those patterns, but first a point or two regarding lazy initialization:

- Lazy initialization is often used when the initialization process is an expensive activity which is not always required. With this approach, the execution of the initialize procedure is deferred to immediately before when it is required.

- In order to improve the start-up performance of an application, one may decide to defer some of the initialization activities.

The first pattern covered by LazyInit<T> is the optimistic lazy instantiation pattern. In this pattern, multiple instances of T may be created but only one of those instances is published for all threads to access. In pseudo code, the implementation of such structure may look similar to this:

public class Singleton

{

static volatile Singleton _instance;

private Singleton()

{

// stuff ...

}

public static Singleton Instance

{

get

{

if (_instance == null)

{

Singleton local = new Singleton();

if (Interlocked.CompareExchange<Singleton>(

ref _instance, local, null) != null)

{

IDisposable disposable = local as IDisposable;

if (disposable != null)

{

disposable.Dispose();

}

return _instance;

}

As you can see, unwanted instances are disposed automatically. This optimistic approach to concurrency removes the need for a full lock, but has the potential of multiple instantiations.

The LazyInit<T> struct offers a similar functionality. Simply create an instance of the LazyInit<T> using its default constructor:

LazyInit<Company> s = new LazyInit<Company>();

// do some stuff here

// ...

Company c = s.Value;

The above example uses reflection to create an instance of Company when the Value property is first accessed. Obviously Company must have a public parameter-less constructor for this to succeed.

It is worth emphasising that the default behaviour of LazyInit does not prevent the instantiation of more than one Company object by multiple threads concurrently accessing the Value property when the value is not yet set. However it guarantees that only one of those instances is published for all threads to access.

Another pattern offered by LazyInit<T> is the pessimistic lazy initialization pattern. This is very similar to the well-known double-check locking pattern and in pseudo code looks like this:

public class Singleton

{

static volatile Singleton _instance;

static object s_lock = new object();

private Singleton()

{

// stuff ...

}

public static Singleton Instance

{

get

{

// double-check locking pattern

if (_instance == null)

{

lock (s_lock)

{

if (_instance == null)

{

_instance = new Singleton();

return _instance;

}

return _instance;

}

As you can see, no more than a single instance of Singleton is ever instantiated. This is achieved at the cost of acquiring a lock.

LazyInit<T> also provides a similar functionally. Here are some variations of this pattern implemented using LazyInit<T>:

Pessimistic Lazy Instantiation:

LazyInit<Company> s = new LazyInit<Company>(

() => new Company(), LazyInitMode.EnsureSingleExecution);

// do some stuff here

// ...

Company c = s.Value;

Pessimistic Lazy Initialization:

In the example below, a database connection is opened only when needed. The connection is then kept open and no other connection is created for this instance of DataAccess:

class DataAccess

{

LazyInit<SqlConnection> cnn;

public DataAccess()

{

cnn = new LazyInit<SqlConnection>(

delegate

{

var c = new SqlConnection(

"Server=.;Database=Northwind;Integrated Security=true");

c.Open();

return c;

}, LazyInitMode.EnsureSingleExecution);

}

public int GetProductCount()

{

// perform a db query: this is when we actually

// create and open a connection to the database

using (var cmd = new SqlCommand(

"select count(*) from products", cnn.Value))

{

return (int)cmd.ExecuteScalar();

}

Another pattern offered by LazyInit<T> allows an initialization per thread such that each thread will get its own value. In this pattern, the value from the initialization is stored in the thread local storage (TLS):

LazyInit<Company> l = new LazyInit<Company>(

() => new Company { ID = Guid.NewGuid() },

LazyInitMode.ThreadLocal);

// l.Value access from the main thread

Console.WriteLine(l.Value.ID);

ThreadPool.QueueUserWorkItem(delegate

{

// l.Value access from a ThreadPool thread

Console.WriteLine(l.Value.ID);

});

In the example above, the main thread and the ThreadPool thread each create a different instance of Company with different IDs:

Now that you have seen some different lazy initialization patterns, it is time to fully introduce the LazyInitMode enum that can be specified as a parameter to the LazyInit<T> constructor:

LazyInitMode Value	Description
AllowMultipleExecution	The initialization function may be executed multiple times if multiple threads race to initialize the value, but only one value will be published for all threads to access.
EnsureSingleExecution	The initialization function will only be executed once, even if multiple threads race to initialize the value. This value will be published for all threads to access.
ThreadLocal	The initialization function will be invoked once per thread such that each thread will get its own published value.

One last point, LazyInit<T> is a value type and does not have the overhead of being a class. However you need to be careful about access patterns. If you accidentally make a copy of the struct, you’ll be copying by value meaning that you will be using a replica rather than the original.

(Thanks to Stephen Toub, Joe Duffy and Ed Essey for their input and support)

How to cancel a task in Parallel FX?

pedramr — Wed, 02 Jan 2008 19:52:00 GMT

Task Parallel Library (TPL) allows you to easily cancel tasks. Effectively you need to call the Cancel method on the task in question. Imagine the simple sample below:

Task task1 = Task.Create(Foo, 10000);

static void Foo(object o)

{

for (int i = 0; i < (int)o; i++)

{

// some code here

}

A task is created and Foo is automatically called. At some point if you decide to cancel the task while it is running, you can call the Cancel method on the task as shown below:

Task task = Task.Create(Foo, 10000);

task.Cancel();

When you call cancel, all you are doing is to set a Boolean field to true. You can query its value through the IsCanceled property:

task.IsCanceled

When Cancel is called, if the task has not yet prepared for execution, it will not run. However if the task has already been prepared and assigned to a thread, it will start executing. The execution is not automatically interrupted and requires manual intervention in order to cancel. Effectively you need to build the cancelation logic into your application:

static void Foo(object o)

{

for (int i = 0; i < (int)o; i++)

{

// Could it have been cancelled?

ThrowIfCurrentTaskCanceled();

// some code here

}

internal static void ThrowIfCurrentTaskCanceled()

{

Task current = Task.Current;

if (current != null && current.IsCanceled)

{

throw new TaskCanceledException(current);

}

As you can see, it is possible to access the current Task and check to see if it has already been cancelled. One thing to remember regarding the Boolean field used by the IsCanceled property is that it is a volatile field meaning that all reads are volatile reads and have acquire semantics which can be expensive.

On a slightly different note, if you are unsure about the spelling of “Cancelled” then take a look at this post.

First look at Parallel FX and self-replicating tasks

pedramr — Wed, 05 Dec 2007 00:54:00 GMT

The Parallel Computing Platform team at Microsoft has recently launched the Parallel Computing Development Centre along with our first CTP of Parallel FX. In here, I will explore some aspects of the framework. If you feel comfortable with the basics of concurrency then read on.

There is obviously PLINQ and the methods defined in the static System.Threading.Parallel class. It is implemented on top of the lower-level System.Threading.Tasks.Task class, which can be used directly to solve parallel problems with greater flexibility and control over the way work is partitioned and distributed.

Let me start by introducing the notion of replicable or self-replicating tasks. A replicable task is a task that its execution can be shared between multiple threads whilst abstracting from the dynamics of work distribution. In other words, you can potentially execute the same action using multiple threads where all executions stride towards achieving a common goal – getting closer to the result of the operation). Follow the following rules when using self-replicating tasks:

- One thread must be able to complete the operation

- It should be possible to complete the operation using multiple threads all executing the same operation at once

- The same set of activities should not be executed multiple times by different threads

Due to its nature, replicable tasks are very powerful and often very difficult to correctly construct. Parallel FX internally benefits from self replicating tasks. For instance Parallel.For and Parallel.ForEach are implemented using replicable tasks.

For example, we could create a simple parallel For in the following manner:

static void SimpleFor(int from, int to, int step, Action<int> body)

{

if (body == null) return;

if (to > from)

{

int sharedIndex = from;

Task task = Task.Create(delegate

{

int i;

while ((i = Interlocked.Add(ref sharedIndex, step)) < to)

{

body(i);

}

TaskCreationOptions.SelfReplicating);

task.Wait();

}

There are a few things to mention here:

- The shared integer and the Interlocked operation are effectively enforcing isolation between threads hence allowing all idle processor cores to participate in the operation

- Task.Create is called with TaskCreationOptions.SelfReplicating option

- If the work carried out by the action is small, cache contention, and performance overhead of calling an Interlocked operation may become an issue

- Depending on the value of step, you could use other Interlocked operations such as Increment, Decrement and CompareExchange to improve efficiency

To avoid possible performance overheads, it would be advisable to process a stride of indices at once:

static void SimpleFor(int from, int to, int step, Action<int> body)

{

if (body == null) return;

if (to > from)

{

const int stride = 10; // take a stride of size 10

int sharedIndex = from;

int increment = stride * step;

Task task = Task.Create(delegate

{

int startIndex;

{

startIndex =

Interlocked.Add(ref sharedIndex, increment) - increment;

for (int i = 0; i < stride && i + startIndex < to; i += step)

{

body(startIndex + i);

}

} while (startIndex + stride < to);

TaskCreationOptions.SelfReplicating);

task.Wait();

}

If you have not already done so, download the December CTP of Parallel Extensions to .NET Framework and let the team know your thoughts through here.