Pedram Rezaei's Ramblings : CDS

Coordination Data Structures – WriteOnce

pedramr — Tue, 03 Jun 2008 01:29:00 GMT

This is an article in a series of blog entries describing a set of new Coordination Data Structures (CDS) introduced in the June 2008 CTP of the Parallel Extensions for .NET Framework.

In C#, when a field declaration includes a readonly modifier, assignments to the fields introduced by the declaration can only occur as part of the declaration or in a constructor in the same class:

readonly Company _company = new Company();

The assignment window is limited to the constructor or the declaration. In many applications however, it is desirable to have the readonly behaviour (set once and read many times) without this limitation. As you might have guessed, the Parallel Extensions now provides a type (System.Threading.WriteOnce<T>) for this exact purpose.

WriteOnce<T> exposes a Value property that can only be set once. This is obviously done in a thread-safe manner:

// this is a struct, so it will not require an explicit construction

WriteOnce<Guid> wo;

wo.Value = Guid.NewGuid();

As you would expect, attempting to reassign the Value property will throw an exception (“The variable is already initialized”):

WriteOnce<T> is a thread-safe type so if trying to set the Value property concurrently, one of the setters will throw the above exception. In order to avoid this, use the TrySetValue method instead:

WriteOnce<Guid> wo;

if (wo.TrySetValue(Guid.Empty))

{

// the value was set successfuly

}

Attempting to access the Value of a WriteOnce<T> before it has been set invalidates the WriteOnce<T> instance for all future access. The instance will be corrupt and can no longer be accessed for future gets and set.

It corrupts the instance because there is a race condition that leads to a read before the write. Proper synchronization should be used to ensure that this race cannot happen. Therefore always ensure that the value is set before accessing it. When you are not certain if the Value is set, you can access it through the TryGetValue. This will return default(T) if the value is not set and will not corrupt the state of the instance for future use:

Guid val;

if (wo.TryGetValue(out val))

{

// the value has been set previously

}

Please note that WriteOnce<T> is a value type (struct), and as such, you need to be careful about access patterns. If you accidentally make a copy of the struct, you’ll be copying by value meaning that you will be using a replica rather than the original. For example in the code below, both assignments to what might appear as the same WriteOnce instance are allowed. The reality is however that wo1 and wo2 are two different instances:

WriteOnce<Guid> wo1;

Task.Create(

s =>

{

WriteOnce<Guid> wo2 = (WriteOnce<Guid>)s;

// assignment (1)

wo2.Value = Guid.NewGuid();

Console.WriteLine(wo2.Value);

}

, wo1);

// assignment (2)

wo1.Value = Guid.NewGuid();

In the example below however, it will not be possible to assign to wo2.Value (assignment 2 will fail). Although wo2 is a replica copy of wo1, internally it is referencing the same value object assigned in (1) which is not cloned:

WriteOnce<Guid> wo1;

// assignment (1)

wo1.Value = Guid.NewGuid();

Task.Create(

s =>

{

WriteOnce<Guid> wo2 = (WriteOnce<Guid>)s;

// assignment (2)

wo2.Value = Guid.NewGuid();

}

, wo1);

(Thanks to Joe Duffy, Ed Essey and Stephen Toub for their inputs and support)

Coordination Data Structures – SpinLock

pedramr — Mon, 02 Jun 2008 20:36:00 GMT

This is an article in a series of blog entries describing a set of new Coordination Data Structures (CDS) introduced in the June 2008 CTP of the Parallel Extensions for .NET Framework.

Waiting on locks usually result in a thread context switch and associated kernel transition which at times can be considered costly. On a multi-processor machine, often it might be more efficient to busy wait for a short period of time instead of paying the cost of performing an expensive context switch and a possible transition to kernel mode.

One approach to busy waiting is to wait in a loop repeatedly checking until the lock becomes available. These types of locks are called spin-locks.

System.Threading.SpinLock provides a mutual exclusion lock primitive where a thread trying to acquire the lock waits in a loop repeatedly checking until the lock becomes available.

When acquiring a spin-lock, it uses the CompareExchange interlocked operation to ensure that the owner of the lock is assigned in a thread-safe manner (this implementation detail is subject to change):

if(Interlocked.CompareExchange(ref _owner, newOwner, owner) == owner && ...)

Failing to become the owner of the lock, other threads will wait using a SpinWait until the lock is released (pseudo code):

public void Enter()

{
SpinWait sw = new SpinWait();
...

while (true)

{

if (!IsOwnerSet())

{

// set the owner - atomic

if (SetOwner())

{

// this thread is the owner so return

return;

}

// spin wait once

sw.SpinOnce();

}

...

}

One point to mention here is the effect of a spin-lock on a single-processor machine. On these systems, the spin-lock will hamper the performance by adding an unnecessary delay into the whole process. It will not result in a deadlock because after executing its quantum, the spinning thread is context-switched or pre-empted by another thread, however the time spent looping has not been productive.

Due to this behaviour, the System.Threading.SpinLock primitive ensures we always yield (switch threads) on single-processor machines instead of using busy waits.

Unlike Monitor, SpinLock publicly offers a set of reliable enter methods for acquiring a lock (i.e. ReliableEnter and TryReliableEnter). You should probably never use the unreliable enter methods because in the case of any asynchronous exceptions such as ThreadAbortException and OutOfMemoryException, you may end up with an orphaned lock that is never released. This is particularly true in ASP.NET since it uses aborts very aggressively.

As a side-note, you may wonder how we achieve this reliability. As it happens, the default CLR host postpones async exceptions inside a finally block until the end of the block. Therefore any part of the code that could result in an orphaned lock can be included inside a finally block:

try

{

}

finally

{

// take the lock in here

// ...

}
Please note that another host could choose to escalate thread aborts to rude thread aborts, which can interrupt finally blocks.

Closer investigating of the SpinLock.Exit method reveals an overload that takes a Boolean. If this Boolean is set to true, SpinLock flushes the write buffers associated with the lock in order to ensure that all processors are immediately made aware that the lock is now available. This is more expensive but will prevent a situation where one processor is given an unfair advantage to reacquire the lock:

SpinLock sharedLock = new SpinLock();

while (true)

{

sharedLock.Enter();

sharedLock.Exit();

// very little between Exit and retry Enter;

// possibly no other processor would see the lock available!

}

There are a few more things to remember when using the SpinLock:

1- When using the default constructor, the thread that owns the lock can enter the same lock and effectively cause a deadlock:

SpinLock l = new SpinLock();
l.Enter(); // non-blocking call

l.Enter(); // blocking call

2- When using the default constructor, any thread can release the lock (This is not a recommended practice and should be avoided in normal circumstances):

SpinLock l = new SpinLock();

l.Enter();

ThreadPool.QueueUserWorkItem(delegate

{

l.Exit();

});

3- In the current implementation, you can change the above behaviour. Simply set the isThreadOwnerTrackingEnabled parameter to true when creating a new instance of SpinLock:

Setting isThreadOwnerTrackingEnabled to true changes the behaviour of the lock by:

- Prohibiting the owner thread from trying to enter the same lock – throws a LockRecursionException; and,

- Ensuring the Exit method is only called by the owner thread – a SynchronizationLockException is thrown otherwise

Something else that’s worth mentioning is that SpinLock is a struct, and as such, you need to be careful about access patterns. If you accidentally make a copy of the struct, you’ll be copying by value meaning that you will be using a replica rather than the original lock. For example in the code below, both the main thread and the ThreadPool thread would successfully enter the lock. The morale of the story is that ideally these are not passed around, and instead they are used as local variables or member fields:

SpinLock sl = new SpinLock();

ThreadPool.QueueUserWorkItem(state =>

{

SpinLock theLock = (SpinLock)state;

theLock.Enter();

}, sl);

sl.Enter();

(Thanks to Stephen Toub, Joe Duffy and Ed Essey for their input and support)

Coordination Data Structures – LazyInit

pedramr — Mon, 02 Jun 2008 20:34:00 GMT

This is an article in a series of blog entries describing a set of new Coordination Data Structures (CDS) introduced in the June 2008 CTP of the Parallel Extensions for .NET Framework.

LazyInit<T> provides support for several common patterns of lazy initialization. In here we will explore some of those patterns, but first a point or two regarding lazy initialization:

- Lazy initialization is often used when the initialization process is an expensive activity which is not always required. With this approach, the execution of the initialize procedure is deferred to immediately before when it is required.

- In order to improve the start-up performance of an application, one may decide to defer some of the initialization activities.

The first pattern covered by LazyInit<T> is the optimistic lazy instantiation pattern. In this pattern, multiple instances of T may be created but only one of those instances is published for all threads to access. In pseudo code, the implementation of such structure may look similar to this:

public class Singleton

{

static volatile Singleton _instance;

private Singleton()

{

// stuff ...

}

public static Singleton Instance

{

get

{

if (_instance == null)

{

Singleton local = new Singleton();

if (Interlocked.CompareExchange<Singleton>(

ref _instance, local, null) != null)

{

IDisposable disposable = local as IDisposable;

if (disposable != null)

{

disposable.Dispose();

}

return _instance;

}

As you can see, unwanted instances are disposed automatically. This optimistic approach to concurrency removes the need for a full lock, but has the potential of multiple instantiations.

The LazyInit<T> struct offers a similar functionality. Simply create an instance of the LazyInit<T> using its default constructor:

LazyInit<Company> s = new LazyInit<Company>();

// do some stuff here

// ...

Company c = s.Value;

The above example uses reflection to create an instance of Company when the Value property is first accessed. Obviously Company must have a public parameter-less constructor for this to succeed.

It is worth emphasising that the default behaviour of LazyInit does not prevent the instantiation of more than one Company object by multiple threads concurrently accessing the Value property when the value is not yet set. However it guarantees that only one of those instances is published for all threads to access.

Another pattern offered by LazyInit<T> is the pessimistic lazy initialization pattern. This is very similar to the well-known double-check locking pattern and in pseudo code looks like this:

public class Singleton

{

static volatile Singleton _instance;

static object s_lock = new object();

private Singleton()

{

// stuff ...

}

public static Singleton Instance

{

get

{

// double-check locking pattern

if (_instance == null)

{

lock (s_lock)

{

if (_instance == null)

{

_instance = new Singleton();

return _instance;

}

return _instance;

}

As you can see, no more than a single instance of Singleton is ever instantiated. This is achieved at the cost of acquiring a lock.

LazyInit<T> also provides a similar functionally. Here are some variations of this pattern implemented using LazyInit<T>:

Pessimistic Lazy Instantiation:

LazyInit<Company> s = new LazyInit<Company>(

() => new Company(), LazyInitMode.EnsureSingleExecution);

// do some stuff here

// ...

Company c = s.Value;

Pessimistic Lazy Initialization:

In the example below, a database connection is opened only when needed. The connection is then kept open and no other connection is created for this instance of DataAccess:

class DataAccess

{

LazyInit<SqlConnection> cnn;

public DataAccess()

{

cnn = new LazyInit<SqlConnection>(

delegate

{

var c = new SqlConnection(

"Server=.;Database=Northwind;Integrated Security=true");

c.Open();

return c;

}, LazyInitMode.EnsureSingleExecution);

}

public int GetProductCount()

{

// perform a db query: this is when we actually

// create and open a connection to the database

using (var cmd = new SqlCommand(

"select count(*) from products", cnn.Value))

{

return (int)cmd.ExecuteScalar();

}

Another pattern offered by LazyInit<T> allows an initialization per thread such that each thread will get its own value. In this pattern, the value from the initialization is stored in the thread local storage (TLS):

LazyInit<Company> l = new LazyInit<Company>(

() => new Company { ID = Guid.NewGuid() },

LazyInitMode.ThreadLocal);

// l.Value access from the main thread

Console.WriteLine(l.Value.ID);

ThreadPool.QueueUserWorkItem(delegate

{

// l.Value access from a ThreadPool thread

Console.WriteLine(l.Value.ID);

});

In the example above, the main thread and the ThreadPool thread each create a different instance of Company with different IDs:

Now that you have seen some different lazy initialization patterns, it is time to fully introduce the LazyInitMode enum that can be specified as a parameter to the LazyInit<T> constructor:

LazyInitMode Value	Description
AllowMultipleExecution	The initialization function may be executed multiple times if multiple threads race to initialize the value, but only one value will be published for all threads to access.
EnsureSingleExecution	The initialization function will only be executed once, even if multiple threads race to initialize the value. This value will be published for all threads to access.
ThreadLocal	The initialization function will be invoked once per thread such that each thread will get its own published value.

One last point, LazyInit<T> is a value type and does not have the overhead of being a class. However you need to be careful about access patterns. If you accidentally make a copy of the struct, you’ll be copying by value meaning that you will be using a replica rather than the original.

(Thanks to Stephen Toub, Joe Duffy and Ed Essey for their input and support)