Parallel Programming with .NET : Coordination Data Structures

Slides from Parallelism Tour

toub — Mon, 02 Nov 2009 02:58:00 GMT

Last week, I had the privilege of touring around Tennessee, Kentucky, Ohio, and Michigan, speaking about the new parallel computing support in Visual Studio 2010 and the .NET Framework 4. Many folks I spoke with were interested in getting a copy of the slide deck I used, so I’ve attached it to this blog post. Enjoy!

Cancellation in Parallel Extensions

mikelid — Mon, 22 Jun 2009 12:00:00 GMT

One of the great features that crosses all of Parallel Extensions types is a consistent approach to cancellation (see http://blogs.msdn.com/pfxteam/archive/2009/05/22/9635790.aspx). In this post we explore some of the ways cancellation is used in Parallel Extensions and explain the guidance we developed.

The new cancellation system is a cooperative approach based on two new types: CancellationTokenSource, which initiates cancellation requests, and CancellationToken which communicates a cancellation request to asynchronous operations and to long-running and blocking method calls.

If you are experimenting with Parallel Extensions, you might like to keep an eye out for the methods that accept a CancellationToken and test them out.

Blocking calls

Some of the Parallel Extension types introduce blocking methods, for example BlockingCollection.Take() and Task.Wait(). The default overloads for these methods will block indefinitely if the condition they are waiting for never occurs. One solution used by these APIs, and many others like them, is to provide a timeout overload so that they may return after some duration and report that the condition did not occur. However, a timeout isn't particularly convenient if you only want to stop waiting if a specific activity occurs, such as a user clicking a 'cancel' button. One possibility is to wait for some amount of time such as 100 milliseconds, check if the button was pressed, and if not then go back to waiting, but this puts the burden on the call-site and must be re-implemented repeatedly; and depending on the frequency of the polling, it may also add unnecessary cost. The types in Parallel Extensions use the new cancellation system and provide overloads that accept a CancellationToken which can cause early termination of blocking methods in response to a specific cancellation request.

Here are some examples of the blocking calls in Parallel Extensions that accept a CancellationToken:

BlockingCollection:
void Add(T item, CancellationToken cancellationToken)
T Take(CancellationToken cancellationToken)

ManualResetEventSlim:
void Wait(CancellationToken cancellationToken)

Task:
void Wait(CancellationToken cancellationToken)
void WaitAll(Task[] tasks, CancellationToken cancellationToken)

In each case, you supply a CancellationToken to the method, and if the CancellationToken is signaled via a call to the associated CancellationTokenSource.Cancel(), then the method will wake up and throw an OperationCanceledException. This frees you from the need to use the timeouts if you are happy to wait until the condition is true or a specific cancellation request occurs. Many of these methods also have overloads that accept both a CancellationToken and a timeout, so that both needs may be accommodated.

Long-running calls

There are also a variety of methods that may simply take a while to complete, particularly if large amounts of data or complex processing is taking place. For example, the following call is likely to take some time, even if individual calls to DoFunc(x) are relatively fast:

Parallel.For(1, 1000000000, (x) => DoFunc(x));

To assist with this, the Parallel APIs have overloads that accept a ParallelOptions class instance that holds a CancellationToken. The machinery inside Parallel.For(...)observes the supplied CancellationToken and will exit early if it sees that the token has be signaled (for more information on exiting loops early, see http://blogs.msdn.com/pfxteam/archive/2009/05/27/9645023.aspx). For example,

CancellationTokenSource cts = new CancellationTokenSource();
ParallelOptions options = new ParallelOptions
{CancellationToken = cts.Token};
Parallel.For(1, 100, options, (x) => DoSlowFunc(x));
// from another thread, call cts.Cancel() to request cancellation.

As you can see, the approach for canceling blocking calls and long-running calls is identical. In both cases you supply a CancellationToken to a specific method call and use the associated CancellationTokenSource to signal a cancellation request.

Digression: Internal cancellation

In certain situations in Parallel Extensions and in other systems, it is necessary to wake up a blocked method for reasons that aren't due to explicit cancellation by a user. For example, if one thread is blocked on blockingCollection.Take() due to the collection being empty and another thread subsequently calls blockingCollection.CompleteAdding(), then the first call should wake up and throw an InvalidOperationException to represent an incorrect usage. A great way to implement this is to use an internal CancellationTokenSource that can wake things up due to internal concerns, and to link it to a second, external CancellationTokenSource that has been supplied by the user. For example, the following captures the essential details of how CompleteAdding() and Take(CancellationToken) are implemented:

class BlockingCollection<T>
{
CancellationTokenSource internalCTS = new CancellationTokenSource();

   public void FinishAdding()
   {
      //...
      internalCTS.Cancel();
   }

   public T Take(CancellationToken externalToken)
   {
      using(CancellationTokenSource linkedTokenSource =
      CancellationTokenSource.CreateLinkedTokenSource(
      internalCTS.Token, externalToken)){
      try
      {
         return InternalTake(linkedTokenSource.Token);
      }
      catch (OperationCanceledException oce)
      {
         if(internalCTS.Token.IsCancellationRequested)
            throw new InvalidOperationException("..msg..");
         else if(externalToken.IsCancellationRequested)
            throw new OperationCanceledException(externalToken);
         else throw;
      }
   }
}

In this fragment, a linked CancellationTokenSource is created and its token passed to the InternalTake() method. This simplifies the implementation for InternalTake() but still allows it to be woken up due to either the external token being signaled or an incorrect concurrent call to CompleteAdding(). If the InternalTake() method throws an OperationCanceledException, we can test the individual tokens to determine what the cause was and take appropriate action. Note that oce.CancellationToken will be equal to linkedTokenSource.Token as this is the token that was actually being observed, but we can look at the original tokens to determine the cause for the linked token being signaled. There is a minor risk of confusion if both sources are signaled simultaneously, but this confusion is benign as it is reasonable to behave as though either source was responsible, and a particular implementation may choose to prioritize one mechanism over the other.

Cooperative cancellation with user-code

Both TPL and PLINQ provide infrastructure that calls back to user code. If the user code is long running or blocking, then it is very useful for the user code to be able to respond to cancellation in a cooperative fashion. TPL and PLINQ achieve this by tracking a supplied CancellationToken, and if they see some user code throw an OperationCanceledException that mentions this specific token, then it is treated as cooperative acknowledgement of cancellation. This means that both TPL and PLINQ understand that the user code did not suffer some catastrophic exceptional conditional, but rather that it is simply responding to cancellation. As a result, TPL and PLINQ constructs will simply exit with an OperationCanceledException of their own, rather than aggregating exceptions from all the participating worker threads and throwing an AggregateException.

PLINQ follows this basic plan; for example, the following query will throw a single OperationCanceledException if the CancellationTokenSource becomes signaled. It doesn't matter whether PLINQ or the user delegate sees the cancellation request first, as the user-delegate exception will be correctly understood by the PLINQ execution engine.

CancellationTokenSource cts = new CancellationTokenSource();
var resultArray =
    data.AsParallel()
        .WithCancellation(cts.Token)
        .Select((x) =>
           {
              if (cts.Token.IsCancellationRequested)
                 throw new OperationCanceledException(cts.Token);
              // ...
           })
        .ToArray();

In both PLINQ and TPL, it is interesting to note that the external cancellation token is not supplied back to the user-delegate via a parameter as may be expected. This would have bloated the number of method overloads considerably and, as shown above, the external token can be conveniently accessed via closures or equivalent techniques.

If the user code throws an OperationCanceledException but one or more "normal" exceptions occur on other threads, then all of the exceptions will be collated into an AggregateException, including the OperationCanceledException. Hence, if only cancellation occurred then a single OperationCanceledException is thrown, but an actual failure will always result in an AggregateException.

PLINQ Queries

PLINQ offers deep support for cancellation and endeavors to ensure that cancellation will be enacted swiftly for any query. For queries that involve only simple and fast user delegates, simply supply a CancellationToken via the WithCancellation() method and PLINQ takes care of the rest. If your query involves long-running or blocking user code, then you should use the cooperative cancellation pattern described in the preceding section. For reasons that are discussed in the next section, PLINQ may not check for cancellation every time it calls a user-delegate, and so long running delegates should check cancellation frequently (as a working rule of thumb, PLINQ will check for cancellation approximately once per one hundred calls to a user delegate). For example, if a delegate call runs for 50ms but does not perform any cancellation checks of its own, then PLINQ may not detect the cancellation request itself for up to 5 seconds, and this could significantly affect a user’s experience. For this reason, we recommend that long-running delegates called from PLINQ queries should check for cancellation as frequently as possible. A good lower bound is to check cancellation once per few thousand IL instructions executed, and an upper bound is to check no less frequently than once per 10ms, but once per 1ms is preferred to ensure snappy response.

The following example demonstrates a user delegate that may run for a while, and so checks cancellation itself every few thousand IL instructions. This provides for timely cancellation support with a negligible cost to performance.

// Assume an external CancellationToken 'token' has been supplied.
int[] data = ...;
var query =
   data.AsParallel()
       .WithCancellation(token)
       .Select((x) =>
          {
             int result = 0;
             for ( int i = 0; i < x; i++ ) {
                for (int j = 0; j < 1000 * x; j++)
                {
                   result += SimpleFunc(i,j);
                }
                token.ThrowIfCancellationRequested();
             }
             return result;})
       .ToArray();

[Update: the call to token.ThrowIfCancellationRequested shown above is not available in Beta1 (but will appear in subsequent releases). This behavior can be acheived via a manual test-and-throw as shown in earlier examples, or via the extension-method approach described in the comments. -Mike]

Summary

The Parallel Extension types use the new cancellation system to provide consistent and rich cancellation features. All of the long-running and blocking methods in Parallel Extensions provide overloads that take a CancellationToken, and any callbacks made to user-code can participate in the process. Application code can also make use of the same ideas and facilities to ensure that cancellation can be a fully supported feature in any situation.

.NET 4 Cancellation Framework

mikelid — Sat, 23 May 2009 06:29:00 GMT

A very interesting addition to .NET 4 is a set of new types that specifically assist with building cancellation-aware applications and libraries. The new types enable rich scenarios for convenient and safe cancellation, and help simplify situations that used to be be difficult and error-prone and non-composable.

The details of the new types are described below, but lets begin with some of the motivating principles that the new types were designed to support:

When any unit of work is commenced, it should have a consistent means to support early termination in response to a cancellation request.
If some unit of work controls various other moving pieces, it should be able to conveniently chain cancellation requests to them.
Blocking calls should be able to support cancellation.
Calls to complex operations, such as MoveNext() on a PLINQ enumerator, should have simple yet comprehensive cancellation support.
When infrastructure makes calls back to potentially long-running user code, it should be possible for the user code to observe and respond to cancellation requests in a cooperative fashion.
Cancellation should be an obvious part of an API with clean and consistent semantics.
Cancellation should not be forceful but instead cooperative.

In many prevailing systems, cancellation has been a secondary feature that rarely gets treated in sufficient detail to enable all of the above principles in a comprehensive fashion. The new types introduced to .NET 4 raise cancellation to be a primary concept for .NET APIs and one that can be cleanly and easily incorporated into any system.

Previous approaches

Cancellation is a feature that finds its way into many APIs and applications either by design or by necessity. An interesting classification of common approaches is Herb Sutter's article "Interrupt Politely", which outlines four distinct techniques. Those four techniques can be distilled down to the two major techniques: The first is to use asynchronous, forced cancellation such as thread-abort, process-kill, or AppDomain-unload. Any asynchronous technique has the common trait that the target of cancellation is caught completely unaware and cannot easily ensure data safety. For many reasons, but particularly due to the risk of partial updates to shared data, the use of asynchronous techniques is strongly discouraged.

The second general technique is cooperative whereby a request for cancellation is communicated to the target in some manner and the target of cancellation observes and enacts the request itself. This is inherently safer, but it does require that all targets of cancellation are fully participating in the process. The most commonly implemented style for this approach involves every potential target object tracking some state that represents whether cancellation has been requested although as we shall see this has some inherent problems and can be improved.

.NET libraries in general follow the cooperative approach and various examples can be found; for example, the BackgroundWorker class in Windows Forms has a member called CancellationPending which is set to true by calling the method
BackgroundWorker.CancelAsync(). Similarly, the classes based on System.Net.WebRequest provide an Abort() method to cancel the current operation and prevent further operations from commencing. A BackgroundWorker has a clear notion of the work that should be canceled, but a WebRequest may have either an active GetResponse() or GetRequestStream() call, and so the target of the cancellation request is not as clear. More generally, if a single object can be used to run distinct operations or if it can run many operations simultaneously, then it becomes very awkward to track cancellation requests on the object itself. And if the object allows the cancellation request to be reset either before or after enactment of a cancellation request, then there are inherent race conditions that can lead to subtle problems.

So although cooperative cancellation is far superior to asynchronous cancellation, the common approach has the following issues:

Data is often imbued with a notion of cancellation, and this is a problematic abstraction. A better model is that specific occurrences of an operation may be cancelable, but data is just data.
Cancellation achieved by direct calls to Cancel() and Abort() methods on specific targets is not groupable or chainable.
It is often necessary to perform some work in addition to setting a Boolean flag when enacting cancellation, but this support is not standardized or ubiquitous.
If an operation on an object makes a callback to user code, it is often awkward to forward the cancellation request made on the parent object to the user code.
If a class tracks cancellation state but allows multiple concurrent operations, or if it allows operations to commence after an earlier operation was canceled , then race-conditions become an inherent issue that frequently leads to subtle bugs.
When cancellation is requested, identified and enacted, the resulting behaviors are not consistent. Some methods return as if nothing happened, some throw OperationCanceledException, some throw custom exceptions, and some return a custom error code.

It is to address these issues that .NET 4 introduces a new approach to cooperative cancellation. And as we will examine in a future blog post, this approach is perfectly suited to the needs of Parallel Extensions and has been adopted across all our types.

New cancellation types

The new types provide a cooperative cancellation framework that is rich and general purpose. Some specific design principles of the new approach are:

Only specific framework types should represent a request for cancellation.
Long-running or blocking methods should accept an explicit parameter that will inform the method of a cancellation request. This is in contrast to mechanisms whereby operations automatically attach themselves to an ambient cancellation-request object, such as one that has been stashed in thread-local-storage. Mechanisms that involve automatic discovery and attachment generally become awkward when the exact specification of the rules are fleshed out.
The response made when cancellation is enacted should be to throw a consistent exception type.

Two new types form the basis of the framework: A CancellationToken is a struct that represents a ‘potential request for cancellation’. This struct is passed into method calls as a parameter and the method can poll on it or register a callback to be fired when cancellation is requested. A CancellationTokenSource is a class that provides the mechanism for initiating a cancellation request and it has a Token property for obtaining an associated token. It would have been natural to combine these two classes into one, but this design allows the two key operations (initiating a cancellation request vs. observing and responding to cancellation) to be cleanly separated. In particular, methods that take only a CancellationToken can observe a cancellation request but cannot initiate one.

When a call is made to CancellationTokenSource.Cancel(), the IsCancellationRequested property of any associated CancellationToken will become true and methods can poll on this to observe the cancellation request. If polling is not possible or not convenient, a callback can be attached via CancellationToken.Register(). A CancellationToken can be copied and passed around and all the copies will simply point back to the original CancellationTokenSource which holds all the actual state regarding cancellation requests, callback lists, etc.

When methods see that CancellationToken.IsCancellationRequested == true, they should respond to this by throwing an OperationCanceledException(cancellationToken). Throwing this exception communicates that the method finished without fully completing its stated purpose and states which CancellationToken it observed. Note that the existing OperationCanceledException class has been augmented with constructors and a property for CancellationToken.

We can now look at some specific examples: the first example shows how to create a CancellationTokenSource, pass its associated CancellationToken to a method, and how to poll on that token and correctly throw an OperationCanceledException. Note that cancellation doesn't need to be checked after every single statement, but it might be checked once per few thousand instructions or so. A general guideline is that cancellation should be checked as frequently as possible subject only to avoiding significant performance impact. Because polling on ct.IsCancellationRequested is very cheap (a single volatile read and a handful of IL instructions), cancellation can be tested frequently without a significant performance impact.

EventHandler externalEvent;
void Example1()
{
   CancellationTokenSource cts = new CancellationTokenSource();
   externalEvent +=
      (sender, obj) => { cts.Cancel(); }; //wire up an external requester
   try
   {
      int val = LongRunningFunc(cts.Token);
   }
   catch (OperationCanceledException)
   {
      //cleanup after cancellation if required...
   }
}

private static int LongRunningFunc(CancellationToken token)
{
   int total = 0;
   for(int i=0; i<1000; i++){
      for (int j = 0; j < 1000; j++)
      {
         total++;
      }
      if(token.IsCancellationRequested){ // observe cancellation
         throw new OperationCanceledException(token); // acknowledge cancellation
      }
   }
   return total;
}

The second example demonstrates using the callback facility on CancellationToken for when polling is not an option.

void BlockingOperation(CancellationToken token)
{
   ManualResetEvent mre = new ManualResetEvent(false);
   //register a callback that will set the MRE
   CancellationTokenRegistration registration =
      token.Register(() => mre.Set());
   using (registration)
   {
      mre.WaitOne();
      if (token.IsCancellationRequested) //did cancellation wake us?
          throw new OperationCanceledException(token);
   } //dispose the registration, which performs the deregisteration.
}

A common pattern in these situations is to use the callback to force whatever condition the blocking call is waiting on, and then immediately check to see if cancellation was the reason the blocking call was woken. Although there is a race condition here that normal waking may occur just before a cancellation request is made, this race is benign as it is always safe to respond to the cancellation request regardless of why the wait finished.

The third example shows how to listen for cancellation via a regular WaitHandle. Under the covers, CancellationToken.WaitHandle is a lazily-allocated ManualResetEvent that becomes set when cancellation is requested.

void Wait(WaitHandle wh, CancellationToken token)
{
   WaitHandle.WaitAny(new [] {wh, token.WaitHandle});
   if (token.IsCancellationRequested) //did cancellation wake us?
     throw new OperationCanceledException(token);
}

And finally, the fourth example shows running multiple asynchronous operations which share a common CancellationToken.

void Example4()
{
   CancellationTokenSource cts = new CancellationTokenSource();
   StartAsyncFunc1(cts.Token);
   StartAsyncFunc2(cts.Token);
   StartAsyncFunc3(cts.Token);
   //...
   cts.Cancel(); // all listeners see the same cancellation request.
}

In particular it is interesting to note that each of the asynchronous functions might pass on the token to other methods and that everyone who is observing the token (or copies of the token) will see the cancellation request when cts.Cancel() is called.

Details

A few details are important for an understanding of the new approach.

A CancellationTokenSource may transition from non-canceled only once and cannot be reset. This prevents various race-conditions that arise deep inside method calls if reset were permitted. At an application level, however, reset functionality is often required and this is achieved by creating a new CancellationTokenSource to replace a used one.

A CancellationToken is a lightweight struct that includes only a single reference back to a CancellationTokenSource. As such, it is the same ‘weight’ as a normal object reference, but because it is a separate type, it can have its own API than provides only read operations.

When a callback is registered to a CancellationToken, the current thread's ExecutionContext is captured so that the callback will be run with the the exact same security context . The capturing of the current thread's synchronization context is optional can be requested via an overload of ct.Register() if required. Callbacks are normally stored and then run when cancellation is requested, but if a callback is registered after cancellation has been requested, the callback will run immediately on the current thread, or via Send() on the current SynchronizationContext if applicable.

When a callback is registered to a CancellationToken, the returned object is a CancellationTokenRegistration. This is a light struct type that is IDiposable, and disposing this registration object causes the callback to be deregistered. A guarantee is made that after the Dispose() method has returned, the registered callback is neither running nor will subsequently commence. A consequence of this is that CancellationTokenRegistration.Dispose()must block if the callback is currently executing. Hence, all registered callbacks should be fast and not block for any significant duration.

Advanced patterns

To finish this introduction to the new cancellation framework we can look briefly at two more advanced patterns of usage.

One situation that arises is that some methods wish to observe two separate tokens. The best approach here is to create a linked CancellationTokenSource that will be signaled if either of the source tokens has become signaled.

The method CancellationTokenSource.CreateLinkedTokenSource() is specifically designed to assist with this and registers the callbacks as required. For example:

void LinkingExample(CancellationToken ct1, CancellationToken ct2)
{
   CancellationTokenSource linkedCTS =
   CancellationTokenSource.CreateLinkedTokenSource(ct1, ct2);
   try
   {
      SlowFunc(linkedCTS.Token);
   }
   catch(OperationCanceledException oce)
   {
      if (ct1.IsCancellationRequested)
      {
         // ...
      }
      else if (ct2.IsCancellationRequested)
      {
         // ...
      }
   }
   linkedCTS.Dispose(); // clean up the linking. required.
}

By using cancellation linking, cancellation-aware methods need only take one CancellationToken parameter. Be aware however that because linking attaches callbacks onto the source tokens, it is very important to clean up properly, otherwise linkedCTS and the callback delegates cannot be garbage collected until the linked sources are also available for collection. This is achieve by explicitly disposing linkedCTS, or via the using() pattern.

Another scenario that arises is to allow library code and user code to cooperate when responding to cancellation. Consider a library method that accepts a CancellationToken, and then calls back to user code. If the user code responds to a cancellation request, it wants to communicate this cleanly to the library code. This is achieved by the user code throwing an OperationCanceledException that mentions the same CancellationToken that the library code was given. Typically the user code will know about the same CancellationToken by getting it through some side channel, e.g. via closure-capture, rather than the library method explicitly passing it on. For example:

private void RunQuery()
{
   int[] data = {1,2,3};
   CancellationTokenSource cts = new CancellationTokenSource();
   var query = data.AsParallel()
                .WithCancellation(cts.Token) // token given to library code
                .Select( (x) => SlowFunc(x, cts.Token) ); // token passed to user code
}

private int SlowFunc(int x, CancellationToken token)
{
   int result
   while(...)
   {
      if (token.IsCancellationRequested)
         throw new OperationCanceledException(token);
      ...
   }
   return result;
}

Notice that the library function (in this case a PLINQ query) and the user code are both observing the same token and that the user-code throws an OperationCanceledException that mentions this common token. This allows the library code to see that the user code has performed cooperative cancellation and has not simply thrown some arbitrary exception. The library may respond to this by behaving exactly as though it had seen the cancellation request itself, rather than thinking the user code failed unexpectedly. It also has the advantage that if the user code throws an OperationCanceledException for some unexpected reason (and mentions a different token or no token at all), it will not be mistakenly interpreted as cooperative cancellation. Because the token is tracked in the OperationCanceledException itself, there is no confusion about the reason for the exception being thrown.

Conclusion

The new cancellation types introduced to .NET 4 provide a new framework for building systems that have rich and consistent cancellation behaviors. This will specifically assist with GUIs, applications, and libraries that manage long running or blocking operations. Over time we expect to see more libraries migrate to this approach and reduce the variation and issues inherent in the current crop of cancellation solutions. Applications and 3rd-party libraries can use these new types to interoperate cleanly with Parallel Extensions and for their own purposes.

Samples for Parallel Programming with the .NET Framework 4

toub — Thu, 21 May 2009 03:36:00 GMT

Along with the release of the .NET Framework 4 Beta 1, we've just published a slew of samples that demonstrate using Parallel Extensions in a variety of ways. You can download these from Code Gallery at http://code.msdn.microsoft.com/ParExtSamples.

These samples include raytracers, a sudoku game, an image colorization algorithm, solvers for the n-queens problem, fractal image generators, and more. The samples include code written to use Parallel Extensions from C#, Visual Basic, F#, and C++/CLI. They include examples of using the Task Parallel Library, PLINQ, new coordination data structures, and even databinding a WPF UI to a collection that has data stored into it from a background thread.

On top of all of this, the samples include a project called "Parallel Extensions Extras", which features a healthy portion of extension methods for .NET 4 types, new data structures to augment what's being shipped in .NET 4, custom schedulers for TPL, and lots of other goodness. Definitely worth a perusal.

Enjoy!

Concurrent, Multi-core Programming on Windows and .NET

toub — Tue, 28 Oct 2008 08:51:00 GMT

Thanks to everyone who attended our PDC pre-conference session yesterday on parallelism and concurrency! We had a wonderful turnout at the event, and David, Joe, and I all had a terrific time. Attached to this post are the slides we presented.

(It turns out that the PDC site does allow you to submit an evaluation for a precon. If you attended, we'd love your feedback.)

Feedback requested: Enumerating Concurrent Collections

toub — Tue, 12 Aug 2008 18:51:18 GMT

The June 2008 CTP of Parallel Extensions contained a first look at some of the work we're doing to augment the .NET Framework with a set of additional coordination data structures that aid in the development of highly concurrent applications. This included two thread-safe collections, ConcurrentQueue<T> and ConcurrentStack<T>, and it's probably not giving away too much to say that we're working diligently on additional collections that aid in key scenarios for parallel programming. For the most part, these collections are similar to the exsiting non-thread-safe counterparts in the .NET Framework, except with APIs designed with concurrency patterns in mind, and of course implementations that tolerate multithreaded access. Some of these designs come naturally and, from an implementation standpoint, are easily enabled. However, there are times when we have to make significant trade-offs in functionality or performance in order to achieve certain designs, and for those situations, it's very important that we have a good understanding of the key scenarios and use cases in which the collections will be used.

One important design point we keep in mind for all of the collections we're working on is how to deal with concurrent modifications while a collection is being enumerated. With the standard collections in the .NET Framework today, modifications to a collection invalidate any outstanding enumerators. For example, consider the following code:

var q = new Queue<int>(new[] { 1, 2, 3, 4, 5 });
foreach (var item in q)
{
if (item <= 5) q.Enqueue(item * 6);
}

This will compile fine, but upon execution, the call to q.Enqueue inside of the foreach loop will throw an InvalidOperationException: "Collection was modified after the enumerator was instantiated." In contrast, if you change the Queue<int> to a ConcurrentQueue<int>, e.g.:

var q = new ConcurrentQueue<int>(new[] { 1, 2, 3, 4, 5 });
foreach (var item in q)
{
if (item <= 5) q.Enqueue(item * 6);
}

no exception will be thrown, and upon examination of the queue after the loop, you'll find it to contain 10 elements. This is an explicit design decision we've made to break the mold from the existing collections, the idea behind such a change in direction being that there are key scenarios where one thread needs to be enumerating while other threads are reading and writing to the collection (of course, as shown in this example, it could be the same thread reading and writing as the one enumerating).

This does raise some questions, of course, especially around the guarantees that can be provided for such an enumeration. If data is added to the list while another thread is enumerating, must that enumerating thread see the new data? If data is removed while another thread is enumerating, must that enumerating thread not see the new data? Is it acceptable for data to be observed twice in the same enumeration or not at all due to the organization of the collection being changed concurrently? And so forth. For some collections, like ConcurrentStack<T> and ConcurrentQueue<T>, we're able to provide moment-in-time snapshot semantics, but for other collection types, doing so isn't necessarily possible without incurring serious performance penalties.

Given this, we're very curious to hear your feedback on a few things:

Given the standard array of collections that exist in .NET, if you had thread-safe versions of them, are there scenarios where you would benefit from being able to enumerate concurrently with other threads modifying the same collection? We already know about some, but we'd love to hear about more.
Assuming you do have scenarios for #1, what are the minimum guarantees you'd need about the data returned in the enumerator for it to be useful? For example, you could say for a thread-safe dictionary that if there are no concurrent modifications (adds/updates/removals), you'll get back in the enumeration exactly what's in the dictionary, and if there are concurrent accesses, you'll never get back something that wasn't in the dictionary, you may not see concurrent adds or updates, and you may still see items that are concurrently removed.
If we did support enumeration on collections that were thread-safe in the face of concurrent modifications, would you ever want the ability to revert to the "throw on modifications" behavior?
Finally, what are the most important collections that you'd like to see thread-safe and scalable counterparts for?

Thanks in advance for any input you may have!

Waiting for Tasks

toub — Wed, 06 Aug 2008 04:52:00 GMT

Parallel Extensions makes it easy to wait for Tasks to complete. Task exposes a Wait method, which can be used trivially:

Task t = Task.Create(...);
...
t.Wait();

Task also exposes several static methods for waiting on an array of tasks, either for all of them to complete or for any of them to complete:

Task t1 = Task.Create(...);
Task t2 = Task.Create(...);
Task t3 = Task.Create(...);
...
Task.WaitAll(t1, t2, t3); // or Task.WaitAny(t1, t2, t3)

Moreover, Tasks have implicit parent/child relationships, and a parent Task will wait for any of its child tasks to complete before it completes:

Task p = Task.Create(delegate
{
    Task c1 = Task.Create(...);
    Task c2 = Task.Create(...);
    Task c3 = Task.Create(...);
});
...
p.Wait(); // will not wake up until c1, c2, and c3 have finished

Sometimes, however, you want to wait for a list of tasks to complete, but you don't necessarily want to have to hold onto references to all of the relevant Task objets. Imagine a scenario where I'm creating tasks for each of an unknown number of data elements to be processed:

IEnumerable<Data> data = ...;
List<Task> tasks = new List<Task>();
foreach(var item in data) tasks.Add(Task.Create(delegate { Process(item); });
Task.WaitAll(tasks.ToArray());

The code is straightforward, but it also requires that I hold onto references to all of the Tasks I've created so that I can wait for all of them. It's interesting to note, however, that a task that has already completed doesn't really need to be waited on. As an example, if I create 10 tasks and 9 of them complete, waiting on the remaining 1 task is sufficient to know that all 10 completed. This means that I only really need to wait on a count of tasks, where each new task increases the count, and where each task completing decreases the count: when the count reaches 0, all tasks have completed.

Sound familiar? This is the domain of a CountdownEvent, which we can use to codify this approach:

IEnumerable<Data> data = ...;
using(CountdownEvent tasksRemaining = new CountdownEvent(1))
{
    foreach(var item in data)
    {
        tasksRemaining.Increment();
        Task.Create(delegate
        {
            Process(item);
            taskRemaining.Decrement();
        });
    }
    tasksRemaining.Decrement(); // to counteract the initial 1
    tasksRemaining.Wait();
}

The CountdownEvent is initialized to a count of 1 to ensure that the event doesn't become signaled before I'm done creating all of the tasks; I remove that 1 count when all tasks have been created. Before creating each Task, I up the count by 1, and when each task finishes, I decrease the count by 1. Then I just wait on the CountdownEvent, which will be signaled when all of the tasks have completed.

Of course, this requires that I incorporate such logic directly into my code, but there's really no need for that. I can extract out this logic into a separate class:

public class TaskWaiter
{
    public CountdownEvent _ce = new CountdownEvent(1);
    private bool _doneAdding = false;

    public void Add(Task t)
    {
        if (t == null) throw new ArgumentNullException("t");
        _ce.Increment();
        t.ContinueWith(ct => _ce.Decrement());
    }

    public void Wait()
    {
        if (!_doneAdding) { _doneAdding = true; _ce.Decrement(); }
        _ce.Wait();
    }
}

Rewriting the previous example is now almost identical, just substituting TaskWaiter for the List<Task>:

IEnumerable<Data> data = ...;
TaskWaiter tasks = new TaskWaiter();
foreach(var item in data) tasks.Add(Task.Create(delegate { Process(item); });
tasks.Wait();

Unlike the previous implementation that uses a List<Task>, this implementation now doesn't need to store references to any of the tasks. For just a few tasks, it's probably not warranted. But if lots of tasks are being create and/or any of the tasks will be long-running such that holding onto tasks could cause them to be promoted to higher GC generations, this could have a noticeable impact on the application's performance.

One neat thing to notice about the implementation of TaskWaiter: it uses ContinueWith. I previously wrote about useful abstractions enabled with ContinueWith, and this is another such abstraction. ContinueWith is used in order to get a bit of extra code to execute when a Task complete, such that the CountdownEvent is properly decremented. This allows me to move that Decrement call out of the Task's body itself (as I had with the original CountdownEvent-based implementation shown earlier), which means this technique can be used with any arbitrary Task you're handed, rather than one you have to create yourself.

There are some downsides to the TaskWaiter approach, however. The biggest that jumps to mind with the current implementation is that we lose the ability to propagate any exceptions that may have occurred from tasks tracked by TaskWaiter. We can fix that by storing references to tasks that complete with unhandled exceptions, and then do a true wait only on those tasks:

public class TaskWaiter
{
    public CountdownEvent _ce = new CountdownEvent(1);
    private bool _doneAdding = false;
    private ConcurrentQueue<Task> _faulted =
        new ConcurrentQueue<Task>();

    public void Add(Task t)
    {
        if (t == null) throw new ArgumentNullException("t");
        _ce.Increment();
        t.ContinueWith(ct =>
        {
            if (ct.Exception != null) _faulted.Enqueue(ct);
            _ce.Decrement();
        });
    }

    public void Wait()
    {
        if (!_doneAdding) { _doneAdding = true; _ce.Decrement(); }
        _ce.Wait();
        if (!_faulted.IsEmpty) Task.WaitAll(_faulted.ToArray());
    }
}

Notice the few changes. A ConcurrentQueue<Task> was added to track any tasks that completed due to exception; this collection needs to be thread-safe, as multiple tasks could fail on different threads and attempt to access the queue from different threads, hence the ConcurrentQueue<Task>. The delegate passed to ContinueWith has been augmented to check to see if the completing Task completed due to an exception, and if it did, the Task is added to the queue. Then in Wait, after waiting on the CountdownEvent, I use Task.WaitAll to wait on all of the tasks that completed due to exceptions. By this point, they've of course already completed, so I'm using Task.WaitAll simply to aggregate and throw all of the exceptions.

There's a subtlety about how I made these changes that's important to keep in mind. Specifically, it's crucial that the faulting task be added to the ConcurrentQueue<T> before the CountdownEvent is decremented; if those lines were reversed in the ContinueWith delegate, the CountdownEvent could become set before the Task was tracked in the queue, in which case the exceptions from that Task may be missed in the call to TaskWaiter.Wait, since the Task may not yet have made it into the queue by the time Task.WaitAll is called.

There are, of course, other optimizations that could be made to an implementation like this one, but we'll save those for another day.

Useful Abstractions Enabled with ContinueWith

toub — Thu, 24 Jul 2008 07:27:00 GMT

In the June 2008 CTP of Parallel Extensions to the .NET Framework, we introduced the ContinueWith method on both Task and Future<T>. ContinueWith is, in effect, a callback, very much like events in .NET. With events, a causal action results in the event being raised, which by default triggers all of the delegates registered with the event to be invoked. ContinueWith supports this, but rather than just registering the callback, it also provides back a Task or Future<T> that represents the callback; that returned Task or Future<T>'s lifecycle is tied specifically to the callback and won't be marked as IsCompleted until that callback completes. While at its core this could be considered a relatively simple idea and is useful for general dataflow in applications, it also enables some important patterns and can serve as the building block for a whole host of larger abstractions.

For example, the June 2008 CTP doesn't provide support for mulit-item continuations (is this something you think we should provide in a future release?): you can create a continuation Task or Future<T> for when one Task or Future<T> finishes, but out-of-the-box you can't create one for when all of multiple Task or Future<T>'s complete. Nevertheless, you can use the basic ContinueWith support to implement this additional support. Here's an example of a ContinueWith method you could implement, where you provide it with multiple Task instances, and the resulting Task will only be scheduled and executed when all of the provided tasks complete (I've ommitted parameter validation and the like to simplify the code):

static Task ContinueWhenAll(
    Action<Task[]> continuation, params Task[] tasks)
{
    var starter = Future<bool>.Create();
    var task = starter.ContinueWith(o => continuation(tasks));

    CountdownEvent ce = new CountdownEvent(tasks.Length);
    Action<Task> whenComplete = delegate {
        if (ce.Decrement()) starter.Value = true;
    };
    foreach (var t in tasks)
        t.ContinueWith(whenComplete, TaskContinuationKind.OnAny,
            TaskCreationOptions.None, true);

    return task;
}

This ContinueWhenAll method tasks two parameters: an Action<Task[]> to execute when all of the provided tasks have completed (supplied with those tasks), and the params array of the Tasks to monitor for completion. It then uses a workaround to create a Task that can be started at an arbitrary time in the future (in the June 2008 CTP, we don't directly support this capability, hence the workaround, but we're planning to in a future release as that capability is useful for a bunch of scenarios, including this one). A delegate is created that will count down from the number of provided tasks every time it's invoked, and when the count reaches 0, it will start the task. ContinueWith is then used to register that whenComplete action as a continuation for all of the parameter tasks. And voila, we now have a ContinueWhenAll method, which can be used like:

Task t1 = ..., t2 = ...;
Task t3 = ContinueWhenAll(
delegate { Console.WriteLine("t1 and t2 finished"); }, t1, t2);

Similarly, a method could be implemented for creating a task continuation for when any of a set of tasks completes (rather than when they all complete):

static Task ContinueWhenAny(
    Action<Task> continuation, params Task[] tasks)
{
    WriteOnce<Task> theCompletedTask = new WriteOnce<Task>();
    var starter = Future<bool>.Create();
    var task = starter.ContinueWith(o =>
        continuation(theCompletedTask.Value));

    Action<Task> whenComplete = t => {
        if (theCompletedTask.TrySetValue(t)) starter.Value = true;
    };
    foreach (var t in tasks)
         t.ContinueWith(whenComplete, TaskContinuationKind.OnAny,
            TaskCreationOptions.None, true);

    return task;
}

This implementation is very similar to ContinueWhenAll. However, the action that's registered as the continuation for each task uses a WriteOnce<T> to store the first Task to complete and to schedule the continuation task when it does. ContinueWhenAny can be used like:

Task t1 = ..., t2 = ...;
Task t3 = ContinueWhenAny(
delegate { Console.WriteLine("t1 or t2 finished"); }, t1, t2);

There are other neat patterns enabled with ContinueWith. One pattern that's starting to become popular with frameworks like the CCR or AsyncEnumerator is to take advantage of C#'s iterator support to make writing code that uses asynchronous operations a bit more like sequential code. While not the primary focus of the Task Parallel Library, ContinueWith can be used to implement such patterns in at least a limited capacity. Consider the following method:

static void RunAsync(IEnumerable<Task> iterator)
{
    var enumerator = iterator.GetEnumerator();
    Action a = null;
    a = delegate
    {
        if (enumerator.MoveNext())
        {
            enumerator.Current.ContinueWith(delegate { a(); });
        }
        else enumerator.Dispose();
    };
    a();
}

This method accepts an IEnumerable<Task>. It retrieves an enumerator from the enumerable and uses that enumerator in a delegate. The delegate moves the enumerator to its next element, and if there is a next element, retrieves it and uses ContinueWith to schedule the delegate (recursively, in a sense) for execution when that current Task completes. When the enumerator reaches the end, it's disposed. With that delegate created, RunAsync simply executes the delegate to get the execution started.

Now imagine I wanted a method that would asynchronously read a file, processing it as it's read in. I can implement such a method as follows:

static IEnumerable<Task> ReadFile(string path)
{
    using (FileStream fs = new FileStream(
         path, FileMode.Open, FileAccess.Read, FileShare.Read, 0x1000, true))
    {
        var enc = new UTF8Encoding();
        byte[] data = new byte[0x1000];
        while (true)
        {
            var pendingRead = CreateFutureFromApm<int>(ac =>
                fs.BeginRead(data, 0, data.Length, ac, null), fs.EndRead);
            yield return pendingRead;
            int bytesRead = pendingRead.Value;
            if (bytesRead <= 0) break;

            Console.WriteLine(enc.GetString(data, 0, bytesRead));
        }
    }
}

ReadFile opens a file stream and continually reads from it asynchronously. The code wraps usage of the FileStream's BeginRead/EndRead methods with a Future<int> (using the example method from http://blogs.msdn.com/8272833.aspx); that Future<int> is yielded from the ReadFile method (which returns an enumerator). We'll execute ReadFile by passing it to RunAsync. When the Future<int> completes, its continuation will be executed, which will cause the RunAsync iterator to MoveNext, thus ending up back in the ReadFile method just after the yield location. The ContinueWhenAll method can be used to allow more complicated asynchronous methods to be written, by wrapping multiple asynchronous invocations and yielding a continuation for when they all complete.

There are a myriad of such interesting patterns that ContinueWith enables. We'd love to hear about any useful ones you come up with.

More Channel 9 Parallel Extensions Goodness

toub — Sat, 07 Jun 2008 07:09:46 GMT

Charles from Channel 9 came over to building 112 last week for an in-depth look at what's new in the June 2008 CTP of Parallel Extensions. We spoke on the subject for an hour and a half or so, which you can see in a two-part series that was posted to Channel 9 yesterday (by the way, Channel 9 team, the new site looks great!):

What better way to spend some of this weekend than to enjoy discussions of Parallel Extensions. :) As always, we're very interested in your feedback, on the CTP itself but also on what we discussed in the videos and on what we should discuss in future videos.

Ray Tracer samples in the June 2008 CTP

mikelid — Thu, 05 Jun 2008 06:33:00 GMT

The June 2008 Community Technology Preview (CTP) of Parallel Extensions to the .NET Framework was released on Monday, and we’re really pleased at the level of excitement in the community that we’re seeing in response. As part of the CTP, we included a variety of demos and samples to help provide a tour of the functionality. If you haven't already, please read the blog entries: Released! Parallel Extensions to the .NET Framework June 2008 CTP, Known Issues in the June 2008 CTP of Parallel Extensions, and What's New in the June 2008 CTP of Parallel Extensions

In this post, I’m going to explore the four separate ray tracer implementations in the samples that demonstrate how to utilize Parallel Extensions to achieve concurrent rendering (note that the actual ray tracing engine itself is not an optimized implementation, as it is intended more for discussion than performance). There were two forms of the base sequential ray tracer, both developed by Luke Hoban, one written in C# using standard loops and other imperative programming constructs, and another implemented in C# via some huge LINQ queries that define the entire execution (read about this implementation at http://blogs.msdn.com/lukeh/archive/2007/04/03/a-ray-tracer-in-c-3-0.aspx)

The samples in the CTP extend these examples in the following ways:

1. The original C# ray tracer is converted to use Parallel Extensions (both the Task Parallel Library and some of the new Coordination Data Structures) to achieve concurrency. This enhanced version was then ported to Visual Basic and F#, demonstrating that Parallel Extensions is usable from any .NET language. See …\Samples\RayTracer\...

2. The LINQ-style ray tracer is converted to use PLINQ to achieve concurrency. See …\Samples\LINQRayTracer

These ray tracer samples involve a fair number of concurrency primitives provided by Parallel Extensions:

System.Threading.Parallel

System.Threading.Task

System.Threading.TaskManager

System.Threading.TaskManagerPolicy

System.Threading.LazyInit<T>

System.Threading.Collections.IConcurrentCollection<T>

System.Threading.Collections.ConcurrentQueue<T>

System.Linq.ParallelQuery

These samples also involve some standard .NET concurrency primitives and patterns:

System.Threading.Interlocked

System.Windows.Forms.Control.BeginInvoke

System.Threading.Monitor (via the lock {…} construct)

Let’s start by exploring the C# ray tracer. Load up Samples\RayTracer\C#\RayTracer.sln, compile and run. If you receive errors regarding System.Threading, please ensure that you have the .NET Framework 3.5 and the Parallel Extensions June 2008 CTP installed. Once the app is running, press the Start button and the ray tracer animation will run in single-threaded mode:

Note that the frames-per-second is displayed in the form’s title bar, and that there are controls to Start/Stop the animation and to turn on parallel rendering. The “Parallel” checkbox enables switching back and forth between sequential and parallel rendering, while the "Show Threads" option (only available when using parallel rendering) performs color-adjustment on the rendered pixels to indicate which thread each pixel was rendered on. Let’s turn on Parallel computation with "Show Threads" and observe the effect, shown here on a quad-core PC:

In this screenshot, we can clearly see four separate colorings, which represent the four underlying threads that are calculating pixel values for the rendered image. Notice also that the allocation of pixels to threads is row-based and that the partitioning of work is not uniform based on row-counts. This is due to Parallel.For()dynamically assigning work to the worker threads to ensure that the computational load remains. The application code itself specifies that the work should be split up row-by-row as this is a good balance between fine-grained work decomposition and the overheads inherent in spooling out small work items. That is, if the work was decomposed to per-pixel work-items, the extra overheads would negatively impact the performance. As it is, parallelizing the row-by-row decomposition provides a 3x improvement when the colorization is turned off. This is a good result, as there is a non-trivial amount of work to be done that cannot be parallelized – note that even though the timing is around the image calculation only, drawing the image to the screen and refreshing the form eats into the cycles available and affects the speed-up factor. If drawing the image is omitted, the speed up is between 3.5x and 4x as expected and hoped for. The moral here is that drawing more images to the screen increases the overheads of the parallel renderer and that we should always keep Amdahl's Law (http://en.wikipedia.org/wiki/Amdahl's_law) in mind when assessing parallel improvements.

The key rendering loops are RenderSequential(), RenderParallel(), and RenderParallelShowingThreads(), all of which appear in Raytracer.cs. Comparing the first two, we see that the only difference is a simple rewriting of the outer loops:

for(int y=0; y < screenHeight; y++){ /* funcBody */ }

Parallel.For(0, screenHeight, y => /* funcBody */ )

A critical part of any parallel application is that the concurrent work (such as the funcBody in the parallel version) is thread-safe. This means that TraceRay() and other work called in the parallel loop must not alter any shared-state unless appropriate thread-synchronization is used or if the updates are known to be safe. You will note that the funcBody's do alter a shared data-structure, rgb[], but that each element in this array will only be written to by a single execution of the funcBody and so no conflict will arise. The developer must always be vigilant that all concurrent work can occur without conflicts. If in doubt, simplify the concurrent work, use locks around risky code, or analyze the code and test to ensure corrrectness.

The application also makes use of a single System.Threading.Task to manage the whole process of running the animation, and to set the options regarding parallelism via a TaskManager. The TaskManager that is provided to this task become the TaskManager.Current for every task that is created inside, and so the configuration options also apply to the Parallel.For() that runs within the task (since Parallel.For() is built using Task, and Task instances by default use TaskManager.Current). Another useful feature of wrapping the animation loop in a task is to provide a simple means of tracking whether the animation is running, and to facilitate cancelling the animation. Diving in to btnStartStop_Click() in MainForm.cs, we see

_renderTask = Task.Create(RenderLoop,

chkParallel.Checked ? _parallelTm.Value : _sequentialTm.Value);

which creates the task to wrap the RenderLoop() and supplies a TaskManager to control the options for parallel execution. The first TaskManager, _parallelTm, will schedule one thread on every core of the machine, whereas _sequentialTm only uses one thread on one core in order to simulate a simple sequential implementation. Actually, each of these variables is actually a LazyInit<TaskManager> which ensure creation of the main type only if the variable is used. This is a minor usage of LazyInit<T> but the construct can be very useful for object creations that are expensive and avoidable; in this example, if you never run a parallel render, you’ll never instantiate the parallel TaskManager. Keep this handy construct in mind for objects that might go unused.

The code for handling the "stop-animation" action includes the following:

_renderTask.ContinueWith(delegate

{

BeginInvoke((Action)delegate

{

chkParallel.Enabled = true;

chkShowThreads.Enabled = chkParallel.Checked;

btnStartStop.Enabled = true;

btnStartStop.Text = "Start";

});

_renderTask.Cancel();

This sets a continuation-handler for the task that is wrapping the RenderLoop(), and this continuation cleans up the GUI and re-enables various controls. Notice that we must use MainForm.BeginInvoke() in order to run the clean-up code on the GUI thread, as tasks run on a background worker thread. After the cleanup continuation is registered, we call Task.Cancel(). Notice that the RenderLoop() is checking for t.IsCanceled in order to detect the cancellation and co-operatively bail out of its animation loop. The standard overload for Task.ContinueWith() will run the continuation whenever the task finishes, whether due to successful termination, failure or cancellation, but there are additional overloads to control when the continuation fires and other task-related options.

There is an interesting system used for managing the bitmaps used for rendering. In particular, allocation of the image bitmaps is expensive, but we don’t want to have to wait for the GUI to display a bitmap before we start computation of the next. One way to solve this is a classic double-buffered solution with one bitmap for scene-drawing and another that is ready for blitting to the screen (this was the approach taken in the sample included with the December 2007 CTP). A more general solution is to keep rendering images at the maximum rate and add them to a queue for blitting. This solution has been implemented by using a set of bitmaps for drawing, and each is sent for blitting by calling Form.BeginInvoke()with an appropriate delegate. In order to reuse bitmaps rather than creating and disposing them repeatedly, an ObjectPool<T> (see ObjectPools.cs) has been built around a ConcurrentQueue<T> that supports object reuse and creation of new objects if no reusable ones are available. The ConcurrentQueue<T> is particularly useful as it provides all the machinery required to make the ObjectPool<T> thread-safe.

The C# ray tracer has been ported to Visual Basic and F#, including all of the items discussed above except for image colorization. To build and run the F# version, download the F# languge support pack from http://research.microsoft.com/fsharp/fsharp.aspx. (To compile the sample, if you’re running on x64, you’ll need to modify the project settings to point to System.Core.dll in the correct location, as it’s a different path than if running on x86.) In particular, notice that F# can directly make direct use of the Parallel Extensions and also gets an excellent performance boost on multi-core PCs:

member this.RenderToArrayParallel(scene, rgb : int[]) =

Parallel.For(0, screenHeight, fun y ->

let stride = y * screenWidth

for x = 0 to screenWidth - 1 do

let color = TraceRay ({Start = scene.Camera.Pos; Dir = GetPoint x y scene.Camera }, scene, 0)

let intColor = color.ToInt ()

rgb.[x + stride] <- intColor)

Finally, a completely different ray tracer implementation is shown in Samples\LINQRayTracer.

When you run this ray tracer, you will notice that it has a more complicated scene and does not perform animation, but as it renders you will see the concurrency in effect as the rows get drawn. The original query was written in the form:

from y in Enumerable.Range(0, screenHeight)

…

select from x in Enumerable.Range(0, screenWidth)

which is a natural 2-D construct to loop over each pixel in each row. In order to make this run concurrently on a per-row basis (as for the other ray tracers), we only have to ask for the outer query to be run in parallel mode:

from y in Enumerable.Range(0, screenHeight).AsParallel()

…

select from x in Enumerable.Range(0, screenWidth)

This sets the query to use PLINQ to partition the work of outer query over all the available cores. The other adjustment is to enumerate the query via

pixelsQuery.ForAll(row =>

{

foreach (var pixel in row)

{

rgb[pixel.X + (pixel.Y * screenWidth)] = pixel.Color.ToInt32();

}

int processed = Interlocked.Increment(ref rowsProcessed);

if (processed % rowsPerUpdate == 0 ||

processed >= screenHeight) updateImageHandler(rgb);

});

which causes the query to run to completion in parallel. Notice that the per-row worker delegate includes thread-safe access to the rowsProcessed counter to be safe in concurrent execution.

What's New in the June 2008 CTP of Parallel Extensions

toub — Mon, 02 Jun 2008 18:45:00 GMT

We've just released a new community technology preview (CTP) of Parallel Extensions to the .NET Framework! You can download it from http://www.microsoft.com/downloads/details.aspx?FamilyId=348F73FD-593D-4B3C-B055-694C50D2B0F3. This release contains a plethora of bug fixes as well as some design changes to address some great feedback from you (note that there are some other changes you've requested and that we're planning to make in future releases, so stay tuned). It also contains a wealth of new functionality, some under the covers, some exposed through new APIs.

Task Parallel Library

The December 2007 CTP included an implementation of the Task Parallel Library (TPL) built on top of a prototype scheduler. As we’ve moved forward with Parallel Extensions, we’ve replaced this prototype with a new scheduler that is built from the ground up to be more robust, efficient, and scalable. The scheduler uses cooperative scheduling and work-stealing to achieve fast, efficient scheduling and maximum CPU utilization. The June 2008 CTP of Parallel Extensions includes this new scheduler, with TPL running on top of it.
While lots of work has been done with the internals of TPL, new functionality has also been exposed through its APIs. A new continuation model has been added to the System.Threading.Tasks.Task and System.Threading.Tasks.Future<T> classes through a new set of ContinueWith methods on both Task and Future<T>, enabling new tasks to be scheduled when previous tasks complete. This is especially powerful for Future<T>, where the continuations are provided with the completed Future<T> and thus with access to its Value, enabling dataflow-oriented solutions, e.g.:

var futureC = Future.Create(() => A()).
                     ContinueWith(a => B(a.Value)).
                     ContinueWith(b => C(b.Value));

Continuations can be scheduled to run under a variety of circumstances, such as when a task completes due to an unhandled exception.

Other new functionality includes a new set of WaitAny methods on Task that serve as counterparts to Task’s WaitAll methods (TaskCoordinator, which existed in the December 2007 CTP as the base class for Task and which exposed the WaitAll methods, has been removed). WaitAny enables a developer to wait for any one of a set of tasks to complete, which can be useful for a variety of scenarios, including speculative execution. Consider a problem where multiple heuristics can be used to solve the problem, but it’s not known a priori which algorithm will solve the problem the fastest. If it’s not feasible to parallelize each of the algorithms, the algorithms could all be run in parallel with one another, and the developer can use WaitAny to wait for the first algorithm to complete, e.g.:

var algorithms = new[] {
    Future.Create(() => Alg1()) ,
    Future.Create(() => Alg2()) ,
    Future.Create(() => Alg3()) ,
    Future.Create(() => Alg4()) };
var result = algorithms[Task.WaitAny(algorithms)].Value;

Several new properties have emerged on the Task class. The Name property that appeared in the December 2007 CTP has been replaced by an Id property, and a Creator property has been added. Creator returns the Task that was current when the Task was created. Creator is similar but is not always identical to the Parent property. New in this CTP is an improved notion of task hierarchies and relationships between parent and child tasks. By default, a Task is parented to the Task that was current when it was created; in this case, both Parent and Creator will return the same Task instance. However, Tasks can be created detached from their creator (by specifying the TaskCreationOptions.Detached value), in which case Parent will return null. This is an important capability, because now by default parent tasks implicitly wait on their children before completing, enabling a more structured form of task parallelism and aiding in a developer’s ability to write a correct, task-based application.

A variety of additional and minor API changes have also surfaced in this CTP. The TaskManagerPolicy class contains new values, representing concepts such as the minimum number of processors to use, the ideal number of processors to use, the ideal number of threads to use per processor, the stack size to use per thread, and the thread priority to assign to threads. At a higher-level, the Parallel.Do method has been renamed to Parallel.Invoke. Significant effort has also been invested in fixing bugs in functionality.

PLINQ

The most important changes in this CTP are implementation details for improving the reliability and debuggability of PLINQ. One such change: whereas PLINQ in the December 2007 CTP release was built on top of the .NET ThreadPool, PLINQ in this CTP is now built on top of the Task Parallel Library. Note, however, that this implementation is minimalistic. The plan is for future releases to take advantage of more of the functionality the Task Parallel Library has to offer.

In addition to implementation changes, new APIs are present in this CTP. First and foremost, the APIs for specifying order-preservation have been revamped. In the previous CTP, order-preservation was enabled on a query using the ParallelQueryOptions enumeration, such as in this example written in Visual Basic:

Public Shared Function Scramble(ByVal words As String()) As String()
    Return (From word In words.AsParallel(ParallelQueryOptions.PreserveOrdering) _
               Select ScrambleWord(word)).ToArray()
End Function

The ParallelQueryOptions enumeration has been removed from the API set, and has been replaced by a new AsOrdered extension method, which can be used as follows:

Public Shared Function Scramble(ByVal words As String()) As String()
    Return (From word In words.AsParallel().AsOrdered() _
               Select ScrambleWord(word)).ToArray()
End Function

An AsUnordered extension method also exists to enable the introduction of “shuffle points” into a query. If a certain portion of a query must be processed with a specific ordering, AsOrdered or OrderBy can be used to ensure that ordering is preserved, while AsUnordered can be used to in effect turn off ordering for portions of the query where performance is more important than ordering.

Other API level changes include the introduction of a new extension method, Zip:

public static IParallelEnumerable<TResult> 
    Zip<TFirst, TSecond, TResult>(
      this IParallelEnumerable<TFirst> first, IEnumerable<TSecond> second,
      Func<TFirst, TSecond, TResult> resultSelector);

Zip “zips together” two enumerables, creating a pairing of an item from each enumeration that is then passed to a function. An enumerable containing the results of that function is returned. This CTP also includes a parallel implementation of the SequenceEquals extension method, which was omitted from the previous CTP. Additionally, the Cast and OfType operators have been updated to conform to a breaking change in LINQ-to-Object’s implementation of these operators that’s available in the .NET Framework 3.5 SP1 (more information about this change is available at http://blogs.msdn.com/ed_maurer/archive/2008/02/16/breaking-change-in-linq-queries-using-explicitly-typed-range-variables.aspx).

Coordination Data Structures

The June 2008 CTP introduces a set of coordination data structures that complement PLINQ and the Task Parallel Library. The System.Threading namespace of the .NET Framework 3.5 already contains a handful of synchronization primitives, such as events, monitors, and mutexes, which are low-level constructs useful in developing multithreaded applications. Many parallel applications, however, would benefit greatly from higher-level constructs such as thread-safe collections, more sophisticated locking primitives, data structures to facilitate work exchange, and types that control how variables are initialized. Parallel Extensions adds a plethora of such constructs to the System.Threading and System.Threading.Collections namespaces. The types included in this CTP are:

System.Threading.CountdownEvent
System.Threading.LazyInit<T>
System.Threading.ManualResetEventSlim
System.Threading.SemaphoreSlim
System.Threading.SpinLock
System.Threading.SpinWait
System.Threading.WriteOnce<T>
System.Threading.Collections.BlockingCollection<T>
System.Threading.Collections.ConcurrentQueue<T>
System.Threading.Collections.ConcurrentStack<T>