Parallel Programming with .NET : Parallel Extensions

Are you using Parallel Extensions? We'd love to know.

toub — Thu, 17 Dec 2009 04:06:00 GMT

Are you using Parallel LINQ (PLINQ), the Task Parallel Library (TPL), or any of the new coordination and synchronization primitives in .NET 4 (or in the Parallel Extensions June 2008 CTP or with the recent Reactive Extensions release)? Are you planning to use or are you already using this support in a production application or library?

We'd love to hear about it, and if you have time, what your experiences have been (good or bad). Please email me at stoub at microsoft dot com... we're looking forward to hearing from you. Thanks!

Updated Beta 2 samples for parallel programming

toub — Mon, 07 Dec 2009 03:59:00 GMT

We've refreshed our Beta 2 samples for parallel programming with the .NET Framework 4. Thanks to the gracious assistance of the fabulous Lisa Feigenbaum and others on the Visual Basic team, in this refresh the majority of the samples are now available not only in C# but also in Visual Basic. The samples are available for download at http://code.msdn.microsoft.com/ParExtSamples.

Enjoy!

PDC09 Parallelism Session Videos Now Available

toub — Sat, 21 Nov 2009 09:05:20 GMT

Attendees at PDC09 this past week were privy to quite a few sessions on parallel computing. Now that the videos of these sessions are online, you can view them as well from the comfort of your own home. Here are some of the key parallelism-related sessions from this past week:

Overview

The State of Parallel Programming

Managed code in Visual Studio 2010

Native code in Visual Studio 2010

HPC Server

Accelerating Applications Using Windows HPC Server 2008

Research and Incubation

Enjoy!

Reactive Extensions and Parallel Extensions

toub — Fri, 20 Nov 2009 02:36:27 GMT

It’s awesome to see the Reactive Extensions to .NET (Rx) live on the DevLabs site. If you haven’t checked out this exciting project, we urge you to do so. Not only is it cool and useful technology, but the download includes a back ported (and unsupported) release of Parallel Extensions for the .NET Framework 3.5 in the form of System.Threading.dll. Rx relies on this, and you can also use it to try out our new parallel programming support for .NET 4 but with your .NET 3.5 projects.

I recently sat down with Wes Dyer from the Rx team to discuss the Reactive Extensions integration with Parallel Extensions. You can view a video of that discussion at Wes Dyer and Stephen Toub: Rx and Px - Working Together.

While unsupported, the 3.5 back port does contain all of the types and constructs present in the .NET 4 supported release. In fact, using the back ported version is very similar to using the bits in .NET 4, albeit with a few known differences:

In order to use the features with .NET 3.5 applications, you will need to add a reference to the back ported standalone DLL named System.Threading.dll. This additional step is not present with .NET 4, since the parallel technology is baked into core CLR components.
The performance of the parallel types is optimized for and predicated on the .NET 4 Framework. You should not assume that an application using the .NET 3.5 version will exhibit similar performance characteristics to one using the supported .NET 4 version. Parallel Extensions extensively leverages advancements made in the .NET 4 ThreadPool and other performance optimizations present in .NET 4.
Unlike in .NET 4, the OperationCanceledException type does not have a constructor that accepts a CancellationToken, nor does the exception have a CancellationToken property. In order to throw an exception to cooperatively acknowledge cancellation within a Task, you may use the ThrowIfCancellationRequested method present on the CancellationToken type. Refer to MSDN documentation for more details.
To handle exceptions thrown due to cancellation, users should catch the OperationCanceledException. Although the exception thrown is of type OperationCancelledException2, users will not be able to catch this type as it is an internal exception derived from OperationCanceledException (necessary as part of the back port to maintain consistency).
The Parallel Task and Parallel Stack tool windows in Visual Studio 2010 will not highlight tasks from the .NET 3.5 back port. These features require .NET 4.
The concurrency visualizations in the Visual Studio 2010 profiler will not show parallel region markers for parallel loops in the .NET 3.5 back port. This is due to the .NET 3.5 version not outputting any of the ETW events output by the .NET 4 release.

Even with these limitations, we hope you enjoy the release! You can also expect further integration between Rx and Parallel Extensions in future drops on the DevLabs site.

Microsoft Biology Foundation now using Parallel Extensions

toub — Fri, 13 Nov 2009 19:32:14 GMT

The Microsoft Biology Initiative (MBI) is a combined project in Microsoft Research focused on two components: the Microsoft Biology Foundation (MBF) and the Microsoft Biology Tools (MBT). Both the framework and the tools are related to the areas of computation biology, genomics, and bioinformatics. MBF is a language-neutral bioinformatics toolkit built as an extension to the Microsoft .NET Framework. Currently it implements a range of parsers for common bioinformatics file formats; a range of algorithms for manipulating DNA, RNA, and protein sequences; and a set of connectors to biological Web services such as NCBI BLAST.

Microsoft Research has released Beta 1 of MBF, including a version that runs on .NET 4 Beta 2 and that’s using Parallel Extensions to provide impressive speedups on some new algorithms that have been added in the release. From the release notes:

“Support for .NET Parallel Extensions. In addition to the complete Beta 1 version of the project being released, we are also providing the Beta 1 - Dev10 Preview release. This version of the code and binaries are utilizing the latest in technology improvements in the .NET 4.0 and Visual Studio 2010 Beta 2 release. The inclusion of a novel multiple sequence alignment algorithm, PAMSAM, provided as an example of how this technology can be used to turn a commodity desktop computer into a valuable research asset.”

If you’re interested in bioinformatics, definitely check this out.

What's New in Beta 2 for PLINQ

essey — Thu, 05 Nov 2009 20:37:00 GMT

Included in the .NET 4 Framework Beta 2 is a more robust and faster version of PLINQ. Between B1 and B2, PLINQ changes have mainly been under the covers, so hopefully no need to rewrite any of your applications to see the improvements.

1. Many improvements to performance and scalability

2. GroupBy and GroupJoin now preserve ordering within the groups

3. Better integration with performance profiling, which now shows PLINQ markers

4. Cancellation consistently works across all PLINQ operators

5. Scoped some of the boundary conditions of PLINQ in .NET 4 in Beta2 of Parallel Extensions as follows:

a. Max Degree of Parallelism changed from 64 to 63

b. Greater consistency regarding maximum input length

i. PLINQ does not support inputs of length > Int32.MaxValue. Queries may throw OverflowException for many operators.

ii. LongCount no longer counts beyond Int32.MaxValue

6. Improved robustness

a. Reliability and concurrency focus

b. More consistent exception handling and error model

7. Other bug fixes

If any of these severely impact your applications, particularly the boundary conditions, please let us know so that we can include that in our planning efforts. For now, we found that it makes the most sense to scope some inputs and give a reliable and predictable experience in this release.

Happy Coding!

What’s new in Beta 2 for the Task Parallel Library? (Part 3/3)

dashih — Wed, 04 Nov 2009 21:52:00 GMT

Last time, we covered Tasks being detached by default and some refactorings in our multiple-Task continuation APIs. The final post of this series will discuss Nested Tasks and Unwrap, a Parallel namespace change, and some changes under the covers.

Nested Tasks and Unwrap

We’ve added the Unwrap APIs to address scenarios that deal with nested Tasks. Before jumping into Unwrap, let’s first talk about nested Tasks, e.g. a Task<Task> or Task<Task<TResult>>. For example:

var nestedTask = Task.Factory.StartNew(() =>
{
    return Task.Factory.StartNew(() =>
    {
        return 42;
    });
});

Nested Tasks commonly lead to unexpected behavior in applications. For example, consider an API that provides the following functionality for asynchronously logging into a web service (like one from Facebook), retrieving a list of friends, and sending an email to each friend.

// Given a user name and password, returns a Task whose

// result is a UserToken object.

public Task<UserToken> LogOn(string username, string password);

// Given a UserToken, returns a Task whose result

// is a collection of Friend objects.

public Task<FriendCollection> GetFriendList(UserToken userToken);

// Given a Friend, subject, and body, returns a Task that

// represents an email sending operation.

public Task SendEmail(Friend friend, string subject, string body);

A user would like to be able to write code like the following, utilizing these APIs and Task continuations:

var userToken = LogOn(user, pass);

var friends = userToken.ContinueWith(_ => GetFriendList(userToken.Result));

var emails = friends.ContinueWith(_ =>

{

var sentMails = new List<Task>();

foreach(var friend in friends.Result)

{

sentMails.Add(SendEmail(friend, subject, body));

}

return Task.Factory.ContinueWhenAll(

sentMails.ToArray(), tasks => Task.WaitAll(tasks));

});

emails.ContinueWith(_ => Console.WriteLine(“All emails sent”));

The bolded code is problematic. Because the GetFriendList method returns a Task<FriendCollection>, the ‘friends’ variable is actually going to be a Task<Task<FriendCollection>>. This will cause a compiler error at the foreach loop, because ‘friends.Result’ will return a Task<FriendCollection> instead of a FriendCollection. The compiler error in this case is a good thing, of course, highlighting a programming error. However, once a developer realizes the type mismatch, he still has to code additional logic to unwrap the ‘friends’ Task so that it returns an actual FriendCollection. This logic is nontrivial, especially if it is to correctly deal with exceptions, cancellation, etc. The last line is also problematic. The emails variable here is actually of type Task<Task>. The outer Task will complete once the inner Task is returned from its body, even if the inner Task hasn’t completed. The net result of this is that “All emails sent” could be written out prior to all of the email tasks actually completing.

Now, you may have noticed that, in Beta 1, we provided special ContinueWith overloads to deal with this type of scenario. For example:

public Task<TResult> ContinueWith<TResult>(

Func<Task, Task<TResult>> continuationFunction);

The Func returns a Task<TResult>, so normally, ContinueWith would return a Task<Task<TResult>>. But this ContinueWith overload does some magic under the covers to return a Task<TResult>. There were a number of reasons why we didn’t like this approach, including:

· Too much magic. It’s hard to explain why one set of ContinueWith overloads is “just different”.

· Only works for ContinueWith. What if user scenarios result in nested Tasks for other Task creation APIs like StartNew, ContinueWhenAll, etc.?

· What if nested Tasks are actually desired? If a user actually wants that Task<Task<TResult>>, he still might unknowingly bind to this magical overload.

Given these reasons, our solution for Beta 2 and beyond is two Unwrap extension methods.

namespace System.Threading.Tasks

{

public static class TaskExtensions

{

public static Task Unwrap(

this Task<Task> task);

public static Task<TResult> Unwrap<TResult>(

this Task<Task<TResult>> task);

}

These methods may be used to transform any Task<Task> or Task<Task<TResult>> into a Task or Task<TResult>, respectively. The transformation performed produces a Task or Task<TResult> that fully represents the original nested Task, including exceptions, cancellation state, etc.

With Unwrap, we can fix the above scenario (note the bolded).

var userToken = LogOn(user, pass);

var friends = userToken.ContinueWith(

_ => GetFriendList(userToken.Result)).Unwrap();

var emails = friends.ContinueWith(_ =>

{

var sentMails = new List<Task>();

foreach(var friend in friends.Result)

{

sentMails.Add(SendEmail(friend, subject, body));

}

return Task.Factory.ContinueWhenAll(

sentMails.ToArray(), tasks => Task.WaitAll(tasks));

}).Unwrap();

emails.ContinueWith(_ => Console.WriteLine(“All emails sent”));

Parallel Namespace Change

We’ve moved the Parallel class from the System.Threading namespace to the System.Threading.Tasks namespace. We found that most applications needed to bring in both namespaces when using TPL, so why not put everything into one namespace? Additionally, Parallel is such a common word (and will likely become more so in the future), and System.Threading such a common namespace, we wanted to reduce the chances of conflict with other .NET types as much as possible.

Here’s a useful IDE tip. Use “Ctrl + .” to automatically bring in the relevant namespace once you’ve typed a class/type name.

Under the Covers

Beta 2 contains quite a few bug fixes not done for Beta 1. All of them were important, but we’ll focus on only two here.

First, Parallel.For and ForEach have been tweaked for better load-balancing with other workloads in the current or other AppDomains. Essentially, the change was to service the parallel loops with Tasks that periodically retired and re-queued themselves, allowing other contenders to grab Threads and make progress.

Second, waiting for Tasks in parent/child relationships has become more efficient. In Beta 1 and before, parent Tasks waited for their children using explicit Waits, so even if a parent completed first, it would burn a thread until all of its children completed. In Beta 2, parent Tasks wait for their children using callbacks; the parent maintains a count for number of children it has, and each child decrements the count as it completes. This waiting logic significantly improves scalability.

That’s it folks! We hope you’ve enjoyed this series. Download Beta 2 and try it out!

Slides from Parallelism Tour

toub — Mon, 02 Nov 2009 02:58:00 GMT

Last week, I had the privilege of touring around Tennessee, Kentucky, Ohio, and Michigan, speaking about the new parallel computing support in Visual Studio 2010 and the .NET Framework 4. Many folks I spoke with were interested in getting a copy of the slide deck I used, so I’ve attached it to this blog post. Enjoy!

What’s new in Beta 2 for the Task Parallel Library? (Part 2/3)

dashih — Tue, 27 Oct 2009 18:02:00 GMT

Last week, we talked about how TPL adopted a new, better cancellation model. Today, we’ll cover a change that makes Tasks Detached by Default, some ContinueWhenAll/Any Refactoring, and the handy UnobservedTaskException event.

Tasks are Detached by Default

In Beta 2, we have changed an important default. Tasks are now created as detached (instead of attached) if no options specify otherwise.

Let’s consider the following code to review the difference between attached and detached Tasks.

Task p = Task.Factory.StartNew(() =>

{

Task c = Task.Factory.StartNew(() =>

{

DoWork();

});

p.Wait();

In Beta 1 and before, since the default options are used, ‘c’ is created as a child Task of Task ‘p’, the parent Task; we refer to this as Task ‘c’ being “attached” to Task ‘p’. This means that the p.Wait() statement will not return until the call to DoWork completes, because parent Tasks do not complete until all of their child Tasks complete. To opt out of this behavior, a user needs to create ‘c’ with the DetachedFromParent option:

Task c = Task.Factory.StartNew(() =>

{

DoWork();

}, TaskCreationOptions.DetachedFromParent);

The original code shown behaves differently in Beta 2. Now, by default, ‘c’ is not related to ‘p’ (it’s “detached” by default), and the p.Wait() statement will return as soon as ‘p’ completes, regardless of the status of Task ‘c’ and thus regardless of when DoWork returns. To opt in to the parent/child relationship, a user needs to create ‘c’ with the AttachedToParent option:

Task c = Task.Factory.StartNew(() =>

{

DoWork();

}, TaskCreationOptions.AttachedToParent);

Here is a summary of the changes:

· Removed the DetachedFromParent option

· Added the AttachedToParent option

· Changed the default behavior so that Tasks do not enlist in parent/child relationships when no options are specified.

There were a number of reasons why we decided that detached is the correct default and to move forward with this change, including:

· Many users were using attached Tasks unknowingly. The vast majority of the time, users create Tasks for simple, asynchronous work. In such scenarios, parent/child relationships (and the implicit waiting) are not needed. We found through many interactions that folks were just going with the default options and were accidentally opting in to this behavior. In the best case, this would only result in a slight performance cost. In the worst case, this would bring with it incorrect behavior that would lead to difficult to diagnose errors.

· Easier migration from ThreadPool.QueueUserWorkItem. Tasks are now the recommended way to queue work to the ThreadPool, but the easiest way to create Tasks resulted in different behavior from QueueUserWorkItem (where there’s no concept of parent/child work items). This change makes Task.Factory.StartNew (with no options) a true replacement for QueueUserWorkItem.

· Additional behavior should be opt-in and pay-for-play. Almost everything in TPL that results in additional behavior is opt-in, e.g. cancellation, LongRunning, PreferFairness. With the Beta 1 default, users opt-out of parent/child relationships. In Beta 2, users opt-in, making it consistent. This makes the extra functionality provided by parent/child relationships pay-for-play, such that you don’t pay the cost for parents implicitly waiting for their children or for exceptions propagating from children to parents unless you need that functionality.

ContinueWhenAny/All Refactoring

We have refactored the set of ContinueWhenAny and ContinueWhenAll overloads to make things more intuitive, consistent, and complete.

To demonstrate the main issue, let’s consider the following overload that was provided in Beta 1.

public class TaskFactory<TResult>

{

public Task<TNewResult> ContinueWhenAny(

Task<TResult>[] tasks,

Func<Task<TResult>, TNewResult> continuationFunction);

}

This confused the meaning of TaskFactory<TResult>, which is meant to create tasks of type Task<TResult>. However, with these overloads, TaskFactory<TResult> could be used to create tasks of type Task<TNewResult>. As an example, consider the code:

Task<int>[] taskOfInts = ...;
Task<string> t = Task<int>.Factory.ContinueWhenAll(taskOfInts, _ => “”);

This compiles and works just fine, but the type parameter mismatch (shown in bold) is certainly odd. To address this, we changed a bunch of overloads, so that instead of taking Task<TResult>s and returning a Task<TNewResult>, they take Task<TAntecedentResult>s and return Task<TResult>s. For example, the overload that replaced the above is:

public Task<TResult> ContinueWhenAny<TAntecedentResult>(

Task<TAntecedentResult>[] tasks,

Func<Task<TAntecedentResult>, TResult> continuationFunction);

And the above example becomes:

Task<int>[] taskOfInts = ...;
Task<string> t = Task<string>.Factory.ContinueWhenAll(taskOfInts, _ => “”);

In addition to this change, we also added, removed, or modified a number of other overloads to make the set consistent and complete. Now, the entire set of ContinueWhenAll and ContinueWhenAny overloads follow these clear rules:

· A TaskFactory creates Tasks, but also provides overloads to create Task<TResult>s.

· A TaskFactory<TResult> only ever creates Task<TResult>s (never Tasks or Task<TNewResult>s).

UnobservedTaskException event

We’ve added an event that fires for every Task exception that goes unobserved. Recall that to “observe” a Task’s exceptions, you must either Wait on the Task or access its Exception property after it has completed. At least one of these actions must be done before the Task object is garbage collected, or its exceptions will propagate (currently this occurs on the finalizer thread).

The new static event resides on the TaskScheduler class, and subscribing to it is straightforward. Here’s an example to log all unobserved exceptions and mark them as observed (preventing them from being propagated).

TaskScheduler.UnobservedTaskException +=

(object sender, UnobservedTaskExceptionEventArgs exceptionArgs) =>

{

exceptionArgs.SetObserved();

LogException(exceptionArgs.Exception);

};

Some customers have complained that TPL’s exception policy is too strict. The UnobservedTaskException event provides an easy way out by allowing you to simply squash all Task exceptions in an application (though using it in this manner is not recommended). The primary reason that we made the addition was to support host-plugin scenarios where a host application can still be perfectly useful in the presence of some truly harmless exceptions (thrown by buggy plugins). These scenarios may be achieved using the UnobservedTaskException event in conjunction with AppDomains to sandbox plugins. Look for a future post that describes this in more detail!

We’re done for now! The 3^rd and final post of this series will cover the new Unwrap APIs, a Parallel namespace change, and some changes under the covers.

What’s new in Beta 2 for the Task Parallel Library? (Part 1/3)

dashih — Mon, 19 Oct 2009 19:42:00 GMT

Visual Studio 2010 and .NET 4 Beta 2 is here! In terms of completeness and readiness for production coding, Beta 2 promises to be much better than Beta 1, and TPL is one component that delivers significant improvements over what was previously available. To get you excited about it, this series of posts details key additions and changes for Beta 2. Enjoy!

In this post, we’re talking about cancellation. A few months ago, our flurry of “What’s new in Beta 1” posts hinted at a new cancellation model and promised more information about it. We made good on that promise with .NET 4 Cancellation Framework, and this post goes further to explain how TPL, specifically, has fully adopted the new model.

The Old Way (Beta 1 and before)

Let’s consider the following code to review the old TPL cancellation model.

Task myTask = Task.Factory.StartNew(() =>

{

for (; ; )

{

if (Task.Current.IsCancellationRequested)

{

Task.Current.AcknowledgeCancellation();

return;

}

...

}

});

// Elsewhere.

myTask.Cancel();

myTask just loops infinitely, checking its IsCancellationRequested property to see if it has been canceled. Elsewhere, cancellation is requested on myTask using the Cancel method. At that point, myTask agrees to get canceled by calling its AcknowledgeCancellation method and returning. This is all necessary, because Task cancellation is cooperative; to enter the Canceled state, outside logic must request cancellation and the Task must acknowledge that cancellation request. However, note that cooperative cancellation is only relevant for already running Tasks. If a Task’s Cancel method is called before it is in the Running state, it will transition directly into the Canceled state.

Waiting on a canceled Task results in a TaskCanceledException wrapped in an AggregateException, hence the try/catch block. Executing this code prints “Canceled” to the console, indicating that myTask was successfully canceled.

This approach works, but we identified a number of problems with it including the following:

1. The cancellation model required exposing a Task.Current static property. This leaks implementation details from libraries that utilize Tasks internally; any code called from the Task can muck with the current Task or take dependencies on its existence. Imagine calling into 3^rd party code and having that code cancel your Tasks, schedule continuations off of them, etc.

2. Anyone with a Task’s reference can request cancellation on it. In many scenarios, it is valuable to separate the ability to check for cancellation and the ability to actually request cancellation.

3. From a cancellation perspective, Tasks didn’t compose well with other APIs that were cancelable, such as executing a cancelable PLINQ query inside of a Task.

4. Tasks were often more expensive than they needed to be, due to needing to track extra cancellation state per Task.

The New Way (Beta 2 and beyond)

The new TPL cancellation model is centered around two types: CancellationTokenSource and CancellationToken. You can read up about them in the post linked to above, but here is a simplified overview of their APIs:

namespace System.Threading
{
    public sealed class CancellationTokenSource
    {
        public void Cancel();
        public CancellationToken Token { get; }
        ...
    }

    public struct CancellationToken
    {
        public Boolean IsCancellationRequested { get; }
        public void ThrowIfCancellationRequested();
        ...
    }
}

Here’s the gist. A CancellationTokenSource contains a CancellationToken, and it can request cancellation on that token using a Cancel method. A CancellationToken can only check if cancellation has been requested on it. Ignore the ThrowIfCancellationRequested method for now; we’ll see why it’s handy later.

Adopting the new model involved not only adding support for these two types, but also ripping out the old model. Here’s a summary of the changes:

· All cancellation-related APIs on the Task class were removed (Cancel, AcknowledgeCancellation, IsCancellationRequested, etc)

· Other APIs that were no longer relevant were removed (Task.Current, Task.Parent, TaskCreationOptions.RespectParentCancellation, etc)

· Overloads that accept CancellationToken were added to many methods (StartNew, ContinueWith, etc)

And now, here’s a table that outlines how achieving cancellation in TPL has changed:

Action	Old Model	New Model

To set up cancellation	Just create a Task	Create a CancellationTokenSource and pass its Token to an API that creates a Task.
To check if cancellation has been requested	Check the IsCancellationRequested property on the relevant Task	Check the IsCancellationRequested property on the CancellationToken that was passed to the API that created the Task
To acknowledge cancellation	Check to ensure that IsCancellationRequested is true, then call the AcknowledgeCancellation() method on the relevant Task	Throw an OperationCanceledException with the task’s CancellationToken
To cancel a tree of tasks	Create all of the tasks as attached tasks and with the RespectParentCancellation flag set, then cancel the root task.	Pass the same CancellationToken to all Tasks, then cancel the associated CancellationTokenSource

To see how this all works, let’s rewrite the code above for Beta 2.

CancellationTokenSource cts = new CancellationTokenSource();

CancellationToken token = cts.Token;

Task myTask = Task.Factory.StartNew(() =>

{

for (; ; )

{

token.ThrowIfCancellationRequested();

...

}

}, token);

// Elsewhere.

cts.Cancel();

A CancellationTokenSource (cts) is initialized, and a CancellationToken (token) is initialized to cts’s Token. myTask still loops indefinitely, but now it calls ThrowIfCancellationRequested. This method just checks the IsCancellationRequested property on a CancellationToken. If true, it throws an OperationCanceledException(token), which is the way to acknowledge cancellation in the new model. Elsewhere, Cancel is called on cts to request cancellation on myTask.

The new model addresses all of the problems with the old model listed above:

1. No more Task.Current (Task.Parent was removed too, as it could be used to get at the current Task).

2. The ability to check for cancellation requests may now be separated from the ability to request cancellation. If only the former is desired, grant access to the CancellationToken only.

3. TPL now uses the new .NET 4 cancellation types. You can use the same CancellationToken{Source} to effect cancellation on multiple Tasks, Parallel.For calls, PLINQ queries, and more, all at once. This provides both better composition and better performance.

Finally, there are other advantages to this model. One is that it is much easier to organize cancellation sets within a large nest of Tasks; just create the right number of tokens and pass them around accordingly. Before, this scenario would have required tedious bookkeeping of many Task references and/or careful architecting of cancellation chains using TaskCreationOptions.RespectParentCancellation.

That’s it for now. The next post in the series will discuss a change regarding Tasks and parent/child relationships.

.NET 4 Beta 2 is here!

toub — Mon, 19 Oct 2009 18:50:05 GMT

The .NET Framework 4 Beta 2 is now available!

MSDN Subscribers can download it today, and it will be generally available for download on Wednesday. More information is available at http://msdn.microsoft.com/en-us/vstudio/dd582936.aspx. Additionally, one of the really exciting things about this Beta release is that it’s “go-live”; more information on that is available at “Going live” with Visual Studio 2010 Beta 2.

As part of .NET 4 Beta 2, of course, comes improved parallelism support. In the coming days and weeks, we’ll be blogging here about some of the exciting updates and additions in this release. Stay tuned…

In the meantime, the MSDN documentation on .NET 4’s parallelism support has been updated. It’s a great place to start learning about all of the functionality available in this release.

You can also learn more about the tools in Visual Studio 2010 for parallel development in the documentation at:

Enjoy!

Parallel Computing Presentations in Michigan, Ohio, Kentucky, and Tennessee

toub — Fri, 16 Oct 2009 15:49:29 GMT

In a week, I’m going to be traveling through Michigan, Ohio, Kentucky, and Tennessee, speaking about parallel computing, Visual Studio 2010, and .NET 4, primarily at corporations during the day and at user groups in the evenings.

If you’re in the area and interested, please do attend, and I look forward to meeting you! A list of events is available at http://blogs.msdn.com/jennifer/archive/2009/10/14/stephen-toub-parallel-computing-tour.aspx.

See you in a week!

Stephen

Task.Wait and “Inlining”

toub — Thu, 15 Oct 2009 15:45:59 GMT

“What does Task.Wait do?”

Simple question, right? At a high-level, yes, the method achieves what its name implies, preventing the current thread from making forward progress past the call to Wait until the target Task has completed, one way or another. If the Task ran to completion, Wait will return successfully. If the Task completed due to faulting or being canceled, it will throw an exception indicating the relevant details (since the waiting code likely expects that all work to be completed by the task ran successfully if Wait returns successfully).

The details are a bit more interesting. Wait could simply block on some synchronization primitive until the target Task completed, and in some cases that’s exactly what it does. But blocking threads is an expensive venture, in that a thread ties up a good chunk of system resources, and a blocked thread is dead weight until it’s able to continue executing useful work. Instead, Wait prefers to execute useful work rather than blocking, and it has useful work at its finger tips: the Task being waited on. If the Task being Wait’d on has already started execution, Wait has to block. However, if it hasn’t started executing, Wait may be able to pull the target task out of the scheduler to which it was queued and execute it inline on the current thread.

To do that, Wait asks for assistance from the target scheduler, by making a call to the TaskScheduler’s TryExecuteTaskInline method. TryExecuteTaskInline can do whatever logic the scheduler needs in order to validate that the current context is acceptable for inlining the Task. If it is, the scheduler tries to execute the task then and there as part of the call to TryExecuteTaskInline (using the base TaskScheduler’s TryExecuteTask method) and returns whether the Task could be executed. If it couldn’t be executed in TryExecuteTaskInline, the scheduler returns false, and Wait will need to block until the Task completes. A Task may not be able to be executed if it’s already been or is currently being executed elsewhere. As an example of this, the default scheduler (based on the ThreadPool) is aggressive about inlining, but only if the task can be efficiently removed from the data structures that hold tasks internally in the ThreadPool (such as if the task is living in the local queue associated with the thread attempting to inline it).

This is also very similar to what happens when Task.RunSynchronously is used to execute a Task (RunSynchronously may be used instead of Task.Start to run a Task currently in the TaskStatus.Created state). RunSynchronously results in the Framework calling the target scheduler’s TryExecuteTaskInline, allowing the scheduler to decide whether to inline or not. If the scheduler chooses not to inline, the Task will instead be queued to the scheduler, and the system will block until the Task is completed.

Of course, schedulers may want to behave differently depending on whether the code is explicitly choosing to inline (e.g. RunSynchronously) or whether the inlining is happening implicitly (e.g. Wait). To support this difference, in addition to accepting as a parameter the Task to be inlined, TryExecuteTaskInline also accepts as a parameter a Boolean taskWasPreviouslyQueued. This value will be false if it is known that the specified task has not yet been submitted to the scheduler, and true otherwise. If RunSynchronously is being used, taskWasPreviouslyQueued will be false; if Wait, it will be true. The default scheduler does, in fact, differentiate between these two cases, in that it always supports explicit inlining through RunSynchronously.

Parallelized Map and Filter Operations

toub — Mon, 12 Oct 2009 16:03:38 GMT

Common operations like map and filter are available in parallelized form through PLINQ, though the names differ. A map can be achieved with PLINQ’s Select operator, and a filter with PLINQ’s Where operator.

For example, I could implement a ParallelMap operation that takes in one array and returns another as follows:

public static TOutput [] ParallelMap<TInput,TOutput>(
TInput [] input, Func<TInput,TOutput> map)
{
return input.AsParallel().Select(map).ToArray();
}

and a ParallelFilter operation could be implemented in a similar fashion, based on PLINQ’s Where operator:

public static TInput [] ParallelFilter<TInput>(
TInput [] input, Func<TInput,Boolean> filter)
{
return input.AsParallel().Where(filter).ToArray();
}

In general, PLINQ is the recommended way to achieve such parallelization. Not only is it useful for individual operations like this, but it’s also a great way to compose operations. For example, if I wanted to write a ParallelFilterAndMap, I could achieve that as follows:

public static TOutput [] ParallelFilterAndMap<TInput,TOutput>(
TInput [] input, Func<TInput,Boolean> filter, Func<TInput,TOutput> map)
{
return input.AsParallel().Where(filter).Select(map).ToArray();
}

It is also possible to build such operations on top of Parallel.*. For example, here’s ParallelMap re-implemented in terms of Parallel.For:

public static TOutput [] ParallelMap<TInput,TOutput>(
    TInput [] input, Func<TInput,TOutput> map)
{
    var output = new TOutput[input.Length];
    Parallel.For(0, input.Length, i => output[i] = map(input[i]));
    return output;
}

and ParallelFilterAndMap re-implemented in terms of Parallel.ForEach and ConcurrentBag<T>:

public static TInput [] ParallelFilterAndMap<TInput,TOutput>(
    TInput [] input, Func<TInput,Boolean> filter, Func<TInput,TOutput> map)
{
    var output = new ConcurrentBag<T>();
    Parallel.ForEach(input, item =>
    {
        if (filter(item)) output.Add(map(item));
    });
    return output.ToArray();
}

As mentioned, in most cases you’re better off using PLINQ for these kinds of operations. However, there may be cases where a Parallel.*-based version is best. For example, let’s say that you wanted to implement an in-place parallel map operation, e.g. a map operation that maps from TData to TData, and that stores the new output into the input array. PLINQ doesn’t modify the input data source, so if you wanted that functionality, you could build it on top of Parallel.For:

public static void ParallelMap<TData>(
TData [] data, Func<TData,TData> map)
{
Parallel.For(0, data.Length, i => data[i] = map(data[i]));
}

There are a few other subtle differences that you could take advantage of by building on Parallel.For/ForEach instead of on PLINQ. For example, PLINQ uses a static number of threads for the processing of a given query (which defaults to Environment.ProcessorCount), whereas the number of threads used by Parallel.* may vary over time as influenced by the ThreadPool’s thread injection and retirement logic. Parallel.* can also be targeted to a specific and custom TaskScheduler, something not supported by PLINQ in .NET 4 (PLINQ always runs on TaskScheduler.Default, which is based on the .NET 4 ThreadPool).

Happy coding.

TaskScheduler.FromCurrentSynchronizationContext

toub — Tue, 22 Sep 2009 19:46:13 GMT

The Task abstractions in .NET 4 run on instances of the TaskScheduler class. Two implementations of TaskScheduler ship as part of the .NET Framework 4. The first is the default scheduler, which is integrated with the .NET 4 ThreadPool and takes advantage of its work-stealing queues. The second is the type of TaskScheduler returned from the static method TaskScheduler.FromCurrentSynchronizationContext.

According to MSDN, SynchronizationContext “provides the basic functionality for propagating a synchronization context in various synchronization models.” What does that really mean? At its core, SynchronizationContext provides two methods, Send and Post, both of which accept a delegate to be executed. Send synchronously invokes the delegate, and Post asynchronously invokes the delegate. That’s it, and the base implementation of SynchronizationContext doesn’t do anything fancier than that:

public virtual void Send(SendOrPostCallback d, object state)
{
d(state);
}

public virtual void Post(SendOrPostCallback d, object state)
{
ThreadPool.QueueUserWorkItem(new WaitCallback(d.Invoke), state);
}

Where things get interesting is when new types are derived from SynchronizationContext, something typically done by a UI framework (though there are other non-UI framework implementations). For folks familiar with Windows Forms and Windows Presentation Framework development, you’re likely aware that UI controls should only be accessed by the thread that created them, almost always the main UI thread. Thus, if a thread doing work in the background wants to update something in the UI, it needs to marshal that work back to the GUI thread so that the controls may be accessed safely. Different UI frameworks expose different ways for accomplishing this marshaling. For example, in Windows Forms, one uses the Invoke or BeginInvoke method of the target Control (or at least a Control created on the same thread as the target Control). In WPF, one uses the target thread’s Dispatcher and corresponding Invoke/BeginInvoke methods. With every UI framework having its own model for marshaling work to a particular “synchronization context”, it becomes difficult to write code that supports this marshaling concept but which is agnostic to the particular environment that it’s in. Enter SynchronizationContext. A new type may be derived from SynchronizationContext such that its Send method synchronously marshals a delegate to the right thread for execution, and Post does the same but asynchronously. If you look at the implementations of the SynchronizationContexts provided by Windows Forms and WPF, that’s exactly what they do, delegating to the relevant Invoke/BeginInvoke methods from Send and Post to marshal the work correctly.

To make it easy to get at the right SynchronizationContext, a UI framework like Windows Forms will publish an instance of its SynchronizationContext-derived class to SynchronizationContext.Current. Code can then grab SynchronizationContext.Current and use it to marshal work, without having to know whether it’s being used from Windows Forms or Windows Presentation Foundation or another similar model.

“TaskScheduler.FromCurrentSynchronizationContext” should now make more sense. This method creates a TaskScheduler that wraps the SynchronizationContext returned from SynchronizationContext.Current. Thus, this gives you a TaskScheduler that will execute Tasks on the current SynchronizationContext. Why is that useful? It means you can create Tasks that are able to access UI controls safely, simply by running them on the right scheduler.

Let’s say that I wanted to load three images from some data source. When those images have been loaded, I want to blend them all together, and then I want to display the result into a PictureBox on my UI. Using Tasks and TaskScheduler.FromCurrentSynchronizationContext, I could write code like the following:

private void Button1_Click(…)
{
    var ui = TaskScheduler.FromCurrentSynchronizationContext();
    var tf = Task.Factory;

    // Load the three images asynchronously
    var imageOne = tf.StartNew(() => LoadFirstImage());
    var imageTwo = tf.StartNew(() => LoadSecondImage());
    var imageThree = tf.StartNew(() => LoadThirdImage());

    // When they’ve been loaded, blend them
    var blendedImage = tf.ContinueWhenAll(
        new [] { imageOne, imageTwo, imageThree }, _ =>
        BlendImages(imageOne.Result, imageTwo.Result, imageThree.Result));

    // When we’re done blending, display the blended image
    blendedImage.ContinueWith(_ =>
    {
        pictureBox1.Image = blendedImage.Result;
    }, ui);
}

This code runs three tasks to load the three input images asynchronously. When all of those have been loaded, again asynchronously some BlendImages method is used to blend the images, taking the three inputs and returning the blended image. Finally, once that’s done, another task is used to render the blended image by storing it into a PictureBox on the UI. Since this modifies a UI control, we need to do it from the UI thread. Thus, we pass a TaskScheduler to the ContinueWith method; this scheduler targets the UI’s SynchronizationContext, and will cause the Task to execute on the UI thread.

TaskScheduler.FromCurrentSynchronizationContext is provided for convenience and because this is a very common need. However, due to TaskScheduler’s extensibility, it’s actually possible to implement this behavior yourself, and in doing so you could modify it to suit your own needs however you see fit.

Let’s say you did want to develop a new SynchronizationContextTaskScheduler. It might look something like this:

public class SynchronizationContextTaskScheduler : TaskScheduler
{
    private ConcurrentQueue<Task> _tasks = new ConcurrentQueue<Task>();
    private SynchronizationContext _context;

    public SynchronizationContextTaskScheduler() :
        this(SynchronizationContext.Current) { }

    public SynchronizationContextTaskScheduler(
        SynchronizationContext context)
    {
        if (context == null) throw new ArgumentNullException("context");
        _context = context;
    }

    protected override void QueueTask(Task task)
    {
        // Add the task to the collection
        _tasks.Enqueue(task);

        // Queue up a delegate that will dequeue and execute a task
        _context.Post(delegate
        {
            Task toExecute;
            if (_tasks.TryDequeue(out toExecute)) TryExecuteTask(toExecute);
        }, null);
    }

    protected override bool TryExecuteTaskInline(
        Task task, bool taskWasPreviouslyQueued)
    {
        return SynchronizationContext.Current == _context &&
                  TryExecuteTask(task);
    }

    public override int MaximumConcurrencyLevel { get { return 1; } }

    protected override IEnumerable<Task> GetScheduledTasks()
    {
        return _tasks.ToArray();
    }
}

Not a lot of code for a fairly powerful thing.

The constructors simply accept the target SynchronizationContext and store it, also initializing a thread-safe queue that will store the tasks to be executed.
The QueueTask method is called whenever the system is providing a Task for this scheduler to execute: this scheduler handles it by storing that Task into a queue, and then Post’ing to the SynchronizationContext a delegate that will pull the next Task from the queue and execute it.
The TryExecuteTaskInline is invoked any time the system wants to run a Task inline on the current thread (either from a call to RunSynchronously or from a Wait attempt): we need to make sure that the call is coming from the same SynchronizationContext as the target, otherwise we may end up running in the wrong context.
We only intended to support SynchronizationContexts that represent a single thread of execution (that’s most common), so we return 1 from MaximumConcurrencyLevel.
And we want the debugger to be able to display tasks scheduled to this scheduler, so we override GetScheduledTasks to return an array of the tasks queued. (This is why we need to explicitly store the queue of tasks; otherwise, we could have simply relied on lambda closures to capture each task to be executed.)