Parallel Programming with .NET

What's New in Beta 2 for PLINQ

essey — Thu, 05 Nov 2009 20:37:00 GMT

Included in the .NET 4 Framework Beta 2 is a more robust and faster version of PLINQ. Between B1 and B2, PLINQ changes have mainly been under the covers, so hopefully no need to rewrite any of your applications to see the improvements.

1. Many improvements to performance and scalability

2. GroupBy and GroupJoin now preserve ordering within the groups

3. Better integration with performance profiling, which now shows PLINQ markers

4. Cancellation consistently works across all PLINQ operators

5. Scoped some of the boundary conditions of PLINQ in .NET 4 in Beta2 of Parallel Extensions as follows:

a. Max Degree of Parallelism changed from 64 to 63

b. Greater consistency regarding maximum input length

i. PLINQ does not support inputs of length > Int32.MaxValue. Queries may throw OverflowException for many operators.

ii. LongCount no longer counts beyond Int32.MaxValue

6. Improved robustness

a. Reliability and concurrency focus

b. More consistent exception handling and error model

7. Other bug fixes

If any of these severely impact your applications, particularly the boundary conditions, please let us know so that we can include that in our planning efforts. For now, we found that it makes the most sense to scope some inputs and give a reliable and predictable experience in this release.

Happy Coding!

What’s new in Beta 2 for the Task Parallel Library? (Part 3/3)

dashih — Wed, 04 Nov 2009 21:52:00 GMT

Last time, we covered Tasks being detached by default and some refactorings in our multiple-Task continuation APIs. The final post of this series will discuss Nested Tasks and Unwrap, a Parallel namespace change, and some changes under the covers.

Nested Tasks and Unwrap

We’ve added the Unwrap APIs to address scenarios that deal with nested Tasks. Before jumping into Unwrap, let’s first talk about nested Tasks, e.g. a Task<Task> or Task<Task<TResult>>. For example:

var nestedTask = Task.Factory.StartNew(() =>
{
    return Task.Factory.StartNew(() =>
    {
        return 42;
    });
});

Nested Tasks commonly lead to unexpected behavior in applications. For example, consider an API that provides the following functionality for asynchronously logging into a web service (like one from Facebook), retrieving a list of friends, and sending an email to each friend.

// Given a user name and password, returns a Task whose

// result is a UserToken object.

public Task<UserToken> LogOn(string username, string password);

// Given a UserToken, returns a Task whose result

// is a collection of Friend objects.

public Task<FriendCollection> GetFriendList(UserToken userToken);

// Given a Friend, subject, and body, returns a Task that

// represents an email sending operation.

public Task SendEmail(Friend friend, string subject, string body);

A user would like to be able to write code like the following, utilizing these APIs and Task continuations:

var userToken = LogOn(user, pass);

var friends = userToken.ContinueWith(_ => GetFriendList(userToken.Result));

var emails = friends.ContinueWith(_ =>

{

var sentMails = new List<Task>();

foreach(var friend in friends.Result)

{

sentMails.Add(SendEmail(friend, subject, body));

}

return Task.Factory.ContinueWhenAll(

sentMails.ToArray(), tasks => Task.WaitAll(tasks));

});

emails.ContinueWith(_ => Console.WriteLine(“All emails sent”));

The bolded code is problematic. Because the GetFriendList method returns a Task<FriendCollection>, the ‘friends’ variable is actually going to be a Task<Task<FriendCollection>>. This will cause a compiler error at the foreach loop, because ‘friends.Result’ will return a Task<FriendCollection> instead of a FriendCollection. The compiler error in this case is a good thing, of course, highlighting a programming error. However, once a developer realizes the type mismatch, he still has to code additional logic to unwrap the ‘friends’ Task so that it returns an actual FriendCollection. This logic is nontrivial, especially if it is to correctly deal with exceptions, cancellation, etc. The last line is also problematic. The emails variable here is actually of type Task<Task>. The outer Task will complete once the inner Task is returned from its body, even if the inner Task hasn’t completed. The net result of this is that “All emails sent” could be written out prior to all of the email tasks actually completing.

Now, you may have noticed that, in Beta 1, we provided special ContinueWith overloads to deal with this type of scenario. For example:

public Task<TResult> ContinueWith<TResult>(

Func<Task, Task<TResult>> continuationFunction);

The Func returns a Task<TResult>, so normally, ContinueWith would return a Task<Task<TResult>>. But this ContinueWith overload does some magic under the covers to return a Task<TResult>. There were a number of reasons why we didn’t like this approach, including:

· Too much magic. It’s hard to explain why one set of ContinueWith overloads is “just different”.

· Only works for ContinueWith. What if user scenarios result in nested Tasks for other Task creation APIs like StartNew, ContinueWhenAll, etc.?

· What if nested Tasks are actually desired? If a user actually wants that Task<Task<TResult>>, he still might unknowingly bind to this magical overload.

Given these reasons, our solution for Beta 2 and beyond is two Unwrap extension methods.

namespace System.Threading.Tasks

{

public static class TaskExtensions

{

public static Task Unwrap(

this Task<Task> task);

public static Task<TResult> Unwrap<TResult>(

this Task<Task<TResult>> task);

}

These methods may be used to transform any Task<Task> or Task<Task<TResult>> into a Task or Task<TResult>, respectively. The transformation performed produces a Task or Task<TResult> that fully represents the original nested Task, including exceptions, cancellation state, etc.

With Unwrap, we can fix the above scenario (note the bolded).

var userToken = LogOn(user, pass);

var friends = userToken.ContinueWith(

_ => GetFriendList(userToken.Result)).Unwrap();

var emails = friends.ContinueWith(_ =>

{

var sentMails = new List<Task>();

foreach(var friend in friends.Result)

{

sentMails.Add(SendEmail(friend, subject, body));

}

return Task.Factory.ContinueWhenAll(

sentMails.ToArray(), tasks => Task.WaitAll(tasks));

}).Unwrap();

emails.ContinueWith(_ => Console.WriteLine(“All emails sent”));

Parallel Namespace Change

We’ve moved the Parallel class from the System.Threading namespace to the System.Threading.Tasks namespace. We found that most applications needed to bring in both namespaces when using TPL, so why not put everything into one namespace? Additionally, Parallel is such a common word (and will likely become more so in the future), and System.Threading such a common namespace, we wanted to reduce the chances of conflict with other .NET types as much as possible.

Here’s a useful IDE tip. Use “Ctrl + .” to automatically bring in the relevant namespace once you’ve typed a class/type name.

Under the Covers

Beta 2 contains quite a few bug fixes not done for Beta 1. All of them were important, but we’ll focus on only two here.

First, Parallel.For and ForEach have been tweaked for better load-balancing with other workloads in the current or other AppDomains. Essentially, the change was to service the parallel loops with Tasks that periodically retired and re-queued themselves, allowing other contenders to grab Threads and make progress.

Second, waiting for Tasks in parent/child relationships has become more efficient. In Beta 1 and before, parent Tasks waited for their children using explicit Waits, so even if a parent completed first, it would burn a thread until all of its children completed. In Beta 2, parent Tasks wait for their children using callbacks; the parent maintains a count for number of children it has, and each child decrements the count as it completes. This waiting logic significantly improves scalability.

That’s it folks! We hope you’ve enjoyed this series. Download Beta 2 and try it out!

Attending PDC09? Meet with the Parallel Computing Platform team.

toub — Tue, 03 Nov 2009 02:55:00 GMT

Are you attending PDC09? Want to meet with members of the Parallel Computing Platform team? See http://blogs.msdn.com/visualizeparallel/archive/2009/11/02/the-parallel-computing-platform-team-pcd-09.aspx for more details.

Samples for Parallel Programming with the .NET Framework 4 Beta 2

toub — Mon, 02 Nov 2009 19:27:00 GMT

We've posted a whole bunch of samples on Code Gallery showcasing how to use the new parallelism support in the .NET Framework 4. You can find them at http://code.msdn.microsoft.com/ParExtSamples. Enjoy!

Slides from Parallelism Tour

toub — Mon, 02 Nov 2009 02:58:00 GMT

Last week, I had the privilege of touring around Tennessee, Kentucky, Ohio, and Michigan, speaking about the new parallel computing support in Visual Studio 2010 and the .NET Framework 4. Many folks I spoke with were interested in getting a copy of the slide deck I used, so I’ve attached it to this blog post. Enjoy!

PLINQ Queries That Run Sequentially

igoro — Sat, 31 Oct 2009 00:40:00 GMT

The goal of PLINQ is to execute computationally intensive LINQ to Objects queries efficiently by splitting up the work across multiple cores on multi-core machines. However, not all queries are equally appropriate for parallelism.

Usually, the best way to use PLINQ is to write short, simple queries with an expensive delegate. This is one example of such query:

var q = src.AsParallel()

.Where(x => ExpensiveFilter(x));

foreach(var x in q) { ... }

And here is another:

    int sum = src.AsParallel()
        .Select(x => ExpensiveFunc(x))
        .Sum();

One design goal behind PLINQ is maximum parity with LINQ to Objects. So, you can combine LINQ operators in all kinds of ways, and PLINQ will correctly execute the query. However, performance characteristics get trickier as queries get more complex.

In some cases, PLINQ may decide to run the query in its entirety or in part sequentially. For example, this query will execute sequentially up to and including the TakeWhile operator:

src.AsParallel()

.Select(x => Foo(x))

.TakeWhile(x => Filter(x))

.Select(x => Bar(x))

.ToArray();

The TakeWhile operator introduces a tricky sequential dependency – whether an element is or isn’t included in the output depends on the result of Filter() on all previous elements in the sequence. There are various ways to execute this query partly in parallel that take different trade-offs. Depending on the selectivity of Filter and the costs of Foo, Bar and Filter, there are different algorithms which may be appropriate.

Whether a particular query executes in part sequentially depends on the combination of operators in the query. The precise rules are fairly complex, but they can be summarized in a simple way. If a query contains one of these operators, PLINQ may decide to run it sequentially:

· First, Last

· Take, Skip

· TakeWhile, SkipWhile

· Concat

· Special overloads of Select, Where and SelectMany that pass position indices into the user delegate

· ElementAt

· Zip

For example, if the Take operator follows a Where operator, PLINQ will execute the Where and the Take sequentially by default. Filtering followed by a Take is a tricky operation to parallelize – different algorithms are appropriate depending on the selectivity of the filter, the size of the argument passed to Take, and other details. However, if the Take is applied straight to an array (or an array followed by a Select), PLINQ can handle the Take operator efficiently simply by restricting execution to a section of the array.

Also, in some queries, SelectMany causes the part of the query that comes before the SelectMany to run sequentially. I haven’t seen a practical example where this would be an issue, though. SelectMany produces multiple output elements for each input, and the part of the query before the SelectMany is normally not the computationally expensive part.

You can prevent PLINQ from falling back to sequential execution by turning on the ForceParallelism mode. In the ForceParallelism mode, PLINQ will always use parallel algorithms to execute the query, even if potentially expensive algorithms will be used. This is desirable if the query contains an expensive delegate:

src.AsParallel()

.WithExecutionMode(
ParallelExecutionMode.ForceParallelism)

.Select(x => Foo(x))

.TakeWhile(x => Filter(x))

.Select(x => Bar(x))

.ToArray();

Alternatively, you can try breaking up the query so that only the simple but computationally expensive part of the query is done in PLINQ, and the rest of the processing is done in LINQ to Objects:

src.Select(x => Foo(x))

.TakeWhile(x => Filter(x))

.AsParallel() // <- only parallelize here

.Select(x => Bar(x))

.ToArray();

This implementation will scale well if the calls to Bar() represent the bulk of the work in the query. This solution is often preferable, as the part of the query that executes in parallel is clearly marked, and thus the code is easier to understand.

What’s new in Beta 2 for the Task Parallel Library? (Part 2/3)

dashih — Tue, 27 Oct 2009 18:02:00 GMT

Last week, we talked about how TPL adopted a new, better cancellation model. Today, we’ll cover a change that makes Tasks Detached by Default, some ContinueWhenAll/Any Refactoring, and the handy UnobservedTaskException event.

Tasks are Detached by Default

In Beta 2, we have changed an important default. Tasks are now created as detached (instead of attached) if no options specify otherwise.

Let’s consider the following code to review the difference between attached and detached Tasks.

Task p = Task.Factory.StartNew(() =>

{

Task c = Task.Factory.StartNew(() =>

{

DoWork();

});

p.Wait();

In Beta 1 and before, since the default options are used, ‘c’ is created as a child Task of Task ‘p’, the parent Task; we refer to this as Task ‘c’ being “attached” to Task ‘p’. This means that the p.Wait() statement will not return until the call to DoWork completes, because parent Tasks do not complete until all of their child Tasks complete. To opt out of this behavior, a user needs to create ‘c’ with the DetachedFromParent option:

Task c = Task.Factory.StartNew(() =>

{

DoWork();

}, TaskCreationOptions.DetachedFromParent);

The original code shown behaves differently in Beta 2. Now, by default, ‘c’ is not related to ‘p’ (it’s “detached” by default), and the p.Wait() statement will return as soon as ‘p’ completes, regardless of the status of Task ‘c’ and thus regardless of when DoWork returns. To opt in to the parent/child relationship, a user needs to create ‘c’ with the AttachedToParent option:

Task c = Task.Factory.StartNew(() =>

{

DoWork();

}, TaskCreationOptions.AttachedToParent);

Here is a summary of the changes:

· Removed the DetachedFromParent option

· Added the AttachedToParent option

· Changed the default behavior so that Tasks do not enlist in parent/child relationships when no options are specified.

There were a number of reasons why we decided that detached is the correct default and to move forward with this change, including:

· Many users were using attached Tasks unknowingly. The vast majority of the time, users create Tasks for simple, asynchronous work. In such scenarios, parent/child relationships (and the implicit waiting) are not needed. We found through many interactions that folks were just going with the default options and were accidentally opting in to this behavior. In the best case, this would only result in a slight performance cost. In the worst case, this would bring with it incorrect behavior that would lead to difficult to diagnose errors.

· Easier migration from ThreadPool.QueueUserWorkItem. Tasks are now the recommended way to queue work to the ThreadPool, but the easiest way to create Tasks resulted in different behavior from QueueUserWorkItem (where there’s no concept of parent/child work items). This change makes Task.Factory.StartNew (with no options) a true replacement for QueueUserWorkItem.

· Additional behavior should be opt-in and pay-for-play. Almost everything in TPL that results in additional behavior is opt-in, e.g. cancellation, LongRunning, PreferFairness. With the Beta 1 default, users opt-out of parent/child relationships. In Beta 2, users opt-in, making it consistent. This makes the extra functionality provided by parent/child relationships pay-for-play, such that you don’t pay the cost for parents implicitly waiting for their children or for exceptions propagating from children to parents unless you need that functionality.

ContinueWhenAny/All Refactoring

We have refactored the set of ContinueWhenAny and ContinueWhenAll overloads to make things more intuitive, consistent, and complete.

To demonstrate the main issue, let’s consider the following overload that was provided in Beta 1.

public class TaskFactory<TResult>

{

public Task<TNewResult> ContinueWhenAny(

Task<TResult>[] tasks,

Func<Task<TResult>, TNewResult> continuationFunction);

}

This confused the meaning of TaskFactory<TResult>, which is meant to create tasks of type Task<TResult>. However, with these overloads, TaskFactory<TResult> could be used to create tasks of type Task<TNewResult>. As an example, consider the code:

Task<int>[] taskOfInts = ...;
Task<string> t = Task<int>.Factory.ContinueWhenAll(taskOfInts, _ => “”);

This compiles and works just fine, but the type parameter mismatch (shown in bold) is certainly odd. To address this, we changed a bunch of overloads, so that instead of taking Task<TResult>s and returning a Task<TNewResult>, they take Task<TAntecedentResult>s and return Task<TResult>s. For example, the overload that replaced the above is:

public Task<TResult> ContinueWhenAny<TAntecedentResult>(

Task<TAntecedentResult>[] tasks,

Func<Task<TAntecedentResult>, TResult> continuationFunction);

And the above example becomes:

Task<int>[] taskOfInts = ...;
Task<string> t = Task<string>.Factory.ContinueWhenAll(taskOfInts, _ => “”);

In addition to this change, we also added, removed, or modified a number of other overloads to make the set consistent and complete. Now, the entire set of ContinueWhenAll and ContinueWhenAny overloads follow these clear rules:

· A TaskFactory creates Tasks, but also provides overloads to create Task<TResult>s.

· A TaskFactory<TResult> only ever creates Task<TResult>s (never Tasks or Task<TNewResult>s).

UnobservedTaskException event

We’ve added an event that fires for every Task exception that goes unobserved. Recall that to “observe” a Task’s exceptions, you must either Wait on the Task or access its Exception property after it has completed. At least one of these actions must be done before the Task object is garbage collected, or its exceptions will propagate (currently this occurs on the finalizer thread).

The new static event resides on the TaskScheduler class, and subscribing to it is straightforward. Here’s an example to log all unobserved exceptions and mark them as observed (preventing them from being propagated).

TaskScheduler.UnobservedTaskException +=

(object sender, UnobservedTaskExceptionEventArgs exceptionArgs) =>

{

exceptionArgs.SetObserved();

LogException(exceptionArgs.Exception);

};

Some customers have complained that TPL’s exception policy is too strict. The UnobservedTaskException event provides an easy way out by allowing you to simply squash all Task exceptions in an application (though using it in this manner is not recommended). The primary reason that we made the addition was to support host-plugin scenarios where a host application can still be perfectly useful in the presence of some truly harmless exceptions (thrown by buggy plugins). These scenarios may be achieved using the UnobservedTaskException event in conjunction with AppDomains to sandbox plugins. Look for a future post that describes this in more detail!

We’re done for now! The 3^rd and final post of this series will cover the new Unwrap APIs, a Parallel namespace change, and some changes under the covers.

Parallelism Talks at PDC09

phillips.joshua — Sat, 24 Oct 2009 00:47:00 GMT

If you’re going to PDC this year, we have four great talks on parallelism coming you’re way and, if you’re not, may we suggest you sign up?

We don’t have the exact dates of the talks yet (we’ll let you know when we do) but here are the talks you won’t want to miss!

Patterns of Parallel Programming: A Tutorial on Fundamental Patterns and Practices for Parallelism
(by Richard Ciapala, Ade Miller, Herb Sutter, and our very own Stephen Toub)
A workshop for experienced developers who are relatively new to parallel computing. Learn how established software patterns can help you build on Microsoft’s Parallel Computing Platform (including deep dives into TPL and PLINQ).

Manycore and the Microsoft .NET Framework 4: A Match Made in Microsoft Visual Studio 2010
(by Stephen Toub)
A deep dive into the System.Threading.Tasks and System.Collections.Concurrent namespaces, cutting-edge concurrency views in the Visual Studio profiler, and debugger tool windows for analyzing the state of concurrent code.

PLINQ: LINQ, but Faster!
(by Igor Ostrovsky and Ed Essey)
Our very own Igor and Ed dive deep into PLINQ via Visual Studio 2010. See what it looks like from the perspective of LINQ developers, the debugging and profiling support, how it's implemented under the covers, and how to best incorporate it into your applications in order to reap the performance benefits of the manycore era.

The State of Parallel Programming
(by supercomputing luminary Burton Smith)
A “relatively recent consensus view about what is needed for productive parallel programming, and why.”

F# for Parallel and Asynchronous Programming
(by Luke Hoban)
Luke will take you through the core concepts of the F# language and show you how ideas like immutability, functional design, async workflows, agents, and more can be used to meet the challenges of today’s real-world applications.

C++ Forever: Interactive Applications in the Age of Manycore
(by Rick Molloy)
Come for a deep dive into the power of actor-based and dataflow programming in Microsoft Visual C++ 2010.

Lighting up Windows Server 2008 R2 Using the ConcRT on UMS
(by Dana Groff)
See examples of how to use C++ and the new Concurrency Runtime (ConcRT) to take advantage of new technologies on Windows Server 2008 R2, such as the ability to scale beyond 64 cores and User-Mode Scheduling (UMS) of threads

Have fun!

Josh Phillips | Program Manager | Parallel Computing Platform |Microsoft

What’s new in Beta 2 for the Task Parallel Library? (Part 1/3)

dashih — Mon, 19 Oct 2009 19:42:00 GMT

Visual Studio 2010 and .NET 4 Beta 2 is here! In terms of completeness and readiness for production coding, Beta 2 promises to be much better than Beta 1, and TPL is one component that delivers significant improvements over what was previously available. To get you excited about it, this series of posts details key additions and changes for Beta 2. Enjoy!

In this post, we’re talking about cancellation. A few months ago, our flurry of “What’s new in Beta 1” posts hinted at a new cancellation model and promised more information about it. We made good on that promise with .NET 4 Cancellation Framework, and this post goes further to explain how TPL, specifically, has fully adopted the new model.

The Old Way (Beta 1 and before)

Let’s consider the following code to review the old TPL cancellation model.

Task myTask = Task.Factory.StartNew(() =>

{

for (; ; )

{

if (Task.Current.IsCancellationRequested)

{

Task.Current.AcknowledgeCancellation();

return;

}

...

}

});

// Elsewhere.

myTask.Cancel();

myTask just loops infinitely, checking its IsCancellationRequested property to see if it has been canceled. Elsewhere, cancellation is requested on myTask using the Cancel method. At that point, myTask agrees to get canceled by calling its AcknowledgeCancellation method and returning. This is all necessary, because Task cancellation is cooperative; to enter the Canceled state, outside logic must request cancellation and the Task must acknowledge that cancellation request. However, note that cooperative cancellation is only relevant for already running Tasks. If a Task’s Cancel method is called before it is in the Running state, it will transition directly into the Canceled state.

Waiting on a canceled Task results in a TaskCanceledException wrapped in an AggregateException, hence the try/catch block. Executing this code prints “Canceled” to the console, indicating that myTask was successfully canceled.

This approach works, but we identified a number of problems with it including the following:

1. The cancellation model required exposing a Task.Current static property. This leaks implementation details from libraries that utilize Tasks internally; any code called from the Task can muck with the current Task or take dependencies on its existence. Imagine calling into 3^rd party code and having that code cancel your Tasks, schedule continuations off of them, etc.

2. Anyone with a Task’s reference can request cancellation on it. In many scenarios, it is valuable to separate the ability to check for cancellation and the ability to actually request cancellation.

3. From a cancellation perspective, Tasks didn’t compose well with other APIs that were cancelable, such as executing a cancelable PLINQ query inside of a Task.

4. Tasks were often more expensive than they needed to be, due to needing to track extra cancellation state per Task.

The New Way (Beta 2 and beyond)

The new TPL cancellation model is centered around two types: CancellationTokenSource and CancellationToken. You can read up about them in the post linked to above, but here is a simplified overview of their APIs:

namespace System.Threading
{
    public sealed class CancellationTokenSource
    {
        public void Cancel();
        public CancellationToken Token { get; }
        ...
    }

    public struct CancellationToken
    {
        public Boolean IsCancellationRequested { get; }
        public void ThrowIfCancellationRequested();
        ...
    }
}

Here’s the gist. A CancellationTokenSource contains a CancellationToken, and it can request cancellation on that token using a Cancel method. A CancellationToken can only check if cancellation has been requested on it. Ignore the ThrowIfCancellationRequested method for now; we’ll see why it’s handy later.

Adopting the new model involved not only adding support for these two types, but also ripping out the old model. Here’s a summary of the changes:

· All cancellation-related APIs on the Task class were removed (Cancel, AcknowledgeCancellation, IsCancellationRequested, etc)

· Other APIs that were no longer relevant were removed (Task.Current, Task.Parent, TaskCreationOptions.RespectParentCancellation, etc)

· Overloads that accept CancellationToken were added to many methods (StartNew, ContinueWith, etc)

And now, here’s a table that outlines how achieving cancellation in TPL has changed:

Action	Old Model	New Model

To set up cancellation	Just create a Task	Create a CancellationTokenSource and pass its Token to an API that creates a Task.
To check if cancellation has been requested	Check the IsCancellationRequested property on the relevant Task	Check the IsCancellationRequested property on the CancellationToken that was passed to the API that created the Task
To acknowledge cancellation	Check to ensure that IsCancellationRequested is true, then call the AcknowledgeCancellation() method on the relevant Task	Throw an OperationCanceledException with the task’s CancellationToken
To cancel a tree of tasks	Create all of the tasks as attached tasks and with the RespectParentCancellation flag set, then cancel the root task.	Pass the same CancellationToken to all Tasks, then cancel the associated CancellationTokenSource

To see how this all works, let’s rewrite the code above for Beta 2.

CancellationTokenSource cts = new CancellationTokenSource();

CancellationToken token = cts.Token;

Task myTask = Task.Factory.StartNew(() =>

{

for (; ; )

{

token.ThrowIfCancellationRequested();

...

}

}, token);

// Elsewhere.

cts.Cancel();

A CancellationTokenSource (cts) is initialized, and a CancellationToken (token) is initialized to cts’s Token. myTask still loops indefinitely, but now it calls ThrowIfCancellationRequested. This method just checks the IsCancellationRequested property on a CancellationToken. If true, it throws an OperationCanceledException(token), which is the way to acknowledge cancellation in the new model. Elsewhere, Cancel is called on cts to request cancellation on myTask.

The new model addresses all of the problems with the old model listed above:

1. No more Task.Current (Task.Parent was removed too, as it could be used to get at the current Task).

2. The ability to check for cancellation requests may now be separated from the ability to request cancellation. If only the former is desired, grant access to the CancellationToken only.

3. TPL now uses the new .NET 4 cancellation types. You can use the same CancellationToken{Source} to effect cancellation on multiple Tasks, Parallel.For calls, PLINQ queries, and more, all at once. This provides both better composition and better performance.

Finally, there are other advantages to this model. One is that it is much easier to organize cancellation sets within a large nest of Tasks; just create the right number of tokens and pass them around accordingly. Before, this scenario would have required tedious bookkeeping of many Task references and/or careful architecting of cancellation chains using TaskCreationOptions.RespectParentCancellation.

That’s it for now. The next post in the series will discuss a change regarding Tasks and parent/child relationships.

.NET 4 Beta 2 is here!

toub — Mon, 19 Oct 2009 18:50:05 GMT

The .NET Framework 4 Beta 2 is now available!

MSDN Subscribers can download it today, and it will be generally available for download on Wednesday. More information is available at http://msdn.microsoft.com/en-us/vstudio/dd582936.aspx. Additionally, one of the really exciting things about this Beta release is that it’s “go-live”; more information on that is available at “Going live” with Visual Studio 2010 Beta 2.

As part of .NET 4 Beta 2, of course, comes improved parallelism support. In the coming days and weeks, we’ll be blogging here about some of the exciting updates and additions in this release. Stay tuned…

In the meantime, the MSDN documentation on .NET 4’s parallelism support has been updated. It’s a great place to start learning about all of the functionality available in this release.

You can also learn more about the tools in Visual Studio 2010 for parallel development in the documentation at:

Enjoy!

Parallel Computing at PDC09 and TechEd Europe 2009

toub — Fri, 16 Oct 2009 17:28:23 GMT

It’s fall, and that means conference season! In November, members of the Parallel Computing Platform team will be descending upon both PDC09 in Los Angeles and TechEd Europe 2009 in Berlin. We’re extremely excited to see all of you there and to discuss the new support for parallel computing that’s coming in Visual Studio 2010, Visual C++ 10, and the .NET Framework 4. Please find us and introduce yourselves… we’re looking forward to speaking with you.

There are a bunch of cool sessions at both conferences having to do with parallel computing. Here’s a rundown of relevant announced sessions at the PDC:

A Computing Platform That Scales	See how Windows Server 2008 R2 represents the latest evolution of the Windows Server operating system and supports high-end hardware systems with large numbers of microprocessors. Learn how new Non-Uniform Memory Access (NUMA) and User-Mode Scheduling (UMS) API's effectively enable application developers to "minimize contention and maximize locality" of system resources relative to thread execution contexts. See how these new platform capabilities are extended via libraries such as the C++ Concurrency Runtime and the .NET Parallel Extensions along with complementary tools support within Visual Studio 2010.
Axum: A .NET Language for Safe and Scalable Concurrency	Axum is an incubation project from Microsoft's Parallel Computing Platform that aims to validate a safe and productive parallel programming model for the .NET framework. It's a language that builds on the principles of isolation, agents, and message-passing to increase application safety, responsiveness, scalability and developer productivity. Come learn about Axum and other advanced concepts we are exploring including data flow networks, asynchronous methods, and protocol-based component interactions.
Data-Intensive Computing on Windows HPC Server with the DryadLINQ Framework	Come get an overview of the DryadLINQ features and runtime environment, and walk through some real-world examples of DryadLINQ programs based on the familiar declarative syntax of LINQ combined with the fault-tolerant distributed graph scheduling of the Dryad runtime. Hear how DryadLINQ provides a programming model and runtime for data-parallel programs running across large clusters and partitioned data sets.
Lighting up Windows Server 2008 R2 Using the ConcRT on UMS	See examples of how to use C++ and the new Concurrency Runtime (ConcRT) to take advantage of new technologies on Windows Server 2008 R2, such as the ability to scale beyond 64 cores and User-Mode Scheduling (UMS) of threads. Learn to write code that scales even in the face of complicated processor architectures, Non-Uniform Memory Access (NUMA), and many-cores. Hear how ConcRT provides mechanisms that allow you to naturally express affinity requirements and schedule work on appropriate nodes without thrashing the system.
PLINQ: LINQ, but Faster!	Multi-core processors are everywhere! Parallel Language Integrated Query (PLINQ) in the Microsoft .NET Framework 4 offers a minimal-code solution to take advantage of this parallel hardware, providing an implementation of the .NET Standard Query Operators that uses parallel execution techniques underneath the simple LINQ programming to make applications run faster! Come for a deep dive into PLINQ via Microsoft Visual Studio 2010. See what it looks like from the perspective of LINQ developers, the debugging and profiling support, how it's implemented under the covers, and how to best incorporate it into your applications in order to reap the performance benefits of the manycore era.
F# for Parallel and Asynchronous Programming	Concurrency is increasingly important for many applications. The shift to manycore brings abundant parallelism to the desktop. The Web, Microsoft Silverlight, and distributed systems increase the demand for asynchronous solutions. F#, a functional and object-oriented language for Microsoft .NET, adds many tools to make parallel and asynchronous programming both fun and easy. Come hear the core concepts of the F# language, and see how ideas like immutability, functional design, async workflows, agents, and more can be used to meet the challenges of today’s real-world applications.
C++ Forever: Interactive Applications in the Age of Manycore	Developing concurrent applications is hard. Making them robust and responsive is even more challenging. The new asynchronous agents library and parallel debugging and profiling support in Microsoft Visual Studio 2010 simplify the situation, enabling you to develop I/O- and computationally-intensive applications that offload work from the UI and that make good use of available resources. Come for a deep dive into the power of actor-based and dataflow programming in Microsoft Visual C++ 2010. Learn how it can help you to write correct, efficient, and scalable applications.
The State of Parallel Programming	Parallel programming has been more difficult than it needs to be, perhaps because its tools have been treated as an “add-on” to serial programming. The objectives of composability and productivity demand something better. Come hear a relatively recent consensus view about what is needed for productive parallel programming, and why.
Manycore and the Microsoft .NET Framework 4: A Match Made in Microsoft Visual Studio 2010	The Microsoft .NET Framework 4 and Visual Studio 2010 include new technologies for expressing, debugging, and tuning parallelism in managed applications. Dive into key areas of support, including the new System.Threading.Tasks and System.Collections.Concurrent namespaces, cutting-edge concurrency views in the Visual Studio profiler, and debugger tool windows for analyzing the state of concurrent code.
Patterns of Parallel Programming: A Tutorial on Fundamental Patterns and Practices for Parallelism	The transition from single-core to multi-core technology is altering computing as we know it, enabling increased productivity, powerful energy-efficient performance, and leading-edge advanced computing experiences. Multi-core and HPC technologies are rapidly moving into the computing mainstream, allowing us to develop applications with improved performance, increased responsiveness, and reduced latency. This workshop is aimed at experienced software developers who are relatively new to the parallel computing space but expect it to become more important to their work. The workshop helps software developers understand the fundamental challenges of parallel computing, that span from the client to the cluster, such as synchronization, shared state, and moving from multi-core to multi-server. Learn how established software patterns can help developers building on both Microsoft’s Parallel Computing Platform—consisting of Task Parallel Library, PLINQ and Coordination Data Structures for .NET development, and Parallel Patterns Library and Concurrency Runtime for C++—and the HPC platform. The presenters describe the patterns in a bigger context, share their experience, and demonstrate implementations of these patterns in examples and demos. Learn how to add these patterns and new technologies to your toolbox.

There are also several good and related sessions at TechEd Europe:

F# for Parallel and Asynchronous Programming	Concurrency is increasingly important for many applications. The shift to manycore brings abundant parallelism to the desktop. The Web, Microsoft Silverlight, and distributed systems increase the demand for asynchronous solutions. F#, a functional and object-oriented language for Microsoft .NET, adds many tools to make parallel and asynchronous programming both fun and easy. This talk quickly reviews the core concepts of the F# language, and shows how ideas like immutability, functional design, async workflows, agents and more can be used to meet the challenges of today's real-world applications.
Parallel Computing for Managed Developers	The microprocessor industry has undergone a major paradigm shift from increasing clock speed with each successive hardware revision to increasing core count and potential for parallelism. This change requires software developers to think parallel in order to extract performance and provide a competitive user experience. This session discusses the new programming model and tools technologies in Visual Studio 2010 and .NET 4.0, including the Task Parallel Library (TPL), Parallel Language Integrated Query (PLINQ), and Concurrent Data Structures (CDS). Developers will also gain a deeper understanding of the advantages of building applications using these task-centric parallel technologies and will see demonstrations of several important patterns for parallelization.
Microsoft Visual C++ 2010: The "Fast" Way towards Modern Windows Applications	Do you love it when your application is the fastest on the block? Do you want to get the most of the Windows platform without re-writing your application and make it even faster in the process? Come and watch an action-packed demo and see how Visual Studio 2010 is a huge release for C++ developers! It's huge because it's focused on making it easier for ISVs who have many million lines of code. We have dramatically improved the scalability and productivity of the IDE and enabled MFC applications to seamlessly light up on Windows Vista and Windows 7. With the advent of highly parallel hardware, our adoption of the new C++ standard ushers in new ways to program scalable native applications.
Building High Performance Parallel Software	In this session, attendees will learn how to parallelize their software to take advantage of the industry shift to multicore hardware architectures. Several key aspects of achieving performance through parallelization will be covered, including identifying opportunities for parallelism, high performance patterns and best practices for parallel coding, effectively employing the new Visual Studio 2010 programming models, and the use of the Visual Studio 2010 Profiler to identify bottlenecks and optimize applications. Examples will be shown in both managed and native code.

Both events should prove to be very exciting. See you there.

Parallel Computing Presentations in Michigan, Ohio, Kentucky, and Tennessee

toub — Fri, 16 Oct 2009 15:49:29 GMT

In a week, I’m going to be traveling through Michigan, Ohio, Kentucky, and Tennessee, speaking about parallel computing, Visual Studio 2010, and .NET 4, primarily at corporations during the day and at user groups in the evenings.

If you’re in the area and interested, please do attend, and I look forward to meeting you! A list of events is available at http://blogs.msdn.com/jennifer/archive/2009/10/14/stephen-toub-parallel-computing-tour.aspx.

See you in a week!

Stephen

Task.Wait and “Inlining”

toub — Thu, 15 Oct 2009 15:45:59 GMT

“What does Task.Wait do?”

Simple question, right? At a high-level, yes, the method achieves what its name implies, preventing the current thread from making forward progress past the call to Wait until the target Task has completed, one way or another. If the Task ran to completion, Wait will return successfully. If the Task completed due to faulting or being canceled, it will throw an exception indicating the relevant details (since the waiting code likely expects that all work to be completed by the task ran successfully if Wait returns successfully).

The details are a bit more interesting. Wait could simply block on some synchronization primitive until the target Task completed, and in some cases that’s exactly what it does. But blocking threads is an expensive venture, in that a thread ties up a good chunk of system resources, and a blocked thread is dead weight until it’s able to continue executing useful work. Instead, Wait prefers to execute useful work rather than blocking, and it has useful work at its finger tips: the Task being waited on. If the Task being Wait’d on has already started execution, Wait has to block. However, if it hasn’t started executing, Wait may be able to pull the target task out of the scheduler to which it was queued and execute it inline on the current thread.

To do that, Wait asks for assistance from the target scheduler, by making a call to the TaskScheduler’s TryExecuteTaskInline method. TryExecuteTaskInline can do whatever logic the scheduler needs in order to validate that the current context is acceptable for inlining the Task. If it is, the scheduler tries to execute the task then and there as part of the call to TryExecuteTaskInline (using the base TaskScheduler’s TryExecuteTask method) and returns whether the Task could be executed. If it couldn’t be executed in TryExecuteTaskInline, the scheduler returns false, and Wait will need to block until the Task completes. A Task may not be able to be executed if it’s already been or is currently being executed elsewhere. As an example of this, the default scheduler (based on the ThreadPool) is aggressive about inlining, but only if the task can be efficiently removed from the data structures that hold tasks internally in the ThreadPool (such as if the task is living in the local queue associated with the thread attempting to inline it).

This is also very similar to what happens when Task.RunSynchronously is used to execute a Task (RunSynchronously may be used instead of Task.Start to run a Task currently in the TaskStatus.Created state). RunSynchronously results in the Framework calling the target scheduler’s TryExecuteTaskInline, allowing the scheduler to decide whether to inline or not. If the scheduler chooses not to inline, the Task will instead be queued to the scheduler, and the system will block until the Task is completed.

Of course, schedulers may want to behave differently depending on whether the code is explicitly choosing to inline (e.g. RunSynchronously) or whether the inlining is happening implicitly (e.g. Wait). To support this difference, in addition to accepting as a parameter the Task to be inlined, TryExecuteTaskInline also accepts as a parameter a Boolean taskWasPreviouslyQueued. This value will be false if it is known that the specified task has not yet been submitted to the scheduler, and true otherwise. If RunSynchronously is being used, taskWasPreviouslyQueued will be false; if Wait, it will be true. The default scheduler does, in fact, differentiate between these two cases, in that it always supports explicit inlining through RunSynchronously.

Parallelized Map and Filter Operations

toub — Mon, 12 Oct 2009 16:03:38 GMT

Common operations like map and filter are available in parallelized form through PLINQ, though the names differ. A map can be achieved with PLINQ’s Select operator, and a filter with PLINQ’s Where operator.

For example, I could implement a ParallelMap operation that takes in one array and returns another as follows:

public static TOutput [] ParallelMap<TInput,TOutput>(
TInput [] input, Func<TInput,TOutput> map)
{
return input.AsParallel().Select(map).ToArray();
}

and a ParallelFilter operation could be implemented in a similar fashion, based on PLINQ’s Where operator:

public static TInput [] ParallelFilter<TInput>(
TInput [] input, Func<TInput,Boolean> filter)
{
return input.AsParallel().Where(filter).ToArray();
}

In general, PLINQ is the recommended way to achieve such parallelization. Not only is it useful for individual operations like this, but it’s also a great way to compose operations. For example, if I wanted to write a ParallelFilterAndMap, I could achieve that as follows:

public static TOutput [] ParallelFilterAndMap<TInput,TOutput>(
TInput [] input, Func<TInput,Boolean> filter, Func<TInput,TOutput> map)
{
return input.AsParallel().Where(filter).Select(map).ToArray();
}

It is also possible to build such operations on top of Parallel.*. For example, here’s ParallelMap re-implemented in terms of Parallel.For:

public static TOutput [] ParallelMap<TInput,TOutput>(
    TInput [] input, Func<TInput,TOutput> map)
{
    var output = new TOutput[input.Length];
    Parallel.For(0, input.Length, i => output[i] = map(input[i]));
    return output;
}

and ParallelFilterAndMap re-implemented in terms of Parallel.ForEach and ConcurrentBag<T>:

public static TInput [] ParallelFilterAndMap<TInput,TOutput>(
    TInput [] input, Func<TInput,Boolean> filter, Func<TInput,TOutput> map)
{
    var output = new ConcurrentBag<T>();
    Parallel.ForEach(input, item =>
    {
        if (filter(item)) output.Add(map(item));
    });
    return output.ToArray();
}

As mentioned, in most cases you’re better off using PLINQ for these kinds of operations. However, there may be cases where a Parallel.*-based version is best. For example, let’s say that you wanted to implement an in-place parallel map operation, e.g. a map operation that maps from TData to TData, and that stores the new output into the input array. PLINQ doesn’t modify the input data source, so if you wanted that functionality, you could build it on top of Parallel.For:

public static void ParallelMap<TData>(
TData [] data, Func<TData,TData> map)
{
Parallel.For(0, data.Length, i => data[i] = map(data[i]));
}

There are a few other subtle differences that you could take advantage of by building on Parallel.For/ForEach instead of on PLINQ. For example, PLINQ uses a static number of threads for the processing of a given query (which defaults to Environment.ProcessorCount), whereas the number of threads used by Parallel.* may vary over time as influenced by the ThreadPool’s thread injection and retirement logic. Parallel.* can also be targeted to a specific and custom TaskScheduler, something not supported by PLINQ in .NET 4 (PLINQ always runs on TaskScheduler.Default, which is based on the .NET 4 ThreadPool).

Happy coding.

Choosing Between the Task Parallel Library and the ThreadPool

dashih — Tue, 06 Oct 2009 02:52:00 GMT

If you’ve been following the development of the Task Parallel Library (TPL), or if you’re an avid reader of this blog, you’re likely aware that the default scheduler in TPL is based on the new and improved ThreadPool in .NET 4. In light of this, though, quite a few folks have asked whether there are any advantages (particularly in regards to performance) to sticking with the existing ThreadPool APIs, e.g. QueueUserWorkItem. This post attempts to briefly answer that question.

First, if your scenario requires the new functionality that Tasks provide, it’s best to use TPL. Not only does this save you the time for rolling and optimizing your own solution, but TPL also has the advantage of being able to hook into internals of the ThreadPool that are not exposed publicly and that can boost performance in a variety of key scenarios. For example, if you need to wait on your work items, Task.Wait (and its variants) benefits from tight integration with the ThreadPool’s work-stealing queues. The story for other features like continuations and cancellation is similar.

Now, if your scenario does not require TPL’s features, such that your workloads are primarily fire-and-forget asynchronous work, then using QueueUserWorkItem may be appropriate. Whether you want to do so, though, still depends on the specific workload. On the one hand, as mentioned above, Tasks take advantage of the new work-stealing queues in the ThreadPool, which can help avoid issues of contention and cache coherency, both of which can lead to performance degradation. On the other hand, additional functionality necessarily means additional overhead. Using Tasks results in larger object allocations, and maintaining a Task’s lifecycle (status, cancellation requests, exception handling, etc.) requires extra work and synchronization. As a result, there are surely some scenarios that will run faster using QueueUserWorkItem.

In conclusion, I’ll reiterate what the CLR team’s ThreadPool developer has already stated:

“Task is now the preferred way to queue work to the thread pool.”

In other words, we recommend that you start with TPL for all of your asynchronous needs. Afterwards, if you suspect that your scenario would benefit from something more lightweight, it should be easy to try out QueueUserWorkItem. If that works better, make the change – it won’t hurt our feelings =).