All about Async/Await, System.Threading.Tasks, System.Collections.Concurrent, System.Linq, and more…
Throughout the development of Parallel Extensions for the .NET Framework 4, we’ve come across a myriad of situations where certain functionality would be useful in developing a particular application or library, but where that functionality isn’t quite encapsulated in the bits we’re shipping. Sometimes this functionality is too application-specific to be included in the core of the Framework, and other times we haven’t been sure how broadly applicable the functionality is, nor did we have the time to fully design it, test it, solicit feedback on it, and so forth.
Luckily, in the vast majority of these cases, we’ve been able to build this functionality on top of the parallelizatino constructs provided in .NET 4, in some cases simply containing .NET 4 types and methods, and in other cases taking advantage of extensibility points provided in .NET 4 for the explicit purpose of enabling developers to provide their own customized functionality where the Framework leaves off. Over time, we’ve amassed a wealth of code of this ilk, and we’ve chosen to expose most of it through the ParallelExtensionsExtras project available as part of our .NET 4 samples, which can be downloaded at http://code.msdn.microsoft.com/ParExtSamples.
ParallelExtensionsExtras is not a fully tested nor stable code base, and we’re continually augmenting it with additional functionality, but it does provide a wealth of code you can use as a starting point for your own solutions or simply as a learning tool to understand how various kinds of functionality might be implemented. Starting today, we’ll be presenting a series of blog posts in which we’ll walk through a subset of this functionality, detailing its implementation and discussing how and when such functionality would be applicable. If you have any requests, questions, or comments along the way, please do let us know. In particular, we’re very interested in knowing what, if any, of this functionality you’d be interested in seeing productized and included in the .NET Framework in the future.
Posts thus far in the series:
How many of the ParallelExtensionsExtras (if any) are supported in Silverlight 4?
Parallel Extensions is not currently in Silverlight 4, and as such these ParallelExtensionsExtras unfortunately don't work on Silverlight 4 either.
We're very interested in understanding the demand for Parallel Extensions in Silverlight, so it'd be helpful for us if you could share information about your need for it there. You can also share your opinion and vote for this support on the Silverlight Feature Suggestions Forum at http://dotnet.uservoice.com/forums/4325-silverlight-feature-suggestions/suggestions/310712-plinq-and-tpl?ref=title.
Can you provide more examples using Parallel.For with functions and Subroutines? None of the samples use this....
John, what specifically are you looking for? Parallel.For accepts as one of its primary parameters a .NET delegate, which can be used to reference an anonymous method, a member function, a static function, etc.
I am really interested to see the custom TaskSchedulers as a part of .NET future enhancements. They are definitely a nice to have.
Thanks for your series of posts.
one note - the link to the code is broken.
I have a question, if I want to queue actions serially (limited to one thread from the threadpool at a time), what should I use for the extensions: SerialTaskQueue or using the OrderedTaskScheduler (which is LimitedConcurrencyTaskScheduler with limit of 1).
how would you compare them?
They're both examples for implementing the same kinds of scenarios using different approaches: neither is "better" per se, and which you'd choose as the basis for a solution would really be based on which approach you feel more comfortable with. I personally like plugging in at the TaskScheduler layer, as it provides for a great deal of flexibility. However, the SerialTaskQueue approach would also allow you to go across schedulers.
Tnx for your reply.
If anyone is interested - I posted the same question on MSDN forums:
We want to be able to start a Task at some future time, and so we were pleased to discover the StartNewDelayed() feature of TaskFactory. This is a keeper!
Stephen, we're also interested in a "ContinueWithDelayed" function that would allow the result of one task to determine how much time to delay before running a second task (see below). How might something like that be written in a similar vein to StartNewDelayed?
// Task1.Result would return an int signifying the number
// of milliseconds to delay before running Task2.
.StartNew(() => Task1())
.ContinueWithDelayed(task1 => task1.Result, () => Task2());
I'm trying to implement a scheduler of sorts that schedules actions to execute at specified periods (not a service, all client side), and I want to make use of the TPL and potentially one or more of these extension classes. The idea is to group together actions with the same period and have one timer (for each period) control their execution to reduce the amount of timers/threads needed. And then invoke the actions in a parallel manner as the periods elapse. Any recommendations on how to accomplish something like this?
I have some ideas on how I could make use of the TPL, but I have a couple questions:
Is there anyway to inject additional arguments into the task that the scheduler can make use of through one of the Factory.StartNew() or Task.Start() methods? Ideally, I would like users to be able to use Factory.StartNew() and pass in my custom scheduler with some additional arguments (e.g. the period in which to execute the action passed in).
Do you have a recommended way of mimicking running the same task multiple times? I would like to run the same action every time the period elapses, but I would like to return a handle to something (maybe a task) that the user can cancel, and/or do further processing on.
CancellationTokenSource tokenSource = ...
var task = PeriodicTaskFactory.StartNew<object>(Task1Action, tokenSource.Token).ContinueWith(task1 => DoSomethingWithResult(task1.Result))
You could pass parameters through to the scheduler via object state on the Task. The scheduler could then look at that via the task's AsyncState property.
However, given your goal of the snippet you shared, I don't think a custom TaskScheduler is the right solution here. I'm not entirely clear on the semantics you want... are you saying you want to run an action repeatedly in a loop with delays in between, and then once cancellation is requested and the loop stops, run a continuation? If so, if you were using the Async CTP or .NET 4.5, you could do such a polling loop with code like:
public static async Task RunPeriodicLoop(
Action action, int millisecondsDelay, CancellationToken cancellationToken)
You could implement similar semantics without the async/await feature and just using ContinueWith in the implementation of this method.
If you're goal is just to do something on a period basis and to be able to cancel that thing, you could also just use a Timer directory, and connect a CancellationToken to the timer to stop it with code something like: cancellationToken.Register(() => timer.Dispose());
I hope that helps.
Thanks a lot Stephen for your feedback! Hopefully I'm not rambling too much here...
First of all, I can't make use of .NET 4.5 for the time being, but would like this code to be easily portable once .NET 4.5 is released. I'll take a look into the CTP; I have not used it before. It is unclear to me how you would implement your snippet with ContinueWith() and what the method would return?
As for your question: "are you saying you want to run an action repeatedly in a loop with delays in between, and then once cancellation is requested and the loop stops, run a continuation?"
- Actually, I would like the optional continuation action to run every time the period elapses, not once the parent 'task' is cancelled and the loop stops. So, with respect to the example snippet I previously posted, if the period was 5 seconds, Task1Action would be executed ~ every 5 seconds and then DoSomethingWithResult(task1.result) would be executed asynchronously each time Task1Action completes.
Basically I would like to extend the current capabilities of Tasks to periodic actions, ideally making use of the TPL and not having to control all of the thread creation and management myself. However, I am open for suggestions, such as Tasks and the TPL not being the optimal solution.
To clarify my requirements a bit more...
- Each time a new 'data point' is requested by our client, an 'action' will be executed on a periodic basis by an object that is responsible for gathering that data and/or doing some further processing on it. This action is typically a very short running operation. (For the sake of not complicating the scenario any more, let's just say that all of that data is readily available on the client.)
- We will have potentially hundreds/thousands of these 'data points' actively requested by a single client at any given time.
- In most cases, the period in which to execute each action (specified by the client when 'registering' one of these actions) will be the same, e.g. 5 seconds. Every time that 5 second interval elapses, each of the actions with that given period should be executed.
- The first time the action gets executed is not important, only that it executes at approximately the specified interval.
- Clients need to be able to unregister/cancel their periodic 'action' at any time.
We are thinking that creating potentially thousands of timers is not the correct solution since that is a lot of overhead. And, since it is not important when a given action first executes, our thinking is to have one timer for each period and then have some mechanism in place for executing all of the registered actions each time that interval elapses. We can consider these actions all independent and therefore we would like to make use of the TPL and execute the actions in parallel (to an extent based on max concurrency settings, etc.).
From an API perspective, I would like the registering of each action to resemble creating/starting a new task since ideally we will be invoking the actions in parallel each time the interval elapses and potentially creating/starting tasks behind the scenes in order to do so. Looking into the different Task creation/starting options, the recommended option is to use Task.Factory.StartNew(...) "for perfromance reasons" and from the task returned from that call, you can provide a continuation task via ContinueWith() and so on. I would like our API to resemble this.
Sorry for my delay in responding.
A key point here is that a Task can only complete once, so you want to create an API where handing out a Task represents one logical operation. That operation could of course have multiple subcomponents, but a Task can only transition from a non-completed state to a completed state once, and once completed, it'll remain completed. Given that, if you want to hand back a task to a consumer registering a periodic Action, it's not clear when you'd complete that task... you couldn't "complete" it each time the action ran, as that would be completing it more than once. It seems then you'd only be able to complete it if the action were ever canceled, and otherwise the action would remain registered "forever", and thus the task would never complete.
Having hundreds or thousands of timers outstanding at a time should not be a problem (though perf of timers was made much better in .NET 4.5), but coalescing into time intervals as you state would also be fine.
Regarding creating tasks, if you're looking to manually complete a task (e.g. create a task for an action, use a Timer to do processing, and complete the task at an appropriate time), you should be looking at the TaskCompletionSource<TResult> type, which allows you to use a Task to represent arbitrary operations.
Alternatively, if you're looking for a type that would allow you to push back multiple signals to the caller, one for each time the operation runs, you might actually be better off with something like IObservable<T>, and then you could consider using Reactive Extensions to process all of these events coming back.
Hi Stephen, is there any chance ParallelExtensionsExtras will be released under a real free license?
I'd love to use it in a MonoTouch project but the license says I have to be writing code for Windows, although it runs just fine on iOS. It's 2012, why don't we get past this already.