All about Async/Await, System.Threading.Tasks, System.Collections.Concurrent, System.Linq, and more…
One of the primary reasons we've released CTPs of Parallel Extensions is to solicit feedback on the design and functionality it provides. Does it provide all of the APIs you need to get your job done? Are there scenarios you wished the APIs supported and that you need to work around in klunky ways? And so forth. We've received some terrific feedback thus far, we've already made changes based on it, we're currently making changes based on it, and we'll continue making changes based on it moving forward.
We continually have discussions internally about additional support we could provide. Frequently these discussions result in our needing to know more about our customers' needs. There's a practically unlimited amount of functionality we could bake into Parallel Extensions, but each additional piece not only requires design, development, testing, and support, but it also can complicate other aspects of the design, potentially slow down other primary scenarios, and so forth. In order to focus our efforts, we need feedback.
Two such discussions occurred recently, and any feedback you provide would be useful in our deciding how to move forward.
The first is about shutting down a TaskManager instance. In the current implementation, we only support one kind of shutdown, which is triggered through a call to Dispose on TaskManager. This implementation is a synchronous invocation that blocks until all of the Tasks previously scheduled to the TaskManager have completed. However, there are other semantics we could potentially implement. For example, we could provide an asynchronous shutdown option, that allowed you to asynchronously call Shutdown, and the TaskManager would only be disposed of when all of the tasks scheduled to it completed. Or with a bit more internal reworking, we could support automatically canceling all of the tasks scheduled to the TaskManager. And again that could be done synchronously (canceling all and then waiting for all) or asynchronously (canceling all and then only cleaning up after the TaskManager when any that were currently executing completed). How useful would such capabilities be to you? In what scenarios would you find them useful or, more importantly, necessary? Would such capabilities be dangerous at all in your scenarios?
The second is about scheduling order. One of the benefits that a work-stealing scheduler (the kind of scheduler employed by the Task Parallel Library) provides is a distribution of work across all cores, where each core prefers to schedule new work to and pull work from its local queue(s). This scheduling can be made extremely efficient and can be made to improve locality and the like by using LIFO ordering, meaning that the task most recently scheduled is the one that will execute first. This is separate from work scheduled to the scheduler from other threads (such as an application's main thread), which will typically still be scheduled in a generally FIFO order. The question, then, is if there are scenarios you might have where you always want that FIFO-ish order, regardless of where the work is scheduled from. Such an option would likely decrease performance in some key scenarios, but it would also be more fair in terms of the order in which work gets executed. Do you have any scenarios that would require such a PreferFairness option? We'd love to hear about them if you do.
I have a scenerio where I would benefit from some kind of guarentee that a task will eventually execute.
I am retrieving images out of a database, and I want to do the loading and decoding asynchronously. My method fires off a task to load the image. For ereyone who askes for the list while the task is executing I keep a record of who they were and return a default image. When the task completes, I signal all the callers that the "final" value is availaible.
This works great on my 4 proc desktop. when I put it on my one proc laptop machine the task never completes.
If I explicitly wait on the task, everything works on both the four proc and the one proc machine.
My requirements are not hard, but I would really like a guarentee that a task will eventually get scheduled, even if nobody ever waits on it.
Some sort of fairness is absolutely necessary.
Without fairness, only kinds of problems you can solve are purely algorithmic. (Where, you do not care if some tasks may take indefinite amount of time to get started). In my opinion, it is very limited scope of very large parallel world out there.
In application domains, where you deal with physical world, such as process controls, test and measurement systems, simulations and even in massively parallel on-line games, tasks are started dynamically (their number is not known at compile time) and it can run for indefinite period of time because they life time is usually governed by external events. Applications in these domains simply cannot use TPL/Task Manager because task starvation for indefinite period of time is never acceptable.
Therefore not only some sort of fairness but additional control over their scheduling is absolutely necessary to use task parallel libraries in any domains outside algorithmic one. Ideally you could have priority based round robin scheduling among tasks. I could also accomplish the same if I could plug-in my own thread-pool under TaskManager, which can anxiously create new threads with the right priority, based on tasks that are getting started, dynamically in controlled fashion. Alternatively, Task Manager could fire an event, if a task is starved for more than x amount of time, which can be used to create more threads by that event subscriber (user) if he wishes to.
Thanks, everyone, for the great feedback!
Darshan, we've implemented an extensibility mechanism very similar to what you're looking for.
Also, based on all the good feedback here, we implemented a fairness mechanism.
I am using VS2010 CTP (published on 10/31/2008) to evaluate task parallel libraries to enhance our scheduler. If viable, I would certainly like to test newer TPL bits that implements fairness scheduling of tasks and or any extensible mechanism to eliminate task starvation. How far is next CTP that will include enhancements of TPL/Task Manager?