All about Async/Await, System.Threading.Tasks, System.Collections.Concurrent, System.Linq, and more…
We exert a good deal of effort ensuring that the APIs we provide are consistent within Parallel Extensions as well as with the rest of the .NET Framework. This is from many angles, including behavior and general design, but also naming. So when there are slight differences in naming, it raises questions.
One occurrence of such a slight naming difference is between the MaxDegreeOfParallelism property on ParallelOptions using by Parallel.For/ForEach/Invoke and the WithDegreeOfParallelism PLINQ extension method. Why isn’t it ParallelOptions.DegreeOfParallelism, or WithMaxDegreeOfParallelism?
The reason has to do with requirements imposed by the design of Parallel and PLINQ, the latter of which is a bit more restrictive in some ways. Parallel works using an under-the-covers concept we refer to as replicating tasks. The concept is that a loop will start with one task for processing the loop, but if more threads become available to assist in the processing, additional tasks will be created to run on those threads. This enables minimization of resource consumption, e.g. if most of the threads in the ThreadPool are busy processing other work, a parallel loop that starts executing will execute with a smaller number of threads, until such time as the threads doing the rest of that processing are freed up and can assist in the execution of the loop. Given this, it would be inaccurate to state that ParallelOptions enables the specification of a DegreeOfParallelism, because it’s really a maximum degree: the loop starts with a degree of 1, and may work its way up to any maximum that’s specified as resources become available.
PLINQ is different. Some important Standard Query Operators in PLINQ require communication between the threads involved in the processing of the query, including some that rely on a Barrier to enable threads to operate in lock-step. The PLINQ design requires that a specific number of threads be actively involved for the query to make any progress. Thus when you specify a DegreeOfParallelism for PLINQ, you’re specifying the actual number of threads that will be involved, rather than just a maximum.
It’s conceivable that in the future, Parallel could be augmented to guarantee a certain number of threads involved, at which point a DegreeOfParallelism could be added, and it’s conceivable that in the future, PLINQ could be augmented to support varying number of threads over the lifetime of a query, at which point a WithMaxDegreeOfParallelism could be added. In the end, the names were chosen to better convey exactly what’s happening in the systems, so that developers can set their expectations correctly.
PingBack from http://microsoft-sharepoint.simplynetdev.com/paralleloptionsmaxdegreeofparallelism-vs-plinq%e2%80%99s-withdegreeofparallelism/
PingBack from http://asp-net-hosting.simplynetdev.com/paralleloptionsmaxdegreeofparallelism-vs-plinq%e2%80%99s-withdegreeofparallelism/
Interesting Finds: May 30, 2009
Сoncise and understandable.
This comment comes right out of the documentation for WithDegreeOfParalellism()
It seems like the documentation contradicts what's explained in this article.
// Sets the degree of parallelism to use in a query. Degree of parallelism is
// the maximum number of concurrently executing tasks that will be used to process
// the query.
// A ParallelQuery on which to set the limit on the degrees of parallelism.
One question: If you add WithMaxDegreeOfParallelism without modifying the original behaviour of the WithDegreeOfParallelism (which makes sense, backward compatibility, and stuff) then what if somebody provides both parameters? Or will the original function get deprecated then?
@Gábor Náray: There are no current plans to add these methods, so it's difficult to say what the outcome would be; we'd need to work through the various options and figure out what best met all of the various constraints.