Is it ok to use nested Parallel.For loops?

re: Is it ok to use nested Parallel.For loops?

Stephen Toub - MSFT — Tue, 20 Mar 2012 03:59:34 GMT

Hi Valera-

Parallel.For is a blocking call in the same way that a for loop is blocking: the caller won't progress past the loop until the loop has finished its work.

It's true that the Parallel.For won't return until the tasks it's used for processing have completed, but as noted in this post, the loop will run the first task it creates synchronously on the current thread, it will only have a single additional task sitting around for an additional thread to pick up, and it'll cancel any outstanding tasks that aren't being used when all of the work for the loop has completed.

Your example of 15 nested Parallel.Fors should not cause thread starvation or deadlock... give it a try.

re: Is it ok to use nested Parallel.For loops?

Valera Kolupaev — Fri, 16 Mar 2012 18:04:45 GMT

Is't Parallel.For is a locking call?

I assume that it locks and waits till child tasks will complete it's execution.

Would't 15 nested Parallel.For cause thread starvation on ThreadPool and deadlock it this case?

re: Is it ok to use nested Parallel.For loops?

Stephen Toub - MSFT — Thu, 15 Mar 2012 16:04:33 GMT

Harry, if/when this occurs, typically it's because either:

1) You have a significant load imbalance in terms of how long it takes to process certain items. It could be that the threads you see running for a while at the end are all processing their last item, which is just taking longer than some of the other items, and thus the other threads have all finished with no more work to be done.

2) The parallel loop partitioned the data in chunks such that, based on the how the data was distributed, some of the threads got starved for data towards the end of the loop. If that's the case, you could control the partitioning in a manner that's better for your data set. The limit of this is ensuring that each thread never takes more than 1 item at a time... to achieve that, you could use a partitioner like the one described at drdobbs.com/.../224600406, or if you're using .NET 4.5, you can use the built-in variant of this via Partitioner.Create(source, EnumerablePartitionerOptions.NoBuffering).

re: Is it ok to use nested Parallel.For loops?

Harry — Thu, 15 Mar 2012 09:51:19 GMT

Could you also explain why Parallel.For does not always utilize all available hardware threads when running long running tasks. E.g. in our case we have 24 long running task we want to run in parallel 8 at a time on a 8 hardware thread machine, but typically at the end of the run less than 8 tasks are running, slowing the total run down considerably. It is as if tasks are not executed even though a thread should be available...