All about Async/Await, System.Threading.Tasks, System.Collections.Concurrent, System.Linq, and more…
Are you using Parallel LINQ (PLINQ), the Task Parallel Library (TPL), or any of the new coordination and synchronization primitives in .NET 4 (or in the Parallel Extensions June 2008 CTP or with the recent Reactive Extensions release)? Are you planning to use or are you already using this support in a production application or library?
We'd love to hear about it, and if you have time, what your experiences have been (good or bad). Please email me at stoub at microsoft dot com... we're looking forward to hearing from you. Thanks!
I'm using Parallel Extensions eveyday since the first CTP. I've sent you my feedback on an e-mail.
I'm using Parallel Extensions too, I'll send you my feedback by email.
I played around with PLINQ and the TPL since it was first released, but nothing real serious. Just experiments so that I'll be familiar with it when it comes out in 4.0.
BUT...I've recently spent a lot of time working with DraydLinq, which implicitly uses PLINQ during its node execution. But I've found the DryadLinq isnt always able to get full parallelism on all cores, so I've been having to dig into PLINQ and the TPL to explicitly parallelize my code across all cores.
I have really grown to like the TPL architecture, and have started to use it in almost all my code.
Good job guys!
Here's why I'm not using Task parallel library since... since ThreadPool in .NET 1.1.
Imagine a real world application (i.e. *not* a hello world window rendering few spheres and a tiled infinite surface). A window with two controls: let’s say control A uses a library to render few spheres and a tiled infinite surface, instantly creating 500 tasks for each line of pixels and control B creates a single task to go to a server to retrieve a value which then gets displayed.
Guess what, the value will get displayed only after everything was rendered in control A.
Conclusion: thread pool (the way it’s done right now) can’t be used in third party libraries or componentized applications with unlimited number of work items for each component.
Yes, you probably can implement your own scheduler or something (wake me up when there’s a public constructor for preemptive TaskScheduler), but it would be *easier* for me to manually spawn four threads (or however many cores you have) for control A and another thread or a work item for control B and plainly do the job there.
Just started using it for some data-parellizable number crunching (where it makes code nicer to read by a long shot) and asynchronous tasks. Nothing very heavy lifting, but the basic API is easy to learn and understand. I'm a fan so far.
Gaston, thanks for the email.
Oronzo, looking forward to receiving your email.
John, what kinds of things are you doing with DryadLINQ? We'd love feedback on that as well... feel free to email me if you'd prefer. Note that the recent DryadLINQ refresh uses the same updated PLINQ that shipped in the Reactive Extensions to .NET... you should hopefully see better core utilization with that refresh of DryadLINQ.
Iso, if you like the Task API but still want a thread for each operation, you can specify TaskCreationOptions.LongRunning; the implementation in .NET 4 will create a Thread for each LongRunning Task. Regarding the kind of scheduing you desire, check out the RoundRobinTaskScheduler in the samples at http://code.msdn.microsoft.com/ParExtSamples... you might also be interested in the article at http://msdn.microsoft.com/en-us/magazine/dd347845.aspx.
Sebastian, thanks for letting us know. If you're comfortable emailing me any more details about either your async code or your number crunching (e.g. what software this is for, what kinds of speedups you've seen, any issues you've hit, etc.), I'd be interested in hearing more.
Thanks for all of the feedback. Everyone else, please do continue to let us know about your Parallel Extensions usage... again, feel free to email me directly.
We'd like to convert to it. We have had something similar in our product for a year now. Would gladly discard and use TPL.
BUT, we need for SL 4. Will it make the cut?
We're using Parallel Extensions inside a brand new application that uses Prism and WCF Data Services. We use TPL in particular to simplify the communication between the WPF client and the remote WCF Data Service.
Our concern now is regarding the support for Silverlight. Could you give us some insight into this?
@Ward Bell and @Michele - One of the most important design considerations for Silverlight is to keep it as streamlined as possible in terms of download size, API volume, and concept count. As such, it primarily focuses on the very highest demanded feature requests. While Parallel Extensions will not be a part of Silverlight 4--and a lot of thinking went into this--it IS a candidate for a future version of Silverlight, if demand is high enough.
It would really help to know as much as possible about how you plan to use Parallel Extensions on Silverlight and how much you hope to gain. I have a system specifically to keep track of requests for exactly this. Please share more of your details at firstname.lastname@example.org so we can best take your needs into account.
Yes, I used TPL. I used it in a data transformation application. look http://www.cnblogs.com/redmoon/archive/2010/01/01/1637608.html (Chinese)
Hey toub, what email do I use to contact you? toub at ms dot com?
Kevin, thanks for letting us know.
John, stoub at microsoft dot com. Thanks in advance!
We are extensively using PLINQ, and I used the Task parallel library in a critical area of an application before resorting back to the old Threading model.
The reason for this is that the application in question must also provide the ability to do distributed processing (such as in a PC farm) as well as local core threaded processes to convert a massive amount of raw data from an external provider into a common format used by many other applications in our company.
I couldn't find a way to steal Tasks out of the a task processing loop and make them eligible to process on another PC without significantly re-writing the task identification process (and even in this scenario, the task processor would have to check to see if the task that's been queued up is already being worked on by another PC and immediately terminate, thus starting another process needlessly.) This is compounded by an issue where there is an enormous start-up cost in terms of time and memory on another PC before the application can begin to assist in the task load. So the parent processing PC has to max out it's processing load before allowing other PCs to assist in the workload.
Admittedly, my method isn't preferred, but I at least know how many processes I've assigned against the count of available cores to roughly determine the processing workload.
I like the TPL because the client code is shorter, easier to follow, and a heck of a lot cleaner and effecient thread hashing code I come up with on my own. It's perfect if all tasks are exclusively local. And while it's possible I've just over looked something, it just doesn't seem to lend itself to the type of implementation I'm doing. At least not in a straightforward way.
As far as your comment about Silverlight, I think the hopeful gain in those applications will be similar to other client-side non-Silverlight applications.
Especially if Silverlight becomes a preferred platform for gaming, there is significant potential for all of the threading utilizations that exist today in other non-silverlight game frameworks (A.I., stat hashing, geometry calculation, etc). And especially if Microsoft doesn't plan on supporting 3D natively in Silverlight, a TPL type of implementation would enable third party developers a more efficient way of developing 3D APIs on their own.
Dan, thanks for the detailed response! Excellent information. I have some follow-up questions, if you don't mind. Could you email me? (stoub at microsoft dot com)
how i can use plinq pfor in silverlight 3, 4