All about Async/Await, System.Threading.Tasks, System.Collections.Concurrent, System.Linq, and more…
Are you using the new System.Threading.Tasks.Dataflow.dll library, either from its CTPs or from the .NET 4.5 Developer Preview or Beta? We'd love to hear about it, and if you have time, what your experiences have been (good or bad). What kind of solution are you building, and how are you using TPL Dataflow in it? Has the library helped from a code maintenance perspective? From a performance perspective? Are there features you’d like to see added in the future?
Please email me at stoub at microsoft dot com. We're looking forward to hearing from you. Thanks in advance!
As soon as the version is released, I'm ready to use it.
Bu before that, I have question.
Is there a way to plug a TPL Dataflow, a buffer Block for instance with a SQL Server Service Broker to have a persistant and reliable system ?
Any work on it ?
Sébastien, thanks for the comment. Regarding your question, you can implement the ITargetBlock<TInput> and ISourceBlock<TOutput> with whatever implementation you like, including wrapping other systems.
A few years ago we @ InKnow wrote a very sophisticate and powerful data-directed server back-end for our Ontology services platform using the CCR; it was a great experience and the CCR in many cases did the job for us. We are in the process of re-writing the whole platform using Dataflow and we are thrilled with the API thus far. We had grown accustomed the CCR way of creating composition of dataflow; we were worried that we would lose that with TPL and Dataflow but we can do the same type of things with TPL and Dataflow no problem. We decided to keep the semantics of Causalities from CCR and have an elegant implementation with Dataflow. We have also redesigned and Predictive Modeling and Decision Support back-end with TPL and Dataflow as well.
Thank you so much for this great technical contribution!
InKnow Technologies, thanks for taking the time to share your experiences!
Hi Stephen - thanks for the great work you have done. I am busy getting to grips with TPL Data flow (did something similar for Java in 98, with pipes and filters, data pumps etc, so I have a fair idea of the complexities you simplified for us developers) and was wondering if we could evolve a Visual Quick Reference Poster for these blocks... I did something as a reminder for me, but I guess other noobs could also benefit... screencast.com/.../4yCb0Wf4B
What I want to do requires a lot of visuals that are not readily available, would be nice to have access to some standard icons for this technology.
I have started a project for USAID using these blocks (deadlines are crazy) had to reverse engineer the wire protocol, no ready for the proxy extender as well as Tee integration into the Azure cloud.
If you have any feedback visuals...would love to hear from you.
I've performed some changes to a my 'bar-code generator program' by using TPL and Dataflow and the things seem to go well.
In practice I reduced sensibly the code involved to keep the flow as asynchronous as possible and even while I'm testing it more and more,
I have the sensation that it is the right way to keep the thing more reliable and simple at best .
I used some of your technique I read in "async tutorials" like TPL with Async CTP and other.
I recon this program was a bit complicated , but now it's incredibly easy to read, more responsive and fast.
Thanks a lot for all that, I think this is the future.
Marineheiro: Thanks for the poster! I'll take a look and will be in touch if I have any feedback.
robo.warrior2: Thanks for sharing the details of your project. I'm glad you've found success with it.
TDF is awsome! i really hope the move out of bcl into nuget doesnt mean there will be less focus on the library and that we'll loose msdn docs and things like that, right now its a little bit difficult to find info on TDF, especially the latest versions (post ctp)
Maybe that wil change though :)
i'm currently using dataflow for a web testing app, where a limited number of browser instances are pooled to do a bunch of page test (navigating to a url, clicking around)
i figured that is a perfect match for dataflow (right?)
Thanks for your feedback. The move out of the BCL into NuGet is to facilitate a faster turnaround of features. So you should expect to see more focus not less. The support and documentation requirements of the OOBs are the same as any framework piece, so you will not be loosing the MSDN documentation. Please let us know what issues you are having difficulty finding info on, I can point to you to the right resources.
It's been just over a years that we been using the TPL and TDF and each day that design and code with it we learn so much more. We use the Dataflow in our Azure compute services to process millions of knowledge sources per second. It is also our in-process messaging paradigm between agent-process-hub (Reasoning Blackboard). Now we are experimenting with TPL and TDF with our WPF Clients.
We need a high-speed asynchronous messaging between Views, ViewModels and our TPL/TDF-centric Repository. We are developing software in the Bio-engineering space where objects can interaction freely across domains and sending messages to objects and controllers is a natural bit of process.
We have had some difficulty in debugging and hope that some the new parallel task debugging capabilities will reduce some of our stress. Our team is in the process of writing some white papers with regards to our experiences wit the TPL/TDF, WCF and WPF.
So far it looks promising! We will keep you posted.
Chief Software Architect