All about Async/Await, System.Threading.Tasks, System.Collections.Concurrent, System.Linq, and more…
Are you using the CCR (Microsoft Robotics' "Concurrency & Coordination Runtime") today in production applications or libraries, and in particular for non-robotics purposes? If so, we’d love to hear about your experiences, and any and all information you’re willing to share would be very welcome.
What do you like about it and the programming model it employs? What don’t you like about it? What features are crucial to you, and what features do you never use? How are you architecting your applications with it? Any key code samples that are representative of your application you’d like to share, or even better, a standalone implementation that highlights how you use it? If you experiemented with it but ended up not using it, why? Etc. If you’re interested and willing to share, please send me an email at stoub at microsoft dot com.
We’re excited to hear from you!
I am sorry to say that, but I've decided to go with Retlang instead of the CCR. Working with Arbiter class isn't funny, since it's syntax looks very ugly.
I blogged about it here: http://www.lanwin.de/2009/10/04/painless-multi-processing-with-retlang/
when i saw the c9 videos about the ccr it did get me very interested but i coulnd never get my head around how to work with it. it all sounded awsome when george chrysanthakopolus described it but i was never "fit it in" to my own apps
also the lack of docs and descriptions how to use it made it even harder..
i got the destinct impression that you had to use their visual programming lanuange to get the most out of ccr and the vpl turned my of since it was so similar WF but not implemented in wf..
had i been able to use wf with te ccr it would have been alot more value for me.. [as well as using it from code ofcourse]
tieing the ccr to robotics studio (unless you used a ~2 year old beta) was also a deal breaker, esp since you had to buy it to try it out
it would be very interesting to hearar more about where the ccr is today and how it relates to tpl. back in the day george talked about how incredibly fast their scheduling was, did the tpl improve that or is it the same/worse? what? :)
No. Unfortunately my code base is mostly in VB, which means I can't use the yield syntax that makes CCR so attractive.
Lanwin, no need to apologize. I'm asking to solicit general feedback, so your referral of retlang is useful. Can you tell me then what it is about retlang you like, what you dislike about it, how you use it in your applications, etc.? Again, if you'd prefer to send me an email directly rather than posting on the blog, you're welcome to do so.
al3891, thanks for the feedback. Can you elaborate on how you would have liked to use it in your apps, and what about your apps made you interested in the CCR (even though you didn't end up using it)? With regards to WCF, how would you have liked to see it integrated with WCF, i.e. what would your ideal model be? As mentioned above, please feel free to email me directly rather than post on the blog; totally up to you. Note that I'm just as interested in understanding why some folks aren't using the CCR as I am in understanding why some are.
Jonathan, understood. Iterator support is something that's been on VB users wish lists for quite some time, and it's definitely on the VB team's radar: http://blogs.msdn.com/lucian/archive/2010/01/28/core1-iterators.aspx. Can I take it from your comment, then, that your interest in the CCR was purely for its use of C# iterators for implementing asynchronous methods, just as other frameworks besides the CCR do? Can you elaborate on the kinds of asynchrony you wanted to implement for which this support would have been useful?
As many I watched some videos about CCR.
But I havn't used it at all because I have no firm understanding what it's designed for exactly and what scenarios it targets.
And also it's not easy to try. CCR is pretty hidden gem inside comercial product Robotics Studio. I have to pay MS for .NET library! We arn't used to such thing ;)
Started with the CCR for distributed work scheduling, then moved into working with Axum (which seems curiously close to the CCR - **laugh**). Overall, we're happy with the paradigms, but the lack of concrete and up-to-date examples does hurt (especially examples that are _not_ robotics-centric). Also evaluating Retlang, as @lanwin pointed out.
Shrike and Mike, thanks for your thoughts.
Mike, when you say using CCR for distributed work scheduling, are you using DSS, or WCF, or some other mechanism for inter-process communication? Or when you say distributed, do you mean between multiple components in the same process? Are there certain CCR features you use more than others? Thanks!
We're looking for a alternative paradigms for work scheduling between many physical machines (not inter-process), for jobs such as cache/data sync'ing (a la Velocity), notification delivery, and distributed calculation / simulations (which would obviously involve distributed aggregation as well, where each node broadcasts its' results to the next node in the data flow). Like I had mentioned, we started with the CCR, which made total sense (ports, arbiters, and dispatchers fit naturally). However, Axum seems to be taking the same technologies to a slightly higher level of abstraction, which is very attractive.
Overall, our focus is on message-oriented work distribution...very much like a lab full of wireless connected Bo-Bots **laugh**.
To answer your other question about connectivity (DSS / WCF), currently we're using WCF between nodes. Works well, but is very close to the "metal" of the solution.
We are developing software with near real-time requirements, and we currently in a process of taking decisions, we have the same feeling like all folks here, truly what we need, but yet does not have Microsoft support like his brothers (i.e. WCF, AXUM, and AZUR).
And we are going on the DSS which is even less spoiled from the CCR.
I hope we will take the right decision and we will have no regrets.
I've used the CCR extensively in two non-robotic applications and am impressed with the performance it can achieve and with the ease with which it is possible to write good quality parallel code.
My first project took my existing Artificial Intelligence library and added a layer to allow parallel processing of Genetic Algorithms and Neural Networks - the result was very impressive and the library was then used as the core of a work scheduling system.
The other project was purely theoretical - I wrote a database storage engine modelled on the SQL Server storage engine (complete with scatter-gather overlapped I/O, devices, filegroups, page caching, indices and tables!).
Originally the code was written using APM style constructs and was very difficult to write and even more difficult to debug. After porting to CCR the codebase was far easier to understand, debug and test. It also ran very well too. (To the point where the storage engine project was used in a slightly modified form for an HTTP download agent)
Ultimately I even wrote a rather complex hierarchical ACID locking implementation all based on the CCR.
The docs could be better but the CCR core is for the most part simple enough to use once you've broken the back of the learning curve!
The feature I used most often was CCR Interleave constructs - probably the most used construct in my code second only to DateTime ports (for timeout support).
I have not used VPL at all and only taken a token look at DSS - I remember thinking that DSS should really be merged with WCF as they looked so similar although I was probably missing the point somewhere...
Actually that reminds me - normally in a CCR project I create a class derived from CcrServiceBase that has some of the functionality found in the DSS service classes (namely the cleanup ports and associated code) just which that intermediate class was in CCR and not in DSS (actually I want the DSS version without the environment) :-)
Anyhow I would love to see the CCR become part of the .NET Framework as it clearly shares some principles with the TPL in .NET 4 (although the resourcing architecture of the CCR looks like it should be a layer on top of TPL...
Right now I'm considering introducing the CCR into an application service I am dealing with - I already have the work queued and scheduled to a pool of threads but the code is custom and brittle...
Keep up the good work but more importantly - let us know what you guys are up to! We're passionate about this toolkit and will remain so as long as we think it's still going places :-)
Oh yeah - I'd love to see a Silverlight port of the CCR - then we can really go to town writing high-performance LOB applications.
Ran, I'd be interested in hearing more about the requirements of your application, what functionality from the CCR you're relying on, etc. If you wouldn't mind emailing me at stoub at microsoft dot com, it'd be great to follow-up.
Adrian, thanks for all of the details; very helpful. As with Ran, I'd be interested in having a follow-up conversation with you, either over mail or on the phone. If you have the time and wouldn't mind emailing me, please do! Looking forward to hearing from you.
The website linked is built using .Net 4.0 MVC 2.0 and is entirely asynchronous via the CCR. From the Webcrawler to each web query.
It isn't meant to be a serious search engine, more of a learning experience.
I built a hybrid LinqToSql / CCR Data Access Layer for the project. I'm now building a Wine Review website on top of this same infastructure (http://www.sipping.com.au - Not active as yet)
I've been using the CCR for a messaging server. The messaging server is used to send enormous amounts of SMS's and MMS's out to the gateways of telecom providers (for opt-in advertising purposes).
The first version of the server was done with threads and although it worked good for small loads, it started to choke with too many connections and things going on. That's when it was starting to get apparent to me that a lot had to be rewritten in an asynchronous style. Something which is messy using callbacks alone.
The new server was architected quite differently. All logical unites are split into separate processes which all tal to each other via a central API. All units are build upon the CCR.
As said earlier, the learning curve is huge for the CCR. Although the port idea is simple, the constructs one can do with arbiters and iterators soon leaves you gasping for air. It also takes quite a mental shift from just doing things one command at a time, but that now you have to do stuff via ports and get data back via return ports.
The documentation is VERY poor. The class references in MSDN are like they have been generated from the source, without any explanation as to why and how one would use it. The overview story on MSDN is from 0-100 in 1 second, and completely illegible except for people with high foreheads.
The only sane piece was the now famous concurrent affairs post, which one has to reread at least 10 times before being able to use it oneself.
anyhow...very cool stuff. Although with all the new parallel things coming out from Microsoft in .Net 4.0 of which none even mention the CCR, I hope I didn't gamble on a dead technology.