Distributed Computing Gets Spruced-Up with CCR/DSS!

asehmi — Wed, 27 Feb 2008 03:00:40 GMT

The reality of distributed systems development today is the complexity that eventually if not immediately bogs it down. Instead of being fun, exciting and easy to develop, distributed systems are often hard and unwieldy to design and work with. We continue to face the same old issues time and again yet never appear to come up with game-changing simplifying solutions to our common issues of complexity which, amongst others, are:

The large variety of participants: consumers, providers, intermediaries, administrators, etc.
Highly heterogeneous systems and architecture, implementation and deployment practices
Highly autonomous and independent systems & services
Highly distributed information sources and services

The challenges developers, architects, testers, etc. face are many. In general we need to be able to:

Apply (re)configuration in a live running system
Coordinate many simultaneous asynchronous interactions
Support high throughputs (and afford low latencies)
Provide simple lifecycle services like starting and stopping components dynamically
Monitor, interact with, and debug a running (distributed) system
Ideally be able to develop the system in simulated conditions or a test environment
Span multiple physically or logically distributed compute units
Re-use components across multiple hardware platforms & devices

A typical solution to these challenges might be to centralise & standardise, but that's a kind of anti-pattern for many types of distributed systems scenarios today and simply serves to:

Reduce the variety of systems, i.e. seeking economies of scale
Remove the positive benefits of heterogeneity in a distributed system
Eliminate the autonomy
Put demands on 'perfect information' which is really hard to do consistently and reliably
Hardwire the connections between components of the system

A better solution is to simply accept diversity and decentralize!

Accept variety, i.e. seeking economies of innovation
Encourage autonomy
Accept heterogeneity
Understand that that custom, personalized, networked information is normal
Imagine an adaptable, open, distributed information system

Recently I've been energized by the power of the Concurrency and Coordination Runtime (CCR) + Decentralized Software Services (DSS) combined features, and dare I say, paradigm shifting concepts to fix a lot of the problems we have today in building distributed systems.

CCR provides concurrent (multi-core) programming support by way of some key pattern implementations and developer conveniences
DSS provides higher level support to create, host, and discover lightweight services
DSS services are oriented to have a state and react to incoming request and updates, and sends update and request to other services
DSS services state can be serialized and a service can migrate to another host
DSS services are naturally RESTful
The Visual Programming Language designer can be used to define, compose, monitor and manage DSS services
CCR/DSS can be hosted in a device and is capable of running under .NET Compact Framework (and .NET Micro Framework coming soon)

Side Note - How does CCR/DSS relate to WCF?

WCF is more about communications infrastructure – standards-based ‘plumbing’ – whereas CCR/DSS is a high level programming model for distributed service-based and resource-oriented systems (however initially targeted to Robotics applications). Some key CCR features such as ports, memory queues, dispatchers, arbiters, etc. are hard to implement and beyond the scope of WCF. A better comparison would be the combo of WCF+WF, but still there would be many features and patterns missing from both sides. Apples to Oranges - just avoid trying to force a comparison and understand the core use cases for each technology set. In my opinion the patterns supported in CCR are also of immense value to WCF+WF developers. I wonder if anyone has implemented CCR patterns in WCF and furthermore provided a DSSP transport protocol binding for WCF? That would be really cool!

Resources

The Microsoft Robotics Studio includes the Concurrency and Coordination Runtime (CCR), Decentralized Software Services (DSS), Visual Programming Language (VPL), and a virtual worlds (robotics) simulation environment! If you want to dive straight into CCR/DSS check out the Wiki, watch a video or two and install Microsoft Robotics Studio.

As part of the Microsoft Robotics Studio (1.5) release, Microsoft has placed the Decentralized Software Services Protocol (DSSP) under its Open Specification Promise. This means that the Microsoft patents for the services-oriented protocol specification may be used by anyone in the world, at no cost, and for any type of development including free software, open source, academic, or commercial, enabling interoperability among a wide variety of devices and applications. The XSD schema for DSSP is also available.

MSRS Blog	Main forum page	DSS forum	CCR forum
VPL forum	Hardware configuration and troubleshooting forum	Simulation forum	Community examples forum

The creators of CCR/DSS are George Chrysanthakopoulos and Henrik Frystyk Nielsen. Search on their names and you'll be very impressed.

CCR/DSS is not just for Robotics - it'll help you spruce-up your enterprise distributed systems architectures too! Like I said earlier, I'm pretty jazzed by this technology and hope to be writing about it again very soon.

Sehmi-Conscious Thoughts : WCF

Distributed Computing Gets Spruced-Up with CCR/DSS!