Sehmi-Conscious Thoughts : Distributed Computing

Architecture Journal #17 - Article on Distributed Embedded Systems

asehmi — Tue, 07 Oct 2008 12:21:01 GMT

I've been working on Distributed Embedded Systems and related communications technology recently so for the "Distributed Computing", Issue #17, of the Microsoft Architecture Journal I wrote a paper on the subject. My Aim was to open up - 'expose' - this very exciting field to the "traditional" distributed systems community. As the industry moves from standalone, smart devices to connected devices and then towards systems of service-oriented connected devices, we will begin to see much more overlap with "traditional" distributed systems design and architecture practices. Not everything we (traditional distributed systems developers and architects) know translates easily to the world of distributed embedded systems, so it's vitally important to understand the different challenges and issues faced, technical imperatives, development strategies, and technology road maps. Hopefully, in my paper, I've done some justice towards answering those questions.

Apart from my own paper there's a fab collections of others. I haven't started reading them as yet, but personally know and respect many of the authors. I know attendees of PDC2008 will be receiving a copy of the Journal, so we've been calling this the "PDC Issue"! The new editorial team are beginning to make their positive marks and you'll notice a new clean, bold look to the Journal as it starts to establish a shared identity with Microsoft other "architecture evangelism" assets. I approve wholeheartedly, but it still hasn't got the minimalism I think we prefer in Europe Western/Northern. But at the end of day it's about content value and that you certainly have in abundance.

http://www.msarchitecturejournal.com/pdf/Journal17.pdf

On another note, my colleague, Yuriy Zaytsev, alerted me to a Japan MITI survey which concluded:

(4) Over 40% of the quality problems with embedded products after shipment are caused by software errors.
The leading cause of the quality problems with embedded products after shipment was software errors (46.3%), followed by hardware design faults (21.8%) and production faults (12.7%). A trend over the past four years is that the percentage of faults during product conceptualization and specification continues to fall year after year. Software errors have remained the major cause of quality problems for the past four years.
(6) Advanced* embedded software engineers are needed.
Although overall shortages are easing, demand is still high for embedded software engineers with advanced skills; the shortage rate of entry-level engineers was 26.3%, while that of advanced (experienced) engineers was 48.4%. Over the three years from 2006, the shortage rate of entry-level engineers fell from 44.2% to 26.3%, showing the largest decline among all the categories. The shortage rate of advanced engineers stood out last year at 64.9%, but the rate declined this year, making the disparity between different skill levels restored to an extent equivalent to that of two years ago.

*Advanced: Capable of implementing technologies, methodologies and businesses while exercising leadership or capable of implementing these while leading internal and external teams by means of standardization and the development of new technologies.
Intermediate: Capable of discovering and solving challenges in business operations while exercising leadership.
Entry-level: Capable of discovering and solving challenges in business operations under the leadership of higher-level engineers.
(7) Software engineering methods have been introduced for improving quality and developing human resources.
The respondents’ most common reason for having adopted solutions developed and provided by the Software Engineering Center was “quality improvement” (cited by more than 75%), followed by “human resource development” (more than 60%). This suggests that many respondents recognize SEC’s solutions as effective for solving major challenges in developing embedded software. About 70% of those who have adopted SEC’s solutions give high marks to them, rating them as being “beyond expectations” or “just as expected.”

The questions that arise for me, especially in the context of realizing Distributed Embedded Systems solutions (i.e. Complex Embedded Systems), are:

Which, if any, of the software development processes, design and architecture methods we use in traditional software development, can be leveraged in Embedded Systems development to mitigate these quality problems and skills shortages?
What is Microsoft doing to address these issues in terms of next generation tool chains for this industry?

Over the next few months we'll certainly be exploring these questions in more detail and report back our learning.

Anyway, here’s wishing you a great read of Journal 17!

JAOO 2008 Presentation

asehmi — Fri, 03 Oct 2008 17:46:19 GMT

In a previous post I listed all the Microsoft folks presenting at JAOO 2008. Well that event has successfully come and gone this week and I'm pleased to share my presentation with the community.

My talk went pretty well and I achieved my objective to explain the relevance of Distributed Embedded Systems to traditional distributed systems developers and architects. I briefly discussed the Device Profile for Web Services and CCR/DSS as good candidates for DES communications infrastructure, choosing to describe these technologies mainly because they would be relevant to my audience in non-DES scenarios too.

The back end of my talk hints at Fabriq4Dss - an implementation of Fabriq (remember that?) on CCR/DSS. I skipped the demos of Fabriq4Dss because I only had 45 minutes for the session, but rest-assured you'll be hearing more about this here soon.

Other highlights of the conference were meeting, Gregor Hohpe and Jeff Sutherland. Gregor attended my session after which I demoed another recent CCR/DSS/VPL project of mine ("PIE") related to his famous book on Enterprise Integration Patterns. We're plotting something together which I hope can be pulled off.

I sat next to Jeff Sutherland at the speaker's dinner. He is of course the co-creator of Scrum and as we chatted he intrigued me with a number of non-S/W development uses of the process. Having myself, just prior to JAOO, attended at a team offsite exploring elements of virtual teaming and organizational design with Dr Homa Bahrami, I thought this conversation with Jeff was sweet serendipity - which in turn reminded me of the late Roger Needham's words, "Serendipity is looking for a needle in a haystack and finding the farmer's daughter." How-so-very-true!

JAOO was a great conference and I'd like to extend my thanks to Beat Schwegler, Microsoft Denmark and the JAOO organizing committee for having me.

P.S. We'll be talking a lot more about these topics at Tech-Ed EMEA Developers 2008, 10-14 November (select Embedded Systems, Architecture tracks).

Gorgeous Fractals with VPL and Distributed DSS Services

asehmi — Mon, 23 Jun 2008 08:43:21 GMT

I think Angel is fast becoming one of VPL's biggest fans. After discussing some ideas with him about simple Map-Reduce scenarios for his agent toolkit and a grid system that I'm building, he went ahead and did it. Check out his distributed Mandelbrot fractal generator. Gorgeous results. A bit over-engineered as a fractal calculator, but that wasn't the point really!

You can get Angel's blog, agent toolkit and sample applications here.

Using the Visual Programming Language (VPL) as a Deployment Tool

asehmi — Mon, 16 Jun 2008 11:43:48 GMT

When building applications using CCR, DSS and VPL one doesn't need to know a priori the precise order of data and message flows between services. Many of the VPL examples provided with the MRDS show services already 'wired' up which can give the impression that one needs to know this information and design one's set of services to be orchestrated as such up front. What if you don't know this information up front but still want to use the VPL? Well, there's the pub/sub pattern supported by DSS out of the box which is very handy and well-understood. Or one can define a 'host' service that acts as a message forwarding system based on its own service directory (perhaps using the one built into DSS). Or one can use an external configuration file that defines the orchestration itself (like a BPML specification, say). Additionally, one could use UPnP device (service) discovery.

But orchestration is not the only thing, what about deployment? Well, the VPL also supports service deployment and this works across multiple physical nodes too!

Angel Lopez has extended his Web Crawler sample and without knowing the orchestration a priori uses VPL simply to create multiple instances of his several service 'agents'. The actual 'orchestration' is emergent as a result of collaboration between his agents and the 'agent host' - in fact he can have several agent hosts in the same Web Crawler application. The agent hosts forward messages to service agents based on their capabilities inferred from a logical 'URI' which gets physically bound when the agents themselves register with their (machine) local hosts. Should more than one agent service offer the same capability, such as web crawling or page downloading, the agent host will automatically load balance across them.

An interesting benefit of using the new VPL (in the April 2008 CTP of MRDS) is its ability to deploy services on multiple DSS nodes which themselves can be hosted on physically different machines. Angel does this with his Web Crawler application and together with the application-level load balancing facility his sample offers a neat and simple outline of potentially powerful solutions that you can build using this approach. I particularly liked the lack of a priori orchestration and the use of VPL as a multi-node deployment tool.

Check out Angel's application here.

CCR/DSS Use Cases in the Enterprise

asehmi — Fri, 13 Jun 2008 00:40:18 GMT

I'm often asked what Enterprise customers are using CCR/DSS in serious, i.e. pretty significant, production deployments. I'm not at liberty to disclose the names of our customers without their permission - a few will be made public soon - but I can say there are quite a lot of application domains where this technology is being applied seriously. Here's a list of scenarios I mined from our internal list of CCR/DSS adoption customers in the Enterprise:

Very large scale web request load balancing & IO handling (online social networks)
Security systems (distributed sensor networks)
Package sorting system (real-time logistics monitoring and control)
Real time processing of seismic data in drilling control (industrial automation & control)
Supply Chain Modeling (simulation and control)
Law enforcement information intercept/inspection (large scale complex event processing)
Automated inventory inspection system in FMCG/retail scenarios (vision processing)
Financial trading systems (transaction flow routing and processing)
Business processes - there is an academic BPEL implementation on CCR
Scientific computing – MPI primitives concisely written as CCR patterns (exchange, shift, double shift)

Hopefully that's useful and piques your interest to look at this Microsoft Robotics (CCR/DSS) stuff deeper.

Fun with Microsoft Robotics Studio

asehmi — Sun, 20 Apr 2008 22:50:19 GMT

Not bad for a Saturday project down in sunny Buenos Aires! My friend and professional geek Angel "Java" Lopez has recently rekindled his love of distributed computing (and Agents) through MRDS and specifically CCR/DSS. We're going to have a lot more fun with this stuff in the coming weeks. In the mean time check out his great little experiment and download the source code here.

Angel concludes:

All these tools opened my brain. Writing DSS service component is a new way of thinking and programming, something I want to explore for years: how to write objects with one-way message processing, in a distributed and parallel way. I envision some system that model Minsky ideas about the human mind.

Well, it’s a new brave world. Dorothy: we are not in Kansas anymore…

I couldn't agree more!

Microsoft Robotics Developer Studio 2008 CTP April Available

asehmi — Thu, 10 Apr 2008 10:33:29 GMT

The high level set of changes is listed below. For me #2 and #3 are golden, but be sure to drill down to find out the details... like async streaming support!

Improved security model with support for Message Level Authorization using Role-based access control allowing to fine-grained control of who can access services and how.
Support for LINQ queries in DSSP subscriptions enabling applications to create complex queries that can be executed close to the data source instead of requiring moving data across the network
Support for creating applications that run on multiple DSS nodes using Visual Programming Language and DSS Manifest Editor. This makes it much simpler to create applications that run across nodes, either on the same machine or across the network. When an application containing multiple nodes is to be started, VPL creates individual deploy packages for each node and fires them up across the network.
Support for record and playback functionality in Visual Simulation Environment enabling it to record the position and orientation of each entity and save that information to a file which later can be replayed.
Support for storing simulation media resources such as texture maps, bitmaps, and mesh files directly as part of the CLR assemblies in the form of embedded resources rather than as external files.
New Simulation floor plan editor which allows the user to put together walls, doors, windows, etc. to represent the interior of a room, house, or other building.

Quoting from Marc Mercuri's blog:

At RoboBusiness today, Microsoft announced the release of Microsoft Robotics Developer Studio 2008 CTP1.

Get the free download here -

http://www.microsoft.com/downloads/details.aspx?FamilyId=EB00C558-2163-45A5-BEFE-531AD48BC525&displaylang=en

Mary-Jo Foley has an article on why you should care ('Why business users should grab a copy of Microsoft's new robotics' toolkit')

See that here - http://blogs.zdnet.com/microsoft/?p=1325

The folks who attended the conference also saw some videos from my new project. CNET references it in the article - "Microsoft's new gig: Virtual host to robot competitions".

See that here - http://www.news.com/8301-10784_3-9914959-7.html.

Outside of this, mum's the word until the clock at www.robochamps.com hit's 0's across the board.

By the way, in addition to RoboChamps, Marc also planned and built Tafiti – the guy is smart!

For further information, you can also check out:

1. MSDN Developer Centre

2. MRDS Team Blog

3. MRDS News Feed

4. MRDS Community Forum

Distributed Computing Gets Spruced-Up with CCR/DSS!

asehmi — Wed, 27 Feb 2008 03:00:40 GMT

The reality of distributed systems development today is the complexity that eventually if not immediately bogs it down. Instead of being fun, exciting and easy to develop, distributed systems are often hard and unwieldy to design and work with. We continue to face the same old issues time and again yet never appear to come up with game-changing simplifying solutions to our common issues of complexity which, amongst others, are:

The large variety of participants: consumers, providers, intermediaries, administrators, etc.
Highly heterogeneous systems and architecture, implementation and deployment practices
Highly autonomous and independent systems & services
Highly distributed information sources and services

The challenges developers, architects, testers, etc. face are many. In general we need to be able to:

Apply (re)configuration in a live running system
Coordinate many simultaneous asynchronous interactions
Support high throughputs (and afford low latencies)
Provide simple lifecycle services like starting and stopping components dynamically
Monitor, interact with, and debug a running (distributed) system
Ideally be able to develop the system in simulated conditions or a test environment
Span multiple physically or logically distributed compute units
Re-use components across multiple hardware platforms & devices

A typical solution to these challenges might be to centralise & standardise, but that's a kind of anti-pattern for many types of distributed systems scenarios today and simply serves to:

Reduce the variety of systems, i.e. seeking economies of scale
Remove the positive benefits of heterogeneity in a distributed system
Eliminate the autonomy
Put demands on 'perfect information' which is really hard to do consistently and reliably
Hardwire the connections between components of the system

A better solution is to simply accept diversity and decentralize!

Accept variety, i.e. seeking economies of innovation
Encourage autonomy
Accept heterogeneity
Understand that that custom, personalized, networked information is normal
Imagine an adaptable, open, distributed information system

Recently I've been energized by the power of the Concurrency and Coordination Runtime (CCR) + Decentralized Software Services (DSS) combined features, and dare I say, paradigm shifting concepts to fix a lot of the problems we have today in building distributed systems.

CCR provides concurrent (multi-core) programming support by way of some key pattern implementations and developer conveniences
DSS provides higher level support to create, host, and discover lightweight services
DSS services are oriented to have a state and react to incoming request and updates, and sends update and request to other services
DSS services state can be serialized and a service can migrate to another host
DSS services are naturally RESTful
The Visual Programming Language designer can be used to define, compose, monitor and manage DSS services
CCR/DSS can be hosted in a device and is capable of running under .NET Compact Framework (and .NET Micro Framework coming soon)

Side Note - How does CCR/DSS relate to WCF?

WCF is more about communications infrastructure – standards-based ‘plumbing’ – whereas CCR/DSS is a high level programming model for distributed service-based and resource-oriented systems (however initially targeted to Robotics applications). Some key CCR features such as ports, memory queues, dispatchers, arbiters, etc. are hard to implement and beyond the scope of WCF. A better comparison would be the combo of WCF+WF, but still there would be many features and patterns missing from both sides. Apples to Oranges - just avoid trying to force a comparison and understand the core use cases for each technology set. In my opinion the patterns supported in CCR are also of immense value to WCF+WF developers. I wonder if anyone has implemented CCR patterns in WCF and furthermore provided a DSSP transport protocol binding for WCF? That would be really cool!

Resources

The Microsoft Robotics Studio includes the Concurrency and Coordination Runtime (CCR), Decentralized Software Services (DSS), Visual Programming Language (VPL), and a virtual worlds (robotics) simulation environment! If you want to dive straight into CCR/DSS check out the Wiki, watch a video or two and install Microsoft Robotics Studio.

As part of the Microsoft Robotics Studio (1.5) release, Microsoft has placed the Decentralized Software Services Protocol (DSSP) under its Open Specification Promise. This means that the Microsoft patents for the services-oriented protocol specification may be used by anyone in the world, at no cost, and for any type of development including free software, open source, academic, or commercial, enabling interoperability among a wide variety of devices and applications. The XSD schema for DSSP is also available.

MSRS Blog	Main forum page	DSS forum	CCR forum
VPL forum	Hardware configuration and troubleshooting forum	Simulation forum	Community examples forum

The creators of CCR/DSS are George Chrysanthakopoulos and Henrik Frystyk Nielsen. Search on their names and you'll be very impressed.

CCR/DSS is not just for Robotics - it'll help you spruce-up your enterprise distributed systems architectures too! Like I said earlier, I'm pretty jazzed by this technology and hope to be writing about it again very soon.