mikechampion's weblog : Web Services and Service Architectures

Service Orientation, the Hype Cycle, and a RESTaurant

mikechampion — Wed, 23 Mar 2005 18:06:00 GMT

Edd Dumbill has a funny retrospective on the SOA hype:

2005 Will Be the Year of SOA -- Are You Ready? (.NET Developer's Journal)
2004: The year of the SOA? (ZapThink, searchwebservices.com)
Predicts 2003: SOA Comes of Age via Web Services (Gartner)
Bowstreet Predicts 2002 Will Be The 'Year of Web Services' (Bowstreet press release)

I remember having a bit of a tiff with someone from ZapThink about the time that "2004: The year of the SOA" piece came out:

Looking back on the OO "revolution" of 15 years or so ago, what would one think of a 1990 analyst report that pointed to a window of opportunity that would be closed by 1992 as C++ and CORBA mature and standardize? That's about where the SOA "revolution" is today.

I would be so nice if just once in my career I see a promising idea NOT go through the hype cycle.

I don't remember the early days of the OO/OOP revolution all that well, but I think we saw a lot of the same kinds of things as are happening with SO/SOA today: endless debates over what OO really means, attempts by vendors to conflate tools with the architectural principles they claim to facilitate, and a long slow process of sorting out the practical ideas that actually work from the prescriptions of the theorists. Whether or not it is true in hindsight about OO, it's my foresight about SO.

In a previous life I spent a fair amount of time trying to make sense out of web services architectures and SOA at W3C and promoting the SOA ideas at the Day Job. With some perspective, and from the vantage point of an organization that has no particular dog in any of these fights, it's interesting to re-examine previous posts about service orientation. I haven't found anything I would disavow, and at least one that I kindof like.

The way I see it, XML and web services just provide standard plumbing

for information. Plumbing is not sexy and it will not solve your

problems, but the *lack* of decent plumbing leads to a rather, uhh,

unsanitary situation. *Standardizing* the plumbing just makes it

easier to mass produce, create a workforce of people who know how to

use it, and generally turn it into a commodity rather than something

you have to worry about.

Anyone marketing SOA or web services as a Silver Bullet will be dodging

ordinary bullets from disgruntled customers. None of this stuff is

going to make "data transparently available everywhere in your

enterprise", but it *is* cheaper and easier to move the data around

and transform it using commodity "plumbing" rather than hand-crafted

"plumbing." That's the value proposition, and it appeals to those who

have been around long enough to know how much of a typical IT budget

has traditionally been spent on the software equivalent of hand-forging

connectors between odd-sized pipes.

In my very last performance as an SO evangelist, I took my best shot at explaining what service orientation was all about, taking a broad perspective and trying hard to avoid buzzwords.

In the real world, we use services all the time -- getting money from banks, ordering food from a restaurant, getting clothes dry cleaned, and so on. What makes these "services" is that we don't need to know anything about banking, cooking, cleaning, etc. in order to use them, we simply request them....
In a nutshell, service orientation is an approach to designing systems in which each component knows only how to request and consume the services provided by other components, and little about their internal algorithms, data structures, stored data formats, query languages, etc.

IMHO this gets at both the promise of SO and its main drawback: It promises to help us think about distributed software systems in ways that are more in synch with the ways our customers -- real business people and consumers -- think of what they want from us. On the other hand, that takes us away from the nice clean world of formats and protocols and into the messy world of actual customer needs and human foibles that tend to trip up nice neat software engineering theories. In short, I agree with the skeptics who don't think that the "Year of SOA" will come anytime soon, but disagree with those who think there is nothing but hype keeping the idea of service orientation aloft.

I hope (i hope i hope i hope) that the "SOAP vs REST" Godzilla-of-permathreads has returned to the ocean after stomping around the Web for a few weeks. The links in the previous sentence point to some wise summaries of where things really stand, which essentially comes down to the fact that both XML over HTTP and the WS-* technologies are useful things to have in one's toolbox, to be used as the situation demands: Pliers and a monkey wrench (XML over HTTP?) for the cheap and cheerful work, a set of carefully calibrated socket wrenches (XSD and WSDL?) for working with carefully engineered machinery, and maybe a pneumatic nut driver (WS-*?) for the industrial strength jobs.

Perhaps it is now time to look at the real differences between thinking of distributed systems as collections of resources to be manipulated vs thinking of them as services to be requested. That seems to be the essence of the real disagreement here -- it's pretty clear to me that services can be invoked with whatever combination of HTTP, XML, SOAP, WS-*, and [some technology to be designated later] is needed for a particular job. On the other hand, I looked back through the posts above and the highly regarded ones by Carlos Perez, and I don't get a good feel for how one would design a resource-oriented or "high REST" application for anything but information resources.

For example, we all know what a service-oriented automated restaurant would look like -- just like the human-powered service-oriented restaurants that have evolved over that last couple hundred years. What about a resource-oriented restaurant? I can see how that would work for fast food place (GET a menu, POST an order, which returns a URI one can GET the status of the order from ....). How about a real full-service restaurant, with a server who handles the details of the complex asynchronous interactions between a table full of people and a busy kitchen? Sure, one could implement those interactions with XML and HTTP, but would a system design that talks about resources and representations really be more sensible than one that talks about services and requests? Any RESTifarians out there want to take on the challenge?

More importantly, are there more real world distributed systems that work like libraries full of information resources to be transferred and manipulated, or more that work like restaurants that supply services which we learn how to request and consume?

"SOAP is Dead" -- if you believe the echo chamber

mikechampion — Sun, 20 Feb 2005 21:10:00 GMT

This is something of a followup to my post on the XML Team weblog last week, taking into consideration some of the feedback. I'm getting just a wee bit annoyed about the echo chamber repeating "SOAP is dead" to itself around the blogosphere, with nobody adding any actual evidence other than the success of various Web applications built mainly with HTTP. There are plenty of people I know and respect, and plenty I don't know but enjoy reading their blogs, making this point. I'll use Carlos Perez's post [someone I don't know personally] as the whipping boy, just to get back at him for "Mike Champion of course is prepared to defend the faith" :-)

To jump to the bottom line, I'm not defending the faith, I'm defending agnosticism. There are a range of technologies available to developers: some simple, some complex; some MS-specific, some Java-specific, some platform-neutral; some Web-based, some protocol-neutral; some tightly coupled, some loosely coupled; some favor up-front design, some are more evolutionary. The choice of one over another in a specific application is a business and engineering decision, not something to be determined by "faith-based programming" in accordance to either the SOAP or REST religion.

Carlos Perez writes:

Isn't SOAP the underpinnings of Web Services, the same technology that was billed as the silver bullet to extinguish our collective integration nightmare?

Uhh, no it's not billed that way. I suppose someone can find Steve Ballmer speeches from 2001 that can be construed that way, but let he whose marketing people are without hyperbole light the first flame :-) If you really want to judge the web services technologies by checking how well they realize the coolest visions of their early advocates, then I suppose SOAP has failed... but so has Java, the Web, and just about everything else I can think of, if judged by those criteria.

The RESTful approach has now born fruit. Applications like BlogLines, Flickr, Mappr, Del.icio.us and 43Things are revealing that proof is in the pudding

I'm not sure these are really RESTful in any meaningful way beyond using HTTP to move information around. [see Steve Maine's post and comments thread] More importantly, it's not at all clear to me how SOAP, WSDL, or WS-* could plausibly be central to these applications even if all the hype had already become reality. There's no client-side software to integrate with other than the browser, and the overwhelmingly obvious way to integrate browser-based applications with services is HTTP. Furthermore, there's no underlying reliable messaging, transaction processing, end-to-end encryption, etc. etc. technology in a typical user's internet environment that the SOAP headers can trigger. (Remember that all the WS-* specs do is describe a standard way to request such services). Maybe in some future world where Indigo and Avalon are pervasive in the Windows world and XUL and some open source SOAP, etc. toolkit are pervasive in the Mac / Linux / Java worlds, one could imagine building this kind of consumer oriented service using WS-*. Until then, the web services technologies don't have much potential to address the challenges faced by a designer of such a service, so seems a bit pointless to note that they aren't used for these applications in practice.

The web services technologies have a place, or at least a great potential, that should not be minimized. People who think that the Web is where almost all the action is might want to spend a little time in the enterprise software world. The dominant vendors there, such as IBM and SAP on the back end, and BEA in the middle, have joined with MS in the web services initiatives because they have determined that the raw XML and HTTP technologies do not do the job by themselves. I honestly don't have a lot of first hand knowledge of whether SOAP is really thriving today in that world, but that's where you want to take its pulse, not in the world of consumer-oriented, browser-based Web applications.

Carols Perez takes me to task:

What Mike seems to be saying is that REST is the Yugo and if you want the features then you have to buy a Rolls Royce. It's a sales pitch used all too commonly. Unfortunately, if I wanted a Rolls Royce I would have purchased a JMS implementation, not a Web Services implementation with a yet to be defined solution for security.

That is exactly the problem: You may have a JMS implementation, but it won't interoperate at the message level with other vendors' JMS implementations, and it won't even be consistent at the API level with all sorts of proprietary message queuing and transaction processing systems that are deeply rooted in enterprises. WS-* is about getting these things to interoperate, leveraging XML's inherent platform and application neutrality, and adding a couple layers to request services from the underlying messaging, security, and transaction processing systems.

Furthermore, the employees of those enterprises do a lot of their actual work with desktop applications, and these currently exchange information with the enterprise systems only with considerable expense and inefficiency. At my previous job , a common use case would be to get data back and forth between a legacy Natural mainframe application and an Excel spreadsheet or other MS Office application. Web services technologies let developers do this kind of thing by running a Natural -> WSDL translation wizard and leveraging the SOAP support built into both Office and various vendors' enterprise middleware. Is this RPC-oriented rather than document-oriented? Tightly coupled? Doing what CORBA tried to do, only with XML underneath? Perhaps, but you'll have to explain to the people whose actual jobs today are made easier by web services why this is a Bad Thing.

With that little rant out of my system, I'd like to point to reponse by Bill de Hora

that I found very thought provoking. I didn't see much that I disagreed with. He confirms my understanding that REST developers don't really want those VisualStudio wizards that MS doesn't offer :-)

My impression of those who are interested in using REST is that they want to be close to wire and to the data and if anything, want to reduce the number of tools in use. REST is a minimalist approach to systems building - when you're using it you're making a decision to some of the heavy lifting on your own

Similarly, I agree (and AFAIK the Indigo people do as well)

there's not much question that WS-* is in need of rationalizing. Entirely natural in the tech sector, competitive pressures have resulted in duplicated effort and excess choice for technology decision makers, which lessens the value of having standards to begin with.

I'm not so sure about

WS can be positioned primarily for those companies whose requirements are effectively off the bell curve

He may be seeing a different bell curve than I do. If we are talking about consumer applications over the web, sure. If we are talking about enterprise scale integration efforts in a world where technologies such as SAP and other ERP systems are pervasive, proprietary message queuing systems are the transport mechanism of choice, COBOL and other mainframe-rooted development languages are remain very important, and millions of terabytes of operational data remain in IMS (a dinosaur-era hierarchical DBMS), then I simply have to disagree. One lesson I learned in my previous job is that while this stuff has little geek mindshare anymore, it is pervasive and persistent in big business. I don't expect a RESTifarian story for mainframe modernization anytime soon!

The bulk of the post is an intriguing discussion of simple technologies and disruptive innovations, drawing heavily on THE INNOVATORS DILEMMA. I agree with Bill's point that while WS is generally marketed as if it were a disruptive technology, it is not really. (Well, OK, maybe it disrupts the moribund EAI industry ....)

Bill's analysis helped me clarify my pushback against all the "SOAP is comatose, put it out of its misery" posts: WS-* is a collection of sustaining innovations intended to incrementally extend and improve existing enterprise infrastructures and PC desktop applications. It's hard to disagree with the proposition that these will eventually be replaced by a disruptive technology that hits a very different price-performance point, and it's quite possible that XML and HTTP will be at the center of that disruption. I'm not at all confident that "eventually" will come anytime soon, however. Great source of ideas for entrepeneurs and experimenters, sure; proven technologies for enterprises, no.

Once again, we're talking about a world where SOAP-based specs are being proposed as standard interfaces to existing enterprise-class technologies that have evolved slowly over decades while the internet has gone through a series of disruptive changes. Many of us jumped on the chance to play with Bloglines, Flickr, and other disruptive applications that are presented as poster children for REST. I doubt if any of us wants our bank or insurance company to risk disruption (in the usual sense of the word) of its business in the name of disruptive innovation. As Bill mentions, CIOs in these industries have been burned so often by unsuccessful projects -- often planned on overly optimistic predictions about the technology fads of yesteryear -- that they are not likely to jump on XML, REST, or agile methods just because they are hot this year (or this decade). IT departments in big organizations which spend a lot of energy fighting spammers and phishing attacks can be forgiven for not wanting to jump on the Next Big Thing that internet technologies such as HTTP and XML offer. They'll migrate when they see their competition getting ahead of them by adopting mature and proven versions of this stuff. Until then, WS-* looks like a promising way to integrate business systems and improve business processes without a lot of "disruption".

I don't know Bill personally, but I do know / admire his colleague Sean McGrath, and I get the impression that their company Propylon is one of the relative few who really know how to apply REST principles to the industrial-strength integration challenges that WS-* was designed for. They are as likely as anyone to create a situation where

...WS specs and tools are being commoditized by disruptive technologies. If so the natural progression for them and the software based on them is to move up the value chain. When the mass market for enterprise technology is either over-served by technology, quality or the price is too high that leaves an opportunity for disruptive technology to take hold at the bottom end of the market.

It will be very interesting to see if they or anyone else pulls this off. I wouldn't be broken hearted, personally, if plain ol' XML disrupts WS-* -- As I said in the team blog post, if the RESTifarians are proven right, it just creates more direct demand for the WebData XML technologies, whereas if the WS people are right, more people will use our stuff via Indigo. I am personally, and I think MS is collectively, hedging the bet by covering both side, and working to better understand the circumstances in which one or the other will really work best. It's an extremely good bet that there are plenty of niches out there for both. MS can and should supply both SOAP-based and Plain O' XML tools; if others want to focus on "REST", go for it. I for one won't suggest that you drink any SOAPy Kool-Aid, although I would appreciate a bit less rhetoric about how tasty the RESTful Kool-Aid is until there are "reference customers" that use that approach instead of WS-* in a scenario that WS-* is intended to handle.

Finally, I was very glad to see James Governor's response to the various commentaries on his article, including mine.

a. niche - my intention was never to say SOAP is dead, merely that is not a necessary condition for web service development. There is a class of applications outside the traditional requirements of high end transactional enterprise development and workflow that just don't require the reinvention of Corba-class integration.
b. the comment there isnt any demand for SOAP refers to comments from the Creative Commons.

It's clear that we're on the same page here: "its all about niches or not."