All About Interop : SOA

Microsoft launches new SOA Website

DotNetInterop — Tue, 30 Oct 2007 19:38:00 GMT

The new Microsoft SOA & Business Process website is up! And I am really pleased to see it.

Mea culpa time: For a long time, we (Microsoft) have not effectively communicated with customers and partners about Service Oriented Architecture. That's why I'm so pleased to see this new website.

By the way, the new website is up on this, the first day of the 2007 SOA & Business Process conference, happening on Microsoft's campus right now. I'm sitting in the keynote address as I type this. The event is sold out! There are 1000 people here to learn about the Real-World approach to SOA that is enabled by the Microsoft application platform.

Why were we so bad at communicating? Have a look back. As SOA became a theme that captured the imagination of people, Microsoft got leery. Other companies started slapping SOA badges on their existing products: "New! Now SOA enabled!" Some companies started producing new products targeted to SOA. Companies built SOA centers of excellence, SOA Swat teams. The attention around SOA grew and grew. SOA became the solution for everything. SOA was the solution to world hunger!

Side note: Later came the backlash. People started realizing, hey, maybe we're over-doing this SOA thing. People want to get more pragmatic on SOA, they want less pie-in-the-sky promise and more practical results. That's a natural, and maybe a necessary, part of the cycle I guess.

Microsoft, for its part, conceived SOA in a technical context, and the way we talked about SOA showed this. We articulated the four tenets of SOA: Boundaries are explicit, services are autonomous, Services share schema and contract, and Service compatibility is negotiated and based on policy (link). Yikes! This is SOA for bit-heads. This is not actually SOA at all! It is how to build web services.

This tech-heavy focus was our error. Certainly we wanted to educate people on how to build services. But that wasn't the only issue around Service Orientation, and maybe wasn't even the most important issue around Service Orientation. Really, SOA was interesting to businesses because of flexibility and agility. SOA had the potential to bring flexibility to a business, and that meant better competitive positioning. We were talking about service contracts and WSDLs, while customers were talking about getting their work done more quickly, or being more responsive to market shifts. We whiffed on this.

But we've seen the error of our ways! SOA remains an important theme for the industry, a key opportunity for customers. And We, Microsoft, have slowly but steadily evolved and matured and broadened our thinking on SOA, and that's a Good Thing. The new SOA website is good evidence of this. It's meant to provide a single point of access for our customers into our SOA thinking, a single place where we talk about the capabilities across the entire Microsoft application platform product set - across Visual Studio, System Center, the .NET Framework, SharePoint Server, BizTalk Server, and more - and how those pieces fit into a SOA strategy. It also provides a place for Microsoft to communicate about SOA-related events, partner activities and offerings, and other content related to SOA that is available throughout the Microsoft.com web.

Guerrilla SOA = Real World SOA?

DotNetInterop — Wed, 19 Sep 2007 21:53:00 GMT

A couple weeks back, in a post entitled "The time may be ripe for ‘Guerrilla SOA’", Joe McKendrick wrote about a meme that Jim Webber of Thoughtworks initially introduced in an interview with Infoq.com. The idea is that if SOA adoption for a company requires a massive mobilization of resources and people, it's often a non-starter. It's impractical to justify the initial investment, and it's impractical to realize actual benefits from that approach. A more reasonable approach may be to adopt a practical, pragmatic approach to employing SOA, with a very incremental, iterative approach. Use SOA in an initial project, and evaluate your results. Then apply the lessons learned and broaden the use. And so on. This is something that I had independently described, although not using the Guerilla metaphor, as a common practice among Microsoft customers in a separate interview on infoq.com.

Are we reaching a tipping point on the real world approach to employing SOA?

on SOA, open standards, open source, and other nonsense

DotNetInterop — Wed, 19 Sep 2007 17:49:00 GMT

I read a blog post by Loraine Lawson that she titled "Biz Talk Server R2 Offers Peek at Microsoft's SOA Strategy". I found a bunch of things in that post disconcerting. Thought I'd respond a little here, not directly to the blog in detail, but more generally about SOA.

SOA does not "demand open source" as the blog post asserts. Many pundits in the industry, whether purposefully or not, equate or conflate "open source" with "open standards". At the risk of stating the obvious, they are not the same. We at Microsoft believe that one key driver of success for SOA is a set of open communications standards - ranging from IP, to HTTP, XML, SOAP, WS-*, RSS, JSON, ATOM and so on. Effective SOA means interconnecting disparate systems, and that does require communications protocol standards. It does not require open source. Can a .NET system interconnect with a appplication running on IBM WebSphere Application Server? Yes. Any open source there? Nope. Can a .NET system interconnect with a PHP app via RSS? Yes! Any open source there? Yes. But, the fact that open source is used, is irrelevant. As I have said many times before, It's the protocols, silly!
Open standards of course are not enough. Also required is a set of infrastructure that can harness the potential of those standards, so that developers can interconnect disparate applications using those standards, so that IT can manage, provision, and govern systems that employ those standards and so on.What’s required is commercial-grade infrastructure in the form of tools, process servers, management suites, development frameworks, registries, and user interface technologies. User interface in a SOA? Absolutely. After all, in the end we want a person to actually see and act on the information that is swirling around within a SOA. We believe as many analysts do (see IDC and Dataquest) that there will be a broad expansion in the market for SOA infrastructure. Some of that infrastructure will be provided in open-source form. For our part, Microsoft is delivering, and will continue to deliver tools, servers, and other infrastructure that enables companies to exploit the advantages available within SOA approaches, as well as architectural guidance and training for how to effectively apply SOA in customer environments.
Some uninformed critics say Microsoft’s SOA infrastructure works only in a "Windows only" or "Windows everywhere" approach. This view doesn’t hold water. As I have said previously, if the claim is that the Microsoft SOA software, like BizTalk Server, Host Integration Server, Visual Studio, SQL Server and so on - runs only on Windows, then this is TRUE. This is not a particularly novel, interesting or relevant observation. Customers don’t demand that their SOA infrastructure run on a particular operating system; They demand that their SOA infrastructure connects readily, and delivers good value. If these people want to imply, that because BizTalk Server (as one example) runs only on Windows, then it can connect only to other Windows-based systems or applications, they are either mis-informed or they are attempting to mislead you. Ninety-two percent of the over 7,000 BizTalk Server customers use the technology to connect with assets on UNIX, Linux and mainframe based systems. Many Microsoft customers connect to SQL Server - running on Windows mind you! - from applications running on iSeries, Linux or Unix. Microsoft has long offered a technology that does nothing but integrate with IBM's iSeries and zSeries technologies. (it accomplishes this via IBM's proprietary protocols, which Microsoft must license). Windows Communication Foundation implements more WS-* standards than any other commercially-available framework, at this time, as far as I am aware. This means WCF connects to disparate heterogeneous systems via standard protocols. "Windows only" is blind ignorance or spin, and either way it is just wrong.

-Dino

Commentary: OASIS is on the wrong path with the SCA standardization effort

DotNetInterop — Tue, 18 Sep 2007 19:05:00 GMT

I am sort of on an SCA kick here. I just saw this post from Dick Weisinger of Formtek. In it he questions the approach OASIS is taking with the SCA family of specifications. Dick implies that SOA is already too complex for most people, and creating six new standards isn't going to simplify anything. I would take that one further and question the very structure of SCA itself. It seems that there are a couple of loosely-related technology areas covered in the SCA specs, why are they all smashed together?

Why isn't the portable communications API for Java being specified in the JCP, where such things are "supposed to' be specified?

Why is that portable API combined in a single spec with a description language for composite artifacts (SCDL)?

Zorro as SOA architect?

DotNetInterop — Tue, 18 Sep 2007 17:09:00 GMT

The latest from Zorro cracked me up. Where is he going with this?

SCA is an endorsement of WCF?

DotNetInterop — Mon, 17 Sep 2007 19:53:00 GMT

huh?

SCA is an endorsement of WCF? Seriously?

I posted earlier in the summer about SCA and made a statement to that effect. That post just got a comment questioning my sanity or intelligence or both (the comment asked if I was in marketing. Can you believe that? The nerve of some people!)

Talking about SCA is difficult. People don't agree. Sometimes they vehemently disagree on stuff that seems like it ought to be very agreeable. Why? I've thought about this, and concluded that the current difficulty in the dialog about SCA, is due to the fact that SCA not well understood, or the understandings people hold about SCA are fairly different. I was in a series of meetings a while back where the topic was SCA, and we had a couple of experts who had entirely opposed views of the content of SCA. These were smart, well-informed people, one of them had actually written parts of the SCA specifications. And yet the disagreement was very stark. How can this be? The only way I can figure it - SCA is still new, the implementations that are out there are fairly new, and there's not a broad, common understanding of concretely, just what it is and what it offers.

The commenter I mentioned wrote this:

First, you said "Microsoft views this part of SCA as a big endorsement of the approach we have taken in Windows Communication Foundation". Then, you say "the primary difference between WCF and SCA is that WCF is about wire-level interop and SCA is about a portable programming model". So can you explain how it's an endorsement of your approach when they aren't about the same thing?

You set up the strawman that "SCA is oriented toward consolidating the disparate communications models and APIs within Java", then you knock it down with WCF.

You then agree isn't what SCA is about communications.[sic] Are you in marketing?

When I used the phrase "big endorsement" I was referring to the part of SCA that unifies the programming model regardless whether you are doing web services, angle brackets, transactions, asynchronous comms, and so on. This is what WCF has already done for .NET. and part of SCA is dedicated to doing this now, specifically for Java. A key observation here is, SCA's programming model is intended to be portable across implementations. The WCF programming model is not. There's only one implementation of WCF, and that is the one in .NET 3.0 (and above). From the perspective of a systems architect hooking stuff together (aka HST) in a heterogeneous system, the programming model used by WCF programmers is irrelevant! Wait, let me extend that - from the systems architect perspective, the programming model used by any programmer of the individual pieces in the system, whether they use VB, Java, C#, or somethign else, ought to be irrelevant. The only thing that should be relevant, from the HST perspective, are the on-the-wire protocols.

I don't grok the rest of the comment, so I'll leave it at that.

Back to my point about people not really agreeing on just what SCA is - there was a previous post from Dana Gardner about Microsoft and SCA. Dana wrote, in part:

The SCA/SDO backers invited Microsoft to join their efforts, they said, to adopt a programming-level of SOA standardization [my emphasis], rather than a Web services level of interoperability. But the members voiced little hope that Microsoft would have a sufficient motivation to move .NET to a programmatic open standards level. SCA/SDO is nonetheless expected to make interoperability between .NET- and non-.NET-based services a natural and rudimentary aspect of SOAs, but at a higher cost — a tax, if you will — due to Microsoft’s separation from the pack.

"programming-level of SOA standardization"? This, to me, is seriously wrongheaded thinking. We've been down this path before. Many times. Common programming model does not mean interop. Come on, people, look at the facts:

J2EE - standardized APIs, but interop between EJBs? Fahgedaboutit. It was always, in theory, a potential benefit of standardizing the APIs. But it never really happened. J2EE people will have to content themselves with the benefit of "sort of portability" - keeping in mind that portability remains partial among implementations, and at this point, is anyone paying attention any more?
CORBA - standardized APIs, but interop was sorely lacking. Even with the advent of the IOR, there was still difficulty getting an Object running within Orbix to intercommunicate with an object running in Visibroker. (Ahh, a blast from the past).
DCE - ahh, going back in time now. Anyone remember DCE? DCE had a set of standardized APIs, C-based, not object oriented. Something like CORBA but for the C programmer, and not the C++ programmer. This was hot stuff before the advent of Java in 1995. Because DCE had a standardized API, programs were portable (mostly) across implementations. One aspect of DCE was the DCE RPC, or Remote Procedure Call API set. This was the commuications layer of DCE. There was an IDL (before CORBA's IDL) and a compiler of the IDL that produced server-side stubs and client-side proxies. Hmm, this sounds vaguely similar to something I've heard of more recently (can anyone say wsdl.exe?).

But once again, interop wasn't magically guaranteed by the consistency of the API across implementations. Interop was guaranteed in DCE by the network data representation, or NDR. In other words, the network data protocol. And trust me on this, there was a ton of work put into the NDR. We had big-endian vs little-endian issues, other encoding issues. All the same issues we have today, but NDR was binary, which meant, for practical purposes, it was opaque to humans. I remember seeing the rpctrace tool for the first time; built on tcptrace, rpctrace was endowed with an understanding of the NDR and so it could show you in real time what it thought the on-the-wire data packets were. Something like Fiddler for HTTP/SOAP traffic today. Even with rpctrace, diagnosing interop issues between, let's say, HP's DCE v1.0 and Transarc's DCE v1.1 on Solaris, was a ... challenge. And as soon as we rolled in encryption (yes, kerberos-based message security based on MD5 hashes and DES encryption (back when DES was good stuff) was integrated with the RPC layer), rpctrace was useless.

Have we learned nothing from the past!??!?!?!?

To me, the promise of web services was that in pursuit of interoperability, we are no longer tilting at windmills trying to produce a single, stable, standardized programming model for all programmers, all devices, all nodes, all languages, and so on. The WS-* work the industry has pursued since 1999 shows that we (vendors, customers, developers, pretty much everybody) recognized that protocols were the sine-qua-non for interop. PROTOCOLS people, not programming models. Protocols, Protocols, Protocols, Protocols, Protocols, Protocols! I say.

And let 1000 flowers bloom! Given a standard protocol, the world can support a myriad of programming models, and they can look like anything they want. As long as each implementation produces the same on-the-wire protocol, they can all intercommunicate. Glory be!

In the post I cited earlier, Mr Gardner asked:

why is standardized interoperability at a programmatic level, embedded within widely accepted SOA open standards, any less beneficial?

That is to say, any less beneficial than a standardized wire protocol. I'll tell you why: Standardizing on a programming model as a way to to boost interoperability has proven to be a failure, repeatedly. [added 1 October 2007: It's a very nifty rhetorical device to coin a phrase such as "standardized interoperability at a programmatic level" but in my view that concept is imaginary. It is a complete fabrication. You cannot "standardize interoperability at a programmatic level." It is not possible. By definition!]

By the way, I wonder sometimes if I am the crazy one, but if so, there are others out there who are similarly demented.

Let's be Clear Here

Look, I am not arguing against the merits of standardized programming models in general. I see what J2EE did for the consortium of Java vendors that drove it - it produced a common programming model, which meant, a single set of skills could be portable across vendor implementations. Yes, I said skills portability, not app portability, because while J2EE promised app portability, it never really happened in practice. As we all know, APIs are not the only thing in an app. An app has lots of metadata, and J2EE did not standardize all of that. An app has administrative interfaces (programmatic and user interfaces) and J2EE did not standardize all of that. As a result, J2EE apps could be "mostly" portable, but never truly portable. [added 1 October 2007: Yes, yes, I know, you can write a JSP app that runs on a bunch of different servlet engines. I am totally clear on that. That to me is interesting, but it is not proof of real world portability of J2EE in general. ]

But skills were mostly portable. That is to say, once a programmer learned the J2EE metaphor, once that person understood MVC and what a session bean was and how a remote EJB method worked, then that knowledge applied across multiple vendor implementations of the J2EE spec. That was valuable, both to the vendors themselves, because they could create a critical mass of devs that could be productive on their product, as well as to the developers, who had more options.

So I see the benefits of a common programming model. But I also see very clearly that a common programming model does not in any way guarantee interoperability. Am I the only one who sees this?

Keep in mind that the SCA spec itself, says that you cannot create cross-domain composites. The composite is a collection of components that are stitched together. According to the SCA spec, those components must run, all in the same SCA container. There is no cross-domain interop provided by SCA, even with its common programming model.

Does this surprise you? It shouldn't!

This begs the question, if a common programming model is beneficial, why shouldn't Microsoft throw in with SCA in pursuit of that? That's a worthwhile question and one I will leave for another post.

Bottom line:

SCA is primarily about programming model consistency among a consortium of vendors. The cast of characters in SCA looks quite similar to the same people who just left the J2EE party, after Sun began exercising the control it had guaranteed itself in writing up the charter of the JCP.
Microsoft does not believe that investing in a common programming model boosts interop, nor do we believe, given dynamic languages, structured languages, and the other advances in programming languages that are occurring today and will continue to occur in the foreseeable future, that it is even practical to enforce a common API.

ps: Yes, I am in marketing. Can you not tell, from the length of this post?

Till next time,

-Dino

IBM WebSphere Registry/Repository and Interop with .NET, part 3

DotNetInterop — Thu, 30 Aug 2007 08:04:00 GMT

Back in May, IBM had posted an article describing how to connect to its WebSphere Service Registry and Repository (WSRR) from .NET applications. A surprise to me. Not long ago I was huffing about lack interop on WSRR. I later amended my comments in a followup post, saying that IBM had documented a SOAP interface to WSRR. But apparently back in May IBM had addressed the question more directly with a nice developer article!

Basic Interop is not the only issue of course. As Anne Thomas Manes pointed out in a comment to the original post, there is a difference between basic interop and integration. WSRR defines its own data model for the artifacts stored in the registry, so basic connection is not good enough. The connecting application needs to have some understanding of the data model, the semantics of what is retrieved from the registry.

Is UDDI sufficient for this? Apparently IBM thinks not.

Talking about SOA to the Business Owners

DotNetInterop — Tue, 28 Aug 2007 22:17:00 GMT

Not really interop, but

here's a discussion of how to talk about SOA to business owners. The view here is that

Business value (agility, quicker deployments, less cost for development) is definitely the place to begin. But be prepared to discuss [the technical aspects of] services and explain why SOA is better than EAI or standard development practices.

Web 2.0 versus WS-* ?

DotNetInterop — Fri, 24 Aug 2007 00:33:00 GMT

Zorro weighs in? http://zorroisb.spaces.live.com/

Architect's Webcast series on SOA

DotNetInterop — Tue, 14 Aug 2007 02:40:00 GMT

The SOA Webcast Series for Architects is now live on microsoft.com.

Comparing SCA to Microsoft platform technologies

DotNetInterop — Thu, 28 Jun 2007 20:29:00 GMT

I posted about SCA previously. My take on it is, SCA isn't really about interop. But many people think it is, so I post about it here, on the interop blog.

Sometimes people ask me to compare SCA with what Microsoft offers. Here's my take.

A large part of SCA is oriented toward consolidating the disparate communications models and APIs within Java. Java does remoting with RMI and RMI/IIOP; web-services via JAX-RPC or JAX-WS; asynchronous invocation with JMS; JCA for packaged apps, and so on. Lots of communication models. SCA proposes to unify them. Within the SCA family, the spec covering Java annotations and APIs, and the spec covering Java component implementation specifically deal with this.

Microsoft views this part of SCA as a big endorsement of the approach we have taken in Windows Communication Foundation, part of the .NET Framework v3.0 that shipped in December 2006. WCF consolidates various disparate communications models in .NET into one generalized programming model, with support for interop protocols (WS-* or web-oriented services), binary optimizations, reliable or asynchronous messaging, pub/sub, and others. Because of the clear value for customers in simplifying communications, WCF is the leading model in the industry for communications frameworks. Unlike SCA, WCF is not a spec; WCF is available now (a free download), and is currently being used in production applications. and, the performance of WCF apps, rocks!

Some of SCA goes beyond the basic communication plumbing, and attempts to address the larger and more complex issues surrounding the description, modeling, assembly and management of distributed systems. These are hard problems; Microsoft has been working in this area for many years, and steadily delivering infrastructure to address customer needs. For example, is shipping technology. In the .NET Framework 3.5, we'll deliver integration between WCF and WF, and the version of Visual Studio mated to .NET 3.5, Visual Studio 2008, will provide modeling tools to truly democratize the construction and deployment of conversational service-oriented applications. System Center 2007, currently shipping, enables management of distributed applications. We expect to continue to steadily deliver evolutionary innovations in these areas over time.