Welcome to MSDN Blogs Sign in | Join | Help

Welcome to The Metaverse

Navigating the service-oriented, identity aware metaverse

News

  • Disclaimer:
    The content of this blog are my own personal opinions and do not necessarily represent Microsoft's position, commitments or strategy. In addition, my thoughts and opinions often change, and as a weblog is intended to provide a semi-permanent point in time snapshot you should not consider out of date posts to reflect my current thoughts and opinions.




    Add to Technorati Favorites
To ESB or not to ESB

There is a discussion raging right now about the notion, value and applicability of an Enterprise Service Bus, or ESB. David Chappell of Sonic Software has written a good, albeit very Java centric book on ESB which I guess not surprisingly leans heavily towards discussing the notion of ESB from the context of the technologies available from Sonic and the Java platform. Sun has recently waded into the game by submitting the Java Business Integration (JBI) draft spec at the Java Community Process (JCP) site. JBI extends J2EE with the technologies necessary for building an ESB.

In the Microsoft world, technologies such as Microsoft BizTalk Server 2004, combined with Microsoft's .NET Frameworks provide most, if not all, of the features that are being proposed as required in order to build an ESB, but give you the ability to compose your system as you need.

I don't want to detract from this work or to get into a competitive slinging match, but I want to ask some questions to this audience:

  1. Does anyone out there want to build their Enterprise infrastructure on a technology from a single vendor, supporting a single platform?
  2. Does anyone out there see benefit from building their Enterprise upon a technology which is closed and proprietary

We are all - ALL of us - responsible for delivering to our customers the ability to integrate our systems with one another, to integrate the systems that our customers pay for and consume, that hold the data that our customers own. Customers are tired of not being able to integrate their systems without jumping through hoops. Why should our customers pay to purchase/license systems that claim to be there to make the customers' lives easier only to find that the systems are opaque and cannot interop with other systems in their environment. This is lunacy and it has to end.

We as technology providers have a responsibility to make our systems interoperable by design. To share a common lingua-franca on the wire. To interop cleanly, securely and reliably. We should not be attempting to do this at the API layer. We should not be trying to achieve this at the technology layer. The ONLY way we're going to achieve broad and true integration and interoperability is by standardizing the wire protocols that we use to enable systems to communicate.

To this end, we, along with many of our partners, competitors and other interested parties formed and work within the Web Services Interop (www.ws-i.org) organization to ensure broad interoperability between our technologies. At long last, our industry is finally growing up and is designing truly interoperable, open, composable wire-level protocols for building secure, reliable, transacted distributed systems.For the first time in this industry's history, we see all the major players sitting down to the table to agree on common wire protocols for the most common scnearios found when building distributed systems. This must continue - there is only upside here.

To which end, if we have wire protocols that permit, even enhance our ability to build truly interoperable distributed systems that communicate via secure and reliable communications infrastructure, and if each node in the system is able to use these technologies, then all nodes in a system should independently be secure and reliable.

What value, therefore, is an ESB? To my mind, an ESB is smart-plumbing to which to attach dumb nodes. To my mind, the ESB approach is less about providing customers with value and more about extending attempts to lock customers into a given platform. How, for example, would I add mainframe nodes, COM+ nodes, Web Service nodes etc. running on different platforms and technologies on top of JBI/Sonic/<enter ESB of choice here> without the use of adaptors and translators? And more importantly, if I chose to change vendors, how would I replace my ESB with that from someone else? Isn't it just crazy to even begin to countenance such an approach?

The WS route makes the nodes themselves smart, reducing the need for underlying smart-plumbing, and ensuring open communications across any platform and device.

Am I missing something here?

Posted: Wednesday, March 23, 2005 1:41 PM by RichTurner666

Comments

Anton said:

With WS, WSE and Indigo, aren't there still some missing "pieces", such as Monitoring, Auditing, Exception Management, Measurement, Definition and monitoring of SLAs, etc.? Isn't this the space that an ESB should cover?

So even though you implement a vendor-specific ESB, it is just an enabler and should itself still support and promote the "common lingua-franca on the wire", and can therefore be replaced at any time.

I may be missing something, because my definition of an ESB is, that it should not quite be as central and "strong" as some of the vendors have defined it.

A related question - Isn't Microsoft's CSF in the ESB space, more so than BizTalk and .NET?
# March 23, 2005 2:53 PM

Richard Turner said:

The Windows platform itself is highly monitorable, exposing many performance counters, statistics, logs, traces etc. for you to gather using your management tool of choice. This only gets better with Indigo too which is instrumented to a degree never before seen on our platform. We do not prescribe the management tool/infrastructure that you MUST use - we leave it up to you which toolset to adopt. We ship MOM for these purposes, but you're free to use third party products if you'd prefer.

In terms of "definition", you will see enormous strides forward when we release Visual Studio 2005 Team System. This capability will only increase in the future as DSI becomes more deeply adopted throughout our products and tools.

I sincerely hope that ESB platforms adopt common, open, non-technology-specific wire protocols (ie: WS-*) in order to perform communications, but then again, if they did, what would be the benefit of ESB if the nodes were all individually able to communicate via such infrastructure?
# March 23, 2005 3:03 PM

Anton said:

Agree, that's why the definition of ESB makes more sense as a concept, rather than a product. So basically just a synonym for EAI.
# March 24, 2005 8:51 AM

Ed Daniel said:

Synonym of EAI, perhaps not! <br> <br><a target="_new" href="http://www.sys-con.com/story/?storyid=48035&amp;DE=1">http://www.sys-con.com/story/?storyid=48035&amp;DE=1</a> <br> <br>Myth #1. ESB is just a new name for EAI. <br>While many IT architecture groups are focusing on building SOAs, they still inevitably beg the question of &quot;how is ESB different from EAI?&quot; An ESB is an infrastructure for building an enterprise SOA, and is capable of being used in a more general way than a conventional EAI broker. According to Forrester Research, an ESB helps enterprises obtain the value of SOA by increasing connectivity, adding flexibility that speeds change, and providing greater control over use of the important resources that it binds. <br> <br>An ESB can be used to handle integration projects that have traditionally been relegated to EAI tools. However, an ESB can also be used for establishing B2B relationships across companies. <br> <br>An ESB provides EAI capabilities, but is based on a fundamentally different architecture that is providing the basis of an industry transition from traditional integration to coordinated service interaction. EAI brokers are historically implemented as a monolithic stack, using centralized hub-and-spoke architecture. <br> <br>An ESB provides the same base functionality as an EAI broker - connectivity, application adapters, routing of messages based on rules, and data transformation engine - yet, in an ESB, these capabilities are themselves SOA based in that they are spread out across the bus in a highly distributed fashion and hosted in separately deployable service containers. This allows the selective deployment of integration broker functionality exactly where you need it, with no additional over-bloating where it's not needed. The distributed nature of the ESB container model allows the independent scalability of integration components, which are plugged into your SOA as event-driven services on an as needed basis. <br> <br>In order for an integration broker to be truly capable of supporting an SOA, and to be considered a true ESB, it would need to have its base functions broken up into their constituent parts, which would then be capable of being separately deployed across the bus while working together in harmony as necessary. <br> <br>Let's use an example of an XSLT-based transformation engine that accepts an incoming XML document and applies an XSLT style sheet to it in order to produce an outgoing document in another XML format. I can tell you that there is nothing that can chew up computing resources more than the parsing and manipulation of XML. If this particular XSLT transformation sits between two popular applications that communicate regularly with each other, then that individual transformation can become a performance and scalability bottleneck. If you are using a monolithic hub-and-spoke integration broker approach, in order to remove the bottleneck and scale up the deployment you would need to either install that integration broker on one big powerful machine, or install the integration broker across multiple machines - just to support that one transform scenario! All the while, the other integration broker capabilities, such as the execution of routing rules, are competing for the same computing resources as the transformation operation. <br> <br>In contrast to the monolithic hub-and-spoke architecture of an integration broker, the foundational core of an ESB provides a distributed services architecture. This architecture is built for integration and has the ability for integration broker functionality, such as message routing, data transformation, and application adapters to be selectively deployed on an as-needed basis.These are separate integration services that are a natural part of an SOA processing pipeline across the bus. <br> <br>An individual XSLT transformation can be deployed as a service in its own ESB service container, and multiple instances of that container can be load-balanced across many machines. If the ESB container implementation is cross-platform, then you can be flexible as to what kinds of machines you spread the transform service across - Linux boxes, Solaris boxes, Windows boxes, and so on. And for those of you who don't find solace in the architectural purity of this discussion, consider this: the ESB vendors who are leading the charge in defining and delivering ESB products are also putting forth a license model where there is no additional cost for deploying as many of these lightweight ESB service containers as necessary to get the job done. <br> <br>The integration services provided by the ESB can be combined with other services into SOA-based processing pipelines that can span business boundaries. The distributed services in an ESB can be combined with itinerary-based routing (see Myth #7) to allow self-directed, message-oriented service interactions, which allow different parts of the ESB to operate independently of one another, without relying on a centralized routing engine. <br> <br>
# March 29, 2005 7:15 AM

Peter Goth said:

Just researching ESB right now but I am looking at where BizTalk and Sharepoint have key roles in an ESB like architecture... It does not seem to be all about code... more about the integration of services. <br> <br>When you look at the out of box integration, orchestration and business rules in BTS with the document flow ability of SPS - both extensible... they seem to be ignored within the ESB space...
# March 31, 2005 6:48 PM

Matt Deacon's WebLog said:

There’s a bit of a buzz going around at the moment relating to ESB; probably the result of someone shaking...
# April 7, 2005 6:58 AM

Barry Talks! said:

Anne Thomas Manes , a senior analyst at the Burton Group (and a colleague of my pal Peter O'Kelly ) has an excellent and penetrating post on the future of the ESB market.
# April 28, 2005 11:20 AM

On the road to Indigo said:

Dave Chappell of Sonic Software and ESB book fame, contacted me last week regarding my thoughts on ESB....
# April 28, 2005 3:24 PM

On the road to Indigo said:

Anne Thomas Manes, Research Director at Burton Group, recently posted an article on her blog discussing...
# May 9, 2005 6:08 PM

Loek said:

Few non-Microsoft people would agree that BizTalk and the .NET framework make for an ESB, since the ESB should consist of separately deployable service containers that provide the required capabilities.
BizTalk has most of these capabilities (routing, transformation, orchestration etc.) but does not consist of separately deployable services.
# May 12, 2005 2:20 AM

Aref Moin said:

BizTalk functionality CAN be deployed as separate assemblies if you go with the BTS 2004. (e.g. transformations) Moroever, it is WS-I compliant, supports a great deal of the WS-* stack including Security, Trust, Policy, RM, etc. without having to write globs of code, is BPEL compliant and consume and expose WS out of the box and can invoke UDDI t-models at run time and design time. Oops, almost forgot, it also gives you tremendous design time efficiencies with an end-to-end debugger, activity monitoring (think complex business process integration debugging). I agree with Richard. It is all about standards. How else will other value add vendors such as AmberPoint and Systinet be able to provide value if they need to navigate proprietory roadblocks? BizTalk is WS-Awesome!!
# May 19, 2005 1:03 PM

Tom Bender said:

I don't think WS-I is the answer -- at least not yet. I prefer Smaller, Faster, Lighter approaches to software devlopment. I do agree 100% that Java can't be the only orchestration implementation --- so to speak. I think a composition of ESB, JBI in a language neutral fashion is the best approach, but to imbed SOAP in the middle of all of this when application protocols like REST meet our needs is a mistake.
# August 10, 2005 12:27 AM

Ramesh Loganathan said:

SOAP or REST, is at some level much lower in the food-chain than what application developers may want. Especially in the context of Rich's original question about enterprises locking themselves onto one vendor if they choose ESB.
If you lok at ESB, especially Sonic's (I understand Sonic ESB very well. As the Technical Director for Sonic Software in India leading the Sonci ESB Tools team here), there is teh infrastructure provided by the vendor and then there is the application. Typical ESB application artifacts will be: services (most likely wrappers, that access some legacy system), XML schemas and transformations, and processes. A few off-the-shelf servcei types may be available like the DB-Service in the case of Sonic ESB.
Services: The Sonic services are already available as a simple Web Service- they can be used as a WS in the processes. Just that these services have the Sonic ESB binding- but this would in no way locks the process using the service to Sonic.
Code: The service code itself is going to be most likely in Java. If designed well, a thin layer that implements sonic interface will alone have the vendor lockin. But then, this layer will be required for any vendor anyway. Web Service? : Even to write a web service, the web service do not just happen. There has to be plumbing that processes teh SOAP and routes the requests to the actual service implementation. Either the aplication platform provides this ability, or the applciation developer must provide this.
Processes: the processes are easily exportable to BPEL. So no vendor lockin here as well
XML schemas & tarnsformations: These are all absolutely standard. Though Sonic supports some extensions to say XSLT, these can easily be not used to ensure that there is no lockin to Sonic.

Ofcourse, in the absence of key standards in the ESB platforms, today theer is no out-of-the box portability from one ESB vendor to another. COme to think of it, there are no common definitions of what ESB is. BUt then, the ESB camp is no worse than MIcrosoft camp- as in the Microsoft camp also the servcies developed are locked into Microsoft. Say, can one take the C# code from .NET and make it run as a web servcei on a J2EE Server? With ESB, a good amount oif it is possible today. More would be possible in the next few releases as WSDL2 and JBI gains more traction. Even today, with soem simple design considerations, the ESB applciation can be kept as vendor neutral as possible. Minimizing the porting effort needed when moving onto a different vendor.

(More on my blog.. http://jroller.com/page/rameshl)
# August 15, 2005 1:05 PM

blat said:

I wouldn't bet the farm on sonic esb. Do yourself a favour try an experiment. Try and find the "sonic esb" developer community.
# April 18, 2006 2:21 PM

Graeme Harker said:

The problem I have with the term "ESB" is the word "bus". In hardware a "bus" is a flat connection highway on which all devices use a single common interface to read and write messages. It's fundamentally a metaphor for something flat where each device can access any other. Hence Ethernet is a "bus".

But today's SOAs are not flat. Services are layered and devices in a SOA connect with each other as graphs and hierarchies. Hence the terms "bus" and "service bus" are not an appropriate metaphors at all.
# June 3, 2006 8:22 AM

richardt said:

Not sure your argument holds water Graeme. Ethernet might be considered a bus, but only for very local scenarios. When data leaves my machine via ethernet and hits the switch, it may or may not continue for some or all of its subsequent segment journeys via ethernet - it might be transformed to or from any number of other transports as it leaves my organization, traverses the internet and eventually hits a server somewhere else.

Besides which, a bus is a shortening of omnibus which essentially means "for everyone" (http://en.wikipedia.org/wiki/Bus)!

I think the Wikipeda definition of "Data Bus" has it right: "In computer architecture, a bus is a subsystem that transfers data or power between computer components inside a computer or between computers" (http://en.wikipedia.org/wiki/Data_bus).
# June 5, 2006 4:50 PM

sparky62 said:

I think that's exactly the point I was making. At the level of a single Ethernet segment (i.e. at the level of MAC addressing) Ethernet is a flat bus. At a higher level in the network stack, higher-level concepts such IP addresses allow routers to forward payloads onto different Ethernets. You'd never dream of building a corporate network with one huge Ethernet segment with every deveice broadcasting to every other device. In just the same way, the bus is too low level a metaphor for today's SOAs. Graeme
# June 6, 2006 3:43 AM

Welcome to The Metaverse said:

I received an email from a long-time follower of my blog that I thought might serve to spark a little

# May 10, 2007 8:22 PM

Weddings said:

There is a discussion raging right now about the notion, value and applicability of an Enterprise Service Bus, or ESB. David Chappell of Sonic Software has written a good, albeit very Java centric book on ESB which I guess not surprisingly leans heavil

# June 6, 2008 6:14 AM

大 兵 said:

学习和研究在企业中实施面向服务架构(SOA),简单回顾SOA和ESB,重点关注微软在SOA领域的相关指导和.NET社区的相关开源的解决方案,和大家一起来探讨如何在企业里实现SOA,期望有实施SOA经验...

# July 16, 2008 4:30 AM
Anonymous comments are disabled
Page view tracker