About: This post is the second in a series of four about Microsoft’s strategy for Enterprise Service Bus.  This post discusses the various definitions of the term ESB, defines the attributes of an ESB and tries to dispel some myths about it

Greetings,

If you want seamless integration between systems implemented on disparate technologies, if you want to plug legacy systems in an SOA environment, if you want to have composite applications based on multiple services, if you want dynamic discovery and binding capabilities, if you want true interoperability, in short if you want to resolve all of your integration nightmares, then ladies and gentlemen I give you the ENTERPRISE SILVER BULLET (ESB).

The ESB is a magical and nebulous piece of software that plugs itself in the middle of your enterprise with neat bidirectional arrows going to boxes labeled ‘mainframe systems’, ‘Java Systems’, ‘.NET systems’ and ‘other legacy assets’, it is all you will ever need to reduce costs, increase your time to market and optimize your IT operations etc. etc. and etc.

As all of you know, despite sounding familiar, most of what I have written is incorrect or minimizes the amount of effort involved in achieving the desired outcome. Unfortunately most of what I have written is based on the descriptions and presentation of ESBs I have come across.

Let us explore a the brief history of ESB and beyond-the-hype attributes of this type of software

A brief history of ESB and its many definitions

The term ESB was first used by Sonic in 2002 to refer to their SonixXQ product that was an XML-enabled MOM product, the product was later re-branded as Sonic ESB. A few months later, Gartner called ESB a strategic investment and then very soon every thing from integration servers to messaging products were re-branded as ESBs, I even attended a presentation where a vendor showcasing their SOA offerings displayed the Ethernet wire as their official ESB strategy.

In addition to the re-branding there was also a plethora of definitions put out, everything from Gartner’s “A Web-services-capable infrastructure that supports intelligently directed communication and mediated relationships among loosely coupled and decoupled biz components”, to Burton’s group “The ESB label simply implies that a product is some type of integration middleware product that supports both MOM and Web services protocols” to IBM’s “To put it bluntly: If you have WebSphere MQ and other WebSphere brokers and integration servers, you have an ESB (Bob Sutor, IBM).”

In general, the market quickly realized the potential of promising a new pill for what was essentially an integration problem. In many cases the companies who bought the ESB products were disillusioned by the amount of effort and customization it takes to ‘plug’ a new system in to an ESB, the licensing costs involved in having an ESB node license in a distributed model, the learning effort involved in creating customizations/extensions for ‘plugging’ legacy systems and the maintenance nightmares. Ironically, in many cases businesses ended up deploying multiple ‘ESBs’ and once you have multiple ESBs what do you need to integrate them?....... Yes you guessed it right; you need another ESB to integrate the ESBs, one BUS to rule them all!

I am not against ESB as an idea, product or integration pattern, I am against setting unrealistic expectations around a buzzword that has resulted in to considerable grief for many early adopters.

Let us explore the basic attributes of this type of software

Attributes of an ESB

There are some attributes of an ESB that all major industry experts have agreed upon, an ESB at the very minimum should support:

1. Brokered communication:  The core purpose of an ESB is to act as an intermediary between systems that need to send and receive data from each other. The notion of an intermediary is essential to the ability of an ESB to provide address indirection, rules-based routing, message transformation and other capabilities. The core value of having brokered communication is to increase loose coupling between systems and externalize integration logic for potential reuse and ease of maintenance

2. Routing: In order to integrate systems in a loosely coupled manner, an ESB typically has some type of name space or directory which resolves a logical service name to a specific implementation at run time. An ESB should also be able to maintain and execute rules for topic-based, property-based or content-based message routing.

3. Basic Web services:  An ESB should at the minimum support basic Web Services standards and their foundation, i.e. it should provide support for WSDL, SOAP over HTTP, TCP/IP and XML

4. Endpoint metadata: An ESB must be capable of storing or accessing information about service interfaces and message schemas. 

5. Non-functional requirements: An ESB should be able to support deployment in a high availability configuration; it should also be scalable and reliable and offer management capabilities

At the very minimum, any decent ESB product will include the above five properties as they form the core value proposition for buying a piece of software that eases integration and interoperability problems in an organization.

In addition, there are other attributes that lend themselves to be placed in an ESB, whether an organization would need some or all of these attributes will differ based on a variety of factors that I plan to discuss in my next post. Some of the other attributes that could be considered with respect to an ESB include

6. Externalizing business processes and rules.   An ESB could contain capabilities that allow for externalizing business process and rules from applications and placing them in the ESB.

7. Business Activity Monitoring. An ESB is a natural place to conduct business activity monitoring, if it is indeed the middleware through which systems are communicating, interacting and interoperating with each other then having capabilities that allow for analysis of the data for various stakeholders can be quiet useful

8. Interoperability and integration.  Web Services are increasingly the ideal choice to implement interoperability between different systems, however, Web Services are sometimes not the best or the most optimal option. An ESB should ideally contain capabilities to facilitate interoperability without Web Services reducing the amount of custom effort required for each type of system.

Based on the above attributes, you can think of an ESB as

Enterprise: The “Enterprise” in ESB refers to the fact that an ESB will generally be installed and managed within one virtual enterprise - one company and possibly some of its customers and suppliers. 

Services: The “Service” in ESB refers to the value added services provided by the ESB, including routing, management and transformation.

Bus: The “Bus” in ESB refers to its support for non-disruptive service substitution. A logical bus topology facilitates these properties as a bus enables any endpoint to talk to any other endpoint in a decoupled manner.

Conclusion

In this article I discussed the history and basic definition of the term ESB. In my opinion rather than buying an ESB and retrofitting your needs around it, you should define what your organization requires from an ESB right now and what you will need from it in future. Based on the needs you can come up with an evaluation criteria that will allow you to choose an ESB (as a product or a pattern, something I will discuss in the next post) that provides the maximum value to your organization.

In the next post I will attempt to describe Microsoft’s strategy for ESB and discuss the future of this type of software based on the new developments in the industry around interoperability and integrations standards.

Best regards,

Mohammad