Service Oriented Architecture and Enterprise Applications

 

Introduction to Service Oriented Architecture

 

Industry analysts predicts[1] that over 80% of the business applications sold between 2005 and 2008 will be based on the principles of Service Oriented Architecture ( SOA) .  SOA preaches a paradigm that departs from the traditional view of encapsulating business logic behind objects and components. Instead, SOA advocates encapsulation based on business processes. This paradigm shift will have pronounced effects on traditional Line of business software as the next generation systems will be woven from services that are implemented on disparate but well known end points. These disparate systems share nothing more than a notion of a common contract and will be often implemented on different technologies, on different hardware systems and maybe even by different organizations. This move to a Service Oriented Architecture will have a significant effect on Enterprise applications as we know them today. This article explores this trend and looks at the various possible implications this shift might bring to the forefront. The first half of the article looks at the general principles of Service oriented Architecture and how it deviates from the traditional view of Object oriented programming. In the later half of the article we will explore the various macro trends that are driving the shift towards SOA.

 

What is SOA ?

The concept of applications is expanding beyond the scope of a single system running on a single device and bounded by a single organization. In response to this, the industry at large, and Microsoft in particular, have expanded beyond object-oriented component designs for application development and deployment. Services are the natural evolution of object oriented and component oriented programming models.

Services are independent units of application logic that expose message-based interfaces that can be accessed across a network. SOAs permit very flexible deployment strategies; rather than requiring that all data and logic reside on a single computer, the service model allows applications to leverage networked computational resources. Services are also independent in the sense that they are deployed, versioned and managed as discreet units. Depending on the application, services implement their own security, transaction and policy boundaries. The SOA approach has gained favor with companies building standards-based service interfaces for new and existing applications. These interfaces enable applications to interoperate or be combined into new, composite applications.

Before we dive into the details of building SOA applications, let us look at some of the basic building blocs of any SOA system.

 

Service

Data

Message

Policy

Logic

 

 


Anatomy of a Service Oriented System

 

Service Interface

Services interact with one another over the network through well established end points called Service Interfaces. The service interface defines and implements the contract between the service and its consumers. Service interfaces contain one or more operations or methods that consumers program against to consume the service. Service interfaces are usually reachable over the network and are recognized by URI ( Universal Resource Identifier ) or can be obtained from a service directory such as UDDI.

A key aspect of designing service interfaces is to decouple the implementation needed to communicate with other systems from the application’s business logic. Service interfaces implement the Service contracts that we saw in the last section. One can view the Service Interfaces as being the implementation of the service contracts.

 

Service implementation

Services are facades over business operations and interact with the external world through the service interfaces described above. The implementation of the service, data it encapsulates and middleware that provide the service’s functionality are collectively called the implementation of the service. SOA does not provide any prescriptions on how to design or build the service’s implementation. This allows the software to leverage its own choice of technologies and tools to implement the service. It could even reuse some of its existing code behind the service interface façade to reuse existing investment.

Service Messages

Services interact with the external world and with other services through messages. Messages encapsulate the data that is contained within the service. There are rules that specify how messages are formulated and intelligent metadata can be added on top of them to make them more sophisticated. The format of the message is not as important as its consistency. Systems use various formats for transmitting messages such as HTTP, TCP, MSMQ, MQ Series etc. There is no right or wring way of communicating with messages and it depends on the system and its uniqueness.

Service Contracts

As we saw earlier, messages the means of communication between services. Interacting services need to know exactly what they are to send, how they are to send it, what to expect, and how to expect it. It is not only necessary to define the messages that can be sent, the format the messages should be in, and how they can be sent, but it is also important to specify the sequence in which these messages should be sent. A contract is the definition, or binding agreement, of all that is sent between two services and of how everything is sent. The current standard that is being promoted by various platform vendors is the Web Services Description Language ( WSDL )

 

Service Policies

Service policies are rules that govern the interaction and the implementation of services. This helps disparate systems to interact with each other by communicating its preferences about security protocols, authentication mechanism, encryption schemas, etc. WS-Policy is the current standard that is used for describing message level policies. In addition to policies that govern the messages and contracts, SOA systems should also communicate and implement policies around the way in which the services are accessed, their versioning, their boundaries etc. There is no standard way to describe system level policies and are often implemented and communicated using SOAP headers.

In this section we will review some of the drivers that Enterprise applications have to be cognizant of while architecting and implementing Service Oriented business applications.

 

 


SOA drivers for Enterprise applications 

 

Now that we have a basic understanding of the concepts behind the Service Oriented Architecture, let’s see why this matters for enterprise line of business ( LOB ) applications. As new technologies evolve and business change, enterprise applications should be designed to take advantage of these new architectural principles and provide an answer to these business challenges. In this section we will review some of the imperatives that will drive the adoption of SOA based systems in the LOB applications ecosystem.

 

  1. The BPM imperative
  2. Integration imperatives
  3. The ‘Real Enough Time” imperative
  4. The smart client imperative  

 

 

BPM

At the business level, BPM is the management of explicit processes from beginning to end. These processes generally contain a long-running set of business activities such as those required to underwrite a policy or deliver an order under varying numbers of business scenarios. BPM solutions are extremely helpful to follow a process methodology like six-sigma or to comply with legal requirements like SOX. These scenarios often require the BPM solutions to map the logical sequence of steps of the process and a SOA based architecture can be an excellent way of abstracting these business processes behind web services facades.

The state of BPM applications today range from home grown solutions to niche solutions those extend into the vertical domains. ERP applications have started incorporating elements of BPM in their suite but they are no where close to providing the end to end functionality that the specialize BPM solutions provide. Analysts predict that all major Enterprise applications will have full featured BPM components in their main stream product lines by the next major version release. Since BPM has multiple uses, from simple personal flow to deep system-to-system flow under performance constraints, it is hard to find a common definition, much less one technology market ready to handle all the needs.

 

Critical elements of BPM applications

There are many components that make up the BPM components of an enterprise software suite. They can be classified into tools that are used to define and model the business process ( classified below as design time tools ) and those that are used to monitor and manage the business processes in real life situations.

 

Design time tools

Rules definition Engine

          Workflow definition engine

          Process modeling tool

 

Runtime tools

          Activity monitors

Event sinks

Execution engine

 

 

BPM Functionality

Scenario

Case for SOA

Rules definition engine

Rules defined for triggering alarms when for an inventory management system

Rules should be associated with a specific business process and its state information. SOA should be used to describe the business process and to provide state information at run time

Workflow Definition

Sequence of steps starting from receiving an RFI to submitting a complete Proposal reflecting “Real Time” data on capabilities ( could be resources/inventory etc )

Workflows are often compositions of sequential or parallel steps that have defined and predictable behavior and outcomes. By exposing these using web Services and abstracting them at a level of “Business Processes”, SOA applications can orchestrate simple and composite workflows.

Process Modeling

 

 

Activity monitoring

Activity monitoring is a natural extension to the rules defined by the rules engine as it pertains to keeping track of changes in the state of data. For ex, in a production line, you might want to monitor the inventory level of a critical item and define “real time” response actions to react to dipping levels

SOA in the Enterprise application world is about breaking down business processes into well defined services. This makes it easy to monitor activities by building the functionality into the services

Event Sinks

Event management provides real-time filtering, correlation and alerts from enterprise applications. Building this intelligence is important in many dimensions. For ex, these events could be analyzed using business intelligence tools to model future behavior. These models can be used as a warning mechanism for business threats and opportunities.

Principles of SOA can be used to

Execution Engine

 

 

 

 

 

 


 

Real Time Enterprise imperative


RTE is about establishing a business process to gain competitive advantage by removing the latency from underlying business practices. Generally speaking this might mean reduction of the half-life periods of business processes to speed up response rates of strategic decisions based on real time data and analysis. The definition of half-life varies greatly depending on the business process on hand. For example, in a order quotation scenario, the half life of responding to an RFI could be a couple of days while the half life of acting upon an inventory trigger in a Supply chain might be few minutes. While the absolute value of the half life is important, it is perhaps more critical to ensure a clear understanding of the context and how an application reacts to the context. Factors that influence the response times are as follows:

  1. Business process awareness
    1. Being aware of the context and the impending associated action is the first step towards acting in a “Real Enough” mode
  2. Data availability
    1. Once the application is aware of the context and knows what action needs to be taken, it should have the data that is required to take the required action. This data could come from structured or non structured sources or even from another extraneous service running somewhere on the internet
  3. Workflow
    1. Building on the previous step, the application should be able to execute the steps in a sequential manner as described by the meta data supporting the context of the business process.
  4. Transactional execution
    1. All the steps of the workflow should be transactional in nature as appropriate as it might involve steps that need to be executed in a safe and isolated manner. Elements of transactional processing like Isolation, Atomicity, Idempotence etc should be preserved
  5. Human intervention
    1. “Real-Enough” time applications are often subjected to scenarios that require human interventions in the workflow. By its very nature, this introduces a latency and unpredictability to the business process
  6. Analytics and Reporting
    1. Once the Business process has been implemented, it is time to reflect on the impact of the data and how it compares with other related pieces of actions over a period of time. This can be very powerful in not only understanding the patterns and behaviors of the Business process on hand but also to predict its behavior in a forward looking manner. Predictive Analytics is the technical term for this process of using historic data to model heuristic patterns that predict future behavior. 
    2. Reporting on the data is a critical piece of reducing the latency of Business processes. This feeds into the analytics described above and should be considered as an important piece of an RTE solution.

 

Business Process

Scenario

Case for SOA

Business Process Awareness

Proposal Generation

Company X receives an RFI for a product that it manufactures and the sales contact generates a proposal by collaborating with their colleagues. A supervisor authorizes the proposal and it is submitted electronically back to the customer

SOA is about defining service facades based on business processes.

Data Availability

The data required for making up a proposal could come from a variety of sources. For ex, product description and specifications for a 1/8” swivel pneumatic valve could come from the product catalog database while the real time inventory status could come from a live Supply chain application’s web service. Customer details and contact information could come from a CRM system and the workflow associated with this business process could come from a meta data file stored in a repository.

SOA is a great architecture for assimilating data from disparate sources. By using a data format independent SOA architecture that relies on XML, applications can pull information from Web Services, data bases or even simple files. A well architected SOA application treats these different data sources almost the same way.

Workflow

Sequence of steps starting from receiving an RFI to submitting a complete Proposal reflecting “Real Time” data on capabilities ( could be resources/inventory etc )

Workflows are often compositions of sequential or parallel steps that have defined and predictable behavior and outcomes. By exposing these using web Services and abstracting them at a level of “Business Processes”, SOA applications can orchestrate simple and composite workflows.

Transactional Execution

If one of the steps in the workflow fails ( or results in an exception ) the system is self-heals and takes appropriate corrective action

A well architected SOA solution should be able to act in a transactional manner. The next section of this document explains this concept in greater detail.

Human intervention

Elements of the workflow that requires human intervention. For example, approval of the sales proposal by a supervisor if the value exceeds $100,000

SOA based systems are inherently “interruptible”.  Describing a workflow is outside the scope of SOA applications but they should be able to consume and participate in well defined workflow scenarios that include human elements of collaboration. Business processes based SOA facades lend themselves to this scenario very well.

Analytics and Reporting

Heuristic analysis of Proposals over a period of time. For ex, how many proposals were created for Oracle to SQL conversions in the Automotive sector over the last 6 months and how this can be used to model future requirements for this skill set

Although SOA applications don’t prescribe analytics and reporting frameworks, it can be used effectively for data aggregation based on business processes. 

 

 

The Smart Client imperative

 

What is a smart client ?

<< >>

<< Intro here >>

Top reasons for the move back to Smart Clients

Offline capabilities

 

Smart Clients help in designing systems that support working in a disconnected state. Being able to work in isolation and connecting back to the live data is an integral part of market requirements for enterprise software. Having a smart client allows applications to support this scenario as it enables superior caching and synchronization capabilities

Rich User experience

The user experience of working with web based applications has its limitations in responsiveness and functionality. Smart clients allows enterprise applications to design user experiences that are much richer and allows faster response times for routine tasks like complex data input validation, richer graphics and data analysis in a manner that is not possible using web based clients

 

Support for multiple form factors 

With the demand for non-pc devices exploding, Enterprise applications have to be agile enough to support multiple devices like Pocket PC, Smart Phones, Tablet PC etc without having to re write the user interface layer. Using the principles of Service Oriented Architecture, the UI layer should be decoupled from the business processes that it drives. This enables the application to develop light weight clients that can consume the services on the back end in a seamless fashion

Technical enhancements supporting smart clients

At one point of time when the notion of thin clients was gaining popular support there were a number of factors that made rich clients too unwieldy and inflexible. Common problems included problems with deployment and patch management, firewall issues in cross organizational scenarios, version management ( aka Dll hell ), client side security issues etc to name a few. With advances in clinet side and server side technologies like .NET, it is much simpler now to write smart client applications that can be deployed over the web. It is also possible to design applications with auto update features that can deliver updated software to clients without user interaction. With these technical barriers breaking down, smart clients are making a compelling case for being the leading UI architecture for the future.

 

Harness the local processing power of the client machine

Web-based applications are often architected so that all processing is handled by the servers. Performing some or all of the work locally with smart client applications means quicker response time and reduced network and server workload, making for more scalable infrastructure. A smart client application can run without any interaction with the network and the server, completely freeing up that infrastructure, or it can go to the server as needed to dynamically download an assembly or to invoke an Extensible Markup Language (XML) Web service. The fact that even some of the work will be done locally—loading and implementing the UI controls, for example—results in a more dynamic user experience and much more efficient use of the infrastructure available, both client and networked.

 

Integration imperatives  

It is inevitable that in large enterprise scenarios, Line of Business applications are not going to exist in a stand alone environment. As technologies mature and many of the business application functionalities of yesteryears reach commodity status, integration is more important in today’s environment than ever before. Application integration is not a new concept in the Enterprise either. It has been around since the early days of data processing and has taken various forms over the course of these years.

Application integration started out with data integration which required the two applications to agree upon a common data format or an intermediary that know how to massage the data flowing between the two applications. This was the most common form of integration in the early days of automated business applications. Then came the wave of integration through pre determined application interfaces at a binary level. This worked fine but had some major draw backs too.

 

Some composite applications use connection-oriented middleware facilities, such as Component Object Model (COM), Common Object Request Broker Architecture (CORBA), Remote Method Invocation (RMI), remote procedure call or TCP/IP sockets. These are tightly coupled and technically synchronous. Other composite applications use connectionless middleware, such as Simple Object Access Protocol (SOAP) on HTTP or message-oriented middleware (MOM). These are loosely coupled and asynchronous in a middleware sense, although when used to implement a composite application, the result is still logically coupled because the client depends on getting some reply from the server to proceed with its work. The server must depend on the client because it cannot begin work until it gets the client's input.

As evidenced by the definitions, Web services are about technology specifications, whereas SOA is a software design principle. Notably, Web services' WSDL is an SOA-suitable interface definition standard: this is where Web services and SOA fundamentally connect. Those who see Web services as architecture regard WSDL as the definitive standard of Web services (others see SOAP as a definitive standard for Web services — this is a view of Web services as a communication method). In practical use, the ubiquitous Web services standards enhance the mainstream appeal of SOA design.

SOA opens up a whole new world of possibilities for leveraging legacy systems. New SOA systems can be constructed by leveraging web services interfaces of existing legacy applications. For the external consumer, the entry points into the system will only be through the service interfaces defined by the SOA system.


Conclusions:

Service Oriented Architecture is an evolution in Enterprise application design and has its roots in the Object Orient programming model. Both the schools of thought promote data encapsulation and abstraction but the similarities end there. With new advances in technology and the changing landscape of business requirements, it is imperative that Enterprise Applications look closely at SOA as its design. SOA provides both the flexibility to meet market demands and the robustness to adapt to the changing technical landscape. Choosing the underlying technology is not as important as the design based on web services and SOA principles.



[1] Gartner – SOBA Apps show their potential  ( SPA-20-7295 )