Recurring Architectural Capabilities - Loosely Coupled Thinking - Site Home

More fluffy draft on yet another fluffy topic...

Again, feedback encouraged.

As we saw earlier, the SOA architectural model is fractal. This means that a service can be used to Expose IT assets (such as a Line of Business system), be Composed into workflows or Business Processes (each of which may also be exposed as a service) and be Consumed by end users, systems or other services. SOA is a fractal, not layered model. While the Abstract SOA Reference Model provides a holistic view of several important SOA concepts, the Expose / Compose / Consume portions of the model should not be interpreted as layers (despite their apparent appearance in the model). Designing a SOA as a set of well-defined tiers (or layers) will constrain the value and flexibility of your services, resulting in dependencies across unrelated components. This is why the Expose / Compose / Consume portions of the model can be thought of as independent architectural initiatives referred to as a Service Implementation Architecture (Expose), a Service Integration Architecture (Compose) and an Application Architecture (Consume). While each of these architectures are designed to be independent of one another, they share a set of five common capabilities

Figure 7: Recurring Architectural Capabilities

Communications

Communications focuses on how messaging is accomplished between senders and receivers. There are a broad array of options and patterns available – from pub/sub and asynchronous to message and service interaction patterns. Services provide an evolutionary approach to building distributed software that facilitates loosely coupled integration and resilience to change. The advent of the WS-* Web Services architecture has made service-oriented software development feasible by virtue of mainstream development tools support and broad industry interoperability. While most frequently implemented using industry standard Web services, service orientation is independent of technology and architectural patterns and can be used to connect with legacy systems as well. Messaging and Services are not a new approach to software design – many of the notions behind these concepts have been around for years. A service is generally implemented as a coarse-grained, discoverable software entity that exists as a single instance and interacts with applications and other services through a loosely coupled (often asynchronous), message-based communication model. Messages tend to be based upon an agreed-upon set of semantics (such as an industry-standard Purchase Order) and serialized using an interoperable, extensible syntax (usually XML, although alternate approaches like JSON, RNC and ASN1 are sometimes used).

Workflow and Process

Workflow and Process are pervasive across multiple layers of an integration architecture – from formal orchestration of processes to flexible ad hoc systems and collaborative workflow across teams. Since business processes are dynamic and evolve with the organization, the workflow that models these processes must be equally adaptable. In addition, effective workflow involves not only process modeling, but also monitoring and analytics in order to respond to exceptions and optimize the workflow system over time.

Data

The lynchpin to success in many integration architectures is the ability to provide Data management. The need to deliver a shared view across disparate, often duplicate sources of data is more important than ever, as businesses strive to achieve a 360-degree view of organizational information. Entity aggregation, master data management, and the ability to make data useful via analytics and mining are crucial elements of integration architecture.

User Experience

Successful integration architectures depend upon both service delivery and the ability to consume services in a rich and meaningful way. Service consumption needs to be contextual, mapping to the natural workflow of employees, customers, and partners. To that end, an integrated User Experience spanning smart clients, rich clients, lightweight Web applications, and mobile devices enables service consumption by the broadest possible audience.

Identity

To support integrated user experiences, customers require the ability to manage the identity lifecycle – providing integrated Single Sign-On (SSO), access management, directory services, and federated trust across heterogeneous systems. Today, many solutions are built using fragmented technologies for authentication and authorization. In the new application model, access decisions and entitlements need to be made at multiple layers and tiers, in which a federated Identity across trust boundaries becomes a key requirement.

Management

During the lifetime of a service, the service most probably changes in different perspectives as listed below. As a result, one service will probably have to be available in several versions.

Difference in interface (e.g. extended interface, but same business object)
Difference in semantics with same interface (business objects changed)
Difference in QoS, e.g. slower but cheaper or high-available but more expensive

Service management encompasses many capabilities, some of which are listed below (we will discuss Management in greater detail in Chapter ??).

A comprehensive solution for change and configuration management, enabling organizations to provide relevant software and service updates to users quickly and cost-effectively.
Reducing the complexity associated with managing the IT infrastructure environment with a focus on lowering the cost of operations.
Centralized backup services capturing changed files to disk. Centralized backup should enable rapid, reliable recovery from disk while providing end-user recovery without IT intervention.
Pre-deployment capacity planning coupled with best-practice guidance and hardware-specific knowledge to help information technology professional make low-risk architectural decisions.
Data warehouse and reporting to help IT better support corporate decision making, improve the quality of service provided, and better administer resources through improved reporting capabilities and management data integration from a broad variety of resources.

Supporting the Common Architectural Capabilities

The five architectural capabilities discussed above are supported by the Microsoft SOA Platform. Support for the common architectural capabilities is discusses in greater detail in Chapter 2.

Figure 8: SOA Capabilities on the Microsoft Platform

Common Architectural Capabilities and the Abstract SOA Model

We can also think of these five common architectural capabilities as set of perspectives for viewing and understanding the Abstract SOA Model. The five architectural capabilities serves as a set of lenses to help us view and better understand the challenges associated with Exposing existing IT investments as services, Composing the services into business processes and Consuming these processes across the organization.

Expose

Service Enablement

Expose focuses on how we design and expose our services. We will most likely start out by enabling our IT investments to be exposed as web services.

As our organization matures we will start adding new services, most likely as proxies for other resources within the organization.

One of the hardest parts of service implementation is deciding where to begin. The are a variety of choices here and there is no single recommendation that will work for everyone. Motion is a methodology that provides some guidance for identifying business capabilities that could be exposes as services.

What are some best practices that one should follow when exposing IT investments as services?

Think big– but start small

Show value at every step along the way – not build it and they will come

Middle-out – not Top-down or bottom-up

Be pragmatic

Vertical-slice – not build it and they will come

Risk-mitigation

Demonstrate value in rapid iterations – not waterfall
Successful customers ‘snowball’

The recurring architectural capabilities provide us with a set of considerations when exposing IT investments as services. Let’s take a quick look at some of the considerations associated with each capability for service exposure (this is by no means a complete list):

Communications

Determining what to expose and how - Avoid falling into the granularity trap – focus on meeting your business requirements
Service Operation Contracts
Message and Data Contracts
Configuration, behaviors and control
SLAs
Governance
Versioning
More…

Workflow and Process

Coordinator services for distributed, long-running processes
Tracking services capable of logging specific events within a workflow
Compensation services

Data

Entity services
Entity Aggregation services: acts as a single point to access information that may exist in multiple systems. An Entity Aggregation service has the following responsibilities:

Acts as a unified source of entities.
Provides a holistic view of an entity.
Provides a holistic view of the entity model—entities and their relationships with other entities
Provides location transparency—consumers of the Entity Aggregation layer do not need to know who owns the information.
Enforces business rules that determine the segments of entities retrieved in a given context.
Determines the system of record for each data element constituting an entity.
Enriches the combined data model across systems—the whole being better than the sum of its parts.
Entity factoring

MDM – exposing data across corporate or departmental boundaries.
Canonical schemas imply that all the services share the same schema, which need not be the case

User Experience

Specialized services for supporting user interfaces (caching resources, communications between UI and services, etc). Service wrappers provide coarse-grained interfaces for user app consumption, lightweight mash-ups, etc.

Identity and Access

Identity Management
Impersonation and Delegation services
Trusted Subsystem - A trusted subsystem model implies that services are trusted to perform specific tasks, such as processing customer orders.
Authentication (Kerberos, SSL)
Role-based access control (RBAC)
Create/revoke trust relationships
Services need to make authorization decisions, such as approving an order submission before performing the business transaction.
The service must know the identity of the end user submitting the order.
Need to flow the identity of the end user is an inherent property of the delegation model, it is not so for the trusted subsystem model and special efforts must be made to include this feature.
To support the notion of trust as defined by the model, the services must at least be able to:

Authenticate / verify identity of upstream / downstream services

Decide if the service is a trusted subsystem for specific functions (including propagating identity claims)

Protect the integrity of the data being communicated between trusted subsystem and services.

Besides application data, application plumbing data, such as the identity claims of the original user, must also be protected so that no man-in-the-middle can modify the identity information that is in transit.

Compose

Service Composition

Compose focuses on how we can combine or aggregate granular services into more complex processes. We will most likely start by using services that expose our existing IT investments.

Service composition results in a new service instance that the rest of the organization can make use of. The composition provides capabilities such as correlated asynchronous service invocation, long running processes and other capabilities for orchestrating autonomous services.

The recurring architectural capabilities provide us with a set of considerations when composing granular services into complex processes. Let’s take a quick look at some of the considerations associated with each capability for service composition (this is by no means a complete list):

Communications
Service interaction patterns
Exposing orchestrations as services
Asynchronous service invocation patterns
Workflow and Process
Transactions
High frequency of change
Business Rules
Service Orchestration
Service Interaction Patterns (SIPs)
Process Externalization
Long Running Processes
Auditing and analytics
Data
Tracking the state of a given workflow instance
Data transformation (ETL)
Reliable message processing and storage
Replication
Synchronization
Metadata repository and Management
Instance reconciliation
Schema reconciliation
Document Replication
Syndication/Aggregation
User Experience
Composite applications (OBAs)
Human workflows (MOSS)
Orchestrations initiate human workflows via SharePoint adapter
Pageflows

Identity and Access
Impersonation and Delegation
Provisioning
Identity Repository synchronization
Approval workflows

Consume

User Experience
Consume focuses on how services and orchestrated processes (which may be exposed as services) are consumed by other services, applications and end-users. Any resource capable of interacting with services can be referred to as a “consumer”. Consumers may appear across the organization in several possible forms:

Lightweight, browser-based applications
Rich Internet applications (RIA) are browser-based applications that can address and cache local and remote resources
Configurable, portal-based user experiences
Applications that are installed on the local machine (such as a custom Windows application)
Corporate business applications with solution-specific extensions (such as Microsoft Office with context-aware activity panes)
Applications designed for mobile devices
Services may act as consumers of other services

Recall that the SOA model is fractal – services may be consumed by other services and service compositions may be exposed as new services. In the last couple of years a “new breed” of consumers has emerged, enabling consumers to be aggregated and consumed by additional consumers. This “new breed” of consumers is usually referred to as a “mashup”. A mashup is a set of services, websites or applications that combine content from multiple resources into a new integrated user experience. Content used by mashups is typically sourced from a third party (such as a service or website) via a public interface or API. Alternative methods of sourcing content for mashups include newsfeeds and JavaScript constructs (JSON).
The recurring architectural capabilities provide us with a set of considerations for User Experience. Let’s take a quick look at some of the considerations associated with each capability for User Experience (this is by no means a complete list):
Communications
Forms-based service consumption
Web parts
Service Registry – check in / check out / search
AJAX, REST
Workflow and Process
Human workflows (MOSS)
Event brokering (CAB)
Page flows
Data
Entities (OBA Business Data Catalog)
Single view of the customer problem
JSON
User Experience
Composite applications (OBAs)
Personalization, user profiles
Portals
Business Intelligence
Reporting
Content aggregation
Declarative UX

Identity and Access
Single Sign-On (password synchronization)
User identification
Role-based access (RBAC)
Directory Services
Password management
Privacy (firewalls, encryption)
Compliance

Summary

In this chapter we provided some useful analogies for understanding the fractal nature of SOA. Services are the fundamental building blocks of SOA, although services do not necessarily need to be web services. Ideally these services should follow the four service design tenets which describe a set of best practices for service scopes, dependencies, communications and policy-based configuration. While these tenets focus upon service design, it is important to realize that services alone are not necessarily solution architecture – Microsoft uses an abstract reference model to describe the various aspects of SOA. The abstract SOA reference model provides three fundamental concepts to help most organizations understand the role that services can play within their solution architectures:

Expose focuses on how existing IT investments are exposed as a set of broad, standards-based services, enabling these investments to be available to a broader set of consumers. A Service Implementation Architecture describes how services are developed, deployed and managed.
Compose focuses on combining services into applications or cross-functional business processes. A Service Integration Architecture describes a set of capabilities for composing services and other components into larger constructs such as business processes.
Consume focuses on delivering new applications that enable increased productivity and enhanced insight into business performance. A Service Oriented Application Architecture describes how “composed services” are made available for consumption through as business processes, new services or new end-user applications.

Each aspect of the Expose / Compose / Consume abstract reference model encompasses a set of five recurring architectural capabilities: Communications, Workflow and Processes, Data, User Experience and Identity. The five architectural capabilities serves as a set of views to better understand the challenges associated with Exposing existing IT investments as services, Composing the services into business processes and Consuming these processes across the organization.