Marty Wasznicky - The "Event Driven Business"

Announcing  - August 2007 Community Release (CTP3) of ESB Guidance  – http://www.codeplex.com/esb

By Marty Wasznicky
Field Program Manager, Connected Systems Division

As a result of our ongoing partnership with Patterns & Practices, we’ve released a new August Community (CTP3) build of the ESB Guidance on ESB Guidance community site, where all future iterations will be released until our final release in October later this year. 

The CTP3 build introduces some new and augmented features such as:

       New Resolver and Adapter Provider Framework provides “messaging” level end point resolution support for

       UDDI

       WS-MetaDataExchange

       Business Rules Engine

       XPath

       STATIC

       Messaging level Transformation services (uses new Resolver Framework).  Allows BizTalk Maps to be executed in pipelines based on the following resolution methods

       UDDI

       WS-MetaDataExchange

       Business Rules Engine

       XPath

       STATIC

       Request-Response support for on and off ramps (partial)

       WCF Adapter integration

       UDDI publishing and query service

       BizTalk runtime query service

       New Dynamic Resolution Sample

       Third party SOA Management and Governance integration:

       SOA Software Integration

       ESB Guidance documentation in CHM format J

The August 2007 CTP3 release focuses on the incorporation of Windows Communication Foundation (WCF), Request-Response Messaging, and the introduction of the Resolver and Adapter Provider Framework. The latter provides the runtime resolution and transformation support to enable both one-way and request-response pure messaging scenarios. The former (specifically the BizTalk WCF adapter) makes it all possible. Hence, key to this release, is the continued refactoring and integration into BizTalk Server 2006 R2—scheduled for release in September 2007. Additionally, we are releasing several infrastructure-based services to support our ongoing development work for the ESB Management Portal, as well as for two key SOA Governance partners, Amberpoint and SOA Software.

Our first goal was simply to ship WCF Adapter versions of our existing On-Ramps and Services, so that—over time—we could retire the older ASMX versions. WCF offers many advantages over ASMX, with the separation of interface and implementation being just one. WCF, specifically the WCF Adapter within BizTalk, goes a long way towards providing full Web Service WS* support for customers, without the need to write significant code. The WCF adapter makes WS-Security and WS-AtomicTransactions as easy as enabling a checkbox. More importantly, its loosely typed nature enables the use of true dynamic-resolution Web Services with BizTalk. This removes the requirement for generating and referencing SOAP Proxies compiled into .NET assemblies for routing to every resolved end-point (as was the case prior to the introduction of the BizTalk WCF Adapter).

Our second goal was to develop a robust architecture for high performance yet flexible runtime resolution and transformation at the messaging level, without the need for the existing Orchestration-based services developed in previous releases of the ESB Guidance. This architecture is required to support full Request-Response pure messaging scenarios. It must also support customers who want to develop their own resolution methods and plug them into our architecture, without having to rewrite our existing code.

The result is the Resolver and Adapter Provider Framework, which supports dynamic loading, caching, and invocation of registered Resolvers. The Resolver determines the end-point configuration and URI, given certain facts. The Adapter Providers then set the specific properties of a BizTalk Adapter. Adapter Providers leverage the same dynamic loading, caching and invocation mechanisms as Resolvers. This framework goes a long way towards enabling us to provide simple messaging scenarios, such as “transform-validate-route”, with all the configuration and information externally driven by policy from a registry or custom repository. Our primary goal with the framework was to develop an architecture that could separate the consumer from the service, enabling us to move towards more policy-driven messaging scenarios.

Our third goal was to develop and introduce some comprehensive, Web-based infrastructure services that both our ESB Management Portal and other third parties can query and leverage. The first of these services, the BizTalk Query Service, exposes BizTalk application, service, messaging, and health information. This service provides users with the ability to ask questions such as:

       "What applications are deployed?"

       "Where are they deployed?"

       "What servers are they running on?"

       "Which services are running and which are stopped?"

       "Which host is the service running under?"

       "What server is the host running on?"

       "Is the host started on server B or server A?"

       "Give me a list of the services deployed since 1/1/2007."

The second of the new services, the UDDI Service, supports querying and modifying the registration information stored in UDDI based registries. 

Finally, we pursued the development of tighter integration with key SOA Governance vendors such as Amberpoint and SOA Software. Customers should not be forced to regard these, as well as our, technologies as isolated "black boxes" with one not able to leverage the features and functions of the other. A seamless integration experience, where one leverage the assets of the other, provides greater synergy, reduces complexity, and will lower the total operational cost of the solution. This can provide immediate benefits to our customers. For example:

       SOA Governance tools and analytics can extend, and provide end-to-end visibility and tracking for the BizTalk native messaging subsystem and solutions; providing customers with a “one management view” of the solution.

       BizTalk can enable SOA Governance vendors to become “transport-agnostic”; providing a functional transport bridge, rules, and orchestration technologies for customer solutions.  

       Integration of these technologies can provide centralized enforcement and management of end-points and policy across the entire solution, regardless of the transport or technology (such as WS*, FTP, FILE, MQSeries, JMS, SAP, Tibco, Peoplesoft, or Siebel) it uses.

We have been working with Amberpoint and SOA Software for several months and are now starting to see the results of our efforts. In fact, SOA Software has almost completed development of their integration code that demonstrates end-to-end SOA governance monitoring and policy enforcement within BizTalk. In this release, we include some of the documentation regarding the new SOA Software Management Point for BizTalk, as well as some screenshots and the URL of their web site where you can obtain more information.

Ultimately, I believe that the team accomplished the goals we set out for this release. We look forward to active feedback from all our users. 

A more thorough description and screen shots of the SOA Software integration with BizTalk can be found below.

Cheers!

SOA Governance Integration

Enterprise-level applications must support robust and reliable management features in order to comply with business requirements, Government legislation, service level agreements (SLAs), and customer and trading partner expectations.

The SOA Software Management Point for BizTalk Server 2006 is an extension to SOA Software’s Web Services Management Point, applied specifically to the BizTalk Server 2006 environment, providing run-time governance for any BizTalk application regardless of transport.

The Management Point integrates natively with SOA Software Service Manager and Workbench products. Unlike the typical Web Services Management Point, this implementation is associated with services provided by BizTalk environment, expressed in terms of BizTalk server Receive Locations and Send Ports. Due to the arbitrary nature of Receive and Send Ports (configured against a variety of BizTalk Adapters) these services  are not necessarily associated with Web Services, but can be treated as such in terms of the SOA Service Manager and SOA Workbench.

Figure 1 shows the SOA Service Manager Web application displaying the Workbench page for an example application. The left-hand tree view allows users to navigate through the applications and services installed within BizTalk, while the right-hand pane allows users to view application details, operations, access ports, categories, rules, and monitoring information.

Figure 1

The SOA Service Manager showing the monitoring features available in the Workbench page

You can monitor all the operations within an application (all Send Ports and Receive Locations), or specific operations, and the Workbench shows a list of the messages passing through the selected Send Ports and Receive Locations. When you double-click on a message in the list, the Workbench displays details of the message, and its context properties (provided that recording is enabled). Alternatively, you can select a specific service and monitor in real time messages passing through that service.

BizTalk Integration

The SOA Service Manager provides Wizards that help you set up the SOA Service Manager for use with BizTalk Server. For example, Figure 2 shows Wizard that helps you install and configure the BizTalk Integration Point that links BizTalk Send Ports and Receive Locations to the SOA Service Manager.

Figure 2

The SOA BizTalk Integration Point installation and configuration Wizard

After installation, the Service Manager can display details of the Send Ports and Receive locations configured by the Wizard. Figure 3 shows the Operations page displaying the name, interface, policy, and number of virtual operations for three Receive Locations and a Send Port.

Figure 3

A list of BizTalk Send Ports and Receive Locations displayed by the SOA Service Manager

Monitoring Policies and Usage Information

The SOA Service Manager provides a mechanism that allows you to create and edit monitoring policies. For example, Figure 4 shows the screen for applying a policy template and activating monitoring for an application.

Figure 4

The SOA Service Manager showing the monitoring features available in the Workbench page

The SOA Service Manager also provides usage information for applications, services, and individual Send Ports and Receive Locations. This information includes charts displaying the relative usage of each operation and usage over time, service usage, and response time information. Figure 5 shows some example.

Figure 5

A selection of the charts that the SOA Service Manager can generate to assist in monitoring server and application behavior and performance

In addition, SOA Service Manager includes features that allow you to view details of individual messages, including the size, response time, operation name, and the associated SOAP messages, as well as any exceptions that may occur in the BizTalk runtime environment, as shown in Figure 6.

Figure 6

The SOA Service Manager showing details of an individual message

SOA Software Service Manager and Workbench products are products from SOA Software, Inc. that integrate with BizTalk Server 2006. For more details of SOA Software and their products, and to download the latest installation and operational instructions, see http://www.soa.com/

Announcing  - June 2007 Community Release of ESB Guidance  – http://www.codeplex.com/esb

By Marty Wasznicky
Field Program Manager, Connected Systems Division

In October of 2006, we announced the preview of the Microsoft ESB Guidance at Microsoft SOA Conference in Redmond. This consisted of the following which enabled Microsoft partners and customers to build large and small-scale ESB solutions:

       Sample code built on BizTalk Server 2006

       Architectural guidance, patterns and practices

       Reusable BizTalk Server ESB and .NET components:

       Dynamic Transformation Service

       Dynamic End Point/Configuration Service

       Itinerary based Routing & Service Invocation

       ESB Portal

       Exception Management framework

       Namespace Resolution Service

       JMS (Java Message Service) pipeline component (IBM JMS over WMQ) 

Since that time we’ve achieved significant momentum:

       Several hundred requests for the ESB Guidance

       Several dozen customers incorporating the guidance into their projects

       Dedicated Microsoft ESB web site: http://www.microsoft.com/biztalk/solutions/soa/esb.mspx

       Dedicated Partner site for delivering the ESB Guidance: http://www.microsoft.com/biztalk/solutions/soa/esbpartners.mspx

       Press on the ESB Guidance from the SOA conference

       http://www.cio.com/blog_view.html?CID=25511

       http://www.crn.com/sections/breakingnews/dailyarchives.jhtml?articleId=193104205

       http://www.infoworld.com/article/06/10/04/HNmsesbsoa_1.html

       http://www.networkworld.com/news/2006/100406-microsoft-offers-esb-guidelines-for.html

       http://www.pcwelt.de/mobile/pda/news20061005/582163/ (German)

       http://www.programmazione.it/index.php?entity=eitem&idItem=34953 (Italian)

Due to the overwhelming demand and popularity of the ESB Guidance, we (the Connected Systems Division) and the Patterns and Practices group have agreed to jointly develop the ESB Guidance moving forward. 

As a result of the iteration of this partnership, we’ve released a new June Community Release build of the ESB Guidance on the new ESB Guidance community site, where all future iterations will be released here until our final release in October later this year. 

Although this June Community Release focused on bug fixes and refactoring the ESB Guidance for BizTalk 2006 R2, future releases will incorporate many new and augmented features such as:

       UDDI publishing and richer resolution options

       WS-MetadataExchange

       Enhanced ESB Portal and Exception Mediation

       Request-Response support for on and off ramps

       WCF Adapter integration

       New Samples and Guidance

       Third party SOA Management and Governance integration:

       Amberpoint Integration

       SOA Software Integration

Through our partnership with Patterns and Practices, we hope to raise the bar of quality and build an extended community to support our customers as they move forward with the ESB Guidance.

With the Patterns and Practices partnership, we incur the benefit of having Don Smith (Product Manager for Patterns & Practices – don.smith@microsoft.com ) take an active role in owning the ESB Guidance Community.  I will be coordinating feature development and community release schedules with Don, ensuring community feedback is incorporated into the ESB Guidance.

Please download the latest Community Release and provide us feedback.  We will be monitoring the discussion forums on the site. 

A more thorough description of the June Community Release can be found below.

Cheers!

About the June 2007 Community Release

This release focuses on refactoring the CTP build to accommodate the augmentation of existing features, as well as the addition of new features to be included within the final release targeted for October 2007.

Some of the changes for this release are:

·         The project namespace changes from Microsoft.BizTalk.ESB to Microsoft.Practices.ESB

·         All schemas follow W3C schema conventions for namespace, such as http://schemas.microsoft.biztalk.practices.esb.com/jms/system-properties

·         Installation scripts and instructions for manual installation are provided

·         All samples now install into the GlobalBank.ESB BizTalk application instead of their own individual applications

·         The following components have been rebuilt to work with the BizTalk Server 2006 R2 Beta 2 and Release Candidate builds:

◦       JMS Pipeline Component

◦       Namespace Pipeline Component

◦       Exception Management API, InfoPath form and Pipeline component

◦       Transformation Service, Helper and Agent

◦       JMS Samples

◦       Exception Management Samples

◦       Namespace Samples

·         This release contains several updates to integrate with .NET version 3.0:

◦       The Transformation Service is exposed as both an ASMX and a WCF service

◦       The Exception Management Fault Service is exposed as both an ASMX and a WCF service

◦       The UDDI helper class now uses native WCF calls and the features of .NET version 3.0. There is no longer a dependency on the Microsoft UDDI service (Microsoft.Uddi.dll)

·         There are a number of fixes for components and services included in this release. For more details, see:

◦       Fixes for the Exception Management API

◦       Fixes for the Transformation Service

◦       Fixes for the JMS Component

◦       Fixes for the Namespace Component

◦       Fixes for the SetContext Component

Note: Although all other components in this release will compile and work with BizTalk Server R2, they still require some re-work. In addition, the ESB Exception Portal does not change in this release, and therefore the Exception Management Framework has not been tested with the older portal. The next community release will contain a prototype of the new portal.

Fixes for the Exception Management API

·         The Exception Management API now supports exceptions that occur in child orchestrations executed using the CALL shape. For example:

◦       Orchestration A calls Orchestration B

◦       Orchestration B contains exception handling code that invokes the CreateFaultMessage method and assigns a fault message

◦       An exception occurs in Orchestration B and the ESB Exception Framework should publish the message

◦       The execution path no longer fails and causes an error within the CreateFaultMessage method

·         InfoPath form rendering exceptions from the ESB Fault Processor pipeline no longer cause an error when scrolling through context properties

·         The ESB processor pipeline now includes whitespace support in the reader

·         The properties for the fault schema change in this release. There is no promotion of FaultDescription, and the description now accepts more than 255 characters

·         Extension of the existing property schema supports additional internal error information

·         The schema namespace changes, and the InfoPath template has been renamed

·         Class references replace the string references for the context property name and namespace used when accessing context properties

·         This release supports MIME-type detection for the body of a message

·         Failed Message Routing scenarios now support flat files and binaries

·         Streaming is implemented for large files

·         Correct identification of the send port now occurs in one way messaging scenarios

·         There is support for two-way messaging scenarios, such as the typical SOAP request-response cycle

·         All project versions are now 1.0.0.1

·         The native BTS stream libraries replace all custom streaming classes

·         A new ContextProperties class allows pipeline components to access property information

·         The orchestration time stamp in now expressed in UTC

·         A new WCF-based Web Service exposes fault information

·         All references to http://org.kp.esb.engine and the "bootcamp" namespaces in InfoPath schemas are removed

·         New scripts update the Resources folder in the BizTalk Server application, and generate an MSI based on the resource artifact files

·         Minor modifications to the code catch more exceptions, in particular by checking for null references

·         The component detects non-HTML and non-text content, and encodes these as base-64 strings

·         The reporting schema and InfoPath template now contain a MIME-type property

·         There is support for rendering the content of XML CDATA sections

·         The schema type for the content block within the reporting template changes from Any to CDATA

Fixes for the Transformation Service

·         The Transformation Service now throws a null exception if the mapName parameter is null.

·         The Transformation Service now has no dependency on the Microsoft UDDI service (Microsoft.Uddi.dll).

Fixes for the JMS Component

·         The strong name key file binding now occurs during the linking phase so that the Manifest phase does not invalidate the signature of the assemblies.

·         The component now uses the Microsoft.Practices.ESB.snk key file included with the source.

·         The JMS Sample uses a new Queue Manager name, and only four queues.

Fixes for the Namespace Component

·         The NamespaceException now inherits from ApplicationException, which corrects the issue where exceptions appear in the Event Log as "Reason: Unknown"

·         Corrections to processing of the ExtractionXPath property solve previous extraction issues

·         A new validation routine introduced prior to pipeline processing ensures that required properties are set at runtime

·         Regular expressions now check that separator property value is valid (or is empty), that namespaces are not in the reserved range (ns0 to ns6), and that the specified namespace prefix is alphanumeric

·         The component now checks that either the XPath or NamespaceBase property is set

·         The component now compares the existing and the new namespaces, and just returns the original message to the pipeline if they are the same

·         The component no longer raises an exception if the XPaths property is null, which allow the use of a static namespace if required

·         Various string comparisons carried out within the component have been corrected

Fixes for the SetContext Component

·         The component uses an updated namespaces for property schemas

·         The component now exposes EndpointUddiServer and MapUddiServer properties

·         The component now contains logic that ensures the value of a SOAP header is not overwritten by a property value provided in the pipeline instance rather than in the SOAP header

·         There are several new trace statements that output the resolved endpoint