BizTalk Adapter Development

"WCF LOB Adapter SDK and the BizTalk Adapter Pack" Whitepaper

mmckeown — Tue, 21 Apr 2009 06:11:00 GMT

When building applications today, it’s hard to consider building something that doesn’t involve connectivity of some sort. Applications require business data and logic that is distributed across several applications or servers. Unfortunately, not all systems provide the same interface to their data and business logic which ultimately forces developers to figure out how to talk to each of those systems. Connecting to a system doesn’t just involve opening up a port on a network address either; we have to work with different message formats, varying security mechanisms, and in many cases custom libraries that rely on proprietary mechanisms. In the end, it’s common for developers to struggle with learning a variety of different programming interfaces, communication protocols, and messaging semantics.

Windows Communication Foundation (WCF) promises to change all of that. WCF provides a unified programming model for building distributed applications using the .NET Framework. WCF was designed to provide a single unified programming model for writing either clients or services while also providing a flexible framework for different styles of communication on the wire. This approach allows developers to focus on writing code in their business domain rather than on learning new networking interfaces or object models. The code you write with WCF always looks the same but you can configure your apps to use different transport protocols like TCP, HTTP, and MSMQ; different message encodings like XML, MTOM, and binary; and varied security options including certificates, passwords, and security tokens.

The goals addressed by WCF are also very similar to the integration goals of Microsoft® BizTalk® Server. Ultimately BizTalk Server is primarily focused on providing an easy-to-manage model for connecting disparate, heterogeneous systems using a variety of different protocols, message formats and security mechanisms without requiring much, if any, code. The WCF LOB Adapter SDK and the BizTalk Adapter Pack whitepaper discusses how the worlds of WCF and BizTalk Server 2006 are fully converging through the WCF LOB Adapter SDK and the BizTalk Adapter Pack.

WCF Adapter Video: How a WCF Receive Adapter Processes an Incoming WCF Message

mmckeown — Wed, 18 Mar 2009 05:06:00 GMT

I just published this WCF adapter video examining, in detail, the flow of an incoming WCF message into a BizTalk receive location that is configured to use a WCF receive adapter. See how this message is converted by the WCF adapter into a BizTalk message and stored in the Messagebox database.

http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=c94f6263-3cd4-43b5-b038-5737a8d0d219

Good as Gold! New WCF Adapter FAQ Whitepaper!

mmckeown — Tue, 10 Mar 2009 23:35:00 GMT

I just published this extensive FAQ with the help of the BizTalk Server product team at http://go.microsoft.com/fwlink/?LinkID=144765.

These FAQs about the Windows Communication Foundation (WCF) adapters consists of the following themes. Pass it on and enjoy!

· General Conceptual: FAQs surrounding general conceptual WCF adapter topics.

· Message Flow and Mapping: FAQs about how a message flows in and out of BizTalk Server using the WCF adapters.

· Development Libraries: FAQs to help you better understand which development libraries to use when developing BizTalk adapters.

· Using WCF Services: FAQs explaining how to access WCF services, and how to extend BizTalk Server using WCF custom bindings and custom behaviors.

· WCF Endpoints: FAQs about WCF endpoints with information about service endpoints, metadata, and how to publish WCF endpoints.

· Service Architecture: FAQs describing the service architecture and when to use a BizTalk pipeline or a WCF behavior, and the types of encoding to select when using the WCF adapters.

· WCF Error Handling: FAQs examining how to handle SOAP faults and errors during processing of messages through the WCF adapters.

"BizTalk Server and WCF Support in Visual Studio 2008" whitepaper just published!

mmckeown — Thu, 12 Feb 2009 02:20:00 GMT

Microsoft® Visual Studio® 2008 provides robust support for solutions that use the Windows® Communication Foundation (WCF) and Microsoft BizTalk® Server 2009. While some of the functionality is clearly visible, some of it may not be evident to a BizTalk application developer how to take full advantage of these options.

The purpose of this whitepaper is to provide a clear and concise summary of the capabilities exposed by Visual Studio in support of BizTalk Server and WCF. That paper contains an explanation for the BizTalk application developer how each feature can help you in your complete BizTalk Server and WCF solution. You can view this as a high-level explanation of how Visual Studio 2008 can help support your BizTalk Server and WCF development and integration.

You can read or download it at http://msdn.microsoft.com/en-us/library/dd443655.aspx.

Troubleshooting BizTalk Server with SOAP Adapter Whitepaper

mmckeown — Wed, 03 Dec 2008 16:54:00 GMT

Microsoft® BizTalk® Server uses the SOAP adapter and orchestrations to receive and send Web service requests. The SOAP adapter enables orchestrations to be published as Web services and consume external Web services. Because configuring and/or using Web services can be complex, BizTalk Server handles many of these complexities on behalf of the user. Even so, there are still issues arising from time to time. This white paper is designed to highlight common issues and to provide guidance for how to resolve them.

http://www.microsoft.com/downloads/details.aspx?FamilyID=0297745b-7636-4f4d-ab44-62d1abe2330d&DisplayLang=en

"Using the Windows Communication Foundation (WCF) Adapters in BizTalk Server" Whitepaper

mmckeown — Mon, 03 Nov 2008 05:10:00 GMT

Written by Aaron Skonnard, this was originally named "Integrating Microsoft BizTalk Server 2006 R2 with the Windows Communication Foundation". It has been renamed to "Using the Windows Communication Foundation (WCF) Adapters in Biztalk Server". It can be found on MSDN at http://msdn.microsoft.com/en-us/library/bb967002.aspx, or directly from the download center at http://www.microsoft.com/downloads/details.aspx?familyid=a976dc7d-2296-4f88-be4d-0d314fca9e59&displaylang=en&tm.

Here are the first few paragraphs from the paper giving a brief introduction of it's content.

Microsoft® BizTalk® Server has long supported Web services as a primary communication technique but the support has been fairly limited to date. For example, developers working with BizTalk Server 2006 can use the built-in SOAP adapter to achieve “basic profile” service integration or the downloadable Web Services Enhancements (WSE) 2.0 adapter when support for the various WS-Security specifications is required. But that’s as far as these existing adapters can take you in today’s maturing service-oriented landscape. Developers working in more modern service-oriented environments, which build on the various WS-* protocols, have been unable to fully utilize BizTalk Server 2006 in such environments since the product lacked support for Windows® Communication Foundation (WCF) within the messaging layer.

Fortunately, with the release of BizTalk Server 2006 R2, developers will find a set of new WCF adapters that offer full integration with the advanced WS-* specifications supported by the WCF runtime. These adapters target the most common WCF communication scenarios and make them easy to tackle. As with the earlier SOAP/WSE adapters, developers can use the tools provided with the WCF adapters to publish WCF services from orchestrations or schemas and to consume external services using a variety of different transport, encoding, and protocol options, offering unparalleled service-oriented flexibility. The remainder of this white paper provides in-depth coverage on using the new WCF adapters.

"Consuming and Hosting WCF Services with Custom Bindings in BizTalk Server" Whitepaper

mmckeown — Mon, 03 Nov 2008 04:49:00 GMT

"Consuming and Hosting WCF Services with Custom Bindings in BizTalk Server is a new WCF whitepaper. You can find the download page from MSDN at http://msdn.microsoft.com/en-us/library/dd203050.aspx, or download it directly from the download center at http://www.microsoft.com/downloads/details.aspx?FamilyID=70badd79-ce7e-4ccb-be5c-3ff74ca3a8c4&displaylang=en.

Here are the first few paragraphs from the paper giving a brief introduction of it's content.

With the realization of the benefits of developing and orchestrating unified business solutions, today’s business organizations adopt the paradigm of Service Oriented Architecture (SOA)-based solution development for the execution of dynamically composed business processes. However, due to the disparate nature of the diverse business applications that exist in an enterprise, many organizations experience daunting challenges in integrating these business applications and services.

This paper discusses the use of Microsoft® BizTalk® Server 2006 R2 and its WCF adapters as a platform for service orientation that alleviates the aforementioned integration challenges. The paper emphasizes the configuration and integration of custom WCF bindings with BizTalk solutions, through the use of the above adapter platform. Furthermore, the paper provides concrete solutions for the identified integration concerns that apply to the orchestration of services through the use of custom WCF bindings.

Another new whitepaper posted! "Using Custom Behaviors with the BizTalk WCF Adapters, Part 1"

mmckeown — Thu, 09 Oct 2008 04:58:00 GMT

The first of two whitepapers entitled “Using Custom Behaviors with the BizTalk WCF Adapters, Part 1"” is now available for download. You can get the Word version of the whitepaper at http://go.microsoft.com/fwlink/?LinkId=129615. Alternatively, it's location in the MSDN library can be located at http://msdn.microsoft.com/en-us/library/cc952299.aspx .

Here is a summary of its content. Enjoy!

Customization of message processing in Microsoft® BizTalk® Server 2006 R2 can be extended through the use of Windows Communication Foundation (WCF) custom behaviors. This article is the first of two parts discussing how the use of WCF custom behaviors within BizTalk Server gives increased control over the message transmission process.

New whitepaper posted! "Modifying Messages with the WCF Adapters"

mmckeown — Wed, 01 Oct 2008 00:14:00 GMT

A new WCF adapter whitepaper entitled “Modifying Messages with the WCF Adapters” is now available for download. You can get the Word version of the whitepaper at http://download.microsoft.com/download/c/f/b/cfbcd6ad-5454-4c23-85d2-0cc3d7627015/WCFModifyingMessages.doc. Alternatively, it's location in the MSDN library can be located at http://msdn.microsoft.com/en-us/library/cc950532.aspx.

Here is a summary of its content. Enjoy!

The Windows Communication Foundation (WCF) adapters enable you to modify both incoming BizTalk® messages and outgoing WCF messages. You can use an XPath expression to specify what part of an incoming WCF message to use as the source of the BizTalk message body, and you can use an XML template to modify the content of an outbound WCF message body.

Adapter Pack Poster from Microsoft

mmckeown — Thu, 24 Jul 2008 20:06:00 GMT

Announcing the Biztalk Adapter Pack Poster

We are happy to announce the availability of the new BizTalk Adapter Pack Poster.

This poster covers interoperability with Line-of-Business Applications using the BizTalk Adapter Pack and the Windows Communication Foundation (WCF) LOB Adapter SDK.

This poster depicts the functionality, components, architecture, and usage/hosting scenarios of BizTalk Adapter Pack 2.0 and of the WCF LOB Adapter SDK. When printed in full scale, this poster size is 38”x 26”.

The poster in PDF format is available for download in the Microsoft Download Center.

We have designed this poster to promote the adoption of the Adapter pack and of the Adapter SDK. This would simplify service enablement and interoperability with Line of Business Applications or any metadata-rich systems.

Three New WCF Adapter Samples

mmckeown — Fri, 11 Apr 2008 04:29:00 GMT

I have recently posted three WCF Adapter samples off the BizTalk Server Developer Center at http://msdn2.microsoft.com/en-us/biztalk/bb608378.aspx, Two samples were released today. The first one is of a WCF message intemediary where an incoming WCF message to BizTalk is dynamically channeled to the correct version of the the WCF service. The second one is shows custom message interception during BizTalk Server's binding processing of an outgoing WCF message. The third sample was published in January and is an example of using pure WCF messaging with the WCF adapters.

Tell me...what do you want for WCF adapter documenation?

mmckeown — Mon, 10 Mar 2008 19:05:00 GMT

Is there a specific WCF adapter sample or whitepaper you feel you would like to see written? If so, please email me directly and let's discuss to see if I can get what is needed any WCF adapter content.Please put on the subject line "WCF Adapter Documentation Recommendation" ,and send to mckeown@microsoft.com. Thank you.

Why does the Adapter say ‘Action is not understood’ even though I am using the binding file generated by the ‘Consume Adapter Service’ wizard?

mmckeown — Wed, 09 Jan 2008 04:54:00 GMT

Believe me, the adapter is telling the truth! It really doesn’t understand the action it got! To understand what’s going on, read on…

When you use the ‘Consume Adapter Service’ wizard to generate LOB schemas in a BizTalk project, you also get a binding file which you can import (using the BizTalk Administration Console) to create the corresponding physical WCF-Custom port (Send or Receive Port, based on whether the contract was Outbound or Inbound respectively) when deploying your BizTalk Application. While this does save you the pain of creating and configuring the port manually, there is a ‘not-so-obvious’ caveat which you need to be aware of. (This applies only to send ports i.e. Outbound Contracts)

The WCF-Custom Adapter allows the SOAP Action to be specified in the following two ways on the static send ports:-

As a single line of text – the port is used for a single intended operation (say ‘ABC Action’).
As an xml, mapping each of a set of ‘Operations’ to the corresponding SOAP Action. This xml looks something like this:- (namespaces omitted for clarity)

<BtsActionMapping>
<Operation Name =”ABC” Action = “ABC Action” \>
<Operation Name = “XYZ” Action = “XYZ Action” \>
</BtsActionMapping>

The binding file created by the ‘Consume Adapter Service’ wizard chooses the second method to specify the action, even if you generate schemas for a single operation.

Now the question is, when a message comes to a port using such a mapping, how does it know what the intended operation was? Or more specifically, how does it know which action to use for which incoming message?

To make this decision, the WCF-Custom adapter compares the message’s BTS.Operation context property with the list of operations in the map (attribute ‘Name’). If a match is found, the corresponding action is used.

However, if the WCF-Custom adapter fails to resolve the operation name from the map, it assumes that the entire thing specified is a single line action (as in 1 above) and sends the whole string to the target adapter binding.

The most common scenario this could happen is when you create an orchestration and forget to change the operation names on the logical ports to appropriate names. By default, when you create a logial port in an orchestration, it’d be created with a single operation called ‘Operation_1’. The next operation you add (either on the same port or while creating another port) would be called ‘Operation_2’ and so on. You need to modify these default names to the appropriate names as the BTS.Operation context property derives it’s value from the operation name on the logical orchestration port which published the message to the message box. (That also means if you are not using orchestration and still want to use the Action-Mapping feature, you’ll probably need to write a custom pipeline component to populate this context property).

So if you are seeing the error mentioned in the title, what is possibly happening is this:-

Your FOO Adapter supports two operations with the corresponding actions being ‘ABC Action’ and ‘XYZ Action’. You generate the schemas and the binding file, create an orchestration (but forget to change the default operation names), tie the message types, deploy the project and import the binding file.

At runtime when you send a message, because you did not change the operation name on the logical port, the BTS.Operation property is set to ‘Operation_1’. The WCF-Custom adapter tries to look up this name in the map you specified on the send port (by importing the binding file). Since it does not find any operation by the name ‘Operation_1’, the WCF-Custom adapter believes that whatever you specified on the port is a single line action and passes the same to the FOO adapter.

So, instead of getting “ABC Action” or “XYZ Action”, which it understands, the FOO adapter gets “<BtsActionMapping><Operation Name =”ABC” Action = “ABC Action” \><Operation Name = “XYZ” Action = “XYZ Action” \></BtsActionMapping>”.

Clearly, it doesn’t understand that action!

Ok, so what do I do to fix this?

In the above example, you should change the operation names on the logical ports in the orchestration to ‘ABC’ and ‘XYZ’ instead of the default ‘Operation_1’, ‘Operation_2’ etc. (Of course, you can also do it the other way round – change the mapping on the port to use names in the orchestration or, if there is just one action to use, specify the action directly on the port instead of using the action map.)
Basically, you need to make sure that the WCF-Adapter is able to resolve the operation name on the message to use the correct action, as explained above.

Using WCF Adapters with SQL Server Integration Services(SSIS)

mmckeown — Wed, 09 Jan 2008 04:50:00 GMT

There are two ways the adapters belonging to Adapter Pack can be used by SQL Service Integration Services:

1. Using ADO layer built on top of base adapters. This option is available with SAP and Siebel adapters.

2. Writing custom SSIS task which directly invokes adapter to communicate with LOB system.

Option 1 covers all scenarios where data need to be moved from LOB system into SQL server. The reason being that ADO layer exposes only Query and Execute interfaces. Using these ADO interfaces, we can only pull out data from LOB system.

If usage scenario is such that data has to be loaded into LOB system, then option 1 doesn't help. But that is the not end of story. We can write custom SSIS task. This custom task essentially wraps the adapter as destination component. Once adapter is exposed as SSIS component, it is available for use in any SSIS scenario.

Wanted - Best Practices for Configuring and Using the new R2 WCF Adapters

mmckeown — Wed, 26 Sep 2007 16:16:00 GMT

I am looking to document "best practice" experiences in configuring and using the new R2 WCF adapters so others can learn from the mistakes/recommendations made by others. If you have anything to contribute I would greatly appreciate it. Either post recommendations here or email me directly at mmckeown@microsoft.com with "WCF Adapter Best Practices" in the heading. I am working to compile a list and publish it by end of October. Thank you!

Download New Windows Communication Foundation (WCF) Line-of-Business (LOB) Adapter SDK

mmckeown — Wed, 26 Sep 2007 16:14:00 GMT

The Microsoft© Windows Communication Foundation (WCF) Line-of-Business (LOB) Adapter SDK provides for a simplified programming model on top of the WCF Channel Model by adding metadata browsing, searching and retrieving functionality, and development tools for generating and testing adapter code. You can download it at http://www.microsoft.com/downloads/details.aspx?FamilyId=56278FDE-B708-469C-987E-DED9C6C5E580&displaylang=en.

How the WCF LOB Adapter SDK works with IChannelFactory and IChannelListener

mmckeown — Wed, 29 Aug 2007 14:19:00 GMT

Q. It seems WCF LOB adapters works different from normal WCF service as there is no WCF Service hosted. Do we only use channelfactory to create custom channel for send wcf message to LOB without channellisterner? I just wonder how WCF LOB adapter run time works behind scene.

WCF Service is hosted using a class called ServiceHost. The ServiceHost encapsulates the functionality regarding bindings, channels, dispatchers and listeners.

WCF LOB Adapter is surfaced to the consumer as a WCF Binding, where the binding is used to create the channel stack. This binding can be considered a peer to other predefined WCF bindings such as BasicHttpBinding, WsHttpBinding, NetTcpBinding, etc and can be set via app.config or in code by the client application when calling a service. This binding contains an ordered set of binding elements, with adapter being the key binding element that derives from TransportBindingElement class. In an outbound scenario, the WCF LOB Adapter SDK runtime uses channel factory to create the adapter (i.e. transport channel). In an inbound scenario, the WCF LOB Adapter SDK runtime uses channel listeners that listen for incoming channels in a service application.

When a .NET application set the endpoint information with the WCF LOB Adapter binding, WCF under-the-covers calls

BindingElement::BuildChannelFactory, if the application uses the adapter in client endpoint or
BindingEement::BuildChannelListener, if the application uses the adapter in service endpoint.

When an application uses the WCF LOB Adapter in an outbound scenario, the WCF LOB Adapter SDK runtime interprets the BindingElement::BuildChannelFactory to build a Channel Factory for a specific type of channel. When ChannelFactory::OnCreateChannel call is made by WCF, WCF LOB Adapter SDK runtime performs some of the following steps –

Checks the type of channel (IOutputChannel, IOutputSesionChannel, IRequestChannel, IRequestSessionChannel) and creates appropriate adapter send channel
Create connection pool
Create metadata cache

Then, on channel’s OnOpen() method, WCF LOB Adapter SDK runtime creates the connection (or uses the connection from pool), builds appropriate outbound handler that implements interface IOutboundHandler and/or IAsyncOutboundHandler and call the Execute/BeginExecute/EndExecute method for message exchange.

Similarly, when an application uses the WCF LOB Adapter in an inbound scenario, the WCF LOB Adapter SDK interprets the BindingElement::BuildChannelListner to accept channels of a specific type from the binding (i.e. the adapter). The adapter checks for the type of channel (IInputChannel, IInputSessionChannel, IReplyChannel, IReplySessionChannel) and creates appropriate adapter receive channel. Then on channel’s OnOpen() method, WCF LOB Adapter SDK retrieves the connection from the pool, builds appropriate inbound handler that implements IInboundHandler and/or IAsyncInboundHandler, and invoke the TryReceive()/WaitForMessage() methods to map the lob-specific method calls to WCF messages. The listening application/service then uses this channel to receive incoming messages.

Check the EchoAdapterSample's TestEchoAdapter_Channel (outbound) and TestEchoAdapter_Channel_Inbound (inbound) projects to see how the channel model is used directly to communicate with the adapter. Then see how the WCF Service Model sample consumer projects TestEchoAdapter_Proxy (outbound) and TestEchoAdapter_Service_Inbound (inbound) hide the channel related details from the end-user. The EchoAdapterSample is available from the WCF LOB Adapter SDK install directory under \Documents\Samples.

WCF LOB Adapter SDK Download

mmckeown — Wed, 29 Aug 2007 14:15:00 GMT

WCF LOB Adapter SDK is now generally available for download . . .

We are pleased to announce the release of WCF LOB Adapter SDK.

Here are some of the related links:

While the team is working on providing the official documentation, see some of the following posts to learn more about WCF LOB Adapter SDK.

The WCF LOB Adapter SDK ships with two samples - EchoAdapterSample and ContosoAdapterSample. The samples are installed as part of the setup in the {install_location}\WCF LOB Adapter SDK\Documents\Samples.

Download WCF LOB Adapter SDK

WCF LOB Adapter SDK and its relationship with WCF Channel Model

mmckeown — Mon, 09 Jul 2007 20:35:00 GMT

In this post, I will show how WCF LOB Adapter SDK relates to WCF Service Model and WCF Channel Model.

WCF Service Model provides an intuitive programming model to create service-oriented interfaces. There is plethora of information available on WCF (aka Indigo) now since its inception few years ago. Typically when you are reading about WCF, the most common usage is using classes/interfaces/attributes in System.ServiceModel namespace.

WCF Channel Model can be used for development of custom protocol and transport channels to communicate with existing systems that expose its functions through proprietary network API. The channels adapt the proprietary network API to WCF.

WCF LOB Adapter SDK utilizes the extensibility features of WCF Channel Model and surfaces an adapter to target line-of-business system as a WCF Binding.

Outbound Message Exchange

Inbound Message Exchange

You can write a custom channel using WCF LOB Adapter SDK and using WCF Channel Model. Here are some of the resources you can use when developing a custom channel (aka creating an adapter) using WCF Channel Model:

1) Channel Model Overview

2) Developing Channels

3) Nicholas Allen’s Blog (an excellent resource)

4) Building Custom Channels

So if you can create a WCF binding using WCF Channel Model, why WCF LOB Adapter SDK? Whatever you can do with WCF LOB Adapter SDK, it is definitely doable with WCF Channel Model.

WCF Channel Model fits in very well for creating transport adapters. Here is an excerpt from Building Custom Channels:

Custom channels are a powerful and versatile extensibility point. You'd write a custom protocol channel if you want to implement a wire protocol. Examples of wire protocols include the WS-ReliableMessaging protocol (WCF already implements that one for you as a channel) and a chunking protocol (see the chunking channel sample below). You'd write a custom transport if you want to send/receive WCF messages over a transport other than the ones we ship in the box (we ship TCP, HTTP, Named Pipes, MSMQ, and PeerChannel). For example, you could write a UDP transport (there's a sample of that in the WCF SDK) or an SMTP transport. Another reason to write a WCF transport is to integrate with an existing system that's not exposed as a Web service. In this case your transport acts as an adapter adapting WCF messages to the existing system's message format or API allowing a WCF client to talk directly to the existing system. An example of that is the WSE 3.0 TCP transport sample.

If you are writing a pure transport and/or protocol channel, Microsoft recommends utilizing WCF Channel Model. WCF LOB Adapter SDK is an application of WCF Channel Model and provides following additional features more suitable for systems that can be described using rich metadata.

1) Code Generation Wizard

It facilitates creation of WCF channels and bindings through Adapter Development Code Generation Wizard. The adapter developer only focuses on three key aspects of building an adapter – connectivity, metadata and message exchange. The channel concepts like message exchange patterns, channel factory, channel listener, binding element, binding, WSDL export/import and Policy export/import are not exposed to the adapter developer.

The adapter developer focuses on mapping of the message from/to their line-of-business messages in inbound and outbound handlers.

2) Metadata Explorer

It provides an ability to generate WSDL / WCF Contract dynamically at design-time. Let’s say you are creating a database adapter with each stored procedure exposed as an operation. The number of stored procedures in the database can vary from implementation to implementation and the parameters/return values can change. Such functionality shouldn’t be captured in a static contract. WCF LOB Adapter SDK promotes inspecting the live line-of-business system for its metadata and allows generation of contract from this live metadata. Even though you can provide metadata support for a WCF binding element in WCF Channel Model using IWsdlExportExtension.ExportContract, WCF LOB Adapter SDK provides the API to enable browsing and searching of metadata. And … when the message flows over the channel during run-time, the code has knowledge of metadata of the message (via SOAP action) and can use it during construction, transformation and verification.

The adapter developer can provide the metadata exploring and export functionality through metadata browse handler, metadata search handler and metadata resolver handler.

3) Connection Pooling

Typically when a line-of-business adapter is created for a database, messaging system and/or packaged application, there is a concept of a connection. WCF LOB Adapter SDK provides an efficient way of using these connections through a pool of connections. Using the WCF Channel Model, the adapter developer can provide own implementation of connection pooling.

The adapter developer can tweak the adapter settings for connection pool and connection pool manager.

4) Adapter Data Reader / Adapter Data Writer

Rather than having to use XmlDictionaryReader / XmlDictionaryWriter to walk through the WCF Message, since the adapter has the knowledge of the metadata of the message, the adapter operation and type readers/writers can get callbacks (e.g. OnAfterOperationParameterWrite) in which the adapter developer can then do line-of-business system specific mapping and transformation.

The adapter developer can use these readers and writers within the message exchange handlers.

Here is a high level synopsis:

Feature	WCF LOB Adapter SDK	WCF Channel Model
Support for one-way message exchange patterns (IOutputChannel / IInputChannel)	Yes	Yes
Support for two-way message exchange patterns (IRequestChannel / IReplyChannel)	Yes	Yes
Support for two-way duplex communications (IDuplexChannel)	No	Yes
Support for sessionful channels	No	Yes
Design-time tools to generate WCF contract	Yes	Yes
Design-time tools to browse and search metadata	Yes	No
Connection pooling	Yes	N/A*
Metadata object model	Yes	N/A*
In-memory metadata cache	Yes	N/A*
Adapter XML data reader and writer	Yes	N/A*

* If the adapter developer is using WCF Channel Model to create the line-of-business system adapter, then the N/A items can be supplemented with developer's own implementation.

Enterprise Application Integration is the secure and orchestrated sharing of processes and/or data between applications which weren’t originally designed for such sharing. To enable such applications to participate in integration processes one has typically had to write an integration adapter or write a web service wrapper around the existing line-of-business system APIs to enable them to be “normalized” for consumption by integration brokers such as BizTalk Server. Adapters traditionally have achieved the normalization by reducing the target systems to appear like “messaging systems” and capture the LOB specific aspects in the actual message that is transferred to the backend systems. While this approach is simple it is practically unusable in regular applications where typically the levels of abstraction are much higher and semantically richer than just exchanging messages. Web services on the other hand offer a much more “human friendly” programming model and the appeal of wide reach and support for several popular WS-* standard protocols. But web services today require a static monolithic service contract declared a priori describing a strongly typed service interface that is the public face of the service.

For LOB systems which have an enormous amount of operations that are constantly changing and evolving it is impractical for such a contract to capture the capabilities of the system with 100% fidelity. One of the more common ways of exposing such a dynamic system to EAI systems is to have a generic InvokeWebMethod() operation exposed as the extensibility mechanism to deal with any new or changing metadata. Here is an example of one such implementation. So the client of the system will invoke the newer methods by passing in the appropriate method name, parameters etc packaged in generic array data structures to be “interpreted” by the InvokeWebMethod() function and turn around and make the actual call. Although this provides the “dynamic” functionality in being able to call any method, it expects the client to be fully aware of the details of the new methods on the LOB system. This is a fundamental violation of the SOA tenets where the boundaries between client and service are expected to be strictly limited to what is exposed in the service contract and nothing more. To expect the client to be deeply aware of the internals of the LOB system leads to tight coupling between service and client and this is total anathema in the SOA world.

What we need in this situation is an intermediary sitting in between the LOB system and the consuming client that lets the client “carve out and construct” a strongly typed service contract dynamically from the live metadata in the LOB system. The WCF LOB Adapter SDK enables the development of adapters which act as an “uber service” that can incarnate any service from the available backend metadata. This allows the clients to keep up with the changing functionality in the LOB system at design-time without being intimately aware of these changes. If the target system changes, the client can easily regenerate updated proxy and tremendously increase the develop-test-deploy web service cycle. This approach also enables a more consumer- friendly experience for exposure of LOB systems with large and unwieldy metadata although the metadata may remain more or less static.

Additionally, adapters written to the WCF LOB Adapter SDK eventually manifest themselves as WCF bindings which can be invoked via WCF Service Programming Model and WCF Channel Programming Model (via the client proxy generated from one such composed contract) thereby combining the advantages of both traditional adapters as well as web services.

Summary

WCF Service Model is the next-generation distributed application technology for developing and consuming services in Windows. The client and service communicate with each other using endpoints.

WCF Channel Model provides extensibility by enabling creation of custom protocol and transport channels for a binding in an endpoint.

WCF LOB Adapter SDK is a utility of WCF Channel Model to surface target line-of-business systems as WCF Bindings to the WCF clients. This means a client can access an existing line-of-business system the same way as they would any other WCF Service. The client is abstracted from having to learn the proprietary access mechanism of the target line-of-business system.

WCF LOB Adapter Usage Patterns

mmckeown — Mon, 09 Jul 2007 20:19:00 GMT

An adapter built using WCF LOB Adapter SDK can be used in variety of topologies and scenarios. Follow your organization’s n-tier architecture principles and design patterns to determine where the adapter can fit best for your requirements.

Usage Pattern 1 – Consume adapter in a .NET managed application (Point-to-Point)

Design Time: In Visual Studio 2005, use in project templates such as Windows Application, Class Library and Console Application.

1.1 Outbound

At design-time, use Add Adapter Service Reference Visual Studio Plug-In to generate the WCF Client Proxy for use in the .NET application.

1.2 Inbound

At design-time, use Add Adapter Service Reference Visual Studio Plug-In to generate the WCF Service Interface (callbacks) for use in .NET application.

Usage Pattern 2 – Consume adapter in BizTalk Server using BizTalk WCF Adapter (Broker, Bus)

Design Time: In Visual Studio 2005, use in project templates such as BizTalk Server Project. Please note that this scenario is only applicable to BizTalk Server 2006 R2, since versions prior to R2 do not have support for WCF.

2.1 Outbound

At design-time, use Consume Adapter Service BizTalk Project Add-In to generate the XML Schema for input and output messages. The Consume Adapter Service BizTalk Project Add-In tool also generates a BizTalk Port Binding Configuration XML File that can be used to configure physical send port(s) at deployment time.

2.2 Inbound

Usage Pattern 3 – Consume adapter in a Windows Workflow Foundation (WF)

Windows Workflow Foundation (WF) provides a general framework for defining workflows and a robust runtime engine that allows you to host and interact with the running workflows. WCF provides a general framework for building connected systems and defining communication between two endpoints. With WCF LOB Adapter, you can use the consistent WCF API to access the line-of-business systems in the business processes within your environment. WF allows you to model and encapsulate your business logic and process, while WCF provides you with messaging infrastructure that gives you ability to distribute your systems. WF is a lightweight component (compared to BizTalk Orchestration) that runs within your application's or component's process.

Note to adapter developers: Workflows will execute asynchronously to the service call, so design appropriately when thinking about returning data from the adapter.

Usage Pattern 4 – Consume adapter in a Web Application

Design Time: In Visual Studio 2005 SP1, use in project templates such as ASP.NET Web Application and ASP.NET Web Site.

According to enterprise architecture guidelines and solution patterns, accessing data from target systems is usually in a separate data access layer. Add Adapter Service Reference Visual Studio Plug-In tool can be used to generate a Common Lanuage Runtime (CLR) WCF Client Proxy to access data from line-of-business target systems.

Usage Pattern 5 – Consume adapter in a Web Service

Design Time: In Visual Studio 2005 SP1, use in project templates such as ASP.NET Web Service, WCF Service and WCF Service Library

In the standard Web Service development cycle, start with either top-down, bottom-up or combination of web service design and use the WCF LOB Adapter to access data from a line-of-business target system. This is especially useful if you are building a Composite Web Service or a workflow using WF that will be deployed as service for integration with other client applications.

If you are -

Building a Web Service (using top-down approach) or

Hosting a workflow as a WCF Service or

Publishing a BizTalk Orchestration as a Web Service

AND there is a need to access data from a line-of-business system within these services, you still need some access mechanism to talk these target system. WCF LOB Adapter can be used to access the data from line-of-business system.

Usage Pattern 6 – Expose the proxy generated from adapter as a Web (WCF) Service hosted in IIS

The WCF LOB Adapter runs 'in-proc' in a client process. If there is a need to access the proxy over the internet using SOAP, the proxy class can be surfaced as a WCF service and hosted in a Web Server.

To accomplish this, you need to create a WCF Service Web Site project and add -

1. .Svc File: This should point to the service implementation class. The "proxy" generated from the adapter contains a service interface and classe that implements this interface. This class can be used as the "service implementation" CLR class in .svc. This file functions very similar is very similar to an .asmx file in classic ASMX web service. It gets placed in the virtual diractory under IIS and is used to identify the code behind the service and the class.

2. Web.Config: This will contain the service endpoint information, where the address is where this service will be hosted, binding is one of the HTTP bindings, contract is the interface generated from the adapter and the service implementation points to the class that implements the interface. The client endpoint information is still needed to communicate with the line-of-business. For the client endpoint, the address contains the location of the line-of-business system, binding is the adapter binding and contract is the interface generated from the adapter. The client endpoint is generated by Add Adapter Service Reference Visual Studio Plug-In.

3. Service Interface and CLR Class: Add Adapter Service Reference Visual Studio Plug-In generates these after the adapter consumers has browsed/searched the metadata hierarchy and selected a few methods of choice.

In current release, it takes a few manual steps to configure, host and publish a line-of-business system proxy generated from the adapter as a WCF Service. Also, special consideration is required for passing the line-of-business Client Credentials to the adapter (can be done using WCF behaviors). A future release of WCF LOB Adapter SDK will include tooling to automate these steps.

Usage Pattern 7 – Consume adapter in Microsoft Office SharePoint Server 2007

One of the major ways to connect to the data and existing applications from Office Applications is through the Business Data Catalog (BDC), a new integration feature in Microsoft Office SharePoint Server (MOSS) 2007. It is shared service and it surfaces business data from back-end data and applications without any coding. BDC is a metadata repository that defines business entities like customers, invoices and purchase orders. Once defined, these entities can be used by the clients. The BDC provides built-in support for displaying data from line-of-business (LOB) applications via databases and Web Services.

Until BDC provides native support for WCF (it is scheduled for a later release), the workaround is to follow the Usage Pattern 6 (substitute Basic Profile Intenet Client with BDC) to expose the data from WCF LOB Adapter as a Web Service. BDC can then integrate with this Web Service. The extra data (de) serialization can be avoided if the WCF LOB Adapter binding can be hosted 'in-proc' with BDC.

Tale of two adapters -- BizTalk WCF Adapter and WCF LOB Adapter

mmckeown — Tue, 08 May 2007 18:44:00 GMT

The Microsoft BizTalk Server 2006 R2 will ship with a BizTalk WCF Adapter. The BizTalk WCF Adapter is a bridge between BizTalk and WCF.

It supports features such as:

Feature		Description
Consume metadata exposed by WCF services		At design-time, in a BizTalk Project, right-click, select Add > Add Generated Items > Consume WCF Service, click on Add to invoke BizTalk WCF Service Consuming Wizard. This wizard will import schemas and BizTalk port binding info XML files into the BizTalk projects. The port binding info XML files can be used to create properly configured send ports for communicating with WCF services.
Send messages to WCF services		BizTalk WCF Adapter includes a WCF transmitter for sending messages to the WCF services. The BizTalk Send Port needs to be configured with WCF client endpoint information (address, binding and contract). In BizTalk Administration Console, select one of the following available transport adapters to configure access to WCF services. - WCF-BasicHttp - WCF-NetNamedPipe - WCF-NetTcp - WCF-WSHttp - WCF-NetMsmq (one-way only) - WCF-Custom
Receive messages from WCF clients		BizTalk WCF Adapter includes a WCF receiver for receiving messages from WCF clients. The BizTalk Receive Location needs to be configured with WCF service endpoint information (address, binding and contract). In BizTalk Administration Console, select one of the following available transport adapters for enabling receiving messages from WCF clients. - WCF-BasicHttp - WCF-NetNamedPipe - WCF-NetTcp - WCF-WSHttp - WCF-NetMsmq (one-way only) - WCF-Custom - WCF-CustomIsolated
Publish BizTalk orchestrations and schemas as WCF services		From Visual Studio, click on Tools from main menu and select BizTalk WCF Service Publishing Wizard. This wizard is also available along with other tools in Start > All Programs > Microsoft BizTalk Server 2006. This feature also allows hosting a non-HTTP service within BTSNtSvc.exe by simply configuring a Receive Location and enabling it within the BizTalk Administration Console (no publishing necessary).

BizTalk WCF Adapter Architecture and WCF LOB Adapter SDK

BizTalk WCF Adapter provides full support for the WS-* stack including the various security, reliable messaging and transaction specifications. The following diagram shows the high level architecture of BizTalk WCF Adapter.

Under the covers, the adapter built using the Adapter SDK i.e. the WCF LOB Adapter, is a transport channel, enriched with the metadata expressed by the adapter. The WCF LOB Adapter is surfaced as a WCF binding. In BizTalk, this adapter is consumed via the BizTalk WCF Adapter.

Here are some key points -

To consume metadata exposed by the WCF LOB Adapter in a BizTalk Project, use the Consume Adapter Service BizTalk Project Add-In. This provides similar functionality to BizTalk Consume WCF Service with an added value of exploring and filtering the adapter metadata. The end-result is generation of XML Schemas and port binding info XML files.
To send messages to the WCF LOB adapter from BizTalk Server, configure Send Port using WCF-Custom Transport Type and select the adapter binding.
To receive inbound messages from the LOB (target system), configure Receive Location using WCF-Custom Transport Type and select the adapter binding

The following diagram shows where the WCF-based adapter fits with the BizTalk WCF Adapter Architecture.

Consume WCF Service Wizard generates port binding configuration files for import into BizTalk Server

mmckeown — Tue, 17 Apr 2007 15:42:00 GMT

Scenario: You have already designed, implemented and hosted a WCF Service. Now, you would like to send (and get) messages to this WCF Service in a BizTalk Orchestration process. BizTalk communicates with WCF applications via BizTalk WCF Adapter, which can be configured through send and receive ports in BizTalk. This post talks about how BizTalk’s Consume WCF Wizard can be used to simplify the port creation process in BizTalk, especially the need of mapping the operation(s) with the SOAP Action(s) within the port to invoke the operation in WCF Service.

Microsoft BizTalk Server 2006 R2 now includes a BizTalk WCF Adapter to provide support for WS-* and transport neutrality by letting the user choose most appropriate Windows Communication Foundation (WCF) binding in BizTalk Port configuration. The BizTalk WCF Adapter can be selected within a Send Port by creating a new port and then selecting a Transport Type. In Receive Port, the BizTalk WCF Adapter can be selected by creating a new Receive Location and then selecting Transport Type from the drop-down box. The Receive Location allows for hosting a non-HTTP WCF Service within BTSNtSvc.exe and can be enabled within the BizTalk Administration Console without need for publishing.

Even though WCF-Custom is the only adapter needed, BizTalk allows other WCF adapters for most common messaging scenarios, as shown in the table below. For example, WCF-Custom can be used as WCF-WSHttp transport by selecting WCF-Custom, clicking on Configure, select Binding tab and selecting the Binding Type wsHttpBinding or customize this transport by using wsDualHttpBinding or wsFederationHttpBinding.

Available Send Port

WCF Adapters

Available Receive Port

WCF Adapters

One-way

WCF-BasicHttp

WCF-Custom

WCF-NetMsmq

WCF-NetNamedPipe

WCF-NetTcp

WCF-WSHttp

WCF-BasicHttp

WCF-Custom

WCF-CustomIsolated

WCF-NetMsmq

WCF-NetNamedPipe

WCF-NetTcp

WCF-WSHttp

Two-way

WCF-BasicHttp

WCF-Custom

WCF-NetNamedPipe

WCF-NetTcp

WCF-WSHttp

WCF-BasicHttp

WCF-Custom

WCF-CustomIsolated

WCF-NetNamedPipe

WCF-NetTcp

WCF-WSHttp

Note: Adapters built using WCF LOB Adapter SDK are consumed within BizTalk using WCF-Custom BizTalk WCF Adapter. Once the adapter is configured, it shows up in the Binding Type drop-down box and can be selected like any other predefined WCF binding.

The following section shows, at a very high level, how a BizTalk WCF Adapter is used within a typical BizTalk programming lifecycle.

· Design-Time Experience

o Create a Visual Studio BizTalk Project

o Use Consume WCF Service Wizard (Right Click > Add > Add Generated Items > Consume WCF Service) to download metadata from a running service using MEX endpoint or import metadata using WSDL.

§ Following files are generated as an outcome of running this wizard.

Metadata

XML Schema Files

Orchestration

BizTalk Orchestration (.odx) file

Port Binding Information Configuration

{ServiceName}_Custom.BindingInfo.xml – Contains port configuration using WCF Custom Adapter

{ServiceName}.BindingInfo.xml – Contains port configuration using other WCF bindings as obtained from the MEX endpoint

These configuration files can be imported in

§ MEX endpoints only with these transport types are supported

o http, https, net.pipe, net.tcp

o Implies users CANNOT use this wizard to import metadata from MEX enabled services that use a custom transport channel. WCF users can publish and retrieve metadata over a custom binding, which may include transports (and schemes) other than the ones defined above.

§ Port Binding Information Configuration Contains operation to action mapping strings

· <BtsActionMapping> <Operation Name=”Operation1” Action=”Action1”/> </BtsActionMapping>

o Build the BizTalk Project

· Deployment-Time Experience

o Deploy the BizTalk Project

o Import the Port Binding Information XML file to auto-generate the ports in BizTalk Server 2006 Administration Console

§ Right click on Applications within BizTalk Group, select Import > Bindings and then navigate to the Visual Studio BizTalk project folder

§ Select the Port Binding Information XML file

o Enlist and start the orchestration (and all its dependencies)

· Execution-Time Experience

o Trigger the orchestration depending on your scenario – e.g. dropping a file in the file drop for receive location to pick it up

See the attached document for screen shots of Consume WCF Service Wizard.

WCF Adapter Content Forthcoming!

mmckeown — Tue, 27 Mar 2007 14:47:00 GMT

Thanks to my co-worker and good friend PM Sonu Arora, I will begin posting content on the WCF LOB Adapter SDK on this site. For more information on this SDK and very useful programming information, add a link to her blog at http://blogs.msdn.com/sonuarora/

The Windows Communication Foundation (WCF) Line-of-Business (LOB) Adapter SDK is a collection of runtime engine and tools to help Adapter Developers in creating service-oriented interfaces to existing LOB applications using WCF. The goal of the WCF LOB Adapter SDK is to facilitate uniform development of reusable metadata-oriented WCF-based adapters that enable enterprise applications, databases and messaging platforms to integrate with each other. The Adapter SDK is based on WCF and it surfaces an adapter to a LOB application as a WCF Binding.

Relationship with BizTalk Server

BizTalk Server ships with BizTalk Adapter Framework which promotes creating custom adapters for use within BizTalk Server. BizTalk adapters use the BizTalk Server Administration Console for adapter management, BizTalk Explorer for adapter configuration, and the Adapter Framework for design-time APIs. WCF LOB Adapter SDK is the evolution of BizTalk Adapter Framework. Since WCF LOB Adapter SDK is based on new architecture, the adapters built to Adapter SDK will continue to co-exist side-by-side with the adapter written using BizTalk Adapter Framework. Microsoft and Microsoft’s Technology Partners have built adapters using BizTalk Adapter Framework. The future direction recommended by Microsoft is to start using WCF LOB Adapter SDK for building new adapters.

Relationship with WCF

The Adapter SDK is based on WCF. It is exposed as a WCF Binding and implements WCF System.ServiceModel.Channels.TransportBindingElement for handling communication between a client and a service. It extends WCF with metadata browse, search and resolve interfaces for Adapter Consumer to selectively compose the Service Description contract. The Adapter SDK enables Adapter Developers to easily create WCF binding(s) and channel(s) for layering existing applications with service-oriented interfaces.

Adapters differ from regular WCF transports due to these main characteristics:

• Adapters are metadata-centric

o Require metadata at run-time

o Require metadata cache management

o To provide rich metadata at design-time they require search/browse/resolve protocol

• Adapters are always connection-oriented

o Connection is very central concept for the adapter

o Require connection pooling and connection life-cycle management

• Adapters are effectively “in-proc WCF Services”

Consuming an adapter is no different than consuming a WCF Service from the perspective of a WCF client and the adapter consumer is not required to learn a new programming model. The client can use either the WCF Service Programming Model or WCF Channel Architecture to communicate with an adapter.

A typical WCF service programming lifecycle begins by a Service Provider defining a static service contract, implementing this contract and hosting the service. Each service is defined entirely by a fixed monolithic contract. This contract needs to be updated by the service provider in order to reflect any changes in the existing application. The Adapter built using WCF LOB Adapter SDK, on the other hand, surfaces the contracts on-demand by consumer. The consumer browses through the available metadata in the adapter using the standard interfaces and then selects the operations and types for inclusion in a dynamic contract. The metadata can be updated when the existing application change providing up-to-date interfaces for the new clients. Adapters are more appropriate in scenarios where a non-WCF based system is being exposed for integration.

For an Adapter Consumer, the adapter can be accessed like a typical WCF Service and the consumer doesn’t have to learn a new programming model. The same adapter developed using the Adapter SDK can be reused in multiple .NET applications including Custom WCF Client Applications, BizTalk Server, SharePoint Server and SQL Server Integration Services. In addition, Adapter SDK provides a common facility for Adapter Developers to expose rich LOB metadata to the Adapter Consumers, who can selectively compose and create dynamic WCF contracts from this available metadata in the adapter.

Goals

Here are three main goals of the WCF LOB Adapter SDK

1. Uniformity

No unified adapter development framework for .NET
Every “integration technology” implements its own adapter framework
Provides an abstraction layer between service oriented world and proprietary application interfaces

2. Reusability

Adapters written using BizTalk adapter framework specific to BizTalk
Duplication of effort – e.g. at least 5 SAP adapters in Microsoft
Surfaces adapter as a WCF binding, widening reach

3. Consumer-Driven Dynamic Contracts

Provides interfaces for browsing, searching and resolving metadata
Provides a service-tier that reflects changes in the existing application type model
Extends WCF to provide an ability to generate dynamic contracts (service description) from available metadata

Three LOB Adapter Webcasts

mmckeown — Tue, 27 Mar 2007 14:36:00 GMT

Host Integration Server 2006 Data Providers DB2 (Level 200) - Ricardo Mendes, Accessing IBM DB2 from BizTalk Server
Host Integration Server 2006 Data Providers for Host Files (Level 200) - Ricardo Mendes Microsoft Visual Studio 2005 and Microsoft Host Integration Server 2006 (including the new Microsoft .NET Framework Data Provider for Host Files) technologies that allow enterprise developers to quickly and efficiently build new .NET Framework applications
Integrating Sales Data with Your LOB Applications Using the Salesforce.com Adapter for BizTalk Server Features and functionality of the Salesforce.com Adapter for Microsoft BizTalk Server 2006 from Two Connect

Your opinion matters!

mmckeown — Thu, 21 Dec 2006 17:05:00 GMT

The BizTalk Server Documentation Team is looking for customer input on the documentation of the operations topics. Specifically they said they "want to know what is missing, incorrect, unclear, or just needs work in any of the Operations topics". Go here for the survey:

http://www.surveymonkey.com/s.asp?u=32373055498

Be as explicit and clear as you possibly can. Don't miss this opportunity to help improve the documentation! Thank you!