Paolo Salvatori's Blog

Customizing and Extending the BizTalk WCF Adapters

leprino — Tue, 17 Nov 2009 11:15:37 GMT

The contents of the following post are taken from a presentation I created and delivered at TechReady 9 in July 2009 and replicated with Stephen Kaufman at Tech-Ed Europe 2009 .

The ABC of WCF

Windows Communication Foundation (WCF) is a runtime and a set of APIs for exchanging messages between components and applications. WCF was designed according to the tenets of service orientation and unified the existing communication technologies like .NET Remoting and ASMX web service into a single programming model and consistent architecture which provides high levels of functionality, interoperability and extensibility. As you know, a typical WCF service can expose one or multiple endpoints. In general, endpoints provide clients access to the functionality offered by a WCF service. Each endpoint consists of 3 properties as depicted in the picture below:

An address that indicates where the endpoint can be found.
A binding that specifies how a client app can communicate with the endpoint.
A contract that identifies the operations exposed by the endpoint.

WCF Architecture: Messaging Runtime

The WCF runtime is divided into 2 primary layers as shown by the following picture:
The Service Layer aka Service Model defines the mechanisms and attributes used by developers to define and decorate service, message and data contracts.
The Messaging Layer is responsible for preparing a WCF message for transmission on the send side and produce a WCF message for the dispatcher on the receive side. The messaging layer accomplishes this task using a Channel Stack. This latter is a pipeline of channel components that handle different processing tasks. Each channel stack is composed of exactly one transport channel, one message encoder, and zero or more protocol channels.

It’s the responsibility of the proxy component on the client side and dispatcher component on the service side to mediate and translate between the two layers. In particular, the proxy component transforms .NET method calls into Message objects, whereas the dispatcher component turns WCF Messages into .NET method calls. WCF uses the Message class to model all incoming/outgoing messages within the Messaging Layer. The message represents a a SOAP Envelope, and therefore it’s composed of a payload and a set of headers. A typical WCF communication can be described as follows:

The client application creates one or more input parameters. Each of these parameters is defined by a data contract.
The client application invokes one of the methods of the service contract exposed by the proxy.
The proxy delivers a WCF Message object to the channel stack.
At this point each protocol channel has a chance to operate on the message before the transport channel uses a message encoder to transmit the final Message as a sequence of bytes to the target service. Each protocol channel can modify the content or the headers of the message to implement specific functionalities or WS-* protocols like WS-AtomicTransaction, WS-Security.
The raw stream of data is transmitted over the wire.
On the service side, the transport channel receives the stream of data and uses a message encoder to interpret the bytes and to produce a WCF Message object that can continue up the channel stack. At this point each protocol channel has a chance to work on the message.
The final Message is passed to the Dispatcher.
The Dispatcher receives the WCF Message from the underlying channel stack, individuates the target service endpoint using the destination address and Action property contained in the Message, deserializes the content of the WCF Message into objects.
Finally the target service method is invoked.

WCF Binding Comparison

Bindings provide a mechanism for configuring channel stacks. A binding defines a precise recipe for building a channel stack using a transport channel, a message encoder, and a set of protocol channels.
WCF ships with several built-in bindings that target common communication scenarios.

Binding Class Name	Transport	Message Encoding	Message Version	Security Mode	Reliable Messaging	Tx Flow*
BasicHttpBinding	HTTP	Text	SOAP 1.1	None	X	X
WSHttpBinding	HTTP	Text	SOAP 1.2 WS-A 1.0	Message	Disabled	WS-AT
NetTcpBinding	TCP	Binary	SOAP 1.2	Transport	Disabled	OleTx
NetNamedPipesBinding	Named Pipes	Binary	SOAP 1.2	Transport	X	OleTx
NetMsmqBinding	MSMQ	Binary	SOAP 1.2	Message	X	X
CustomBinding	You decide	You decide	You decide	You decide	You decide	You decide

You should select the most appropriate binding based on your needs:

For example, BasicHttpBinding is designed for interoperability with simple web services that conform to the SOAP 1.1 message version.
Instead the WSHttpBinding is designed for interoperability with more advanced web services that might leverage different WS-* protocols. Both of these bindings use HTTP for the transport and the text message encoding because they are designed for maximum interoperability.
NetTcpBinding and NetNamedPipeBinding, on the other hand, are designed for efficient and performant communication with other WCF applications across machines or on the same machine respectively.
If you need maximum flexibility and you need to use one or multiple custom protocol channels at runtime, you can use the CustomBinding that gives you the possibility to decide which binding elements compose your binding.

Bindings have different characteristics in terms of response time and throughput, so the general advice to increase performance is using the NetTcpBindind and NetNamesPipeBinding whenever possible.

BizTalkServiceInstance Class

Each WCF Receive Location in BizTalk Server is indeed an instance of a WCF Service Class called BizTalkServiceInstance hosted by a separate instance of a ServiceHost-derived class

the BtsServiceHost for RLs running in an in-process host
the WebServiceHost for RLs running in an isolated host

When you “enable” a WCF receive location, the adapter initializes and opens the ServiceHost, which dynamically builds the WCF runtime components within the BizTalk service process (BtsNtSvc.exe for in in-process host, w3wp.exe for an isolated host). This includes the channel stack, dispatcher, and the singleton service instance. WCF Receive and Send Ports are message-type agnostic or if you prefer untyped. This design comes in handy when you need to configure a single Receive Port to accept numerous message types or versions that you can normalize (via BizTalk maps or using a custom pipeline component) into a common message type before being posted the BizTalkMsgBoxDb. However, this design also implies that the WCF adapters will need to build on generic service contracts in order to remain message-type agnostic within the WCF implementation.

WCF Receive Locations have the responsibility to receive the incoming message bytes, perform any necessary SOAP and WS-* processing, and publish the message (or some part of it) to the message box. The WCF Adapters create a separate ServiceHost and singleton service object of this class for each receive location to handle client requests for the lifetime of the BizTalk Host instance running WCF receive locations. The service object uses multiple threads to process messages concurrently unless the WCF-NetMsmq receive locations are used with the Ordered processing property being selected.. As shown in the picture below, the class is decorated with the ServiceBehavior attribute:

[ServiceBehavior(InstanceContextMode = InstanceContextMode.Single, ConcurrencyMode = ConcurrencyMode.Multiple)]
internal sealed class BizTalkServiceInstance : ITwoWayAsync, 
                                               ITwoWayAsyncVoid, 
                                               IOneWayAsync, 
                                               IOneWayAsyncTxn, 
                                               ITwoWayAsyncVoidTxn
{
    ...
}

In particular, you can note that:

InstanceContextMode = InstanceContextMode.Single
ConcurrencyMode = ConcurrencyMode.Multiple

Hence, all incoming messages to a WCF Receive Location are received and processed by a single well-known instance of the BizTalkServiceInstance class. This allows to avoid service activation/deactivation costs and improve performance/scalability.

The BizTalkServiceInstance class implements multiple untyped, generic service contracts.

Each contract was designed for a different scope (as suggested by their name):

OneWay vs TwoWay Message Exchange Pattern
Transactional vs Non-Transactional Communication

[ServiceContract(Namespace = "http://www.microsoft.com/biztalk/2006/r2/wcf-adapter")]
public interface IOneWayAsync
{
    // Methods
    [OperationContract(AsyncPattern = true, IsOneWay = true, Action = "*")]
    IAsyncResult BeginOneWayMethod(Message message, AsyncCallback callback, object state);
    [OperationContract(IsOneWay = true, Action = "BizTalkSubmit")]
    void BizTalkSubmit(Message message);
    void EndOneWayMethod(IAsyncResult result);
}

[ServiceContract(Namespace = "http://www.microsoft.com/biztalk/2006/r2/wcf-adapter")]
public interface IOneWayAsyncTxn
{
    // Methods
    [OperationContract(AsyncPattern = true, IsOneWay = true, Action = "*")]
    IAsyncResult BeginOneWayMethod(Message message, AsyncCallback callback, object state);
    [OperationContract(IsOneWay = true, Action = "BizTalkSubmit")]
    void BizTalkSubmit(Message message);
    void EndOneWayMethod(IAsyncResult result);
}

[ServiceContract(Namespace = "http://www.microsoft.com/biztalk/2006/r2/wcf-adapter")]
public interface ITwoWayAsync
{
    // Methods
    [OperationContract(AsyncPattern = true, IsOneWay = false, Action = "*", ReplyAction = "*")]
    IAsyncResult BeginTwoWayMethod(Message message, AsyncCallback callback, object state);
    [OperationContract(IsOneWay = false, Action = "BizTalkSubmit")]
    Message BizTalkSubmit(Message message);
    Message EndTwoWayMethod(IAsyncResult result);
}

[ServiceContract(Namespace = "http://www.microsoft.com/biztalk/2006/r2/wcf-adapter")]
public interface ITwoWayAsyncVoid
{
    // Methods
    [OperationContract(AsyncPattern = true, IsOneWay = false, Action = "*", ReplyAction = "*")]
    IAsyncResult BeginTwoWayMethod(Message message, AsyncCallback callback, object state);
    [OperationContract(IsOneWay = false, Action = "BizTalkSubmit")]
    void BizTalkSubmit(Message message);
    void EndTwoWayMethod(IAsyncResult result);
}

[ServiceContract(Namespace = "http://www.microsoft.com/biztalk/2006/r2/wcf-adapter")]
public interface ITwoWayAsyncVoidTxn
{
    // Methods
    [TransactionFlow(TransactionFlowOption.Mandatory)]
    [OperationContract(AsyncPattern = true, IsOneWay = false, Action = "*", ReplyAction = "*")]
    IAsyncResult BeginTwoWayMethod(Message message, AsyncCallback callback, object state);
    [TransactionFlow(TransactionFlowOption.Mandatory), OperationContract(IsOneWay = false, Action = "BizTalkSubmit")]
    void BizTalkSubmit(Message message);
    void EndTwoWayMethod(IAsyncResult result);
}

All the methods exposed by these service contracts are generic, asynchronous and untyped. In fact, as shown in the picture above, each of this method is decorated by the OperationContract attribute and in particular:

AsyncPattern = True indicates that an operation is implemented asynchronously using a Begin<methodName> and End<methodName> method pair in a service contractAction = “*” means that the method accepts a message with any Action.
ReplyAction = “*” means that the method can return a message with any Action.
Every method accepts as parameter or returns a generic WCF Message.

As a consequence, each WCF Receive Location can accept multiple message types and versions that can be normalized into a canonical format using a different map before being published to the MessageBox.
Also Send Ports are message-type agnostic.

One-Way WCF Receive Locations

When you define a One-Way WCF Receive Location that uses the NetMsmqBinding, the underlying WCF service will expose an endpoint which uses the IOneWayAsync service interface. If instead the One-Way WCF Receive Location is not configured to use NetMsmqBinding, the underlying WCF service will expose an endpoint using the ITwoWayAsyncVoid contract interface that returns an acknowledgment message. Otherwise, when you define a two-way WCF receive location, the underlying WCF service will expose an endpoint using the ITwoWayASync service interface.

As you can note in the picture above, the IOneWayAsync service contract has a single logical operation named OneWayMethod that matches all incoming messages (Action=”*”). It’s implemented with the WCF asynchronous programming model that ties two methods – BeginOneWayMethod and EndOneWayMethod – together via AsyncPattern=true.
ITwoWayAsyncVoid is similar in design except that it also returns a WCF Message object (notice the addition of ReplyAction=”*”) and the operation is not marked as one-way. The only difference with ITwoWayAsync is that it also returns a Message object in order to properly model two-way ports. The following picture depicts what happens when a message is received and processed through a One-way WCF Receive location.

The Raw Message Data are sent over the wire.
The Transport Channel receives and decodes the incoming stream of bytes and creates a WCF Message that is processed through the Channel Stack.
The WCF message is passed on to the Dispatcher
The WCF message is passed on to the BizTalkServiceInstance.
Based on the RL configuration, the entire SOAP Envelope, the Body of the SOAP message or a specific Xml Element is used as content of the BizTalk message.
The BizTalk message is processed through the pipeline.
A Map is eventually applied.
The BizTalk message is published to the MessageBox.

WCF Adapters

The WCF Adapters provided by BizTalk correspond 1:1 to the most commonly used WCF bindings.

Adapter Name	WCF Binding Name	When to use?
WCF-BasicHttp	BasicHttpBinding	When you need interoperability with WS-I Basic Profile 1.1 services, such as those created with ASP.NET Web services (ASMX) or other first-generation service frameworks
WCF-WSHttp	WSHttpBinding	When you need interoperability with more advanced services that leverage WS-* protocols, such as those created with WCF or other modern service frameworks
WCF-NetTcp	NetTcpBinding	When you need efficient inter-machine communication with other WCF applications
WCF-NetNamedPipe	NetNamedPipeBinding	When you need efficient intra-machine communication with other WCF applications
WCF-NetMsmq	NetMsmqBinding	When you need durable, asynchronous communication with other WCF applications (using MSMQ as the underlying transport)
WCF-Custom	Any	When you need to define a custom binding configuration for an “in-process” host
WCF-CustomIsolated	Any	When you need to define a custom binding configuration for an “isolated” host – this is only a receive adapter, not used on send ports

WCF-Custom-WCF-CustomIsolated Adapters

The WCF-Custom and WCF-CustomIsolated adapters offer you complete control over the channel stack and behaviors configuration, and as a consequence, they are the only WCF adapters you really need. Compared with the other WCF Adapters, they are the only ones to provide the possibility to:

Implement and exploit extensibility points.
Have full access to properties exposed by bindings/behaviors.
Enable the use of the bamInterceptor endpoint behavior.
Export/Import the binding configuration.
Disable a receive location on failure.
Run an http-based RL within an in-process host.
Use bindings (e.g. wsDualHttpBinding) for which a WCF Adapter does not exist.

WCF allows developers to modify and extend its standard runtime behavior with several extensibility points, some of which are not supported by BizTalk Server: for example you cannot customize the BizTalkServiceInstance class, create you own ServiceHost or create a custom ContractBehavior. BizTalk Server supports the following extensibility points:

Custom Behaviors

Service Behaviors: they enable the customization of the entire service runtime including the ServiceHost.
Endpoint Behaviors: they enable the customization of service endpoints and their associated EndpointDispatcher. The most common technique is creating an endpoint behavior to add a Message Inspector to the collection of Message Inspectors used at run time by a proxy/dispatcher. This allows to intercept an in-flight incoming/outgoing WCF message before this latter is passed/transmitted to the intended operation/target service, inspect and eventually change its content (headers or body).

Custom Binding Elements, Channels Binding Elements, Channels, Channel Factories, Binding Element Extension, etc. This technique is more complex, because in order to register and execute at runtime a custom channel within the channel stack used by a WCF Receive Location or Send Port, it’s necessary to create and properly register multiple components:

Custom BindingElementExtensionElement.
Custom BindingElement.
Custom ChannelFactory
Custom Channel.

Custom Bindings: the WCF LOB Adapter SDK allows developers to create new bindings to use with WCF-Custom and WCF-CustomIsolated Adapters. For example, the WCF based SQL Adapter (WCF-SQL) is based on a new binding called sqlBinding that is properly registered in the machine.config when installing the Adapter.

In my previous post called How to Throw Typed Fault Exceptions from Orchestrations Published as WCF Services I created 2 different Endpoint Behaviors to customize the default runtime behavior of the WCF-Custom/WCF-CustomIsolated Adapter. In my next post I will show how to create a custom channel to debatch (on a WCF Send Port) an inbound message containing a collection of operations and make multiple calls to the underlying WCF service, one for each operation.

Conclusion

I strongly suggest you to read the following whitepapers to have a better insight in WCF Adapters and their extensibility points.

The Using the Windows Communication Foundation (WCF) Adapters in BizTalk Server whitepaper describes the use of the WCF adapters in BizTalk Server.
The Consuming and Hosting WCF Services with Custom Bindings in BizTalk Server whitepaper provides an in-depth explanation on how to use the BizTalk Server WCF adapters for hosting and consuming Windows Communication Foundation (WCF) services with custom bindings. The paper compiles a series of lessons learned based on a real-world implementation of integrating a custom WCF binding with the WCF adapters. Furthermore, key industrial infrastructure concerns such as transaction management and security are discussed in the context of integrating custom WCF bindings and behaviors with the BizTalk Messaging Engine. Also emphasized are a few pragmatic paradigms such as the use of dynamic ports in consuming WCF services. Finally, some key integration challenges are discussed to streamline the correct use of the adapters for solving complex business problems.

Moreover, I recommend to download and review the following WCF Adapters samples on MSDN:

Using Custom Behaviors with the BizTalk WCF Adapters: Extend WCF services through custom behaviors and the WCF adapters.
Using a Custom Behavior to Secure a Receive Location with the WCF Adapters: Secure access to a BizTalk receive location by using a custom authorization behavior with the WCF adapters.
Using a Pipeline Intermediary with the WCF Adapters: Use an intermediary BizTalk Server custom pipeline to dynamically channel an incoming WCF client call to the correct version of a WCF service.
Using Transactions with Message Interception and the WCF Adapters: Custom message interception during BizTalk Server’s binding processing of an outgoing WCF message.
Using BizTalk Server Messaging with the WCF Adapters: Use BizTalk Server messaging and the WCF adapters to send WCF messages from a WCF client application to a WCF service.
Using Dynamic Send Ports with the WCF Adapters: Configure dynamic send ports by using WCF adapter context properties for the WCF-NetTcp transport and for the WCF-Custom transport with the WCF-NetTcp binding.
Handling Exceptions in Orchestrations with the WCF Adapters: Handle typed fault exceptions in orchestrations.
Using Custom SOAP Headers with the WCF Adapters: Access custom SOAP headers in inbound WCF messages, and add custom SOAP headers to outbound WCF messages.
Programmatically Submitting a WCF Message Using the WCF Adapters: Submit a WCF message to an orchestration exposed as a WCF service. The WCF adapter uses an XPath command to extract specific SOAP property elements from the incoming message.
Using the Messages Tab with the WCF Adapters: Use the Messages tab in the Transport Properties dialog box of a WCF adapter to control which part of an incoming WCF message is mapped to the body of the resulting BizTalk message.
WCF Adapter Walkthroughs: Get sample code for the WCF adapter walkthroughs outlined in the WCF Adapter Walkthroughs topic of the MSDN Library.

Using Visual Studio 2005/2008 To Generate Load Against a Two-Way Request-Response WCF Receive Location: ERRATA CORRIGE

leprino — Sat, 26 Sep 2009 00:06:03 GMT

Today, while I was driving south to Tuscany, I realized that my post had the following defects:

I forgot to include the text of the configuration file used by the project;
The code of the HandleException and CloseObjects methods had some imperfections, so I fixed it;
One of you (Pablo Alarcon Garcia, thanks mate!) dropped me an email to inform me that I committed a very common error in C#: I used "throw ex;" command rather than "throw;" to re-throw an exception within every catch block of the InvokeTwoWayWCFReceiveLocation method. Indeed, this is not really an error as the value of the exception properties (Message, InnerException, etc.) is preserved when using a “throw ex;” command. Conversely, the value of the StackTrace property is reinitialized to refer the “throw ex;” command within the catch block instead of the line of code that originally raised the exception. If you want to re-throw the exception currently handled by a catch block and preserve the value of its StackTrace property, use the throw statement without arguments: "throw ex;". See the following link for more information on this topic.

So I corrected my post and uploaded the new code here. Sorry folks. :-)

Using Visual Studio 2005/2008 To Generate Load Against a Two-Way Request-Response WCF Receive Location

leprino — Thu, 24 Sep 2009 08:26:09 GMT

Introduction

We all know that testing is very important for every kind of solution. However, testing and in particular performance testing is often neglected for time or budget reasons. One of the things that I have found when working with customers is that the amount of performance testing done on a project tends to fall into a couple of categories. The first and in my opinion the worst is when no performance testing is done at all. I would call this a write once, go live and cross your fingers approach. The next type is when load/stress testing is performed right at the end of the project lifecycle, typically a few weeks before the go live date. If the application is not able to meet the expected performance goals in terms of scalability and latency, there’s a serious risk that architects and developers will have to spend a considerable amount of time to redesign, rewrite and test large portions of the solution.

Sometimes performance testing is done using a rig that does not accurately reflect the production environment, while in other cases I have seen people using different adapters that do not match those used in the production environment. All these bad practices and factors expose the project to the risk of delays and failure.

BizTalk Server is used in mission and business critical scenarios where it is not just part of the business, it is actually running the entire business. Overlooking performing testing increases the risk of failure which can cause not just technical problems but political problems as well.

The way to minimize the risk of a BizTalk project (and of any software project in general) is to adopt a strong and consistent testing strategy throughout the development phase and before deploying the final solution into production. For this reason, the BizTalk Customer Advisory Team adopts a consistent and reusable methodology when running performance labs with customers. This process is fully documented in the BizTalk Server Performance Testing Methodology of the Microsoft BizTalk Server 2009 Performance Optimization Guide.

The best approach is to leverage a test framework like BizUnit to automate test runs and possibly schedule their execution over night. Another key factor is to individuate the right tool to generate load against the application. To this purpose you can use the BizTalk LoadGen 2007 as I explained in my previous article. The LoadGen tool offers the following important features and benefits to provide a simple, generic, reliable, and consistent tool for generating message transmission loads. By default, LoadGen supports several transport protocols:

Transport component	Description
File transport	The File transport creates files including message data in the destination folder.
HTTP transport	The HTTP transport sends messages with the POST method to the destination location that the BizTalk HTTP receive adapter hosts.
MQSeries transport	The MQSeries transport sends messages to the target MQSeries queue.
MSMQLarge transport	The MSMQLarge transport sends messages to the BizTalk MSMQT receive locations. Native message queuing cannot process a message with a body larger than 4 megabytes (MB).
MSMQ transport	The MSMQ transport sends messages smaller than 4 megabytes (MB) to the target MSMQ queue.
SOAP transport	The SOAP transport sends messages with the POST method to the target Web service.
WSE transport	The WSE transport sends messages to the target Web service hosted by Web Services Enhancements (WSE) 2.0.
WSS transport	The WSS transport sends messages with the PUT method to the target Windows SharePoint Services site.
WCF transport	The WCF transport sends message data to the target Windows Communication Foundation (WCF) service.

However, LoadGen is a great tool, but it suffers of the following problems:

Test definition is entirely done through XML configuration files. LoadGen does not provide any UI to model and define load tests.
It doesn’t provide the possibility to create rules to generate warnings when a give performance counter exceeds a configurable threshold.
Generating high loads against a BizTalk application is an intensive task and can easily lead to run out of system resources (CPU, Memory, Disk I/O, Network, etc.). For this reason, when running a performance lab against an enterprise level application is necessary to arrange multiple Test Agent machines in order to generate the expected load. Even though you can start several instances of LoadGen on separate machines, you can’t coordinate their work in a centralized manner. Multiple instances of LoadGen can be started using a script or the LoadGenExecuteStep class provided by BizUnit, but there’s no way to monitor their execution using a console.
When invoking a Two-Way Receive Location, LoadGen provides the possibility to save response times to a log file, but not to a centralized database. Of course, when the test is over, it’s possible to collect the log files produced by the different Test Agents, each running on a separate machine, and upload their contents to a SQL Server table with a Bulk Insert. This allows to conduct an in-depth analysis of the test results and eventually run a set of queries to generate statistics and compare key performance indicators of different test runs. Nevertheless, this process must be manually built as it is not provided out of the box by LoadGen.

Sometimes customers/partners create a custom multi-threaded application in order to generate load against their BizTalk Application, correlate request and response messages and persist response times to a relational database. This approach requires time and resources to design and develop the multi-threaded application capable to generate the load against the BizTalk platform and store test results to a custom repository. Another common technique adopted by customers is to use a dedicated product like HP LoadRunner to measure end-to-end performance, diagnose application and system bottlenecks and tune for better performance. So the question arises: is there any Microsoft product that I can use to perform stress and load tests in a easy way? The answer is yes.

Visual Studio Team System Test Edition

In my opinion, the best solution to design, configure, execute and monitor performance tests against BizTalk is to leverage Visual Studio Team System Test Edition that includes a suite of tools that allow to conduct several kinds of tests (unit tests, load tests, web tests, etc.). In particular, Visual Studio Team System Test Edition provides a tool for creating and running load tests. For more information on this topic see Understanding Load Tests. Visual Studio Team System Test Edition provides the following advantages:

It enables developers to set up a group of computers that generates simulated load for testing. The group consists of a single Controller and one or more Test Agents, each running on a separate machine. Collectively, this group is called a Rig. In particular, a Test Agent is the part of the Rig that is used to run tests and generate simulated load. For more information on this topic see Controllers, Agents, and Rigs.
When you define a load test, you can choose between two different load patterns:
- Constant: the constant load pattern is used to specify a user load that does not change during the load test. For example, when you run a smoke test on a Web application, you might want to set a light, constant load of 10 users.
- Step: the step load pattern is used to specify a user load that increases with time up to a defined maximum user load. For stepping loads, you specify the Initial User Count, Maximum User Count, Step Duration (seconds), and Step User Count.For example a Step load with an Initial User count of one, Maximum User Count of 100, Step Duration (seconds) of 10, and a Step User Count of 1 creates a user load pattern that starts at 1, increases by 1 every 10 seconds until it reaches 100 Users.
It’s possible to create a Load Test Results Repository to store the information gathered during a test run. This repository contains performance counter data and information about any errors. See Load Test Results Repository for more information on this topic.
When you create a load test, Visual Studio Team System Test Edition lets you specify multiple counter sets. A counter set is collection of performance counters that are useful to monitor during a load test run. Counter sets are part of the load test and apply to all its scenarios. They are organized by technology, for example, ASP.NET, IIS or SQL counter sets. Visual Studio allows developers to create their own Counter Sets to track any performance counters. Counter sets can be defined to gather performance counter data from all the machines composing the performance lab kit. (BizTalk Server, SQL Server, Test Agents, etc.).

This feature is extremely helpful because during a test run it allows to collect performance counter data (BizTalk Server, SQL, IIS, etc.) in a central storage and to create multiple diagrams to visualize the most relevant counters. When the test run is over, the summary page allows to browse and analyze test results.

How to use Visual Studio to generate load against a Two-Way WCF Receive Location

Recently I published an article in 2 parts (Part1, Part2) where I compare 4 different techniques to process an XLANGMessage within a method exposed by a business component invoked by an orchestration. In particular, I created a Two-Way Synchronous version and a One-Way Asynchronous version for each of these patterns. All of the 4 synchronous patterns share the same architecture depicted below:

Message Flow:

A WCF-BasicHttp or WCF-Custom Request-Response Receive Location receives a new CalculatorRequest xml document from the Test Agent/Client Application.
The XML disassembler component within the XMLTransmit pipeline promotes the Method element inside the CalculatorRequest xml document. The Message Agent submits the incoming message to the MessageBox (BizTalkMsgBoxDb).
The inbound request starts a new instance of a given orchestration.
The orchestration invokes a method exposed by a business component.
The method in question returns a response message.
The orchestration publishes the CalculatorResponse message to the MessageBox (BizTalkMsgBoxDb).
The response message is retrieved by the Request-Response WCF-BasicHttp or WCF-Custom Receive Location.
The response message is returned to the Test Agent/Client Application.

In order to generate the load against the WCF-Basic and WCF-Custom Receive Locations I proceeded as follows:

I started creating a Test Project called WCFLoadTest.

Then I created a Test Class called WCFLoadTest. This class is decorated with the Microsoft.VisualStudio.TestTools.UnitTesting.TestClassAttribute to indicate that the class contains test methods.
I defined a Test Method for each combination of the following factors:
- Receive Location/Transport used to submit messages to BizTalk (WCF-NetTcp, WCF-BasicHttp)
- Message Size (small, medium, large)
- Orchestration Pattern (XmlDocumentOrchestration, StreamOrchestration, MessageClassesOrchestration, CustomBTXMessageOrchestration).
Each combination of the factors above corresponds to a separate test case. I used the following naming convention for test methods:
- <Transport><Message Size><Orchestration Pattern>
I created an helper method called InvokeTwoWayWCFReceiveLocation to allow test methods to exchange messages with a Request Response WCF Receive Location. This methods expects the following input parameters
- endpointConfigurationName: the name of one of the service endpoints defined in the app.config configuration.
- requestMessageFolder: the path of the folder containing the request documents.
- requestMessageName: the name of the request document to send.
- messageVersion: specifies the version of SOAP and WS-Addressing of the request and response messages.
- sessionMode: indicates the support for reliable sessions (Allowed, Required, NotAllowed).
Each test method invokes the InvokeTwoWayWCFReceiveLocation passing a different combination of parameters.

The client endpoints used by the InvokeTwoWayWCFReceiveLocation method to exchange messages with the WCF Receive Locations are defined in the configuration file (App.config) of the project.

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
    <system.serviceModel>
    <!-- Bindings used by client endpoints -->
        <bindings>
            <netTcpBinding>
                <binding name="netTcpBinding"
                         closeTimeout="01:10:00"
                         openTimeout="01:10:00"
                         receiveTimeout="01:10:00"
                         sendTimeout="01:10:00"
                         transactionFlow="false"
                         transferMode="Buffered"
                         transactionProtocol="OleTransactions"
                         hostNameComparisonMode="StrongWildcard"
                         listenBacklog="100"
                         maxBufferPoolSize="1048576"
                         maxBufferSize="10485760"
                         maxConnections="200"
                         maxReceivedMessageSize="10485760">
                    <readerQuotas maxDepth="32"
                                  maxStringContentLength="8192"
                                  maxArrayLength="16384"
                                  maxBytesPerRead="4096"
                                  maxNameTableCharCount="16384" />
                    <reliableSession ordered="true"
                                     inactivityTimeout="00:10:00"
                                     enabled="false" />
                    <security mode="None">
                        <transport clientCredentialType="Windows" protectionLevel="EncryptAndSign" />
                        <message clientCredentialType="Windows" />
                    </security>
                </binding>
            </netTcpBinding>
            <basicHttpBinding>
                <binding name="basicHttpBinding"
                         closeTimeout="00:10:00"
                         openTimeout="00:10:00"
                         receiveTimeout="00:10:00"
                         sendTimeout="00:10:00"
                         allowCookies="false"
                         bypassProxyOnLocal="false"
                         hostNameComparisonMode="StrongWildcard"
                         maxBufferSize="10485760"
                         maxBufferPoolSize="524288"
                         maxReceivedMessageSize="10485760"
                         messageEncoding="Text"
                         textEncoding="utf-8"
                         transferMode="Buffered"
                         useDefaultWebProxy="true">
                    <readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384"
                        maxBytesPerRead="4096" maxNameTableCharCount="16384" />
                    <security mode="None">
                        <transport clientCredentialType="None" proxyCredentialType="None"
                            realm="" />
                        <message clientCredentialType="UserName" algorithmSuite="Default" />
                    </security>
                </binding>
            </basicHttpBinding>
        </bindings>
        <client>
      <!-- Client endpoints used by client excahnge messages with the WCF Receive Locations -->
            <endpoint address="net.tcp://localhost:8123/calculator"
                      binding="netTcpBinding"
                      bindingConfiguration="netTcpBinding"
                      contract="System.ServiceModel.Channels.IRequestChannel"
                      name="netTcpCalculatorServiceReceiveLocationEndpoint" />
            <endpoint address="http://localhost/CalculatorServiceReceiveLocation/CalculatorService.svc"
                      binding="basicHttpBinding"
                      bindingConfiguration="basicHttpBinding"
                      contract="System.ServiceModel.Channels.IRequestChannel"
                      name="basicHttpCalculatorServiceReceiveLocationEndpoint" />
        </client>
    </system.serviceModel>
    <appSettings>
    <!-- Folder containing test messages -->
        <add key="testMessageFolder" value="C:\Projects\HandleXLANGMessages\WCFLoadTest\TestMessages" />
    </appSettings>
</configuration>

The following table contains the code of the WCFLoadTest class:

#region Copyright
//-------------------------------------------------
// Author:  Paolo Salvatori
// Email:   paolos@microsoft.com
// History: 2009-09-20 Created
//-------------------------------------------------
#endregion

#region Using Directives
using System;
using System.IO;
using System.Diagnostics;
using System.Text;
using System.Configuration;
using System.Collections.Generic;
using System.Linq;
using System.Xml;
using System.ServiceModel;
using System.ServiceModel.Channels;
using Microsoft.VisualStudio.TestTools.UnitTesting;
#endregion

namespace Microsoft.BizTalk.CAT.Samples.HandleXLANGMessages.WCFLoadTest
{
    /// <summary>
    /// Summary description for UnitTest1
    /// </summary>
    [TestClass]
    public class WCFLoadTest
    {
        #region Constants
        private const int MaxBufferSize = 2097152;
        private const string Source = "WCF Load Test";
        private const string Star = "*";
        private const string TestMessageFolderParameter = "testMessageFolder";
        private const string TestMessageFolderDefault = @"C:\Projects\HandleXLANGMessages\WCFLoadTest\TestMessages";
        private const string TestMessageFolderFormat = @"Test Message Folder = {0}";
        private const string NetTcpCalculatorServiceReceiveLocationEndpoint = "netTcpCalculatorServiceReceiveLocationEndpoint";
        private const string BasicHttpCalculatorServiceReceiveLocationEndpoint = "basicHttpCalculatorServiceReceiveLocationEndpoint";
        private const string SmallCustomBtxMessageOrchestration = "SmallCustomBtxMessageOrchestration.xml";
        private const string SmallMessageClassesOrchestration = "SmallMessageClassesOrchestration.xml";
        private const string SmallStreamOrchestration = "SmallStreamOrchestration.xml";
        private const string SmallXmlDocumentOrchestration = "SmallXmlDocumentOrchestration.xml";
        private const string MediumCustomBtxMessageOrchestration = "MediumCustomBtxMessageOrchestration.xml";
        private const string MediumMessageClassesOrchestration = "MediumMessageClassesOrchestration.xml";
        private const string MediumStreamOrchestration = "MediumStreamOrchestration.xml";
        private const string MediumXmlDocumentOrchestration = "MediumXmlDocumentOrchestration.xml";
        private const string LargeCustomBtxMessageOrchestration = "LargeCustomBtxMessageOrchestration.xml";
        private const string LargeMessageClassesOrchestration = "LargeMessageClassesOrchestration.xml";
        private const string LargeStreamOrchestration = "LargeStreamOrchestration.xml";
        private const string LargeXmlDocumentOrchestration = "LargeXmlDocumentOrchestration.xml";
        #endregion

        #region Private Instance Fields
        private TestContext testContextInstance;
        #endregion

        #region Private Static Fields
        private static string testMessageFolder = null;
        #endregion

        #region Public Instance Constructor
        public WCFLoadTest()
        {
        }
        #endregion

        #region Public Static Constructor
        static WCFLoadTest()
        {
            try
            {
                testMessageFolder = ConfigurationManager.AppSettings[TestMessageFolderParameter];
                if (string.IsNullOrEmpty(testMessageFolder))
                {
                    testMessageFolder = TestMessageFolderDefault;
                }
            }
            catch (Exception ex)
            {
                Trace.WriteLine(ex.Message);
                EventLog.WriteEntry(Source, ex.Message, EventLogEntryType.Error);
            }
        }
        #endregion

        #region Public Properties
        /// <summary>
        ///Gets or sets the test context which provides
        ///information about and functionality for the current test run.
        ///</summary>
        public TestContext TestContext
        {
            get
            {
                return testContextInstance;
            }
            set
            {
                testContextInstance = value;
            }
        }
        #endregion

        #region Test Methods
        [TestMethod]
        public void BasicHttpSmallCustomBtxMessageOrchestration()
        {
            InvokeTwoWayWCFReceiveLocation(BasicHttpCalculatorServiceReceiveLocationEndpoint,
                                           testMessageFolder,
                                           SmallCustomBtxMessageOrchestration,
                                           MessageVersion.Soap11,
                                           SessionMode.Allowed);
        }

        [TestMethod]
        public void BasicHttpSmallMessageClassesOrchestration()
        {
            InvokeTwoWayWCFReceiveLocation(BasicHttpCalculatorServiceReceiveLocationEndpoint,
                                           testMessageFolder,
                                           SmallMessageClassesOrchestration,
                                           MessageVersion.Soap11,
                                           SessionMode.Allowed);
        }

        [TestMethod]
        public void BasicHttpSmallStreamOrchestration()
        {
            InvokeTwoWayWCFReceiveLocation(BasicHttpCalculatorServiceReceiveLocationEndpoint,
                                           testMessageFolder,
                                           SmallStreamOrchestration,
                                           MessageVersion.Soap11,
                                           SessionMode.Allowed);
        }

        [TestMethod]
        public void BasicHttpSmallXmlDocumentOrchestration()
        {
            InvokeTwoWayWCFReceiveLocation(BasicHttpCalculatorServiceReceiveLocationEndpoint,
                                           testMessageFolder,
                                           SmallXmlDocumentOrchestration,
                                           MessageVersion.Soap11,
                                           SessionMode.Allowed);
        }

        [TestMethod]
        public void BasicHttpMediumCustomBtxMessageOrchestration()
        {
            InvokeTwoWayWCFReceiveLocation(BasicHttpCalculatorServiceReceiveLocationEndpoint,
                                           testMessageFolder,
                                           MediumCustomBtxMessageOrchestration,
                                           MessageVersion.Soap11,
                                           SessionMode.Allowed);
        }

        [TestMethod]
        public void BasicHttpMediumMessageClassesOrchestration()
        {
            InvokeTwoWayWCFReceiveLocation(BasicHttpCalculatorServiceReceiveLocationEndpoint,
                                           testMessageFolder,
                                           MediumMessageClassesOrchestration,
                                           MessageVersion.Soap11,
                                           SessionMode.Allowed);
        }

        [TestMethod]
        public void BasicHttpMediumStreamOrchestration()
        {
            InvokeTwoWayWCFReceiveLocation(BasicHttpCalculatorServiceReceiveLocationEndpoint,
                                           testMessageFolder,
                                           MediumStreamOrchestration,
                                           MessageVersion.Soap11,
                                           SessionMode.Allowed);
        }

        [TestMethod]
        public void BasicHttpMediumXmlDocumentOrchestration()
        {
            InvokeTwoWayWCFReceiveLocation(BasicHttpCalculatorServiceReceiveLocationEndpoint,
                                           testMessageFolder,
                                           MediumXmlDocumentOrchestration,
                                           MessageVersion.Soap11,
                                           SessionMode.Allowed);
        }

        [TestMethod]
        public void BasicHttpLargeCustomBtxMessageOrchestration()
        {
            InvokeTwoWayWCFReceiveLocation(BasicHttpCalculatorServiceReceiveLocationEndpoint,
                                           testMessageFolder,
                                           LargeCustomBtxMessageOrchestration,
                                           MessageVersion.Soap11,
                                           SessionMode.Allowed);
        }

        [TestMethod]
        public void BasicHttpLargeMessageClassesOrchestration()
        {
            InvokeTwoWayWCFReceiveLocation(BasicHttpCalculatorServiceReceiveLocationEndpoint,
                                           testMessageFolder,
                                           LargeMessageClassesOrchestration,
                                           MessageVersion.Soap11,
                                           SessionMode.Allowed);
        }

        [TestMethod]
        public void BasicHttpLargeStreamOrchestration()
        {
            InvokeTwoWayWCFReceiveLocation(BasicHttpCalculatorServiceReceiveLocationEndpoint,
                                           testMessageFolder,
                                           LargeStreamOrchestration,
                                           MessageVersion.Soap11,
                                           SessionMode.Allowed);
        }

        [TestMethod]
        public void BasicHttpLargeXmlDocumentOrchestration()
        {
            InvokeTwoWayWCFReceiveLocation(BasicHttpCalculatorServiceReceiveLocationEndpoint,
                                           testMessageFolder,
                                           LargeXmlDocumentOrchestration,
                                           MessageVersion.Soap11,
                                           SessionMode.Allowed);
        }

        [TestMethod]
        public void NetTcpSmallCustomBtxMessageOrchestration()
        {
            InvokeTwoWayWCFReceiveLocation(NetTcpCalculatorServiceReceiveLocationEndpoint,
                                           testMessageFolder,
                                           SmallCustomBtxMessageOrchestration,
                                           MessageVersion.Default,
                                           SessionMode.Allowed);
        }

        [TestMethod]
        public void NetTcpSmallMessageClassesOrchestration()
        {
            InvokeTwoWayWCFReceiveLocation(NetTcpCalculatorServiceReceiveLocationEndpoint,
                                           testMessageFolder,
                                           SmallMessageClassesOrchestration,
                                           MessageVersion.Default,
                                           SessionMode.Allowed);
        }

        [TestMethod]
        public void NetTcpSmallStreamOrchestration()
        {
            InvokeTwoWayWCFReceiveLocation(NetTcpCalculatorServiceReceiveLocationEndpoint,
                                           testMessageFolder,
                                           SmallStreamOrchestration,
                                           MessageVersion.Default,
                                           SessionMode.Allowed);
        }

        [TestMethod]
        public void NetTcpSmallXmlDocumentOrchestration()
        {
            InvokeTwoWayWCFReceiveLocation(NetTcpCalculatorServiceReceiveLocationEndpoint,
                                           testMessageFolder,
                                           SmallXmlDocumentOrchestration,
                                           MessageVersion.Default,
                                           SessionMode.Allowed);
        }

        [TestMethod]
        public void NetTcpMediumCustomBtxMessageOrchestration()
        {
            InvokeTwoWayWCFReceiveLocation(NetTcpCalculatorServiceReceiveLocationEndpoint,
                                           testMessageFolder,
                                           MediumCustomBtxMessageOrchestration,
                                           MessageVersion.Default,
                                           SessionMode.Allowed);
        }

        [TestMethod]
        public void NetTcpMediumMessageClassesOrchestration()
        {
            InvokeTwoWayWCFReceiveLocation(NetTcpCalculatorServiceReceiveLocationEndpoint,
                                           testMessageFolder,
                                           MediumMessageClassesOrchestration,
                                           MessageVersion.Default,
                                           SessionMode.Allowed);
        }

        [TestMethod]
        public void NetTcpMediumStreamOrchestration()
        {
            InvokeTwoWayWCFReceiveLocation(NetTcpCalculatorServiceReceiveLocationEndpoint,
                                           testMessageFolder,
                                           MediumStreamOrchestration,
                                           MessageVersion.Default,
                                           SessionMode.Allowed);
        }

        [TestMethod]
        public void NetTcpMediumXmlDocumentOrchestration()
        {
            InvokeTwoWayWCFReceiveLocation(NetTcpCalculatorServiceReceiveLocationEndpoint,
                                           testMessageFolder,
                                           MediumXmlDocumentOrchestration,
                                           MessageVersion.Default,
                                           SessionMode.Allowed);
        }

        [TestMethod]
        public void NetTcpLargeCustomBtxMessageOrchestration()
        {
            InvokeTwoWayWCFReceiveLocation(NetTcpCalculatorServiceReceiveLocationEndpoint,
                                           testMessageFolder,
                                           LargeCustomBtxMessageOrchestration,
                                           MessageVersion.Default,
                                           SessionMode.Allowed);
        }

        [TestMethod]
        public void NetTcpLargeMessageClassesOrchestration()
        {
            InvokeTwoWayWCFReceiveLocation(NetTcpCalculatorServiceReceiveLocationEndpoint,
                                           testMessageFolder,
                                           LargeMessageClassesOrchestration,
                                           MessageVersion.Default,
                                           SessionMode.Allowed);
        }

        [TestMethod]
        public void NetTcpLargeStreamOrchestration()
        {
            InvokeTwoWayWCFReceiveLocation(NetTcpCalculatorServiceReceiveLocationEndpoint,
                                           testMessageFolder,
                                           LargeStreamOrchestration,
                                           MessageVersion.Default,
                                           SessionMode.Allowed);
        }

        [TestMethod]
        public void NetTcpLargeXmlDocumentOrchestration()
        {
            InvokeTwoWayWCFReceiveLocation(NetTcpCalculatorServiceReceiveLocationEndpoint,
                                           testMessageFolder,
                                           LargeXmlDocumentOrchestration,
                                           MessageVersion.Default,
                                           SessionMode.Allowed);
        }
        #endregion

        #region Helper Methods
        public void InvokeTwoWayWCFReceiveLocation(string endpointConfigurationName,
                                                   string requestMessageFolder,
                                                   string requestMessageName,
                                                   MessageVersion messageVersion,
                                                   SessionMode sessionMode)
        {
            ChannelFactory<IRequestChannel> channelFactory = null;
            IRequestChannel channel = null;
            XmlTextReader xmlTextReader = null;
            Message requestMessage = null;
            Message responseMessage = null;

            try
            {

                channelFactory = new ChannelFactory<IRequestChannel>(endpointConfigurationName);
                channelFactory.Endpoint.Contract.SessionMode = sessionMode;
                channel = channelFactory.CreateChannel();
                string path = Path.Combine(requestMessageFolder, requestMessageName);
                xmlTextReader = new XmlTextReader(path);
                requestMessage = Message.CreateMessage(messageVersion, Star, xmlTextReader);
                TestContext.BeginTimer(requestMessageName);
                responseMessage = channel.Request(requestMessage);
                channel.Close();
                channelFactory.Close();
            }
            catch (FaultException ex)
            {
                HandleException(ref channelFactory,
                                ref channel,
                                ex);
                throw;
            }
            catch (CommunicationException ex)
            {
                HandleException(ref channelFactory,
                                ref channel,
                                ex);
                throw;
            }
            catch (TimeoutException ex)
            {
                HandleException(ref channelFactory,
                                ref channel,
                                ex);
                throw;
            }
            catch (Exception ex)
            {
                HandleException(ref channelFactory,
                                ref channel,
                                ex);
                throw;
            }
            finally
            {
                TestContext.EndTimer(requestMessageName);
                CloseObjects(channelFactory,
                             channel,
                             xmlTextReader,
                             requestMessage,
                             responseMessage);
            }
        }

        private void HandleException(ref ChannelFactory<IRequestChannel> channelFactory,
                                     ref IRequestChannel channel,
                                     Exception ex)
        {
            try
            {
                if (channelFactory != null)
                {
                    channelFactory.Abort();
                    channelFactory = null;
                }
                if (channel != null)
                {
                    channel.Abort();
                    channel = null;
                }
                Trace.WriteLine(ex.Message);
                EventLog.WriteEntry(Source, ex.Message, EventLogEntryType.Error);
            }
            catch (Exception)
            {
            }
        }

        private void CloseObjects(ChannelFactory<IRequestChannel> channelFactory,
                                     IRequestChannel channel,
                                     XmlTextReader xmlTextReader,
                                     Message requestMessage,
                                     Message responseMessage)
        {
            try
            {
                if (channelFactory != null)
                {
                    channelFactory.Close();
                }
                if (channel != null)
                {
                    channel.Close();
                }
                if (xmlTextReader != null)
                {
                    xmlTextReader.Close();
                }
                if (requestMessage != null)
                {
                    requestMessage.Close();
                }
                if (responseMessage != null)
                {
                    responseMessage.Close();
                }
            }
            catch (Exception)
            {
            }
        }
        #endregion
    }
}

Each test method corresponds to a separate Unit Test. Visual Studio permits to create a Load Test as a composition of one or multiple unit tests. Explaining in detail how creating a Load Test is out of the scope of the present post. This process is fully documented on MSDN at the following link: Walkthrough: Creating and Running a Load Test. However, for the sake of the completeness I’m going to describe the steps necessary to create a Load Test based on a single Unit Test:

Right-click the Test project, expand the Add menu and select the Load Test menu item as shown in the picture below:

This opens a a Wizard that walks you through the creation of a Load Test. Click the Next button. See Creating Load Tests for more information on this topic.

Enter a name for the load test scenario and select one of the available think time profiles (I always choose ‘Do not use think times’ to increase the load against the tested application). Click the Next button. See How to: Specify Scenarios for more information on this step.

Choose a load pattern for your load test and configure parameters applicable to your choice. Click the Next button. See How to: Specify Load Patterns for more information on this step.

Choose a Test Mix and click the Next button. See How to: Specify Test Mix for more information on this step.

A load test contains one or more scenarios, each of which contains one or more Unit Tests. Press the Add button to add one or multiple Unit Tests to your scenario. See How to: Specify Browser Mix for more information on this step.

Select one or multiple Unit Tests and click the OK button. One again, each Test Method corresponds to a separate Unit Test.

If you selected multiple Unit Tests, as in the picture below, you can select your preferred browser mix by adjusting the sliders in the Distribution column, or by typing the percentage values directly into the % column, then click the Next button to advance to the next step.

The following dialog allows you specify which computers and counter sets to monitor during a load test run. After completing the wizard, you can define additional counter sets. This is very helpful because it allows to select any performance counters exposed by a product (BizTalk Server in our case) or by your application. When you have done, click the Next button to advance to the next step. See How to: Specify Counter Sets for more information on this step.

In this dialog, you can specify Run settings that affect the entire load test. The run settings determine the length of the test, warm-up duration, maximum number of error details reported, sampling rate, connection model (Web tests only), results storage type, validation level and SQL tracing. The run settings should reflect the goals of your load test. For more information, see About Run Settings. When you have done, click the Finish button to complete the Load Test creation. See How to: Specify Run Settings for more information on this step.

At this point you can open the newly created Load Test, right click the Counter Sets and select Add Custom Counter Set from the drop menu. Rename your new Counter Set ‘BizTalk Server’.

Now, right click the BizTalk Server Counter Set and click the Add Counters menu item to add the most relevant BizTalk Server, System, .NET and WCF performance counters:
Repeat the last step to include the most relevant performance counters on the SQL machine hosting the BizTalkMsgBoxDb. In the list below, the name of the SQL Server instance hosting the BizTalkMsgBoxDb is MSSQL$BTS:

SQL Server performance counters
\\.\MSSQL$BTS:Access Methods\*
\\.\MSSQL$BTS:Buffer Manager\*
\\.\MSSQL$BTS:Buffer Node(*)\*
\\.\MSSQL$BTS:Buffer Partition(*)\*
\\.\MSSQL$BTS:Databases(*)\*
\\.\MSSQL$BTS:General Statistics\*
\\.\MSSQL$BTS:Latches\*
\\.\MSSQL$BTS:Locks(*)\*
\\.\MSSQL$BTS:Memory Manager\*
\\.\MSSQL$BTS:SQL Statistics\*
\\.\MSSQL$BTS:Transactions\*
\\.\MSSQL$BTS:Wait Statistics(*)\*
\\.\MSSQL$BTS:Plan Cache\*

System performance counters
\\.\Objects\*
\\.\Memory\*
\\.\Paging File(*)\*
\\.\PhysicalDisk(*)\*
\\.\Process(*)\*
\\.\Processor(*)\*
\\.\Network Interface(*)\*
\\.\TCPV4(*)\*
\\.\LogicalDisk(*)\*
\\.\System\*

Take into account that the WCF adapters do not provide their own performance counters. However, you can monitor the performance counters of Windows Communication Foundation (WCF) to gauge the performance of the WCF Receive Locations. To use the WCF performance counters for the WCF receive locations, you have to enable the performance counters for the host instances running the receive locations. See WCF Adapters Performance Counters for more information on this topic.
Another interesting feature provided by Visual Studio is the possibility to define one or multiple threshold rules on individual performance counters to monitor system resource usage or the respect of performance-related SLAs during a load test. Counter set definitions contain predefined threshold rules for many key performance counters, but you can create specific rules for any performance counter. See About Threshold Rules for more information on this topic.

Let’s make an example using my use case. During the scope phase of my performance lab I specify the following performance goal:

Latency:

warning: response time < 2 sec for 90% of messages
critical: response time < 5 sec for 99% of messages

In order to be notified during a test run that a response time exceeded one the threshold above, I can create a custom threshold rule. Nice, but how can I accomplish this task and which performance counter should I use? Let’s start asking to this latter question. The suitable performance counters for measuring the latency of a Request-Response WCF Receive Location are the following:
- \\.\BizTalk:Messaging Latency(<Host running your WCF Receive Location>)\Request-Response Latency (sec)
- \\.\ServiceModelService 3.0.0.0(<the service corresponding to your WCF Receive Location>)\Calls Duration
- \\.\ServiceModelEndpoint 3.0.0.0(<the endpoint used by your WCF Receive Location>)\Calls Duration
- \\.\ServiceModelEOperation 3.0.0.0(<the endpoint used by your WCF Receive Location>)\Calls Duration
In order to create a threshold on the Request-Response Latency (sec) performance counter, you can expand the BizTalk Server Counter Set and browse to the counter. Now right click the corresponding node and select the Add Threshold Rule from the drop menu:

In the Add Threshold Rule dialog, select Compare Constant in the left panel and assign:
- True to the Alert If Over property to indicate that a threshold violation occurs when the performance counter value is greater then the threshold value.
- 2 to the Warning Threshold Value property.
- 5 to the Critical Threshold Value property.

Click the OK button to confirm. See How to: Add a Threshold Rule for more information on this topic

You can repeat the same process and set a Threshold Rule to be alerted when the throughput of your application goes below a certain threshold. A good candidate to measure the throughput of a BizTalk application composed by one or multiple orchestration is the \\.\XLANG/s Orchestrations(<Host>)\Orchestration completed/sec performance counter. However, in this case I will set the value of the Alert If Over property to False to indicate that a threshold violation occurs when the performance counter value is less then the threshold value (in the sample below 20 and 10 are respectively the Warning and Critical Threshold Values).

At runtime, threshold violations can be reviewed clicking the threshold violation link under the Load Test toolbar. If the counter in question is exposed by a Graph, each threshold violations is signaled with a yellow triangle.

At this point I’m ready to launch a load test, so I can just press the start button. In order to create a custom Graph to monitor most relevant counters I proceed as follows:

I click the Add New Graph button on the Load Test toolbar (highlighted in the picture below), I specify a name for my new Graph in the Enter Graph Name dialog and then I press the OK button to confirm.

Now, it’s sufficient to select, drag & drop counters from the Counter panel to the newly created Graph. For the sake of I completeness, during my demo I created a graph called BizTalk Server and I dragged and dropped the following performance counters:
- \\.\BizTalk:Messaging Latency(BizTalkServerIsolatedHost)\Request-Response Latency (sec)
- \\.\BizTalk:Messaging (BizTalkServerIsolatedHost)\Documents received/sec
- \.\XLANG/s Orchestrations(BizTalkServerApplication)\Orchestration completed/sec
- \.\BizTalk:Message Box:General Counters(biztalkmsgboxdb:babo)\Spool Size
I strongly suggest to repeat the process and create a custom Graph to monitor the most relevant performance counters exposed by the SQL Server instance or instances that host the most solicited BizTalk Server databases:
- BizTalkMsgBoxDb and any additional MessageBox.
- BizTalkDTADb if global tracking is turned on.
- BAMPrimaryImport if the application makes use of BAM.

and any custom databases used by the application.

When the load test is over, you can review the results on the Summary Page:

You can also click the links in the Test Results and Transaction Results tables to navigate to a page where you can review more in detail test results, errors and threshold violations.

Summary

Visual Studio Test Edition is a powerful suite to rapidly build Unit and Load Tests to conduct performance and stress tests against your BizTalk application. We assumed that the application in question was exposed via a WCF Two-Way Receive Location, but the approach exposed in the present post can be customized and extended to other scenarios where the application makes use of other Adapter to expose its services. Here you can download the Test Project I used to conduct tests against the HandleXLANGMessages application. See Part1 and Part2 for more information.

4 Different ways to process an XLANGMessage within an helper component invoked by an orchestration Part 2

leprino — Mon, 21 Sep 2009 17:02:43 GMT

In the first part of the article we introduced and analyzed 4 different techniques to process an XLANGMessage within a business component invoked by an orchestration. In the second part we’ll compare these design patterns from a performance perspective. To this purpose I conducted some tests against the asynchronous version of each use case to measure and compare their latency and throughput. In particular, I used a floodgate approach when running test cases in order to isolate orchestration processing from message receiving and sending. To achieve this result I used BizTalk LoadGen 2007 followed the steps below at each test run.

I ran the bts_CleanupMsgbox stored procedure to delete any unprocessed or suspended messages and any pending service instances within the BizTalkMsgBoxDb.
I stopped the BizTalkServerApplication host instance that is responsible for running orchestrations.
I stopped the BizTalkServerSendHost host instance that is responsible for running send ports.
I stopped the FILE Send Port which consumes response messages generated by orchestrations. This way the send port subscription remains active and the response documents published to the BizTalkMsgBoxDb by the asynchronous orchestrations become suspended (resumable) messages.
I stopped all the orchestrations using the BizTalk Administration Console.
I created and launched a script that exploited BizTalk LoadGen 2007 to generate a configurable amount of messages within a given folder.
I started another in-process host instance called BizTalkServerReceiveHost.
A OneWay FILE Receive Location hosted by BizTalkServerReceiveHost was used to read and publish those messages to the BizTalkMsgBoxDb.
I started an instance of the Performance Monitor (perfmon.exe) to monitor performance counters. In particular, I used the \\.\BizTalk:Message Box:General Counters(biztalkmsgboxdb:babo)\Spool Size counter to confirm that all messages were successfully published to the BizTalkMsgBoxDb.
At the this point, on the BizTalk Administration Console I started the orchestration used by the current test case.
I clicked the Suspended Services Instances link on the Group Overview page within the BizTalk Administration Console to retrieve all suspended messages.
I right-clicked the summary row containing the total number of resumable messages.
I launched a script that:
- stopped the BizTalkServerReceiveHost;
- started the BizTalkServerApplication;
- started the performance log to measure relevant BizTalk, SQL and System performance counter.
I resumed all messages. This operations started the real test run as each resumed message was processed by a separate orchestration instance.

In order to automate the steps above, I created a bunch of scripts that you can download here along with the code.
I conducted my tests using 3 different message sizes:

Small Size Message: 10 operations, 3 KB.
Medium Size Message: 1,000 operations, 249 KB.
Large Size Message: 10,000 operations, 2,485 KB.

For the sake of completeness, I conducted tests on my laptop, not really the kind of harness that should be used for performance testing:

Windows Server 2008 R2 64-bit.
SQL Server 2008 SP1 EE 64-bit .
BizTalk Server 2009 64-bit EE
1 dual-core CPU.
4 GB RAM

The tables below report test results. I encourage you to perform long running tests on a production-like environment composed of multiple machines to obtain more significant and relevant results. Take into account that I had just the time to run a couple of tests for each use case. Nevertheless, the results obtained are quite interesting and in line with expectations: in fact they confirm that using an XmlDocument to process the content of an XLANGMessage is extremely handy and powerful, but it can easily lead to high memory usage and performance degradation, especially when dealing with large messages.

Small Size Messages

Test Case	# Messages/Test	# Operations/Message	Message Size (Kb)	Elapsed Time (Sec)	Messages/Sec
AsyncXmlDocumentOrchestration	5,000	10	3	104	47.204
AsyncStreamOrchestration	5,000	10	3	87	58.085
AsyncMessageClassesOrchestration	5,000	10	3	71	71.434
AsyncCustomBTXMessageOrchestration	5,000	10	3	66	75.099

Medium Size Messages

Test Case	# Messages/Test	# Operations/Message	Message Size (Kb)	Elapsed Time (Sec)	Messages/Sec
AsyncXmlDocumentOrchestration	10,000	1,000	249	240	42.009
AsyncStreamOrchestration	10,000	1,000	249	224	46.476
AsyncMessageClassesOrchestration	10,000	1,000	249	188	54.936
AsyncCustomBTXMessageOrchestration	10,000	1,000	249	181	61.396

Large Size Messages

Test Case	# Messages/Test	# Operations/Message	Message Size (Kb)	Elapsed Time (Sec)	Messages/Sec
AsyncXmlDocumentOrchestration	1,000	10,000	2,485	182	5.521
AsyncStreamOrchestration	1,000	10,000	2,485	84	11.905
AsyncMessageClassesOrchestration	1,000	10,000	2,485	93	10.964
AsyncCustomBTXMessageOrchestration	1,000	10,000	2,485	78	12.718

Diagrams

The following diagrams summarize the results contained in the tables above and visualize the differences among the various use cases in terms of latency (elapsed time) and scalability (messages/sec).

Latency

Throughput

4 Different ways to process an XLANGMessage within an helper component invoked by an orchestration Part 1

leprino — Thu, 10 Sep 2009 19:44:23 GMT

One of the most common scenarios in BizTalk applications is when an orchestration receives and processes an incoming XML document to produce a result message. Sometimes this latter can be generated just transforming the inbound message with a map, but in another cases the orchestration has to invoke a method exposed by a helper component which contains the necessary business logic to process the request document and produce a new XML response message. Usually the signature of this method is similar to the following code snippet:

public XmlDocument ProcessRequestReturnXmlDocument(XLANGMessage message)

As you can see, the method above does not return an XLANGMessage object as expected and the reason is quite straightforward: the only constructor exposed by the XLANGMessage class contained in the Microsoft.XLANGs.BaseTypes assembly is protected and inaccessible to user code. The XmlDocument is commonly used by developers to read the entire content of an XML message part using a single line of code (document.DocumentElement.OuterXml)and to access the value of one or multiple elements using an XPath expression and the SelectSingleNode/SelectNodes methods exposed by the class. Using an instance of the XmlDocument class to manipulate the content of an XLANGMessage is a flexible and handy technique, but it can easily lead to out of memory exceptions when dealing with large messages or with a significant amount of medium-size messages within the same host process. In fact, the use of an XmlDocument instance forces the message content to be entirely loaded into memory in order to build the object graph for the DOM and the total amount of memory used by a single instance of this class can grow up to 10 times the actual message size. See the following articles for more information on this topic:

“Optimizing Orchestration Performance” article in the Microsoft BizTalk Server 2009 Performance Optimization Guide.
“Optimizing Memory Usage with Streaming” article in the Microsoft BizTalk Server 2009 Performance Optimization Guide.
“Optimizing Pipeline Performance” article in the Microsoft BizTalk Server 2009 Performance Optimization Guide.
“Orchestration Performance” article on Jon Flanders' Blog for some performance numbers when using the DOM.
“Improving XML Performance” chapter in the Improving .NET Application Performance and Scalability guide.

Some months ago I decided to create a sample to compare the different techniques that can be used when invoking an helper component within an orchestration to process the request message and produce a response XML document. The 4 use cases described below have been fully tested on both BizTalk Server 2006 R2 and BizTalk Server 2009. Below you can find a pointer to the BizTalk Server 2009 version of the code. All the use cases are exposed by the same 2 Request-Response WCF Receive Locations:

HXM.Calculator.WCF-BasicHttp.ReceiveLocation: a WCF-BasicHttp Receive Location hosted by the BizTalkServerIsolatedHost.
HXM.Calculator.WCF-Custom.ReceiveLocation: a WCF-Custom Receive Location hosted by an in-process host which uses a CustomBinding composed of the BinaryMessageEncodingBindingElement + TcpTransportBindingElement.

The 4 test cases implement the same scenario using a different approach. They all process the operations contained within the CalculatorRequest message and return a CalculatorResponse document containing results.

CalculatorRequest

<CalculatorRequest mlns="http://Microsoft.BizTalk.CAT.Samples.HandleXLANGMessages.Schemas.CalculatorRequest">
  <Method>XmlDocumentOrchestration</Method>
  <Operations>
    <Operation>
      <Operator>+</Operator>
      <Operand1>82</Operand1>
      <Operand2>18</Operand2>
    </Operation>
    <Operation>
      <Operator>-</Operator>
      <Operand1>30</Operand1>
      <Operand2>12</Operand2>
    </Operation>
    <Operation>
      <Operator>*</Operator>
      <Operand1>25</Operand1>
      <Operand2>8</Operand2>
    </Operation>
    <Operation>
      <Operator>\</Operator>
      <Operand1>100</Operand1>
      <Operand2>25</Operand2>
    </Operation>
      <Operation>
      <Operator>+</Operator>
      <Operand1>100</Operand1>
      <Operand2>32</Operand2>
    </Operation>
  </Operations>
</CalculatorRequest>

ResponseMessage

<CalculatorResponse xmlns="http://Microsoft.BizTalk.CAT.Samples.HandleXLANGMessages.Schemas.CalculatorResponse">
      <Status>Ok</Status>
      <Results>
            <Result>
                  <Value>100</Value>
                  <Error>None</Error>
            </Result>
            <Result>
                  <Value>18</Value>
                  <Error>None</Error>
            </Result>
            <Result>
                  <Value>200</Value>
                  <Error>None</Error>
            </Result>
            <Result>
                  <Value>4</Value>
                  <Error>None</Error>
            </Result>
            <Result>
                  <Value>132</Value>
                  <Error>None</Error>
            </Result>
      </Results>
</CalculatorResponse>

I created a BizTalk project called Schemas to create the XML schemas that define and model the structure of the CalculatorRequest and CalculatorResponse messages. Then I created another project called Orchestrations and I implemented a different orchestration for each technique:

XmlDocumentOrchestration: the helper component invoked by the orchestration uses an XmlDocument object and in particular the SelectNodes method to read data of each operation within the incoming request message and another XmlDocument instance to create the outbound response document which contains results.
StreamOrchestration: the helper component invoked by the orchestration uses a VirtualStream and an XmlReader to read the content of the request message and uses a VirtualStream and an XmlWriter to generate the response message.
MessageClassesOrchestration: the helper component invoked by the orchestration deserializes the CalculatorRequest message to an instance of the CalculatorRequest .NET class, loops through the collections of operations and stores results within a new instance of the CalculatorResponse class. This latter is then serialized to an XML stream within the orchestration.
CustomBtxMessageOrchestration: This is probably the most tricky and interesting test case. In fact, the helper component uses an XmlReader to process the data contained in the CalculatorRequest message and a uses VirtualStream and and XmlWriter to generate the response message. However, instead of returning a Stream object as the method invoked by the StreamOrchestration, this method creates and returns a brand new XLANGMessage. See below for more details.

I created an asynchronous, one-way version for each of the above use cases. In this case requests are submitted to BizTalk via a one-way FILE Receive Location while the response documents generated by each orchestration are persisted to another folder using a one-way FILE Send Port.

AsyncXmlDocumentOrchestration: the helper component invoked by the orchestration uses an XmlDocument object and in particular the SelectNodes method to read data of each operation within the incoming request message and another XmlDocument instance to create the outbound response document which contains results.
AsyncStreamOrchestration: the helper component invoked by the orchestration uses a VirtualStream and an XmlReader to read the content of the request message and uses a VirtualStream and an XmlWriter to generate the response message.
AsyncMessageClassesOrchestration: the helper component invoked by the orchestration deserializes the CalculatorRequest message to an instance of the CalculatorRequest .NET class, loops through the collections of operations and stores results within a new instance of the CalculatorResponse class. This latter is then serialized to an XML stream within the orchestration.
AsyncCustomBtxMessageOrchestration: This is probably the most tricky and interesting test case. In fact, the helper component uses an XmlReader to process the data contained in the CalculatorRequest message and a uses VirtualStream and and XmlWriter to generate the response message. However, instead of returning a Stream object as the method invoked by the StreamOrchestration, this method creates and returns a brand new XLANGMessage. See below for more details.

Let’s review each test case in detail. Take into account that the synchronous and asynchronous version of each use case exploit exactly the same code, so in the code section I will refer only to the synchronous version.

XmlDocumentOrchestration

The following picture depicts the architecture of the XmlDocumentOrchestration test case.

Message Flow:

A WCF-BasicHttp or WCF-Custom Request-Response Receive Location receives a new CalculatorRequest xml document from the Test Agent/Client Application.
The XML disassembler component within the XMLTransmit pipeline promotes the Method element inside the CalculatorRequest xml document. The Message Agent submits the incoming message to the MessageBox (BizTalkMsgBoxDb).
The inbound request starts a new instance of the XmlDocumentOrchestration. This latter uses a Direct Port receive the CalculatorRequest messages with the Method promoted property = “XmlDocumentOrchestration”.
The XmlDocumentOrchestration invokes the ProcessRequestReturnXmlDocument method exposed by the RequestManager helper component. This object processes within a loop all the operations contained in the inbound XLANGMessage. In particular, the ProcessRequestReturnXmlDocument loads the entire message into an XmlDocument object and uses the XmlDocument.SelectNodes(XPathExpression) method to retrieve the operations contained in the inbound document.
The ProcessRequestReturnXmlDocument method returns the response message as an XmlDocument.
The XmlDocumentOrchestration publishes the CalculatorResponse message to the MessageBox (BizTalkMsgBoxDb).
The response message is retrieved by the Request-Response WCF-BasicHttp or WCF-Custom Receive Location.
The response message is returned to the Test Agent/Client Application.

Orchestration:

The following picture shows the structure of the test XmlDocumentOrchestration.

AsyncXmlDocumentOrchestration

The following picture depicts the architecture of the AsyncXmlDocumentOrchestration test case.

Message Flow:

A One-Way FILE Receive Location receives a new CalculatorRequest xml document from the Client Application or Loadgen.
The XML disassembler component within the XMLTransmit pipeline promotes the Method element inside the CalculatorRequest xml document. The Message Agent submits the incoming message to the MessageBox (BizTalkMsgBoxDb).
The inbound request starts a new instance of the AsyncXmlDocumentOrchestration. This latter uses a Direct Port receive the CalculatorRequest messages with the Method promoted property = “AsyncXmlDocumentOrchestration”.
The AsyncXmlDocumentOrchestration invokes the ProcessRequestReturnXmlDocument method exposed by the RequestManager helper component. This object processes within a loop all the operations contained in the inbound XLANGMessage. In particular, the ProcessRequestReturnXmlDocument loads the entire message into an XmlDocument object and uses the XmlDocument.SelectNodes(XPathExpression) method to retrieve the operations contained in the inbound document.
The ProcessRequestReturnXmlDocument method returns the response message as an XmlDocument.
The AsyncXmlDocumentOrchestration publishes the CalculatorResponse message to the MessageBox (BizTalkMsgBoxDb).
The response message is retrieved by a One-Way FILE Send Port.
The response message is written to an output folder by the One-Way FILE Send Port.

Orchestration:

The following picture shows the structure of the test AsyncXmlDocumentOrchestration.

Code

The XmlDocumentOrchestration receives a request message and returns a response document through a Request-Response Direct Bound Logical Port. The Filter Expression defined on the Activate Receive Shape is configured to receive all the CalculatorRequest messages which Method promoted property equals ‘XmlDocumentOrchestration’. The Expression Shape called BusinessLogic contains the following code:

logHelper.WriteLine("[XmlDocumentOrchestration] Request message received.");
xmlDocument = requestManager.ProcessRequestReturnXmlDocument(requestMessage);
logHelper.WriteLine("[XmlDocumentOrchestration] Request message successfully processed.");

while the Message Assignment Shape simply assigns the XmlDocument returned by the helper component to the response message.

responseMessage = xmlDocument;

This is probably the most common approach used by developers to create and assign an XML document to a new XLANGMessage. Let’s see the code of the method invoked by the XmlDocumentOrchestration:

ProcessRequestReturnXmlDocument Method

public XmlDocument ProcessRequestReturnXmlDocument(XLANGMessage message)
{
    XmlDocument responseDocument = new XmlDocument();
    List<Response> responseList = new List<Response>();
    string op = null;
    string status = Ok;
    string error = null;
    double operand1 = 0;
    double operand2 = 0;
    double value = 0;
    bool ok = true;
    bool succeeded = true;
    int i = 0;

    try
    {
        logHelper.WriteLine("[RequestManager][XmlDocumentOrchestration] Request message received.");
        XmlDocument xmlDocument = (XmlDocument)message[0].RetrieveAs(typeof(XmlDocument));
        if (xmlDocument != null)
        {
            XmlNodeList nodeList = xmlDocument.SelectNodes(OperationXPath);
            if (nodeList != null && nodeList.Count > 0)
            {
                bool parsed1 = true;
                bool parsed2 = true;
                for (i = 0; i < nodeList.Count; i++)
                {
                    if (nodeList[i].HasChildNodes)
                    {
                        succeeded = true;
                        error = None;
                        value = 0;
                        for (int j = 0; j < nodeList[i].ChildNodes.Count; j++)
                        {
                            switch (nodeList[i].ChildNodes[j].LocalName)
                            {
                                case "Operator":
                                    op = nodeList[i].ChildNodes[j].InnerText;
                                    break;
                                case "Operand1":
                                    parsed1 = double.TryParse(nodeList[i].ChildNodes[j].InnerText, out operand1);
                                    break;
                                case "Operand2":
                                    parsed2 = double.TryParse(nodeList[i].ChildNodes[j].InnerText, out operand2);
                                    break;
                            }
                        }
                        if (parsed1 && parsed2)
                        {
                            switch (op)
                            {
                                case "+":
                                    value = operand1 + operand2;
                                    break;
                                case "-":
                                    value = operand1 - operand2;
                                    break;
                                case "*":
                                    value = operand1 * operand2;
                                    break;
                                case "/":
                                    value = operand1 / operand2;
                                    break;
                                default:
                                    error = string.Format(OperationUnknownErrorMessageFormat, op);
                                    status = OperationsFailed;
                                    ok = false;
                                    succeeded = false;
                                    break;
                            }
                        }
                        else
                        {
                            succeeded = false;
                            ok = false;
                            status = OperationsFailed;
                            if (!parsed1)
                            {
                                error = string.Format(OperandIsNotANumberMessageFormat, 1, i + 1);
                            }
                            if (!parsed2)
                            {
                                if (parsed1)
                                {
                                    error = string.Format("{0}\r\n{1}", error, string.Format(OperandIsNotANumberMessageFormat, 2, i + 1));
                                }
                                else
                                {
                                    error = string.Format(OperandIsNotANumberMessageFormat, 2, i + 1);
                                }
                            }
                        }
                        if (succeeded)
                        {
                            logHelper.WriteLine(string.Format(OperationFormat, "XmlDocumentOrchestration", operand1, op, operand2, value));
                        }
                        else
                        {
                            logHelper.WriteLine(error);
                        }
                        responseList.Add(new Response(error, value));
                    }
                }
            }
        }
        StringBuilder builder = new StringBuilder();
        using (XmlWriter writer = XmlWriter.Create(builder))
        {
            writer.WriteStartDocument();
            writer.WriteStartElement("CalculatorResponse", CalculatorResponseNamespace);
            writer.WriteStartElement("Status", CalculatorResponseNamespace);
            writer.WriteString(status);
            writer.WriteEndElement();
            writer.WriteStartElement("Results", CalculatorResponseNamespace);
            for (i = 0; i < responseList.Count; i++)
            {
                writer.WriteStartElement("Result", CalculatorResponseNamespace);
                writer.WriteStartElement("Value", CalculatorResponseNamespace);
                writer.WriteString(responseList[i].Value.ToString());
                writer.WriteEndElement();
                writer.WriteStartElement("Error", CalculatorResponseNamespace);
                writer.WriteString(responseList[i].Error);
                writer.WriteEndElement();
                writer.WriteEndElement();
            }
            writer.WriteEndElement();
            writer.WriteEndElement();
        }
        string text = builder.ToString();
        responseDocument.LoadXml(text);
        if (ok)
        {
            logHelper.WriteLine("[RequestManager][XmlDocumentOrchestration] Response message successfully processed.");
        }
        else
        {
            logHelper.WriteLine("[RequestManager][XmlDocumentOrchestration] Request failed.");
        }
    }
    catch (Exception ex)
    {
        logHelper.WriteLine(ex.Message);
        responseDocument.LoadXml(string.Format(ErrorMessageFormat, ex.Message));
    }
    finally
    {
        message.Dispose();
    }
    return responseDocument;
}

Comment

As already highlighted above, using an XmlDocument to process the content of an XLANGMessage is extremely handy and powerful, but it can easily lead to high memory usage, especially when dealing with large messages.

StreamOrchestration

The following picture depicts the architecture of the StreamOrchestration test case.

Message Flow:

A WCF-BasicHttp or WCF-Custom Request-Response Receive Location receives a new CalculatorRequest xml document from the Test Agent/Client Application.
The XML disassembler component within the XMLTransmit pipeline promotes the Method element inside the CalculatorRequest xml document. The Message Agent submits the incoming message to the MessageBox (BizTalkMsgBoxDb).
The inbound request starts a new instance of the StreamOrchestration. This latter uses a Direct Port receive the CalculatorRequest messages with the Method promoted property = “StreamOrchestration”.
The StreamOrchestration invokes the ProcessRequestReturnStream method exposed by the RequestManager helper component. This object processes within a loop all the operations contained in the inbound XLANGMessage. In particular, the ProcessRequestReturnStream uses an XmlReader object to read and process the operations and an XmlWriter and a VirtualStream objects to produce the response message.
The ProcessRequestReturnStream method returns the response message as a Stream (VirtualStream) object.
The StreamOrchestration publishes the CalculatorResponse message to the MessageBox (BizTalkMsgBoxDb).
The response message is retrieved by the Request-Response WCF-BasicHttp or WCF-Custom Receive Location.
The response message is returned to the Test Agent/Client Application.

Orchestration:

The following picture shows the structure of the test StreamOrchestration.

AsyncStreamOrchestration

The following picture depicts the architecture of the AsyncStreamOrchestration test case.

Message Flow:

A One-Way FILE Receive Location receives a new CalculatorRequest xml document from the Client Application or Loadgen.
The XML disassembler component within the XMLTransmit pipeline promotes the Method element inside the CalculatorRequest xml document. The Message Agent submits the incoming message to the MessageBox (BizTalkMsgBoxDb).
The inbound request starts a new instance of the AsyncStreamOrchestration. This latter uses a Direct Port receive the CalculatorRequest messages with the Method promoted property = “AsyncStreamOrchestration”.
The AsyncStreamOrchestration invokes the ProcessRequestReturnStream method exposed by the RequestManager helper component. This object processes within a loop all the operations contained in the inbound XLANGMessage. In particular, the ProcessRequestReturnStream uses an XmlReader object to read and process the operations and an XmlWriter and a VirtualStream objects to produce the response message.
The ProcessRequestReturnStream method returns the response message as a Stream (VirtualStream) object.
The AsyncStreamOrchestration publishes the CalculatorResponse message to the MessageBox (BizTalkMsgBoxDb).
The response message is retrieved by a One-Way FILE Send Port.
The response message is written to an output folder by the One-Way FILE Send Port.

Orchestration:

The following picture shows the structure of the test AsyncStreamOrchestration.

Code

The StreamOrchestration receives a request message and returns a response document through a Request-Response Direct Bound Logical Port. The Filter Expression defined on the Activate Receive Shape is configured to receive all the CalculatorRequest messages which Method promoted property equals ‘StreamOrchestration’. The Expression Shape called BusinessLogic contains the following code:

logHelper.WriteLine("[StreamOrchestration] Request message received.");
stream = requestManager.ProcessRequestReturnStream(requestMessage);
logHelper.WriteLine("[StreamOrchestration] Request message successfully processed.");

As the name suggests, the ProcessRequestReturnStream method exposed by the RequestManager component receives the CalculatorRequest message as XLANGMessage input parameter and returns a Stream object containing the response document. In particular, the method adopts a streaming approach to process the incoming message and generate the response message:

It uses a VirtualStream and an XmlReader to read the content of the request message.
It uses a VirtualStream and an XmlWriter to generate the response message.

ProcessRequestReturnStream Method

public Stream ProcessRequestReturnStream(XLANGMessage message)
{
    VirtualStream stream = null;
    List<Response> responseList = new List<Response>();
    string op = null;
    string status = Ok;
    string error = null;
    double operand1 = 0;
    double operand2 = 0;
    double value = 0;
    bool ok = true;
    bool succeeded = true;
    int i = 0;

    try
    {
        logHelper.WriteLine("[RequestManager][StreamOrchestration] Request message received.");
        using (VirtualStream virtualStream = new VirtualStream(bufferSize, thresholdSize))
        {
            using (Stream partStream = (Stream)message[0].RetrieveAs(typeof(Stream)))
            {
                using (XmlReader reader = XmlReader.Create(partStream))
                {
                    while (reader.Read() && ok)
                    {
                        if (reader.LocalName == "Operator" &&
                            reader.NodeType == XmlNodeType.Element)
                        {
                            succeeded = true;
                            error = None;
                            value = 0;
                            op = reader.ReadElementContentAsString();
                            reader.MoveToContent();
                            operand1 = reader.ReadElementContentAsDouble();
                            reader.MoveToContent();
                            operand2 = reader.ReadElementContentAsDouble();
                            i++;
                            switch (op)
                            {
                                case "+":
                                    value = operand1 + operand2;
                                    break;
                                case "-":
                                    value = operand1 - operand2;
                                    break;
                                case "*":
                                    value = operand1 * operand2;
                                    break;
                                case "/":
                                    value = operand1 / operand2;
                                    break;
                                default:
                                    error = string.Format(OperationUnknownErrorMessageFormat, op);
                                    status = OperationsFailed;
                                    ok = false;
                                    succeeded = false;
                                    break;
                            }
                            if (succeeded)
                            {
                                logHelper.WriteLine(string.Format(OperationFormat, "StreamOrchestration", operand1, op, operand2, value));
                            }
                            else
                            {
                                logHelper.WriteLine(error);
                            }
                            responseList.Add(new Response(error, value));
                        }
                    }
                }
            }
        }
        if (ok)
        {
            stream = new VirtualStream(bufferSize, thresholdSize);
            using (XmlWriter writer = XmlWriter.Create(stream))
            {
                writer.WriteStartDocument();
                writer.WriteStartElement("CalculatorResponse", CalculatorResponseNamespace);
                writer.WriteStartElement("Status", CalculatorResponseNamespace);
                writer.WriteString(status);
                writer.WriteEndElement();
                writer.WriteStartElement("Results", CalculatorResponseNamespace);
                for (i = 0; i < responseList.Count; i++)
                {
                    writer.WriteStartElement("Result", CalculatorResponseNamespace);
                    writer.WriteStartElement("Value", CalculatorResponseNamespace);
                    writer.WriteString(responseList[i].Value.ToString());
                    writer.WriteEndElement();
                    writer.WriteStartElement("Error", CalculatorResponseNamespace);
                    writer.WriteString(responseList[i].Error);
                    writer.WriteEndElement();
                    writer.WriteEndElement();
                }
                writer.WriteEndElement();
                writer.WriteEndElement();
            }
            stream.Seek(0, SeekOrigin.Begin);
            logHelper.WriteLine("[RequestManager][StreamOrchestration] Response message successfully processed.");
        }
    }
    catch (Exception ex)
    {
        logHelper.WriteLine(string.Format("[RequestManager][StreamOrchestration] {0}", ex.Message));
        stream = new VirtualStream(bufferSize, thresholdSize);
        using (XmlWriter writer = XmlWriter.Create(stream))
        {
            writer.WriteStartDocument();
            writer.WriteStartElement("CalculatorResponse", CalculatorResponseNamespace);
            writer.WriteStartElement("Status", CalculatorResponseNamespace);
            writer.WriteString(ex.Message);
            writer.WriteEndElement();
            writer.WriteEndElement();
        }
        stream.Seek(0, SeekOrigin.Begin);
    }
    finally
    {
        message.Dispose();
    }
    return stream;
}

VirtualStream and ReadOnlySeekableStream

The ReadOnlySeekableStream can be used to wrap and read the content of a non-seekable stream in those cases where after reading it’s necessary to reposition the cursor at the beginning of the stream. While reading the content of the wrapped stream, the ReadOnlySeekableStream copies its content to a temporary file created in the folder specified by the TMP and/or TEMP environment variables. However, the class exposes a public constructor which allows to specify a different persistence stream as alterative to the FileStream (e.g. VirtualStream, MemoryStream):

public ReadOnlySeekableStream (Stream source, Stream persist, int maxBuffer)

The VirtualStream in its turn is very useful when dealing with large messages within a custom pipeline component or a helper component invoked by an orchestration. In fact, if the document size is bigger than a certain threshold, the default is 1MB, the message is persisted to a temporary file. This file is created inside a folder identified by the TEMP environment variable of the service account used to run the current host instance. So, when parsing, mapping and dealing in general with large messages, it’s a good practice to move the location of the temporary folder for the BizTalk service account to a dedicated local disk separate from the volume hosting the Windows OS. The VirtualStream class exposes a particular constructor (see below) which enables to specify the size of the internal buffer and the size of this threshold, so you can set the best value based on a given scenario. It’s a good practice to expose these 2 variables, the buffer size and the threshold size as properties of custom pipeline components or helper components.

public VirtualStream (int bufferSize, int thresholdSize)

The use of the VirtualStream and ReadOnlySeekableStream (both classes are contained in the Microsoft.BizTalk.Streaming.dll assembly) can be combined to provide both “seekability” and “overflow to file system” capabilities to custom pipeline components and helper classes. This accommodates the processing of large messages without loading the entire message into memory. The following code could be used in a pipeline component to implement this functionality.

int bufferSize = 0x280;
int thresholdSize = 0x100000;
Stream vStream = new VirtualStream(bufferSize, thresholdSize);
Stream seekStream = new ReadOnlySeekableStream(inboundStream, vStream, bufferSize);

As you can see, the VirtualStream is used as persistence storage for the ReadOnlySeekableStream. In this way, if the length of the original stream is lower the threshold size, its content will be persisted to a MemoryStream, otherwise it will be written to a temporary file.
As a rule of thumb, if the original stream is seekable, there's no need to use the ReadOnlySeekableStream and you can use just the VirtualStream. However, some of the streams used by the Messaging Engine are forward-only, non-seekable streams, so when it's necessary to read the content of a message within a custom pipeline component and reposition the cursor at the beginning of the stream at the end, it's a good practice combining the use of the VirtualStream and ReadOnlySeekableStream as shown by the code snippet above. See Optimizing Pipeline Performance for more information on this topic.

Response Message Assignment

To assign the content of the stream returned by the ProcessRequestReturnStream to the response message, the Message Assignment Shape (see the code below) invokes the SetResponse method exposed by another helper component called ResponseManager.

responseMessage = null;
responseManager.SetResponse(responseMessage, stream);

This SetResponse method uses the LoadFrom method exposed by the XLANGPart class to assign the content of the stream to the message part of the response document.

public void SetResponse(XLANGMessage message, Stream stream)
{
    try
    {
        if (stream != null &&
            message != null &&
            message.Count > 0)
        {
            if (stream.CanSeek)
            {
                stream.Seek(0, SeekOrigin.Begin);
            }
            message[0].LoadFrom(stream);
        }
    }
    catch (Exception ex)
    {
        Debug.WriteLine(string.Format("[ResponseManager] {0}", ex.Message));
    }
    finally
    {
        message.Dispose();
    }
}

Comment

This streaming approach used by StreamOrchestration is extremely useful and performant especially when dealing with large messages.

MessageClassesOrchestration

The following picture depicts the architecture of the MessageClassesOrchestration test case.

Message Flow:

A WCF-BasicHttp or WCF-Custom Request-Response Receive Location receives a new CalculatorRequest xml document from the Test Agent/Client Application.
The XML disassembler component within the XMLTransmit pipeline promotes the Method element inside the CalculatorRequest xml document. The Message Agent submits the incoming message to the MessageBox (BizTalkMsgBoxDb).
The inbound request starts a new instance of the MessageClassesOrchestration. This latter uses a Direct Port receive the CalculatorRequest messages with the Method promoted property = “MessageClassesOrchestration”.
The MessageClassesOrchestration invokes the ProcessRequestReturnObject method exposed by the RequestManager helper component. This object processes within a loop all the operations contained in the inbound XLANGMessage. In particular, the ProcessRequestReturnObject deserializes the CalculatorRequest message into a new instance of the CalculatorRequest .NET class, processes al the operations contained in this object and accumulates the results inside an instance of the CalculatorResponse .NET class.
The ProcessRequestReturnObject method returns an instance of the CalculatorResponse .NET class.
The MessageClassesOrchestration publishes the CalculatorResponse message to the MessageBox (BizTalkMsgBoxDb).
The response message is retrieved by the Request-Response WCF-BasicHttp or WCF-Custom Receive Location.
The response message is returned to the Test Agent/Client Application.

Orchestration:

The following picture shows the structure of the test MessageClassesOrchestration.

AsyncMessageClassesOrchestration

The following picture depicts the architecture of the AsyncMessageClassesOrchestration test case.

Message Flow:

A One-Way FILE Receive Location receives a new CalculatorRequest xml document from the Client Application or Loadgen.
The XML disassembler component within the XMLTransmit pipeline promotes the Method element inside the CalculatorRequest xml document. The Message Agent submits the incoming message to the MessageBox (BizTalkMsgBoxDb).
The inbound request starts a new instance of the AsyncMessageClassesOrchestration. This latter uses a Direct Port receive the CalculatorRequest messages with the Method promoted property = “AsyncMessageClassesOrchestration”.
The AsyncMessageClassesOrchestration invokes the ProcessRequestReturnObject method exposed by the RequestManager helper component. This object processes within a loop all the operations contained in the inbound XLANGMessage. In particular, the ProcessRequestReturnObject deserializes the CalculatorRequest message into a new instance of the CalculatorRequest .NET class, processes al the operations contained in this object and accumulates the results inside an instance of the CalculatorResponse .NET class.
The ProcessRequestReturnObject method returns an instance of the CalculatorResponse .NET class.
The AsyncMessageClassesOrchestration publishes the CalculatorResponse message to the MessageBox (BizTalkMsgBoxDb).
The response message is retrieved by a One-Way FILE Send Port.
The response message is written to an output folder by the One-Way FILE Send Port.

Orchestration:

The following picture shows the structure of the test AsyncMessageClassesOrchestration.

Code

The MessageClassesOrchestration receives a request message and returns a response document through a Request-Response Direct Bound Logical Port. The Filter Expression defined on the Activate Receive Shape is configured to receive all the CalculatorRequest messages which Method promoted property equals ‘MessageClassesOrchestration’. The Expression Shape called BusinessLogic contains the following code:

logHelper.WriteLine("[MessageClassesOrchestration] Request message received.");
calculatorResponse = requestManager.ProcessRequestReturnObject(requestMessage);
logHelper.WriteLine("[MessageClassesOrchestration] Request message successfully processed.");

The ProcessRequestReturnObject method invoked by the MessageClassesOrchestration expects an XLANGMessage object as inbound parameter. The code of the method retrieves the content of the message part as instance of the CalculatorRequest class. In this case the RetrieveAs method exposed by the XLANGPart class deserializes the content of the message part and creates an instance of the aforementioned class.

ProcessRequestReturnObject Method

public CalculatorResponse ProcessRequestReturnObject(XLANGMessage message)
{
    CalculatorResponse response = new CalculatorResponse();
    Operation operation = null;
    string error = null;
    double value = 0;
    bool ok = true;
    bool succeeded = true;

    try
    {
        logHelper.WriteLine("[RequestManager][MessageClassesOrchestration] Request message received.");
        CalculatorRequest request = (CalculatorRequest)message[0].RetrieveAs(typeof(CalculatorRequest));
        if (request != null &&
            request.Operations != null &&
            request.Operations.Count > 0)
        {
            response.Status = Ok;
            for (int i = 0; i < request.Operations.Count; i++)
            {
                operation = (Operation)request.Operations[i];
                error = None;
                value = 0;
                succeeded = true;
                switch (operation.Operator)
                {
                    case "+":
                        value = operation.Operand1 + operation.Operand2;
                        break;
                    case "-":
                        value = operation.Operand1 - operation.Operand2;
                        break;
                    case "*":
                        value = operation.Operand1 * operation.Operand2;
                        break;
                    case "/":
                        value = operation.Operand1 / operation.Operand2;
                        break;
                    default:
                        error = string.Format(OperationUnknownErrorMessageFormat, operation.Operator);
                        response.Status = OperationsFailed;
                        ok = false;
                        succeeded = false;
                        break;
                }
                if (succeeded)
                {
                    logHelper.WriteLine(string.Format(OperationFormat, "MessageClassesOrchestration", operation.Operand1, operation.Operator, operation.Operand2, value));
                }
                else
                {
                    logHelper.WriteLine(error);
                }
                response.Results.Add(new Result(value, error));
            }
        }
        else
        {
            response.Status = RequestDoesNotContainAnyOperationsMessage;
        }
        if (ok)
        {
            logHelper.WriteLine("[RequestManager][MessageClassesOrchestration] Response message successfully processed.");
        }
        else
        {
            logHelper.WriteLine("[RequestManager][MessageClassesOrchestration] Request failed.");
        }
    }
    catch (Exception ex)
    {
        logHelper.WriteLine(string.Format("[RequestManager][MessageClassesOrchestration] {0}", ex.Message));
        response.Status = string.Format("[RequestManager][MessageClassesOrchestration] {0}", ex.Message);
        response.Results = null;
    }
    finally
    {
        message.Dispose();
    }
    return response;
}

In this case the RetrieveAs method exposed by the XLANGPart class deserializes the content of the message part and creates an instance of the CalculatorRequest class. The code for both the CalculatorRequest and CalculatorResponse classes was obtained running the xsd.exe tool to build XML serializable objects for each schema.

xsd /c /n:"Microsoft.BizTalk.CAT.Samples.HandleXLANGMessages.BusinessLogic" "C:\Projects\HandleXLANGMessages\Schemas\CalculatorRequest.xsd"

Indeed, I customized the code returned by the xsd.exe tool to replace arrays with Lists, but anyway… not really a big deal!

CalculatorRequest Class

namespace Microsoft.BizTalk.CAT.Samples.HandleXLANGMessages.BusinessLogic
{
    [Serializable]
    [XmlType(AnonymousType = true, Namespace = "http://Microsoft.BizTalk.CAT.Samples.HandleXLANGMessages.Schemas.CalculatorRequest")]
    [XmlRoot(Namespace = "http://Microsoft.BizTalk.CAT.Samples.HandleXLANGMessages.Schemas.CalculatorRequest", IsNullable = false)]
    public partial class CalculatorRequest
    {
        #region Private Fields
        private string method;
        private List<Operation> operations = new List<Operation>();
        #endregion

        #region Public Properties
        public string Method
        {
            get
            {
                return this.method;
            }
            set
            {
                this.method = value;
            }
        }

        [XmlArrayItem("Operation", Type=typeof(Operation), IsNullable = false)]
        public List<Operation> Operations
        {
            get
            {
                return this.operations;
            }
            set
            {
                this.operations = value;
            }
        } 
        #endregion
    }

    [Serializable]
    [XmlType(AnonymousType = true, Namespace = "http://Microsoft.BizTalk.CAT.Samples.HandleXLANGMessages.Schemas.CalculatorRequest")]
    public partial class Operation
    {
        #region Private Fields
        private string op;
        private double operand1;
        private double operand2;
        #endregion

        #region Public Constructors
        public Operation()
        {
        }
        #endregion

        #region Public Properties
        public string Operator
        {
            get
            {
                return this.op;
            }
            set
            {
                this.op = value;
            }
        }

        public double Operand1
        {
            get
            {
                return this.operand1;
            }
            set
            {
                this.operand1 = value;
            }
        }

        public double Operand2
        {
            get
            {
                return this.operand2;
            }
            set
            {
                this.operand2 = value;
            }
        } 
        #endregion
    }
}

CalculatorResponse Class

namespace Microsoft.BizTalk.CAT.Samples.HandleXLANGMessages.BusinessLogic
{
    [Serializable]
    [XmlType(AnonymousType=true, Namespace="http://Microsoft.BizTalk.CAT.Samples.HandleXLANGMessages.Schemas.CalculatorResponse")]
    [XmlRoot(Namespace="http://Microsoft.BizTalk.CAT.Samples.HandleXLANGMessages.Schemas.CalculatorResponse", IsNullable=false)]
    public partial class CalculatorResponse
    {
        #region Private Fields
        private string status;
        private List<Result> results = new List<Result>();
        #endregion

        #region Public Properties
        public string Status 
        {
            get 
            {
                return this.status;
            }
            set 
            {
                this.status = value;
            }
        }
        
        [XmlArrayItem("Result", Type=typeof(Result), IsNullable=false)]
        public List<Result> Results 
        {
            get 
            {
                return this.results;
            }
            set 
            {
                this.results = value;
            }
        }
        #endregion
    }

    [Serializable]
    [XmlType(AnonymousType=true, Namespace="http://Microsoft.BizTalk.CAT.Samples.HandleXLANGMessages.Schemas.CalculatorResponse")]
    public partial class Result
    {
        #region Private Fields
        private double value;
        private string error; 
        #endregion

        #region Public Constructors
        public Result()
        {
            this.value = default(double);
            this.error = default(string);
        }

        public Result(double value, string error)
        {
            this.value = value;
            this.error = error;
        }
        #endregion

        #region Public Properties        
        public double Value
        {
            get
            {
                return this.value;
            }
            set
            {
                this.value = value;
            }
        }
        
        public string Error
        {
            get
            {
                return this.error;
            }
            set
            {
                this.error = value;
            }
        } 
        #endregion
    }
}

Response Message Assignment

To assign the content of the CalculatorResponse object returned by the ProcessRequestReturnObject to the response message, the Message Assignment Shape (see the code below) just assign the calculatorResponse object to the message. The XLANG Engine accepts and interprets this particular syntax as a call to the LoadFrom method exposed by the XLANGPart which in its turn serializes the object instance into an XML stream.

responseMessage = calculatorResponse;

Comment

Using the xsd.exe tool you can create a .NET class for each XML schema used by the solution. This approach allows to deserialize an inbound message into an instance of a custom entity class or to to serialize an object to create an XML instance. This technique allows developers to use an object oriented approach when dealing with messages as they can exploit the properties and methods exposed by classes to access and manipulate data instead of using XPath expressions. This approach is valid until the size of messages exchanged and processed by a BizTalk Application is relatively small. As the message size increases, the cost in terms of CPU and Memory usage for deserializing xml messages into objects and serializing objects into xml messages can grow significantly and this technique loses effectiveness.

CustomBtxMessageOrchestration

The following picture depicts the architecture of the CustomBtxMessageOrchestration test case.

Message Flow:

A WCF-BasicHttp or WCF-Custom Request-Response Receive Location receives a new CalculatorRequest xml document from the Test Agent/Client Application.
The XML disassembler component within the XMLTransmit pipeline promotes the Method element inside the CalculatorRequest xml document. The Message Agent submits the incoming message to the MessageBox (BizTalkMsgBoxDb).
The inbound request starts a new instance of the CustomBtxMessageOrchestration. This latter uses a Direct Port receive the CalculatorRequest messages with the Method promoted property = “CustomBtxMessageOrchestration”.
The CustomBtxMessageOrchestration invokes the ProcessRequestReturnXLANGMessage method exposed by the RequestManager helper component. This object processes within a loop all the operations contained in the inbound XLANGMessage. In particular, the ProcessRequestReturnXLANGMessage uses an XmlReader object to read and process the operations and an XmlWriter and a VirtualStream objects to produce the response message.
The ProcessRequestReturnXLANGMessage method returns a new XLANGMessage.
The CustomBtxMessageOrchestration publishes the CalculatorResponse message to the MessageBox (BizTalkMsgBoxDb).
The response message is retrieved by the Request-Response WCF-BasicHttp or WCF-Custom Receive Location.
The response message is returned to the Test Agent/Client Application.

Orchestration:

The following picture shows the structure of the test CustomBtxMessageOrchestration .

AsyncCustomBtxMessageOrchestration

The following picture depicts the architecture of the AsyncCustomBtxMessageOrchestration test case.

Message Flow:

A One-Way FILE Receive Location receives a new CalculatorRequest xml document from the Client Application or Loadgen.
The XML disassembler component within the XMLTransmit pipeline promotes the Method element inside the CalculatorRequest xml document. The Message Agent submits the incoming message to the MessageBox (BizTalkMsgBoxDb).
The inbound request starts a new instance of the AsyncCustomBtxMessageOrchestration. This latter uses a Direct Port receive the CalculatorRequest messages with the Method promoted property = “AsyncCustomBtxMessageOrchestration”.
The AsyncCustomBtxMessageOrchestration invokes the ProcessRequestReturnXLANGMessage method exposed by the RequestManager helper component. This object processes within a loop all the operations contained in the inbound XLANGMessage. In particular, the ProcessRequestReturnXLANGMessage uses an XmlReader object to read and process the operations and an XmlWriter and a VirtualStream objects to produce the response message.
The ProcessRequestReturnXLANGMessage method returns a new XLANGMessage.
The AsyncCustomBtxMessageOrchestration publishes the CalculatorResponse message to the MessageBox (BizTalkMsgBoxDb).
The response message is retrieved by a One-Way FILE Send Port.
The response message is written to an output folder by the One-Way FILE Send Port.

Orchestration:

The following picture shows the structure of the test AsyncCustomBtxMessageOrchestration.

Code

The CustomBtxMessageOrchestration receives a request message and returns a response document through a Request-Response Direct Bound Logical Port. The Filter Expression defined on the Activate Receive Shape is configured to receive all the CalculatorRequest messages which Method promoted property equals ‘CustomBtxMessageOrchestration ’. The Expression Shape called BusinessLogic contains the following code:

logHelper.WriteLine("[CustomBtxMessageOrchestration] Request message received.");
responseMessage = requestManager.ProcessRequestReturnXLANGMessage(requestMessage);
logHelper.WriteLine("[CustomBtxMessageOrchestration] Request message successfully processed.");

As the name suggests, the ProcessRequestReturnXLANGMessage method exposed by the RequestManager component receives the CalculatorRequest message as XLANGMessage input parameter and returns a XLANGMessage object containing the response document. But…wait a minute, the XLANGMessage class do not exposes any public constructor. So how can we create a brand new XLANGMessage within an helper component? Well, there’s a trick underneath and we’ll see the details in a moment. The ProcessRequestReturnXLANGMessage method adopts the same streaming approach and mostly the same code used by the ProcessRequestReturnStream method to process the incoming message and generate the response message:

It uses a VirtualStream and an XmlReader to read the content of the request message.
It uses a VirtualStream and an XmlWriter to generate the response message.

However, instead of returning a Stream it returns an XLANGMessage… or better… it returns an instance of the CustomBTXMessage class which inherits from the BTXMessage contained in the Microsoft.XLANGs.BizTalk.Engine assembly. Therefore, the XLANGMessage object returned by the ProcessRequestReturnXLANGMessage method can be directly assigned to the response message in the Expression Shape without the need to introduce a separate Message Assignment Shape in the orchestration.

The XLANGMessage indeed is an abstract class so whenever an orchestration invokes a method exposed by a business component and passes an XLANGMessage as parameter, the real type of the object is MessageWrapperForUserCode. Now the trick to create and return an XLANGMessage object as result of a method call is to create a custom class which inherits from a BizTalk-provided class that in its turn inherits from the XLANGMessage. So i chose to create a custom class called CustomBTXMessage that inherits from the BTXMessage class. See the code below:

CustomBTXMessage Class

namespace Microsoft.BizTalk.CAT.Samples.HandleXLANGMessages.Utilities
{
    [Serializable]
    public sealed class CustomBTXMessage : BTXMessage
    {
        public CustomBTXMessage(string messageName, Context context)
            : base(messageName, context)
        {
            context.RefMessage(this);
        }
    }
}

Said that, the code of the ProcessRequestReturnXLANGMessage looks as follows:

ProcessRequestReturnXLANGMessage Method

public XLANGMessage ProcessRequestReturnXLANGMessage(XLANGMessage requestMessage)
{
    CustomBTXMessage customBTXMessage = null;
    XLANGMessage responseMessage = null;
    VirtualStream stream = null;
    List<Response> responseList = new List<Response>();
    string op = null;
    string status = Ok;
    string error = null;
    double operand1 = 0;
    double operand2 = 0;
    double value = 0;
    bool ok = true;
    bool succeeded = true;
    int i = 0;

    try
    {
        logHelper.WriteLine("[RequestManager][CustomBtxMessageOrchestration] Request message received.");
        using (XmlReader reader = (XmlReader)requestMessage[0].RetrieveAs(typeof(XmlReader)))
        {
            while (reader.Read() && ok)
            {
                if (reader.LocalName == "Operator" &&
                    reader.NodeType == XmlNodeType.Element)
                {
                    error = None;
                    value = 0;
                    succeeded = true;
                    op = reader.ReadElementContentAsString();
                    reader.MoveToContent();
                    operand1 = reader.ReadElementContentAsDouble();
                    reader.MoveToContent();
                    operand2 = reader.ReadElementContentAsDouble();
                    i++;
                    switch (op)
                    {
                        case "+":
                            value = operand1 + operand2;
                            break;
                        case "-":
                            value = operand1 - operand2;
                            break;
                        case "*":
                            value = operand1 * operand2;
                            break;
                        case "/":
                            value = operand1 / operand2;
                            break;
                        default:
                            error = string.Format(OperationUnknownErrorMessageFormat, op);
                            status = OperationsFailed;
                            ok = false;
                            succeeded = false;
                            break;
                    }
                    if (succeeded)
                    {
                        logHelper.WriteLine(string.Format(OperationFormat, "CustomBtxMessageOrchestration", operand1, op, operand2, value));
                    }
                    else
                    {
                        logHelper.WriteLine(error);
                    }
                    responseList.Add(new Response(error, value));
                }
            }
        }
        stream = new VirtualStream(bufferSize, thresholdSize);
        using (XmlWriter writer = XmlWriter.Create(stream))
        {
            writer.WriteStartDocument();
            writer.WriteStartElement("CalculatorResponse", CalculatorResponseNamespace);
            writer.WriteStartElement("Status", CalculatorResponseNamespace);
            writer.WriteString(status);
            writer.WriteEndElement();
            writer.WriteStartElement("Results", CalculatorResponseNamespace);
            for (i = 0; i < responseList.Count; i++)
            {
                writer.WriteStartElement("Result", CalculatorResponseNamespace);
                writer.WriteStartElement("Value", CalculatorResponseNamespace);
                writer.WriteString(responseList[i].Value.ToString());
                writer.WriteEndElement();
                writer.WriteStartElement("Error", CalculatorResponseNamespace);
                writer.WriteString(responseList[i].Error);
                writer.WriteEndElement();
                writer.WriteEndElement();
            }
            writer.WriteEndElement();
            writer.WriteEndElement();
        }
        stream.Seek(0, SeekOrigin.Begin);
        if (ok)
        {
            logHelper.WriteLine("[RequestManager][CustomBtxMessageOrchestration] Response message successfully processed.");
        }
        else
        {
            logHelper.WriteLine("[RequestManager][CustomBtxMessageOrchestration] Request failed.");
        }

    }
    catch (Exception ex)
    {
        logHelper.WriteLine(string.Format("[RequestManager][CustomBtxMessageOrchestration] {0}", ex.Message));
        stream = new VirtualStream(bufferSize, thresholdSize);
        using (XmlWriter writer = XmlWriter.Create(stream))
        {
            writer.WriteStartDocument();
            writer.WriteStartElement("CalculatorResponse", CalculatorResponseNamespace);
            writer.WriteStartElement("Status", CalculatorResponseNamespace);
            writer.WriteString(ex.Message);
            writer.WriteEndElement();
            writer.WriteEndElement();
        }
        stream.Seek(0, SeekOrigin.Begin);
    }
    finally
    {
        customBTXMessage = new CustomBTXMessage("CalculatorResponse", Service.RootService.XlangStore.OwningContext);
        customBTXMessage.AddPart(string.Empty, "Body");
        customBTXMessage[0].LoadFrom(stream);
        responseMessage = customBTXMessage.GetMessageWrapperForUserCode();
        if (requestMessage != null)
        {
            requestMessage.Dispose();
        }
    }
    return responseMessage; 
}

Comment

This technique is tricky and combines the performance of the streaming approach with the ability to directly return an XLANGMessage. In the second part of the article, I will present the results of the performance tests I conducted to measure and compare the latency and throughput of the design patterns discussed in this post.

Code

You can find the code for BizTalk Server 2009 here, while at the following link you can download the Visio document I used to create my diagrams! Please, let me know your feedbacks! Enjoy! :-)

BizTalk Server 2009 Performance on Hyper-V and Physical Deployments

leprino — Mon, 31 Aug 2009 16:37:15 GMT

My friend and colleague Ewan Fairweather will deliver a presentation during which he will cover all the performance testing results he/we obtained in the lab this year on physical and Hyper-V systems. If you would like to understand the performance differences of BizTalk Server 2009 on Microsoft SQL Server 2008, Windows Server 2008, and Hyper-V platforms, this webcast is for you.

Register for the upcoming TechNet Webcast: BizTalk Server 2009 Performance on Hyper-V and Physical Deployments (Level 300)

Language(s): English.
Product(s): Microsoft BizTalk Server.
Audience(s): IT Generalist.
Duration: 60 Minutes
Start Date: Tuesday, September 01, 2009 1:00 PM Pacific Time (US & Canada)

Event Overview

As part of the Microsoft BizTalk Server 2009 launch wave, the BizTalk Rangers and Technology Adoption Program (TAP) teams are conducting a performance lab to compare Microsoft BizTalk Server 2006 R2, BizTalk Server 2009 (physical), and BizTalk Server 2009 on a Hyper-V platform. In this webcast, you hear from the product group about the performance characteristics of solutions deployed on different platforms. We outline the tests that we have performed and the results that we obtained, and we share common best practices for deployment. If you would like to understand the performance differences of BizTalk Server 2009 on Microsoft SQL Server 2008, Windows Server 2008, and Hyper-V platforms before the official guides are published, then this webcast is for you.

Presenter: Ewan Fairweather, Program Manager, Microsoft Corporation

Ewan Fairweather has worked for Microsoft for four years. He currently works as a program manager in the Microsoft BizTalk Server product group on the Customer Advisory Team (CAT). Prior to this, Ewan spent over three years working for Microsoft U.K. on the Premier Field Engineering team where he worked with enterprise customers, helping them to maintain and optimize their BizTalk applications. This involved providing both proactive and reactive on-site assistance within the U.K. and the rest of Europe. Ewan has also worked in a dedicated capacity on some of the world's largest BizTalk deployments, predominantly within financial services.

Ewan coauthored the successful Professional BizTalk Server 2006 (Wrox, 2007) and has written many white papers for Microsoft including the "Microsoft BizTalk Server Performance Optimization Guide," which is available on the Microsoft Developers Network (MSDN) Web site.

BizTalk Server 2009 – End to End Performance Testing

leprino — Fri, 17 Jul 2009 16:06:21 GMT

At the end of April 2009, I was invited by Microsoft Sweden for the Launch Event of BizTalk Server 2009. During this event I gave a speech about BizTalk Server 2009 End to End Performance Testing. In this session I introduced the methodology and process that the BizTalk Customer Advisory Team follows to achieve consistent results during Performance Labs. The entire process along with the suggested optimizations will be soon delivered in the BizTalk Server 2009 Optimization Guide. In the meantime, you can watch the presentation I took on Channel9:

SOA Patterns with BizTalk Server 2009 by Richard Seroter

leprino — Thu, 04 Jun 2009 10:37:53 GMT

Recently I had the chance to review an excellent book called SOA Patterns with BizTalk Server 2009 by Richard Seroter who maintains a popular and appreciated blog on BizTalk. The book discusses the core principles of SOA and shows how to effectively apply them in a BizTalk solution. Richard provides a good insight in WCF Adapters and explains with clear samples how to exploit their functionalities to implement synchronous and asynchronous communication patterns. The author makes a superb introduction of technologies such as the new WCF SQL Adapter, UDDI and the ESB Guidance 2.0. SOA Patterns with BizTalk Server 2009 is definitely a must have book for all BizTalk developers, especially for those who intend to design and implement a complex SOA platform using BizTalk Server 2009. You can download for free the Chapter 9: New SOA Capabilities in BizTalk Server 2009: WCF SQL Server Adapter from the Packt website. This chapter covers the following topics:

Executing composite transactions
Polling for data
Using SQL Server Query notification
Consuming the adapter from outside BizTalk Server

I sincerely suggest this book to all BizTalkers. ;-)

How to Throw Typed Fault Exceptions from Orchestrations Published as WCF Services

leprino — Fri, 22 May 2009 19:29:22 GMT

In general, a WCF Web Service can return two types of SOAP faults: typed and untyped SOAP faults.

Typed Faults

In order to throw a typed fault, a service operation must be decorated with a System.ServiceModel.FaultContractAttribute that specifies a fault data contract defined earlier. The following code snippet shows a WCF web service called HelloWorld which implements the IHelloWorld service or contract interface. This interface exposes a single synchronous operation called SayHello that is decorated with the FaultContractAttribute. The attribute declares that the SayHello method can throw a typed fault exception defined by the CustomError data contract class. The implementation of the SayHello method in the HelloWorld class checks if the incoming request is null and in this case it throws an error of type FaultException<CustomError>.

namespace Microsoft.BizTalk.CAT.Samples.PublishTypedFaults.Services
{
    [Serializable]
    [DataContract(Name = "HelloWorldRequest", Namespace = "http://microsoft.biztalk.cat/10/helloworld")]
    public class HelloWorldRequest
    {
       ...
    }

    [Serializable]
    [DataContract(Name = "HelloWorldResponse", Namespace = "http://microsoft.biztalk.cat/10/helloworld")]
    public class HelloWorldResponse
    {
        ...
    }

    [Serializable]
    [DataContract(Name = "CustomError", Namespace = "http://microsoft.biztalk.cat/10/customerror")]
    public class CustomError
    {
        ...
    }

    [ServiceContract(Name = "HelloWorld", Namespace = "http://microsoft.biztalk.cat/10/helloworld")]
    public interface IHelloWorld
    {
        [OperationContract(Action = "SayHello", ReplyAction = "SayHello", ProtectionLevel = ProtectionLevel.None)]
        [FaultContract(typeof(CustomError), Action = "CustomErrorFault")]
        HelloWorldResponse SayHello(HelloWorldRequest request);
    }

    [ServiceBehavior(Name = "HelloWorld", Namespace = "http://microsoft.biztalk.cat/10/helloworld")]
    public class HelloWorld : IHelloWorld
    {
        [OperationBehavior]
        public HelloWorldResponse SayHello(HelloWorldRequest request)
        {
            if (request == null || string.IsNullOrEmpty(request.Name))
            {
                throw CreateFault(NameCannotBeNullOrEmpty);
            }
            return new HelloWorldResponse(string.Format(SayHelloFormat, request.Name));
        } 

         private FaultException<CustomError> CreateFault(string message)
        {
             ...
            return fault;
        }
    }
}

Untyped Faults

Untyped SOAP faults are those that do not require a service operation to be decorated with a FaultContractAttribute that specify a custom data contract class.

BizTalk Orchestrations and Typed Faults

BizTalk Server 2006 R2 and BizTalk Server 2009 allow handling typed fault contracts when consuming WCF services from within orchestrations. The steps necessary to implement this technique are clearly described in the article “How to Handle Typed Fault Contracts in Orchestrations” on MSDN. Instead, WCF adapters actually do not support processing typed fault contract exceptions within orchestrations published as WCF services. However, untyped SOAP faults can always be returned by orchestrations or pipelines. For more information on this topic review the article “How to Throw Fault Exceptions from Orchestrations Published as WCF Services” on MSDN.

This constraint arises from how the Receive Handler of WCF Adapters is actually implemented. The WCF Receive Adapter instantiates a singleton instance of the BizTalkServiceInstance class for each WCF Receive Location. This class can be found in the Microsoft.BizTalk.Adapter.Wcf.Runtime.dll assembly. In particular, as you can see in the picture below, the BizTalkServiceInstance class is decorated with the attribute ServiceBehavior(InstanceContextMode=InstanceContextMode.Single, ConcurrencyMode=ConcurrencyMode.Multiple).

Therefore, all the incoming requests towards a given WCF Receive Location are are managed by the same singleton object. When you enable a WCF Receive Location, the Adapter initializes and opens a dedicated instance of a ServiceHost-derived class (WebServiceHost), which dynamically builds the WCF runtime components within the in-process or isolated host process. This includes the channel stack, dispatcher, and generic service instance. Almost all of the WCF adapters can be hosted within the BizTalk service process itself – the only exception is WCF-CustomIsolated, which must be used in a BizTalk isolated host by design. Even the HTTP adapters can be hosted in-process now. The WCF Adapters build the generic service contracts shown in the table below. Each service contract implemented by the BizTalkServiceInstance covers a different scenario.

[ServiceContract(Namespace = "http://www.microsoft.com/biztalk/2006/r2/wcf-adapter")]
public interface IOneWayAsync
{
    // Methods
    [OperationContract(AsyncPattern = true, IsOneWay = true, Action = "*")]
    IAsyncResult BeginOneWayMethod(Message message, AsyncCallback callback, object state);
    [OperationContract(IsOneWay = true, Action = "BizTalkSubmit")]
    void BizTalkSubmit(Message message);
    void EndOneWayMethod(IAsyncResult result);
}

[ServiceContract(Namespace = "http://www.microsoft.com/biztalk/2006/r2/wcf-adapter")]
public interface IOneWayAsyncTxn
{
    // Methods
    [OperationContract(AsyncPattern = true, IsOneWay = true, Action = "*")]
    IAsyncResult BeginOneWayMethod(Message message, AsyncCallback callback, object state);
    [OperationContract(IsOneWay = true, Action = "BizTalkSubmit")]
    void BizTalkSubmit(Message message);
    void EndOneWayMethod(IAsyncResult result);
}

[ServiceContract(Namespace = "http://www.microsoft.com/biztalk/2006/r2/wcf-adapter")]
public interface ITwoWayAsync
{
    // Methods
    [OperationContract(AsyncPattern = true, IsOneWay = false, Action = "*", ReplyAction = "*")]
    IAsyncResult BeginTwoWayMethod(Message message, AsyncCallback callback, object state);
    [OperationContract(IsOneWay = false, Action = "BizTalkSubmit")]
    Message BizTalkSubmit(Message message);
    Message EndTwoWayMethod(IAsyncResult result);
}

[ServiceContract(Namespace = "http://www.microsoft.com/biztalk/2006/r2/wcf-adapter")]
public interface ITwoWayAsyncVoid
{
    // Methods
    [OperationContract(AsyncPattern = true, IsOneWay = false, Action = "*", ReplyAction = "*")]
    IAsyncResult BeginTwoWayMethod(Message message, AsyncCallback callback, object state);
    [OperationContract(IsOneWay = false, Action = "BizTalkSubmit")]
    void BizTalkSubmit(Message message);
    void EndTwoWayMethod(IAsyncResult result);
}

[ServiceContract(Namespace = "http://www.microsoft.com/biztalk/2006/r2/wcf-adapter")]
public interface ITwoWayAsyncVoidTxn
{
    // Methods
    [TransactionFlow(TransactionFlowOption.Mandatory), OperationContract(AsyncPattern = true, IsOneWay = false, Action = "*", ReplyAction = "*")]
    IAsyncResult BeginTwoWayMethod(Message message, AsyncCallback callback, object state);
    [TransactionFlow(TransactionFlowOption.Mandatory), OperationContract(IsOneWay = false, Action = "BizTalkSubmit")]
    void BizTalkSubmit(Message message);
    void EndTwoWayMethod(IAsyncResult result);
}

As you can easily see in the code above, all the service contracts implemented by the BizTalkServiceInstance Class are generic and asynchronous. In fact, one-way service operations are decorated with [OperationContract(AsyncPattern = true, IsOneWay = true, Action = "*")] attribute, while request-response methods are decorated with the [OperationContract(AsyncPattern = true, IsOneWay = false, Action = "*", ReplyAction = "*")] attribute. Besides, they receive a generic inbound message of type System.ServiceModel.Channels.Message and eventually return a message of the same type. As a consequence, these methods can accept any request message and eventually process any reply message. However, since these operations are generic and untyped, they are not decorated by any FaultContractAttribute. As a consequence, WCF Receive Locations cannot return typed fault exceptions. Since every WCF Receive Location is implemented as an instance of the BizTalkServiceInstance and this class cannot be customized or inherited to expose typed service contracts, BizTalk Server does not natively support throwing typed fault exceptions within orchestrations published as WCF services or more in general throwing typed fault exceptions within a WCF Receive Location.

At this point a question arises: is there any workaround to throw typed fault exceptions within orchestrations published as WCF services? The answer, fortunately, is yes.

The Solution

The WCF-Custom and WCF-CustomIsolated Adapters provided by BizTalk Server 2006 R2 and BizTalk Server 2009 allow to define a custom WCF binding configuration to meet your precise communication needs. These adapters can be used to define custom WCF configurations for Send Ports or Receive Locations and extend their standard functionality using custom components as endpoint and service behaviors, message inspectors, custom bindings, binding elements, channels, etc. Therefore, I decided to create the following custom components:

Microsoft.BizTalk.CAT.Samples.PublishTypedFaults.WsdlExportExtensions assembly: flattens and extends the WSDL generated by the BizTalk WCF Service Publishing Wizard to enable WCF Receive locations to expose typed soap faults.
Microsoft.BizTalk.CAT.Samples.PublishTypedFaults.MessageInspector assembly: intercepts a reply message and when this latter is a fault, it creates and returns a typed fault message.

The following picture depicts the architecture of the proof of concept I implemented to test these components:

A WCF-CustomIsolated Request-Response Receive Location receives a new xml document from a client application.
The XML disassembler component within the XMLTransmit pipeline promotes the MsgType context property. The Message Agent submits the incoming message to the MessageBox (BizTalkMsgBoxDb).
The inbound request starts a new instance of the HelloWorld orchestration type.
The orchestration checks the incoming HelloWorldRequest message, and if the Name element is null or empty, it creates and return a new fault message of type CustomError. Otherwise, the orchestration returns a reply message of type HelloWorldResponse.
The HelloWorld orchestration publishes the reply message to the MessageBox (BizTalkMsgBoxDb).
The response message is retrieved by the WCF-CustomIsolated Request-Response Receive Location.
The reply message is processed by the CustomErrorMessageInspector component. If the response is a fault message, it creates a new typed fault message using the configuration data set on the Receive Location.
The reply message is returned to the client application.

The following picture shows the XML Schema which defines the CustomError typed fault.

The following figure shows the configuration of the WCF-CustomIsolated Request-Response Receive Location.

The Receive Location exposes an endpoint that uses the WsHttpBinding and make use of the serviceThrottling service behavior (see System.ServiceModel.Description.ServiceThrottlingBehavior for more information) and 2 custom endpoint behaviors:

customError
wsdlExport

These components must be configured in the machine.config as follows:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    ...
  <system.serviceModel>
      ...
    <extensions>
      <behaviorExtensions>
          ...
        <add name="customError" type="Microsoft.BizTalk.CAT.Samples.PublishTypedFaults.MessageInspector.CustomErrorBehaviorExtensionElement,
                                      Microsoft.BizTalk.CAT.Samples.PublishTypedFaults.MessageInspector, Version=1.0.0.0, Culture=neutral,
                                      PublicKeyToken=d7f63d8d08d8f3a2" />
        <add name="wsdlExport" type="Microsoft.BizTalk.CAT.Samples.PublishTypedFaults.WsdlExportExtensions.WsdlExportBehaviorExtensionElement,
                                     Microsoft.BizTalk.CAT.Samples.PublishTypedFaults.WsdlExportExtensions, Version=1.0.0.0, Culture=neutral,
                                     PublicKeyToken=d7f63d8d08d8f3a2" />
      </behaviorExtensions>
    </extensions>
  </system.serviceModel>
</configuration>

The Include exception detail in faults option must be set on the Messages tab to enable the WCF Receive Location to return the error detail in fault messages. Below you can see the code of the CustomErrorMessageInspector component. The constructor retrieves configuration data from the Receive Port configuration, while the IDispatchMessageInspector::BeforeSendReply method intercepts the reply message and if this latter is a fault, it generates a typed fault exception using the configuration data.

namespace Microsoft.BizTalk.CAT.Samples.PublishTypedFaults.MessageInspector
{
    /// <summary>
    /// This class can be customized to create a message inspector.
    /// </summary>
    public class CustomErrorMessageInspector : IDispatchMessageInspector, IClientMessageInspector
    {
        #region Private Constants
        ...        
        #endregion

        #region Private Fields
        ...        
        #endregion

        #region Public Constructors
        public CustomErrorMessageInspector(bool enabled,
                                           bool traceEnabled,
                                           int maxBufferSize,
                                           string typedFaults)
        {
            this.enabled = enabled;
            this.traceEnabled = traceEnabled;
            this.maxBufferSize = maxBufferSize;
            this.typedFaults = typedFaults;

            try
            {
                if (!string.IsNullOrEmpty(typedFaults))
                {
                    TypedFaults faultData = SerializationHelper.XmlDeserialize(typedFaults, typeof(TypedFaults)) as TypedFaults;
                    if (faultData != null &&
                        faultData.FaultList != null &&
                        faultData.FaultList.Count > 0)
                    {
                        for (int i = 0; i < faultData.FaultList.Count; i++)
                        {
                            if (!string.IsNullOrEmpty(faultData.FaultList[i].Action) &&
                                !string.IsNullOrEmpty(faultData.FaultList[i].Name))
                            {
                                faults.Add(string.Format(MessageTypeFormat, 
                                                         faultData.FaultList[i].Namespace, 
                                                         faultData.FaultList[i].Name ?? string.Empty), 
                                                         new CustomErrorFaultInfo(faultData.FaultList[i].Action, 
                                                                                  faultData.FaultList[i].Reason));
                            }
                        }
                    }
                }
            }
            catch (Exception ex)
            {
                EventLog.WriteEntry(Source, ex.Message, EventLogEntryType.Error);
            }
        }
        #endregion

        #region IDispatchMessageInspector Members
        public object AfterReceiveRequest(ref Message request, IClientChannel channel, InstanceContext instanceContext)
        {
            return request.Headers.Action;
        }

        public void BeforeSendReply(ref Message reply, object correlationState)
        {
            try
            {
                if (enabled &&
                    reply.IsFault &&
                    string.Compare(((string)correlationState), TransferGetNamespace, true) != 0)
                {
                    string action = correlationState as string;
                    MessageFault fault = MessageFault.CreateFault(reply, maxBufferSize);
                    string detail = fault.Reason.ToString();
                    CustomErrorMessageFault faultMessage = new CustomErrorMessageFault(faults, 
                                                                                       new FaultCode(FaultCodeName), detail, traceEnabled);
                    Message message = Message.CreateMessage(reply.Version, faultMessage, null);
                    message.Headers.CopyHeadersFrom(reply.Headers);
                    message.Properties.CopyProperties(reply.Properties);
                    if (string.IsNullOrEmpty(faultMessage.Action))
                    {
                        message.Headers.Action = action;
                    }
                    else
                    {
                        message.Headers.Action = faultMessage.Action;
                    }
                    reply = message;
                }
            }
            catch (Exception ex)
            {
                EventLog.WriteEntry(Source, ex.Message, EventLogEntryType.Error);
                throw ex;
            }
            return;
        }
        #endregion

        #region IClientMessageInspector Members
        public void AfterReceiveReply(ref Message reply, object correlationState)
        {
            return;
        }

        public object BeforeSendRequest(ref Message request, IClientChannel channel)
        {
            return null;
        }
        #endregion
    }
}

The following table contains the XML snippet used as configuration data for the CustomErrorMessageInspector component. As you can see, it allows defining multiple typed faults in a declarative way. At runtime it will use this data to recognize a fault reply message and accordingly set the Action and Reason properties of the fault message returned.

<TypedFaults xmlns="http://microsoft.biztalk.cat/10/typedfaults">
    <TypedFault>
        <Action>SayHello</Action>
        <Name>CustomError</Name>
        <Namespace>http://microsoft.biztalk.cat/10/customerror</Namespace>
        <Reason>Custom Error</Reason>
    </TypedFault>
</TypedFaults>

If you set the TraceEnabled property to true, at runtime the CustomErrorMessageInspector component will produce a trace that you can intercept and review with a tool such as Debug as shown in the picture below.

The WCF-Receive Location can be created using the BizTalk WCF Service Publishing Wizard. In particular, this allows to create a WCF Receive Location to expose the HelloWorld orchestration as a WCF service. Moreover, setting the Enable metadata endpoint option it’s possible to enable the WCF Receive Location to expose a Metadata Endpoint.

This allows a developer to use Visual Studio or a tool such as svcutil to generate a proxy class to invoke the WCF Receive Location. Now, let’s say that you want to create a WCF client application to invoke the WCF Receive Location. Within Visual Studio you can create a new Windows Application, right-click the project inside the Solution Explorer and then select Add Service Reference. If the WCF Receive Location exposes a Metadata Endpoint, this operation will create a proxy class to invoke the corresponding WCF web service. However, if you review the code and in particular the contract interface, you’ll realize that the SayHello method is not decorated with a FaultContractAttribute. This is due to the fact, that since BizTalk Server does not support throwing typed fault exceptions, the native wsdl returned by the Metadata Endpoint exposed by the WCF Receive location does not contain any soap fault message. In order to sort out this problem, you can adopt one of the following techniques:

Create and expose a custom wsdl that contains the definition of your typed soap faults.
Create a custom endpoint behavior to dynamically modify the wsdl produced by BizTalk.

The first technique is quite straightforward:

You manually define a custom wsdl using a text or xml editor.
You publish the resulting wsdl file to IIS (e.g. http://localhost/samples/service.wsdl)
You configure your WCF-Custom or WCF-CustomIsolated Receive Location to expose metadata and in particular the newly created wsdl file.

In order to accomplish the finale step, you can proceed as follows:

Start the BizTalk Server Administration Console.
Open your WCF-Custom/WCF-CustomIsolated Receive Location.
Click the Configure button in the Transport section.
Click the Behavior tab.
Right-click the Service Behavior node and choose to Add Extension.
In the Select Behavior Extension window, choose the serviceMetadata component.
Set the value of the httpGetEnabled property of the serviceMetadata behavior to True (see the picture below).
Set the value of the externalMetadataLocation property of the serviceMetadata behavior to the url of your hand-built wsdl (see the picture below).

The second technique consists in creating a custom component called WsdlExtensions to modify at run-time the wsdl returned by the the Metadata Endpoint exposed by a WCF-Custom/WCF-CustomIsolated Receive location. This component is implemented as an a custom endpoint behavior and it can be used along with any WCF-Custom or WCF-CustomIsolated Receive Location, as shown in the picture below.

The WsdlExtensions property exposed by the component accepts an XML snippet that allows to specify how the wsdl has to be customized at runtime.

<WsdlExtensions xmlns="http://microsoft.biztalk.cat/10/wsdlextensions">
    <Prefix>bts</Prefix>
    <XmlSchemas>
        <XmlSchema>
            <Name>CustomError</Name>
            <Namespace>http://microsoft.biztalk.cat/10/customerror</Namespace>
        </XmlSchema>
    </XmlSchemas>
    <Messages>
        <Message>
            <Name>HelloWorld_SayHello_CustomErrorFault_FaultMessage</Name>
            <Namespace>http://microsoft.biztalk.cat/10/customerror</Namespace>
            <Parts>
                <Part>
                    <Name>detail</Name>
                    <Element>CustomError</Element>
                </Part>
            </Parts>
        </Message>
    </Messages>
    <PortTypes>
        <PortType>
            <Name>HelloWorld</Name>
            <Operations>
                <Operation>
                    <Name>SayHello</Name>
                    <Faults>
                        <Fault>
                            <Name>CustomErrorFault</Name>
                            <Message>HelloWorld_SayHello_CustomErrorFault_FaultMessage</Message>
                        </Fault>
                    </Faults>
                </Operation>
            </Operations>
        </PortType>
    </PortTypes>
</WsdlExtensions>

As shown in the picture above, the components allows to define the following information:

The prefix for the namespace of new messages created inside the wsdl.
One or multiple Xml Schemas each defining a different typed fault. These schemas must be deployed to BizTalk. At runtime, the component is able to retrieve the content of each schema from the BizTalkMgmtDb that subsequently is inserted in the outbound wsdl.
One or multiple fault messages, each containing one or multiple parts.
One or multiple operation-fault associations. At runtime the component search through the original wsdl structure and creates faults accordingly.

At runtime the WsdlExtensions component validates the XML configuration specified on the port and if the TraceEnabled property is set to true, it produces a trace with a tool such as Debug as shown in the picture below.

The following picture shows the wsdl produced by the WsdlExtensions component at runtime. In particular, the parts added by the component are highlighted by a red frame.

Note: Flattening the wsdl to include the XML schemas used to define messages allows to make the wsdl document compatible with non-Microsoft development environments. In fact, Visual Studio supports import and include directives in a wsdl, but some non-Microsoft development environments are not able to consume the wsdl exposed by a WCF web service.

Note: the XML Schemas defining the structure of the XML configuration data consumed at runtime respectively by the CustomErrorMessageInspector and by the WsdlExportEndpointBehavior can be found inside the BehaviorSchemas project. The corresponding assembly doesn’t need to be deployed to BizTalk Server.

Proof Of Concept in Action

In order to test the WCF custom components, I created a WinForms application. In particular, when I leave the Name textbox blank and press the Call button, the HelloWorld orchestration returns a fault message. The client application intercepts the error with a catch (FaultException<HelloWorldBizTalk.CustomError> ex) block and opens a MessageBox to show the detail of the exception.

Show me the code!

At this point, you are probably saying:

- ok, cool, where’s the code?

Well, I packaged the code in 2 different flavors:

- BizTalk Server 2009

- BizTalk Server 2006 R2

Obviously, everything there are many ways to implement the same functionalities, so I’d be glad if you could leave a message on my blog and let me know how you used/customized/improved my code. Enjoy! ;-)

BizTalk server and protocol transition

leprino — Tue, 20 Jan 2009 20:26:45 GMT

Consider the following scenario:

You have to implement a BizTalk Server application that has to interact with one or more downstream systems. Request messages are submitted by a front-end application which authenticates callers using the Windows Integrated Security. These messages are received by one or more SOAP or WCF Receive Locations which use the Windows Integrated Security to authenticate the original caller. The orchestration and/or the Send Ports (WCF, SOAP, SQL) responsible to process incoming requests need to impersonate the user account credentials of the original caller whilst invoking one or more downstream systems (e.g. Web Service, SQL Server database).

Enterprise Single Sign-On (SSO) that is part of BizTalk Server since the 2004 release provides services to enable credential mappings and single sign-on to end users for enterprise application integration (EAI) solutions. The SSO system maps Microsoft Windows accounts to back-end credentials. SSO simplifies the management of user IDs and passwords, both for users and administrators. It enables users to access back-end systems and applications by logging on only once to the Windows network. If the BizTalk Server application is able to properly authenticate the original caller, her/his credentials can be mapped to a username/password couple that can be used to access a downstream system (SAP, mainframe, etc.) on her/his behalf. What about if the target system in question is a web service which makes use of the Windows Integrated Security in order to authenticate and authorize incoming requests or SQL Server database configured to use the Windows Integrated Security? In this case, you should create a SSO Affiliate Application for each target system and map each domain account to its username/password credentials. The major drawback of this approach is that you should use the Password Change Notification Service to receive password changes directly from domain controllers and keep password in sync with the Active Directory. But there’s another issue with this approach: the SSO support for WCF and SOAP Send Ports include the Digest and Basic Authentication but not the Windows Integrated Security (see Single Sign-On Support for the WCF Adapters for more information on this topic). As a consequence, even if you manage to successfully impersonate the original caller, the Send Handler of the WCF and SOAP Adapters won’t be able to use these credentials when invoking the downstream service.

The solution to this problem is to use the Kerberos protocol extensions provided by Windows Server 2003 and in particular the Kerberos Protocol Transition and Constrained Delegation. The objective of this article is not to provide full coverage of this topic. For more information on this subject, see the following articles:

The following section provides an introduction to the Protocol Transition and Constraint Delegation.

Protocol Transition and Constraint Delegation

The Protocol Transition and Constraint Delegation can be used in managed code to initialize a valid WindowsIdentity object to impersonate any domain account just using its user principal name in order to accomplish one of the following tasks:

Authorization checks.
Impersonation.
Access remote resources.

The first two operations can be implemented independently of each other, but the third operation requires that you use Protocol Transition when you are not using Kerberos authentication. In other words, constrained delegation has two configurations; one that requires Kerberos authentication and another that works with any authentication protocol. When you use the configuration that supports any authentication protocol, you must first implement protocol transition with impersonation before you implement constrained delegation.

The next section provides details on the extensions themselves. After you have an understanding of these extensions, see the “Implementation” section to learn how to implement the operations described earlier.

New Kerberos Extensions

As previously mentioned, Windows Server 2003 provides two new Service-for-User (S4U) Kerberos extensions that support protocol transition and constrained delegation. Protocol transition and constrained delegation can be used independently of each other, but they are often used together to implement the scenario described in the introduction.

Protocol Transition

The S4U2Self Kerberos extension can be used to initialize a WindowsIdentity object with the user principal name of a valid Windows account in Active Directory. The password associated with the user principal name is not required. This feature allows you to transition from any authentication protocol into the Kerberos authentication protocol. This operation is accomplished by using the ticket-granting ticket (TGT) of a service account to request a service ticket for itself. The service account in this case is the one associated with the Web service that performs the protocol transition. The service ticket that is returned from the ticket-granting service (TGS) contains identity and principal information for the user whose ID was sent with the request.

The new WindowsIdentity that is initialized with this service ticket can then be used to perform role-based authorization checks. In addition, when used with constrained delegation, this new identity can be used to access downstream resources. There are limitations to what this new identity is allowed to do that are based on the privileges of the service account. These limitations are discussed in the “Implementation” section later in this technical supplement.

Constrained Delegation

The S4U2Proxy Kerberos extension provides an implementation of constrained delegation that allows you to use a Kerberos service ticket — instead of a TGT — to request another service ticket. Delegation is considered to be constrained because the identity (service account) that is used to request the service ticket must be configured to access a specific service. Constrained delegation works with or without protocol transition. The primary restriction is that the service account used to request a Kerberos service ticket must be configured to access the requested service. In addition, the service account must be able to impersonate the client prior to calling the service. For example, when you use Windows integrated authentication with impersonation, the default Web server’s computer account can be configured for constrained delegation without making any changes to the Internet Information Services (IIS) process account. A restriction of protocol transition is that the Web server’s computer account cannot be used for constrained delegation without modifying the IIS process account or better the service account used by the Application Pool. The reason for this is that the default IIS process account (which is the NT AUTHORITY\NETWORK SERVICE account on Windows 2003 Server) does not have necessary privileges to implement impersonation using the WindowsIdentity object that was created during protocol transition. Instead of modifying the default IIS process account, you can also use a different service account for the Application Pool.

Protocol Transition

.NET Framework applications can implement protocol transition by creating an instance of the WindowsIdentity object with a User Principal Name (UPN), which is similar to an e-mail address. For example, if the user ID is john and the corresponding Active Directory domain is contoso.com, the UPN is john@contoso.com.

It is also possible to use protocol transition to initialize a WindowsIdentity object using a common Active Directory account for trusted subsystem implementations. This type of approach is normally used when you want to improve scalability with resources that use object or connection pooling based on the credentials that were used to access them. For example, connection pooling with SQL Server will work only if a common identity is used. As a result, the following two primary scenarios are associated with protocol transition in Windows:

Transitioning from a different authentication protocol, such as X.509 client certificates, into the Kerberos protocol.
Transitioning from custom authentication by using a common identity for trusted subsystem implementations.

For more information on these topics you can review Protocol Transition with Constrained Delegation Technical Supplement.

Protocol Transition and BizTalk Server

In the following section I’ll introduce the 3 scenarios implemented to demonstrate how exploiting the Protocol Transition technique in a BizTalk Server project.

Scenario 1 – Messaging Only Scenario with a SOAP Receive Location

The picture below depicts the architecture design and message flow of the first scenario.

Fig.1: Architecture Design of the first Scenario

1. A WinForm client application submits a Request to a Request-Response Soap Receive using the following code.

    Private void btnSoap_Click(object sender, EventArgs e)
    {
        SoapReceiveLocation.HelloWorldService client = null;
        try
        {
        client = new SoapReceiveLocation.HelloWorldService();
        client.Credentials = CredentialCache.DefaultCredentials;
        SoapReceiveLocation.RequestMessage request = new 
                SoapReceiveLocation.RequestMessage();
        request.Message = txtRequest.Text;
        SoapReceiveLocation.ResponseMessage response = 
                client.SayHello(request);
        if (response != null &&
            !string.IsNullOrEmpty(response.Greeting))
        {
            WriteToLog(response.Greeting);
        }
        }
        catch (Exception ex)
        {
        MessageBox.Show(ex.Message, 
                ExceptionCaption, 
                MessageBoxButtons.OK, 
                MessageBoxIcon.Error);
        }
    }

Fig.2: Client code to invoke the Soap Receive Location

As you can see in the code above, the client invokes the Soap Receive Location through an instance of a Soap proxy class which inherits from SoapHttpClientProtocol and uses the Integrated Windows authentication with the credentials of the current user. The Soap Receive Location is hosted by a custom authentication trusted isolated host called BizTalkServerTrustedIsolatedHost (see Fig.3). BizTalk Server trusts authentication trusted hosts to place the sender security ID (SSID) on messages that the trusted host is queuing that map to users other than to the host. For more information about authentication trusted hosts, see Authenticating the Sender of a Message on MSDN.

Fig.3: The Soap Receive Location is hosted by an authentication trusted host.

The virtual directory hosting the Soap Receive Location on IIS is configured to use the Integrated Windows authentication (see Fig.4).

Fig.4: Integrated Windows authentication configured on the RL virtual directory.

In addition, the Soap Receive Location was configured to use the Enterprise Single Sign-On (see Fig.5).

Fig.5: The SSO was enabled on the Soap Receive Location.

In this way, when the Soap Receive Location is invoked by the client application, the Soap Adapter assigns the client credentials (DOMAIN\USER) to the BTS.WindowsUser message context property. Note that this property is accessible within an orchestration via the Microsoft.BizTalk.XLANGs.BTXEngine.OriginatorSID property.

2. The Message Agent submits the incoming request message to the MessageBox (BizTalkMsgBoxDb).

3. The inbound request is consumed by a Solicit Response WCF-Custom Send Port. This latter uses a Filter Expression to receive all the documents published by the Receive Port hosting the Soap Receive Location.

4. The inbound message is mapped to the request format by the downstream HelloWorldService web service. This latter is hosted by IIS and exposes a single WsHttpBinding endpoint.

5. The WCF-Custom Send Port invokes the underlying HelloWorldService WCF service that in the scenario is hosted by a separate Application Pool (w3wp.exe) on the same IIS instance. The WCF-Custom Send Port uses a CustomBinding which contains the following binding elements (see Fig.9):

ProtocolTransitionBindingElement.
TextMessageEncodingBindingElement,
HttpsTransportBindingElement.

This ProtocolTransitionBindingElement binding element is a custom binding which injects a custom channel in the channel stack used by the WCF Send Handler to send out the request message. The binding element exposes multiple properties which allow controlling the channel behavior at run time. For example, the binding element can be configured to read the client credentials from a given message context property setting the value of the WindowsUserPosition to Context. In this case, the name and namespace of the source context property must be specified respectively with the ContextPropertyName and ContextPropertyNamespace properties exposed by the custom binding element. By default, the context property used to read the client account is the http://schemas.microsoft.com/BizTalk/2003/system-properties#WindowsUser promoted in this scenario by the Soap Adapter on the inbound Receive Location, but it could be read from a custom message context property. Instead, if the client account must be read from the Xml request message using an XPath Expression specified in the WindowsUserXPath property of the binding element, the value of the WindowsUserPosition property must be set to Message. At run time, the custom channel called InspectingRequestChannel reads the client account name from the context property or from the message and impersonates the original caller before passing the call to the innerChannel. When reading the client account name from the message context, the custom channel exploits a feature provided by the WCF Send Handler: when creating the System.ServiceModel.Channels.Message from the inbound IBaseMessage, this latter writes all the message context properties in the Properties of the newly WCF message. So to to read the name of the client user is sufficient to use the following lines of code within the custom channel.

    private const string ContextPropertyKeyFormat = "{0}#{1}";
    ...
    string contextPropertyKey = string.Format(ContextPropertyKeyFormat, 
                                              contextPropertyNamespace, 
                                              contextPropertyName);
    if (message.Properties.ContainsKey(contextPropertyKey))
    {
        windowsUser = message.Properties[contextPropertyKey] as string;
    }

Fig.6: Code snippet from the helper class invoked by the custom channel.

A helper component invoked by the custom channel transforms the original account name (DOMAIN\USER) into the equivalent User Principal Name (USER@DOMAIN). Finally the custom channel impersonates the client user using the following code:

    public Message Request(Message message, TimeSpan timeout)
    {
        string upn = null;
        ...
        try
        {
            ...
            upn = InspectingHelper.GetUserPrincipalName(...);
            if (!string.IsNullOrEmpty(upn))
            {
                WindowsIdentity identity = new WindowsIdentity(upn);
                impersonationContext = identity.Impersonate();
            }
            reply = this.InnerChannel.Request(request);
        }
        catch (Exception ex)
        {
            Debug.WriteLineIf(traceEnabled, string.Format(MessageFormat, ex.Message));
            throw ex;
        }
        finally
        {
            if (impersonationContext != null)
            {
                impersonationContext.Undo();
                Debug.WriteLineIf(traceEnabled, string.Format(UndoneFormat, upn ?? Unknown));
            }
        }
        return reply;
    }

Fig.7: Request method of the InspectingRequestChannel custom channel class.

The custom binding element, along with other components (binding element extension, channel, channelfactory etc.) used to extend the default behavior of the WCF Send Port, is contained in a custom assembly. In order to let BizTalk to use the custom binding element inside a WCF-Custom Send Port, it’s necessary to register the corresponding binding element extension inside the machine.config configuration file with the following section.

    <system.serviceModel>
        <extensions>
            <bindingElementExtensions>
                <add name="protocolTransition" type="Microsoft.BizTalk.Rangers.ProtocolTransition.WCFCustomChannel.InspectingBindingExtensionElement,
                 Microsoft.BizTalk.Rangers.ProtocolTransition.WCFCustomChannel, Version=1.0.0.0, Culture=neutral, PublicKeyToken=874a60d7e5a4dd9b" />
            </bindingElementExtensions>
        </extensions>
    </system.serviceModel>

Fig.8: How to configure Binding Element Extension in the machine.config.

To configure the WCF-Custom Send Port to use protocolTransition binding element, the custom binding element extension class needs to be configured in the machine.config. This functionality is implemented by the InspectingBindingExtensionElement class which returns the InspectingBindingElement class to indirectly handle the message modification. InspectingBindingElement is the only class that must be public. The factory, listener, and channels can all be internal implementations of the public run-time interfaces. The protocolTransition custom binding element is associated with a BizTalk Server Send Port under the properties of the adapter in the WCF-Custom Transport Properties dialog box (see Fig.9). On the Binding tab, you can select a binding provided by the .NET Framework 3.0, in our scenario the CustomBinding. Then in the Binding panel it’s possible to select any message encoding and transport element along with other custom or standard binding elements. In our case we selected the InspectingBindingElement (protocolTransition), TextMessageEncodingBindingElement and HttpsTransportBindingElement. The assembly Microsoft.BizTalk.Rangers.ProtocolTransition.WCFCustomChannel containing the custom binding element and channel needs to be installed into the GAC.

Fig.9: Binding configuration of the WCF-Custom Send Port.

6. The HelloWorldService WCF service returns a response message.

7. The incoming response message is mapped to the format expected by the client application.

8. The transformed response message is published to the MessageBox.

9. The response message is retrieved by the Request-Response Soap Custom Receive Location which originally received the request call.

10. The response message is returned to the client WinForm application.

Scenario 2 – Messaging Only Scenario with a WCF-Custom Receive Location

The picture below depicts the architecture design and message flow of the second scenario.

Fig.10: Architecture Design of the second Scenario

In this sample the Soap Receive Location was replaced by a WCF-Custom Receive Location.

1. A WinForm client application submits a Request to a Request-Response Soap Receive using the following code.

    private void btnBizTalk_Click(object sender, EventArgs e)
    {
        WCFReceiveLocation.HelloWorldServiceClient client = null;

        try
        {
            client = new WCFReceiveLocation.HelloWorldServiceClient();
            client.ClientCredentials.Windows.AllowedImpersonationLevel = TokenImpersonationLevel.Impersonation;
            WCFReceiveLocation.RequestMessage request = new WCFReceiveLocation.RequestMessage();
            request.Message = txtRequest.Text;
            WCFReceiveLocation.ResponseMessage response = client.SayHello(request);
            if (response != null && !string.IsNullOrEmpty(response.Greeting))
            {
                WriteToLog(response.Greeting);
            }
        }
        catch (FaultException ex)
        {
            MessageBox.Show(ex.Message, 
                            ExceptionCaption, 
                            MessageBoxButtons.OK,
                            MessageBoxIcon.Error);
        }
        catch (Exception ex)
        {
            MessageBox.Show(ex.Message,
                            ExceptionCaption,
                            MessageBoxButtons.OK,
                            MessageBoxIcon.Error);
        }
        finally
        {
            if (client != null)
            {
                try
                {
                    client.Close();
                }
                catch (Exception)
               {
               }
            }
        }
    }

Fig.11: Client code to invoke the Soap Receive Location

As you can see in the code above, the client invokes the Soap Receive Location through an instance of a WCF proxy class and uses the Integrated Windows authentication with the credentials of the current user. The WCF-Custom Receive Location is hosted by an authentication trusted in-process host called BizTalkServerTrustedHost (see Fig.12).

Fig.12: WCF-Custom Receive Location Host configuration.

This WCF-Custom Receive Location uses the NetTcpBinding and is configured to use the Transport level security (see Fig.13).

Fig.13: WCF-Custom Receive Location Binding configuration.

In addition, the WCF-Custom Receive Location was configured to use the Enterprise Single Sign-On (see Fig.14).

Fig.14: The SSO was enabled on the Soap Receive Location.

In this way, when the WCF-Custom Receive Location is invoked by the client application, the WCF-Custom Adapter assigns the client credentials (DOMAIN\USER) to the BTS.WindowsUser message context property. Note that this property is accessible within an orchestration via the Microsoft.BizTalk.XLANGs.BTXEngine.OriginatorSID property. The original WCF Adapter contained in BizTalk Server 2006 R2 has a bug that is the Receive Handler doesn’t assign the client credentials to the BTS.WindowsUser context property. The product group made patch was done to fix this problem, but this package is not publicly available. By the way, even without the fix, it’s possible to read and promote the client user account. To accomplish this task, you need to create a custom endpoint behavior which injects a message inspector in the execution chain of the WCF Receive Location. The endpoint behavior can be configured on the WCF-Custom Receive Location on the Behavior tab. The message inspector needs to implement the methods defined by the IDispatchMessageInspector interface. In particular, the AfterReceiveRequest method is executed after the WCF Receive Adapter has received a request message. This function can be used to read the client Windows identity name from the current service security context and to write this information in a custom header identified by a name and namespace that can be configured in the endpoint configuration.

    public object AfterReceiveRequest(ref Message request, 
                                      IClientChannel channel,
                                      InstanceContext instanceContext)
    {
        ...
        string windowsUserName = ServiceSecurityContext.Current.WindowsIdentity.Name;
        request.Headers.Add(MessageHeader.CreateHeader(contextPropertyName,
                                                       contextPropertyNamespace,
                                                       windowsUserName));
        ...
        return null;
    }

Fig.15: The custom interceptor used to read the client user account.

If you define a message context property in a custom PropertySchema with the same name and namespace as the header created by the message inspector component at the previous step, the WCF Adapter will transform the custom header in a written context property with the same name and namespace when transforming the inbound WCF message (System.ServiceModel.Channels.Message) in an IBaseMessage instance. Unofrtunately, the value of the header will be contained in a <name xmlns=”…”></name> Xml snippet, so it’s necessary to create a custom pipeline component and a custom receive pipeline in order to extract the information from the custom context property and sets its value without the Xml start and end tags. The solution developed for to this article contains the code for the endpoint behavior, custom pipeline component and receive pipeline that you eventually need to set on the WCF-Custom Receive Location.

2. The Message Agent submits the incoming request message to the MessageBox (BizTalkMsgBoxDb).

4. The inbound message is mapped to the request format by the downstream HelloWorldService web service. This latter is hosted by IIS and exposes a single WsHttpBinding endpoint.

5. The WCF-Custom Send Port invokes the underlying HelloWorldService WCF service that in the scenario is hosted by a separate Application Pool (w3wp.exe) on the same IIS instance. For more details about this step, see the point number 5 of scenario 1.

6. The HelloWorldService WCF service returns a response message.

7. The incoming response message is mapped to the format expected by the client application.

8. The transformed response message is published to the MessageBox.

9. The response message is retrieved by the Request-Response Soap Custom Receive Location which originally received the request call.

10. The response message is returned to the client WinForm application.

Scenario 3 – Orchestration with an Inline Send Port

The picture below depicts the architecture design and message flow of the third scenario.

Fig.16: Architecture Design of the third Scenario.

This scenario uses a technique commonly known as Inline Send Ports pattern: orchestrations do not use any logical ports bound to physical ports to invoke underlying services (e.g. WCF, Soap, Http services) or access data repositories (e.g. SQL Server, Oracle, Mainframe) but in their place they use custom .NET components (e.g. an instance of a SOAP or WCF proxy class to invoke a downstream web service or a ADO.NET component to invoke a SQL or an Oracle database) invoked inside an Expression Shape. Note that not using adapters and physical ports, the application will not benefit from their functional capabilities such as batching, retries, correlation sets initialization, declarative configuration and secondary transports. The advantage of this technique is that for each Solicit Response call made by the orchestration, it allows to eliminate 4 roundtrips to the MessageBox:

Orchestration Logical Port --> Request --> MessageBox
MessageBox --> Request --> Physical Send Port
Physical Send Port --> Response --> MessageBox
MessageBox --> Response --> Orchestration Logical Port

In the third scenario the request message is received by the same Soap Receive Location used by the first scenario, but in this case the incoming document is consumed and processed by an orchestration. This latter directly invokes the downstream HelloWorldService WCF service using an instance of a WCF proxy.

1. A WinForm client application submits a Request to a Request-Response Soap Receive using the following code. See point 1 of the first scenario for more information on the code used by the client application and the Soap Receive Location configuration.

2. The Message Agent submits the incoming request message to the MessageBox (BizTalkMsgBoxDb).

3. The inbound request is consumed by new instance of the CallHelloWorld orchestration.

Fig.17: The structure of the orchestration used by the third scenario.

4. The orchestration instance invokes the SayHello static method exposed by the OrchestrationHelper class inside the WCF Call Expression Shape.

 stream = Microsoft.BizTalk.Rangers.ProtocolTransition.Helpers.OrchestrationHelper.SayHello(
                                                               RequestMessage(Microsoft.BizTalk.XLANGs.BTXEngine.OriginatorSID), 
                                                               RequestMessage.BodyPart.Message);

Fig.18: Helper component call inside the CallHelloWorld orchestration.

5. The SayHello method invoked by the orchestration impersonates the user account specified in the originatorSID parameter and then invokes the HelloWorld WCF service using the following code.

    private const string UserPrincipalNameFormat = "{0}@{1}";
    ...
    public static Stream SayHello(string originatorSID, string message)
    {
        WindowsImpersonationContext impersionationContext = null;

        try
        {
            if (!string.IsNullOrEmpty(originatorSID))
            {
                string[] parts = originatorSID.Split(new char[] {Path.DirectorySeparatorChar});
                if (parts != null && parts.Length > 1)
                {
                    string upn = string.Format(UserPrincipalNameFormat, 
                                               parts[1], 
                                               domainFQDN);
                    WindowsIdentity identity = new WindowsIdentity(upn);
                    impersionationContext = identity.Impersonate();
                }
            }
            RequestMessage request = new RequestMessage();
            request.Message = message;
            HelloWorldClient client = new HelloWorldClient();
            ResponseMessage response = client.SayHello(request);
            string greeting;
            if (response != null && !string.IsNullOrEmpty(response.Greeting))
            {
                greeting = response.Greeting;
            }
            else
            {
                greeting = NoGreeting;
            }
            return WriteStream(greeting);
        }
        catch (FaultException<GreetingFault> ex)
        {
            return WriteStream(ex.Detail.Message);                
        }
        catch (Exception ex)
        {
            return WriteStream(ex.Message);
        }
        finally
        {
            if (impersionationContext != null)
            {
                impersionationContext.Undo();
            }
        }
    }

Fig.19: The code of the SayHello method called by the CallHelloWorld orchestration.

The client endpoint used by the HelloWorldClient WCF proxy class is defined in the BizTalk Server configuration file (BTSNTSvc.exe.config) as follows:

<system.serviceModel>
    <bindings>
        <wsHttpBinding>
            <binding name="HelloWorldBinding" 
                     closeTimeout="00:01:00"
                     openTimeout="00:01:00"
                     receiveTimeout="00:10:00"
                     sendTimeout="00:01:00"
                     bypassProxyOnLocal="false"
                     transactionFlow="false"
                     hostNameComparisonMode="StrongWildcard"
                     maxBufferPoolSize="524288"
                     maxReceivedMessageSize="65536"
                     messageEncoding="Text"
                     textEncoding="utf-8"
                     useDefaultWebProxy="true"
                     allowCookies="false">
            <readerQuotas maxDepth="32" 
                          maxStringContentLength="8192"
                          maxArrayLength="16384"
                          maxBytesPerRead="4096" 
                          maxNameTableCharCount="16384" />
            <reliableSession ordered="true"
                             inactivityTimeout="00:10:00"
                             enabled="false" />
            <security mode="Transport">
            <transport clientCredentialType="Windows" 
                       proxyCredentialType="None"
                       realm="" />
            <message clientCredentialType="Windows"
                     negotiateServiceCredential="true"
                     establishSecurityContext="true" />
            </security>
            </binding>
        </wsHttpBinding>
    </bindings>
    <client>
        <endpoint address="https://domokun/HelloWorldService/HelloWorldService.svc"
                  binding="wsHttpBinding"
                  bindingConfiguration="HelloWorldBinding"
                  contract="IHelloWorld"
                  name="HelloWorldBinding">
        </endpoint>
    </client>
</system.serviceModel>

Fig.20: The system.serviceModel configuration in the BTSNTSvc.exe.config file.

6. The HelloWorldService WCF service returns a response message to the helper component.

7. The orchestration creates a response message.

8. The response message is published to the MessageBox.

9. The response message is retrieved by the Request-Response Soap Custom Receive Location which originally received the request call.

10. The response message is returned to the client WinForm application.

Summary

This article described how using the Protocol Transition technique in a BizTalk Server project to impersonate the original caller authenticated with the Windows Integrated security at the transport level. At the same time the samples shown demonstrate the WCF extensibility points provided by the WCF Adapters and the use of the Inline Send Pattern. You can download the code here.

Microsoft BizTalk Server 2006 R2 Hyper-V Guide

leprino — Wed, 23 Jul 2008 19:30:40 GMT

The BizTalk Customer Advisory Team and BizTalk UE team released the first edition of the “Microsoft BizTalk Server 2006 R2 Hyper-V Guide”.

The Microsoft BizTalk Server 2006 R2 Hyper-V Guide is the third deliverable in a series of guides intended to provide easily accessible, hands-on guidance to our customer and partner community. This 145 page guide is available on MSDN, TechNet and as a separate DOCX or PDF download alongside the already available “Microsoft BizTalk Server Operations Guide” and “Microsoft BizTalk Server Performance Optimizations Guide”

The guide provides relevant information to IT professionals to enable them to make educated decisions about the advantages and tradeoffs of using Windows Server 2008 Hyper-V to virtualize BizTalk Server environments. This guidance was derived from a 6 week performance lab conducted by the BizTalk Customer Advisory Team and Premier Field Engineering. It will be refreshed for future versions of Microsoft BizTalk Server.

The key sections of the guide are:

Getting Started: provides conceptual information about Hyper-V, the virtualization technology introduced with Windows Server 2008, and an introduction to the Hyper-V architecture.
Deploying BizTalk Server on Hyper-V: describes the steps that were followed to set up the lab environment, which was used to compare the performance of a BizTalk Server solution running on Hyper-V environment to the same BizTalk Server solution running on comparable physical hardware.
Evaluating BizTalk Server Performance on Hyper-V: details the important considerations when measuring the performance of a BizTalk Server solution running on a Hyper-V virtualized environment.
Testing BizTalk Server Performance on Hyper-V: provides detailed results of four distinct testing scenarios that compare the performance of a BizTalk Server solution running on Hyper-V environment to the same BizTalk Server solution running on comparable physical hardware.

The target audience for this guide is Microsoft field, partner organizations, and customers who plan, deploy, and maintain BizTalk Server installations.

Full MSDN URL: http://msdn.microsoft.com/en-us/library/cc768518.aspx

Full TechNet URL: http://technet.microsoft.com/en-us/library/cc768518.aspx

Synchronous To Asynchronous Flows Without An Orchestration

leprino — Mon, 21 Jul 2008 19:31:00 GMT

The Issue

Some months ago I had the chance to work with a customer who asked me a very nteresting and tricky question: does BizTalk Server 2006 R2 support synchronous-to-asynchronous messaging only flows, without the need of an orchestration to couple a Two Way Request Response Receive Location with a One-Way Send Port to send out the request and a One-Way Receive Location to get response back from the invoked system?

The use case the customer wanted to reproduce was the following:

· A Two-Way SOAP Receive Location receives a request message and publishes it to the BizTalkMsgBoxDb.

· The message is consumed by a MQ Series One-Way Send Port which writes the document into an outgoing MQ queue.

· The response message is received by a given MQ Series One-Way Receive Location which reads the document from an incoming MQ Queue and publishes the message to the BizTalkMsgBoxDb.

· The response message is finally received by the Two-Way SOAP Receive Location which received the original request and then the document is returned to the caller.

I started to create a POC of this scenario, using the FILE adapter in place of the MQ Series Adapter and folders in place of MQ queues. In the majority of BizTalk applications that use a Request-Response receive port, the request message is used to start a given Orchestration or it is directly relayed to a solicit-response Send Port. When the Orchestration or Send Port provides the response message, this message is routed back to the Two-Way Receive Location that submitted the original request message. These two design patterns are fully supported by the product group and the picture below depicts the case where a Two-Way Request Response Receive Location is directly coupled with a Two-Way Solicit Response Send Port which is responsible for invoking an external web service.

But what about our case in which the request and response are respectively sent and received to/from an external target system not using a Two-Way Solicit-Response Send Port, but using a One-Way Send Port while and a One-Way Receive Location? The standard solution is using an Orchestration to correlate the request submitted to the external system with the response received from this latter. But as you can see from the picture below, this pattern requires 8 roundtrips to the BizTalkMsgBoxDb to complete, 4 message writes and 4 message reads.

In other words, this approach requires multiple roundtrips to the MessageBox to publish and consume messages to/from the BizTalkMsgBoxDb and to hydrate and dehydrate the internal state of the orchestration and all these operations can dramatically increase the contention on SQL and reduce the overall throughput and speed of the application.

So the following question spontaneously emerged: is there any technique to implement the same behavior getting rid of the orchestration? This pattern would allow to eliminate 2 messages publications and 2 message reads and this would allow to speed up the execution and to decrease the contention on the BizTalkMsgBox.

The solution to this problem rests in understanding how BizTalk matches a response message to an initial request message. After you understood the internal subscription matching mechanism implemented by BizTalk, the second step is to individuate the context properties that need to be propagated along and promoted in the One-Way Receive Location in order to allow the Two-Way Receive Location to receive the response when this latter is published to the BizTalkMsgBox.

When a message is received through a Request-Response Receive Location and published to the BizTalkMsgBoxDb, an instance subscription is created to allow the Receive Location to receive the response back when this latter is published to the MessageBox. This subscription expression is always composed of the pmRRCorrelationToken promoted property that is in the request message's context, and a RouteDirectToTP promoted property with a value of True.

In particular the pmRRCorrelationToken is a string containing the name of the BizTalk node which received the request, the process id of the host instance which received the message and a GUID which represents a correlation id. For more info on these and other context properties you can review the following article on MSDN http://msdn.microsoft.com/en-us/library/ms966048.aspx.

I made some tests and I quickly realized that those properties were not sufficient to correlate a response back to the original request. In other words, even if the subscription expression is composed of pmRRCorrelationToken and the RouteDirectToTP properties, promoting them with the expected value in the message context of the response document before publishing this latter is not sufficient to pass message back to the Receive Location. It was clear that it was necessary to propagate some additional context properties, even if they were not used by the subscription expression. After some attempts, I was able to identify the set of context properties that were necessary to correlate a response message back and I was able to create a sample were a synchronous WCF Request Response is couple to 2 asynchronous One-Way FILE Ports.

The Solution

I created a sample BizTalk application called SyncToAsync containing a Two-Way Receive Port called SyncToAsync.Request.ReceivePort. Subsequently, I created a WCF-NetTcp Request Response Receive Location called SyncToAsync.Request.WCF-NetTcp.ReceiveLocation and a WinForm driver application to submit requests to it. Using the WCF Service Publishing Wizard I created a MetadataExchange Endpoint for the web service exposed by the WCF Receive Location and then I used the Add Service Reference command provided by Visual Studio 2005 to create a proxy class in order to submit messages to it. I selected the XmlReceive and PassThruTransmit pipelines on the SyncToAsync.Request.WCF-NetTcp.ReceiveLocation to process respectively the incoming request and the outgoing response messages. In particular, The XmlReceive pipeline contains the Xml Disassembler which is responsible to probe and individuate the message type and to promote this information along with other context properties. The second step was creating a One-Way FILE Send Port that subscribed all the messages published by the former Receive Location. In order to that I just used the BTS.ReceivePortName = SyncToAsync.Request.ReceivePort predicate as Filter expression. At this point I enabled the SyncToAsync.Request.WCF-NetTcp.ReceiveLocation WCF Receive Location and maintained disabled the SyncToAsync.Request.FILE.SendPort Send Port. Then, using the .NET driver application, I submitted some messages to the BizTalk application. Since no subscribers were active, those messages were suspended. Therefore, using the Administration Console I could review the promoted properties of the incoming request message. After some attempts, I realized that the properties highlighted below (CorrelationToken, EpmReqRespCorrelationToken, ReqRespTransmitPipelineID, IsRequestResponse) plus the RouteDirectToTP that is not promoted on the original request message, were the properties that had to be propagated and subsequently promoted on the One-Way FILE Receive Location used to publish the response to MessageBox.

For the demo I created a very simple XML Schema to represent the basic attributes of a Book entity. I called the schema below InboundBook and this represents the format of the incoming message sent by the .NET Driver application and received by the WCF Receive Location.

Then I created an extended version of the above XML schema in order to demote some of the aforementioned context properties inside the outgoing message. I called this schema OutboundBook. As you can see in the picture below, this schema is composed of a Header and a Body:

· The Header contains the CorrelationToken, EpmRRCorrelationToken and ReqRespTransmitPipelineID elements. In particular, this latter will contain the GUID of the transmit pipeline used by the WCF Receive Location to process the outbound response message. I promoted these 3 elements respectively to the properties with the same name exposed by the BTS.bts_system_properties schema contained in the Microsoft.BizTalk.GlobalPropertySchemas assembly. In this way, if on the Send Port I use any send pipeline containing the XML Assembler component (e.g. the XmlTransmit standard pipeline), at run-time, when the pipeline is executed, the value of those message context properties are demoted to those elements.

· The Body contains the payload of the message, that is the ISBN and Title elements.

Then, I created a map to transform an InboundBook message into a OutboundBook document on the FILE Send Port.

For the response message, I created an XML Schema to represent the author of a certain book. Even in this case, the schema is composed of a Header and Body elements. The Header contains the following elements:

· CorrelationToken.

· EpmRRCorrelationToken.

· ReqRespTransmitPipelineID.

· IsRequestResponse.

· RouteDirectToTP.

Each of those elements are promoted to the context properties having the same name and contained in the Microsoft.BizTalk.GlobalPropertySchemas assembly. In particular, the IsRequestResponse and RouteDirectToTP elements are defined as boolean and they have a Fixed value equal to 1.

Finally, I created a One-Way Receive called SyncToAsync.Response.ReceivePort and a corresponding FILE Receive Location called SyncToAsync.Response.FILE.ReceiveLocation. I set the standard XmlReceive pipeline on this Receive Location which contains the XmlDisassembler component that is responsible for promoting contained in the Header section of an Author message.

Inside the configuration file of the Driver application you can specify the location of the inbound and outbound folder used respectively by the FILE Receive and Send Ports. The client application used a FileSystemWatcher instance to trigger when a file is published by the Send Port on the outbound folder. When this event happens, a dialog opens where you can review the book information and enter the name and surname of the corresponding author. When you press the Submit button, an instance of the Author message is created for you by the Driver application, the value of the Header elements of the OutboundBook message are copied to the corresponding elements inside the Header of the Author message and the value of the IsRequestResponse and RouteDirectToTP elements is set to true. Then, the Author message is written in the inbound folder where it is received by the SyncToAsync.Response.FILE.ReceiveLocation which promotes the context properties and publishes the message to MessageBox. When the XML document is published, subscriptions are evaluated and the response message is passed to the WCF Receive Location which returns the message to the client application. The following picture depicts the overall architecture of the demo.

You can download the code here.

Conclusions

This technique is a good way to couple a synchronous Request Response Port with 2 Asynchronous One-Way Ports without using an Orchestration. This approach, compared with the standard approach which makes use of an Orchestration to implement the same behavior, allows saving 4 roundtrips to the MessageBox. Therefore, this pattern allows to decrease the contention on the SQL Server instance running the master BizTalkMsgBoxDb and speed up the overall message processing. By the way, I don’t think that the Product Group would support this solution, even if it doesn’t require any custom component. In fact, it uses context properties that were not probably meant to be used by developers, even if the BizTalk programmers’ community knows and uses them. Remember in fact that Microsoft could in any moment release a fix or a service pack that could change the behavior of the subscriptions evaluation and of the request-response correlation, invalidating this technique. On the other hand, this eventuality is not likely for the following reasons:

· The use of the request-response correlation mechanism is quite consolidated.

· Microsoft is working on R3 and the Oslo, and I don’t think they would have time and will to re-engineer this mechanism.

I have also some ideas on how to modify/customize the technique I described to obtain the same behavior in a slightly different way: for example, instead of including all the necessary context properties in the header of the response message and promote all of them, you could promote the EpmRRCorrelationToken, CorrelationToekn and RouteDirectToTP, since the IsRequestReponse and ReRespTransmitPipelineID are originally written properties. The solution I implemented is probably the fastest one, but it requires all the 5 properties to be propagated and promoted. Another approach could be to save the value of these properties on a DB with a custom pipeline component in the transmit pipeline run by the One-Way Send Port and retrieve them in the receive pipeline on the One-Way Receive Location. This mechanism would allow you to propagate just one unique identifier or correlation id that should be repeated in the response message. If you don’t want to use the EpmRRCorrelationToken or the CorrelationToken, you could even use an existing ID already present in the request and response documents as the OrderID or CustomerID. This approach is less performant (since you need to access a custom DB on the send and receive pipelines) but more flexible and less intrusive, and it’s relatively easy to implement.

MSDN Blogs