Paolo Salvatori's Blog

Handling Topics, Queues and Relay Services with the Service Bus Explorer Tool

Paolo Salvatori — Thu, 24 Nov 2011 16:49:07 GMT

The Windows Azure Service Bus Community Technology Preview (CTP), which was released in May 2011, first introduced queues and topics. At that time, the Windows Azure Management Portal didn’t provide a user interface to administer, create and delete messaging...(read more)

How to integrate a BizTalk Server application with Service Bus Queues and Topics

Paolo Salvatori — Thu, 03 Nov 2011 09:19:37 GMT

Microsoft BizTalk Server enables organizations to connect and extend heterogeneous systems across the enterprise and with trading partners. The Service Bus is part of Windows Azure AppFabric and is designed to provide connectivity, queuing, and routing...(read more)

Microsoft BizTalk 2010: Line of Business Systems Integration

Paolo Salvatori — Sun, 18 Sep 2011 08:07:12 GMT

I want to recommend a book I reviewed this weekend: Microsoft BizTalk 2010: Line of Business Systems Integration . Written by 5 BizTalk Server experts Microsoft BizTalk 2010: Line of Business Systems Integration is a new book from Packt. It is a guide...(read more)

How to use a WCF custom channel to implement client-side caching in a Windows Azure-hosted application

Paolo Salvatori — Wed, 14 Sep 2011 05:42:26 GMT

Introduction Some months ago I created a custom WCF protocol channel to transparently inject client-side caching capabilities into an existing WCF-enabled application just changing the its configuration file. Since I posted my first post on this subject...(read more)

Exploring Topics and Queues by Building a Service Bus Explorer Tool–Part 1

Paolo Salvatori — Fri, 08 Jul 2011 15:06:08 GMT

Introduction WARNING Due to the breaking changes introduced by the production version of the Service Bus Messaging API with respect to the CTP version, this version of the Service Bus Explorer tool doesn’t compile against the API contained in the Windows...(read more)

A Broker Pipeline Component for End-To-End Ordered Delivery

Paolo Salvatori — Wed, 01 Jun 2011 01:22:00 GMT

Introduction Recently I had chance to work with a health care company that makes an extensive use of the BizTalk Accelerator for HL7 for processing a large amount of messages containing clinical and administrative information. Health care companies need...(read more)

How to use a WCF custom channel to implement client-side caching

Paolo Salvatori — Tue, 05 Apr 2011 17:05:01 GMT

Introduction A couple of months ago Yossi Dahan told me that one of his customers in the UK was searching for a solution to transparently cache the response messages resulting from a WCF call. I immediately thought that this design pattern could be implemented...(read more)

Exploring Windows Azure Storage APIs By Building a Storage Explorer Application

Paolo Salvatori — Mon, 21 Feb 2011 16:23:10 GMT

Introduction The Windows Azure platform provides storage services for different kinds of persistent and transient data: Unstructured binary and text data Binary and text messages Structured data In order to support these categories of data, Windows Azure...(read more)

Intellisense when editing Service Bus configuration files? Yes please!

Paolo Salvatori — Mon, 07 Feb 2011 18:38:43 GMT

The problem The setup of Windows Azure AppFabric SDK adds the elements reported in the table below to the configuration contained in the the machine.config files under the following folders: %windir%\Microsoft.NET\Framework\v4.0.30319\Config %windir...(read more)

How to use Duplex MEP to communicate with BizTalk from a .NET application or a WF workflow running inside AppFabric Part 3

Paolo Salvatori — Thu, 20 Jan 2011 15:30:00 GMT

Introduction In the first article of the series we discussed how to exchange messages with an orchestration via a two-way WCF Receive Location using the Duplex Message Exchange Pattern . This form of bi-directional communication is characterized by the...(read more)

How to use Duplex MEP to communicate with BizTalk from a .NET application or a WF workflow running inside AppFabric Part 2

Paolo Salvatori — Thu, 20 Jan 2011 10:22:00 GMT

How to use Duplex MEP to communicate with BizTalk from a .NET application or a WF workflow running inside AppFabric Part 1

Paolo Salvatori — Mon, 22 Nov 2010 23:37:00 GMT

Introduction Some time ago, I was asked by a customer whether or not BizTalk WCF Adapters support the Duplex Message Exchange Pattern. The latter is characterized by the ability of both the service and the client to send messages to each other independently...(read more)

How to use AppFabric Auto-Start feature to avoid warm-up delays for IIS-hosted WCF Receive Locations

Paolo Salvatori — Sat, 30 Oct 2010 06:15:00 GMT

Scenario During recent years, I had the chance to work with many customers and I realized that one of the most recurring problems they usually experience is due to long start-up times: indeed, this is not a BizTalk specific issue, it’s rather...(read more)

How to Boost Performance of the ESB Routing and Transform Services – Part 3

Paolo Salvatori — Fri, 24 Sep 2010 18:13:17 GMT

Introduction In the first article of the series, we discussed the advantages and disadvantages of using Dynamic Send Ports in a BizTalk application and in particular within a solution that makes an extensive use of the ESB Toolkit. We also talked...(read more)

Large Message Transfer with WCF-Adapters Part 2

Paolo Salvatori — Thu, 09 Sep 2010 09:13:00 GMT

Introduction In the first part of the article we have introduced and analyzed the problem regarding the transmission and reception of a large message via a WCF Receive Location. In particular, we discussed the difference between the Buffered and Streamed...(read more)

How to Boost Performance of the ESB Routing and Transform Services – Part 2

Paolo Salvatori — Mon, 12 Jul 2010 17:09:00 GMT

Introduction In the first part of the article, we discussed the advantages and disadvantages of Dynamic Send Ports and we introduced two custom components that allow, respectively, to use Static Send Ports instead of Dynamic Send Port with an ESB Itinerary...(read more)

How to Boost Performance of the ESB Routing and Transform Services – Part 1

Paolo Salvatori — Wed, 07 Jul 2010 09:37:49 GMT

Introduction Recently I had chance to work with a couple of customers that make extensive use the ESB Toolkit (specifically, V 2.0 in this case) in their BizTalk solutions, and to exchange ideas with some of you regarding the alleged performance problems...(read more)

Video Recordings from the meeting of the Swedish BizTalk User Group

Paolo Salvatori — Tue, 08 Jun 2010 22:37:06 GMT

Below you can find a link to the video recording of the presentations that Mikael Håkanssons and Ewan Fairweather delivered at the meeting of the Swedish BizTalk User Group. Instrument your BizTalk Server Optimizing and verifying your BizTalk Server installation...(read more)

Large Message Transfer with WCF-Adapters Part 1

Paolo Salvatori — Tue, 25 May 2010 08:20:51 GMT

Introduction This is the first in a series of articles that will introduce and explore a couple a patterns to transfer and process large messages using WCF-Adapters. Generally speaking, an end to end BizTalk solution adopts the FTP Adapter or one of its...(read more)

Packt's SOA book Promotion

Paolo Salvatori — Wed, 28 Apr 2010 07:11:46 GMT

I publish the following announcement by Packt Publishing as some of you might be interested in their promotions: Innovative and fast-growing IT publisher, Packt Publishing ( www.PacktPub.com ) has announced a series of giveaways and promotions, including...(read more)

Yet on BizTalk Impersonation With WCF Adapters

Paolo Salvatori — Thu, 15 Apr 2010 14:33:32 GMT

Scenario

Last year I wrote a post on how using BizTalk Server 2006 R2/2009 and Protocol Transition to impersonate the original caller when invoking a downstream service that uses the Windows Integrated Security. Recently, one customer posed the following question to my colleague, Tim Wieman:

Can I create a WCF Send Port that is able to impersonate a statically defined domain user, other than then the service account of the host instance process, when calling a downstream WCF service that exposes a BasicHttpBinding/WsHttpBinding endpoint configured to use the Transport security mode ?

The answer is:

Yes, when using the Basic or the Digest authentication scheme.
No, when using the Windows or NTLM client credential type.

To verify this constraint, you can proceed as follows:

Open the BizTalk Administration Console.
Create a new WCF-BasicHttp or WCF-WsHttp Static Solicit-Response Send Port
Click the Configure button.
Select the Security tab.
Choose the Transport security mode.

At this point, if you select the Basic or Digest transport client credential type from the corresponding drop-down list:

You can click the Edit button in the User name credentials section, as highlighted in the picture below.

You can specify the Username and Password of the account that the Send Port will impersonate when invoking the target WCF service. As an alternative, you can leverage the Single Sign-On to redeem a ticket and pick a user at runtime from a certain affiliate application.

Instead, if you select the Ntlm or Windows transport client credential type from the corresponding drop-down list, the Edit button in the User name credentials section is greyed out, as shown in the picture below:

So at this point some of you might ask yourselves:

How can I impersonate a statically-defined user, different from the service account of the host process running my WCF Send Port, when invoking an underlying WCF service that uses the Transport security mode along with the Ntlm or Windows authentication scheme?

The answer is straightforward, you can achieve this objective using the WCF-Custom adapter and writing a custom WCF channel. Indeed, I didn’t create a new component from scratch, I just used grabbed some code from MSDN, and extended the component I wrote one year ago for my previous post on BizTalk and Protocol Transition . In particular, I made the following changes:

I extended the following components:
- InspectingBindingExtensionElement
- InspectingBindingElement
- InspectingChannelFactory
- InspectingRequestChannel
to expose two additional properties:
- WindowsUserName: gets or sets the domain account in the form of DOMAIN\Username that the Send Port will impersonate at runtime.
- WindowsUserPassword: gets or sets the password of the domain account.
Then I extended the WindowsUserPositionEnum type and therefore the WindowsUserPosition property to include a new mode called Static. As a consequence, the custom channel at runtime will retrieve the client credentials in a different way depending on the value of the WindowsUserPosition property exposed by the InspectingBindingExtensionElement component:
- Context: username will be read from the message context property identified by the ContextPropertyName and ContextPropertyNamespace properties exposed by the InspectingBindingExtensionElement. In this case, the environment must be properly configured to use Protocol Transition. See my previous post for more details.
- Message: username will be read from the message using the XPath expression contained in the WindowsUserXPath property. Even in this case, the environment must be properly configured to use Protocol Transition. See my previous post for more details.
- Static: the custom channel will use the client credentials contained in the WindowsUserName and WindowsUserPassword to impersonate the corresponding domain account before invoking the downstream WCF service. Note: this pattern requires the component to uses the client credentials to invoke the LogonUser function at runtime, but it does not require to configure the BizTalk environment for Protocol Transition.
I finally extended the InspectingRequestChannel and InspectingHelper classes to support the new Static mode. In particular, the custom channel at runtime performs the following steps to impersonate a given domain account declaratively defined in the WCF Send Port configuration:
- It calls the LogonUser static method exposed by the InspectingHelper class which in turn invokes the LogonUser Windows function which returns a token handle.
- The channel creates a new WindowsIdentity object using the constructor that accepts the user token returned by the previous call.
- Then, it invokes the Impersonate method exposed by the WindowsIdentity object.
- The channel invokes the underlying channel.
- In the finally block, when the call is complete, the channel invokes the Undo method on the WindowsImpersonationContext object returned by the Impersonate method, an then it invokes the CloseHandle static method exposed by the InspectingHelper class which in turn invokes the CloseHandle Windows function.

For your convenience, I report below the new code for the InspectingRequestChannel and InspectingHelper classes (I purposely omitted parts for ease of reading):

InspectingHelper class

#region Copyright
//-------------------------------------------------
// Author:  Paolo Salvatori
// Email:   paolos@microsoft.com
// History: 2008-09-17 Created
//-------------------------------------------------
#endregion

#region Using Directives
using System;
using System.Diagnostics;
using System.Configuration;
using System.Runtime.InteropServices;
using System.Security.Principal;
using System.Security.Permissions;
using System.ServiceModel;
using System.ServiceModel.Channels;
using System.ServiceModel.Configuration;
using System.DirectoryServices.ActiveDirectory;
using System.Xml;
using System.IO;
using System.Text;
using Microsoft.BizTalk.XPath;
#endregion

namespace Microsoft.BizTalk.CAT.Samples.ProtocolTransition.WCFExtensionLibrary
{
    /// <summary>
    /// This class exposes the logic to impersonate another user using the Protocol Transition mechanism.
    /// </summary>
    public class InspectingHelper
    {
        #region DllImport
        [DllImport("advapi32.dll", SetLastError = true, CharSet = CharSet.Unicode)]
        public static extern bool LogonUser(String lpszUsername, String lpszDomain, String lpszPassword,
            int dwLogonType, int dwLogonProvider, ref IntPtr phToken);

        [DllImport("kernel32.dll", CharSet = CharSet.Auto)]
        public extern static bool CloseHandle(IntPtr handle);
        #endregion

        #region Private Constants
        ...
        // The following constants are used when calling the LogonUser external function
        private const int LOGON32_PROVIDER_DEFAULT = 0;
        //This parameter causes LogonUser to create a primary token.
        private const int LOGON32_LOGON_INTERACTIVE = 2;
        #endregion

        #region Private Static Fields
        private static string domainFQDN = string.Empty;
        #endregion

        #region Public Static Constructor
        static InspectingHelper()
        {
            try
            {
                Domain domain = Domain.GetComputerDomain();
                if (domain != null)
                {
                    domainFQDN = domain.Name;
                }
            }
            catch (Exception ex)
            {
                Debug.WriteLine(string.Format(MessageFormat, ex.Message));
            }
        }
        #endregion

        #region Static Public Methods
        public static string GetUserPrincipalName(ref Message message, 
                                                  WindowsUserPositionEnum windowsUserPosition,
                                                  string contextPropertyName,
                                                  string contextPropertyNamespace,
                                                  string windowsUserXPath,
                                                  string windowsUserName,
                                                  string windowsUserPassword,
                                                  int maxBufferSize,
                                                  bool traceEnabled,
                                                  out string userName,
                                                  out string domainName)
        {
            string windowsUser = null;
            domainName = null;
            userName = null;
            try
            {
                switch (windowsUserPosition)
                {
                    case WindowsUserPositionEnum.Message:
                        if (message != null &&
                            !string.IsNullOrEmpty(windowsUserXPath))
                        {
                            MessageBuffer messageBuffer = message.CreateBufferedCopy(maxBufferSize);
                            if (messageBuffer == null)
                            {
                                throw new ApplicationException(MessageBufferCannotBeNull);
                            }
                            Message clone = messageBuffer.CreateMessage();
                            if (message == null)
                            {
                                throw new ApplicationException(CloneCannotBeNull);
                            }
                            message = messageBuffer.CreateMessage();
                            if (message == null)
                            {
                                throw new ApplicationException(MessageCannotBeNull);
                            }
                            XmlDictionaryReader xmlDictionaryReader = clone.GetReaderAtBodyContents();
                            if (xmlDictionaryReader == null)
                            {
                                throw new ApplicationException(XmlDictionaryReaderCannotBeNull);
                            }
                            XPathCollection xPathCollection = new XPathCollection();
                            if (xPathCollection == null)
                            {
                                throw new ApplicationException(XPathCollectionCannotBeNull);
                            }
                            XPathReader xPathReader = new XPathReader(xmlDictionaryReader, xPathCollection);
                            if (xPathReader == null)
                            {
                                throw new ApplicationException(XPathReaderCannotBeNull);
                            }
                            xPathCollection.Add(windowsUserXPath);
                            bool ok = false;
                            while (xPathReader.ReadUntilMatch())
                            {
                                if (xPathReader.Match(0) && !ok)
                                {
                                    windowsUser = xPathReader.ReadString();
                                    ok = true;
                                }
                            }
                        }
                        break;
                    case WindowsUserPositionEnum.Context:
                        if (string.IsNullOrEmpty(contextPropertyName))
                        {
                            throw new ApplicationException(ContextPropertyNameCannotBeNull);
                        }
                        if (string.IsNullOrEmpty(contextPropertyNamespace))
                        {
                            throw new ApplicationException(ContextPropertyNamespaceCannotBeNull);
                        }
                        string contextPropertyKey = string.Format(ContextPropertyKeyFormat, 
                                                                  contextPropertyNamespace, 
                                                                  contextPropertyName);
                        if (message.Properties.ContainsKey(contextPropertyKey))
                        {
                            windowsUser = message.Properties[contextPropertyKey] as string;
                        }
                        else
                        {
                            throw new ApplicationException(string.Format(NoContextPropertyFormat, contextPropertyKey));
                        }
                        break;
                    case WindowsUserPositionEnum.Static:
                        windowsUser = windowsUserName;
                        break;

                }
                if (!string.IsNullOrEmpty(windowsUser))
                {
                    string[] parts = windowsUser.Split(new char[] { Path.DirectorySeparatorChar });
                    if (parts != null &&
                        parts.Length > 1)
                    {
                        domainName = parts[0];
                        userName = parts[1];
                        Debug.WriteLineIf(traceEnabled, string.Format(CreatingUPNFormat, windowsUser));
                        string upn = string.Format(UserPrincipalNameFormat, parts[1], domainFQDN);
                        Debug.WriteLineIf(traceEnabled, string.Format(UsingUserPrincipalNameFormat, upn));
                        return upn;
                    }
                }
            }
            catch (Exception ex)
            {
                Debug.WriteLineIf(traceEnabled, string.Format(MessageFormat, ex.Message));
                throw ex;
            }
            return null;
        }

        public static bool LogonUser(string userName, 
                                     string domainName, 
                                     string windowsUserPassword,
                                     bool traceEnabled,
                                     ref IntPtr tokenHandle)
        {
            Debug.WriteLineIf(traceEnabled, string.Format(StartLogonUserFormat, 
                                                          domainName ?? Unknown, 
                                                          userName ?? Unknown));
            // Call LogonUser to obtain a handle to an access token.
            bool ok = LogonUser(userName, 
                                domainName, 
                                windowsUserPassword,
                                LOGON32_LOGON_INTERACTIVE, 
                                LOGON32_PROVIDER_DEFAULT,
                                ref tokenHandle);
            if (traceEnabled)
            {
                if (ok)
                {
                    Debug.WriteLineIf(traceEnabled, string.Format(LogonUserSucceededFormat, 
                                                                  domainName ?? Unknown, 
                                                                  userName ?? Unknown));
                }
                else
                {
                    Debug.WriteLineIf(traceEnabled, string.Format(LogonUserFailedFormat, 
                                                                  domainName ?? Unknown, 
                                                                  userName ?? Unknown));
                }
            }
            return ok;
        }

        public static bool CloseHandle(IntPtr tokenHandle,
                                       string userName,
                                       string domainName,
                                       bool traceEnabled)
        {
            Debug.WriteLineIf(traceEnabled, string.Format(StartCloseTokenFormat, 
                                                          domainName ?? Unknown, 
                                                          userName ?? Unknown));
            bool ok = CloseHandle(tokenHandle);
            if (traceEnabled)
            {
                if (ok)
                {
                    Debug.WriteLineIf(traceEnabled, string.Format(CloseTokenSucceededFormat, 
                                                                  domainName ?? Unknown, 
                                                                  userName ?? Unknown));
                }
                else
                {
                    Debug.WriteLineIf(traceEnabled, string.Format(CloseTokenFailedFormat, 
                                                                  domainName ?? Unknown, 
                                                                  userName ?? Unknown));
                }
            }
            return ok;
        }
        #endregion
    }
}

InspectingRequestChannel class

public class InspectingRequestChannel : InspectingChannelBase<IRequestChannel>,
                                        IRequestChannel
{
    ...

    public Message Request(Message message, TimeSpan timeout)
    {
        Message reply = null;
        string upn = null;
        WindowsImpersonationContext impersonationContext = null;
        IntPtr tokenHandle = new IntPtr(0);
        string userName = null;
        string domainName = null;

        try
        {
            if (componentEnabled)
            {
                WindowsIdentity identity = null;
                upn = InspectingHelper.GetUserPrincipalName(ref message,
                                                                windowsUserPosition,
                                                                contextPropertyName,
                                                                contextPropertyNamespace,
                                                                windowsUserXPath,
                                                                windowsUserName,
                                                                windowsUserPassword,
                                                                maxBufferSize,
                                                                traceEnabled,
                                                                out userName,
                                                                out domainName);
                if (windowsUserPosition == WindowsUserPositionEnum.Static)
                {

                    // Call LogonUser to obtain a handle to an access token.
                    bool returnValue = InspectingHelper.LogonUser(userName, 
                                                                  domainName, 
                                                                  windowsUserPassword,
                                                                  traceEnabled,
                                                                  ref tokenHandle);
                    // Protocol Transition is not necessary in this case
                    identity = new WindowsIdentity(tokenHandle);
                }
                else
                {
                    if (!string.IsNullOrEmpty(upn))
                    {
                        // Protocol Transition must be properly configured,
                        // otherwise the impersonation will fail
                        identity = new WindowsIdentity(upn);
                    }
                }
                Debug.WriteLineIf(traceEnabled, string.Format(ImpersonatingFormat, upn));
                impersonationContext = identity.Impersonate();
                Debug.WriteLineIf(traceEnabled, string.Format(ImpersonatedFormat, upn));
            }
            Debug.WriteLineIf(traceEnabled, CallingWebService);
            reply = this.InnerChannel.Request(message);
            Debug.WriteLineIf(traceEnabled, WebServiceCalled);
        }
        catch (Exception ex)
        {
            Debug.WriteLineIf(traceEnabled, string.Format(MessageFormat, ex.Message));
            throw ex;
        }
        finally
        {
            if (impersonationContext != null)
            {
                impersonationContext.Undo();
                Debug.WriteLineIf(traceEnabled, string.Format(ImpersonationUndoneFormat, upn ?? Unknown));
            }
            if (tokenHandle != IntPtr.Zero)
            {
                InspectingHelper.CloseHandle(tokenHandle,
                                             userName,
                                             domainName,
                                             traceEnabled);
            }
        }
        return reply;
    }
}

Test Case

To test my component, I created the following test case:

WinForm driver application submits a new request to a WCF-NetTcp Request-Response Receive Location

The Message Agent submits the incoming request message to the MessageBox (BizTalkMsgBoxDb).
The inbound request is consumed by a Solicit Response WCF-Custom Send Port. This latter uses a Filter Expression to receive all the documents published by the Receive Port hosting the WCF Receive Location.
The inbound message is mapped to the request format by the downstream HelloWorldService web service. This latter is hosted by IIS and exposes a single WsHttpBinding endpoint.
The WCF-Custom Send Port impersonates the user statically defined in the Port configuration and invokes the underlying HelloWorldService WCF service that in the scenario is hosted by a separate Application Pool (w3wp.exe) on the same IIS instance.
The HelloWorldService WCF service returns a response message.
The incoming response message is mapped to the format expected by the client application.
The transformed response message is published to the MessageBox.
The response message is retrieved by the Request-Response WCF Custom Receive Location which originally received the request call.
The response message is returned to the client WinForm application.

The following picture shows the binding configuration of the WCF Send Port used to communicate with the HelloWorldService.

Finally, the picture below reports the trace captured during a test run.

Conclusions

As I explained in one of my recent posts, using WCF extensibility points allows you customize in-depth the default behavior of BizTalk WCF Adapters. In particular, the WCF-Custom Adapter provides the possibility to specify the customize the composition of the binding and hence of the channel stack that will be created and used at runtime to communicate with external applications.

In this article we have seen how to exploit this characteristic to workaround and bypass a constraint of WCF Adapters. As usual, I had just a few hours to write the code and write the article, so should you find an error or a problem in my component, please send me an email or leave a comment on my blog, thanks!

You can find the new version of the code here.

How To Boost Message Transformations Using the XslCompiledTransform class Extended

Paolo Salvatori — Thu, 08 Apr 2010 12:07:00 GMT

Problem Statement

Some of you posted the following feedback regarding my post How To Boost Message Transformations Using the XslCompiledTransform class:

"Very nice post. I just have a question. How would you handle multiple input messages in a map? I have a map that has 2 input source messages that i would like to unit test. The BTS expression shape has transform(outmsg) = map(msg1, msg2), but i have not yet found a way to do it in a C# class."

Solution

Well, before answering to this question, I'll briefly explain how BizTalk Server handles maps with multiple input messages. BizTalk Server uses a trick to manage this situation. In fact, when you create a transformation map with multiple source document, BizTalk uses an envelope to wrap the individual input messages. For those of you who love to disassemble BizTalk code with Reflector, the envelope is created at runtime by the private class called CompositeStreamReader that can be found within the assembly Microsoft.XLANGs.Engine. In particular, the ConstructCompositeOutline method uses the code reported in the table below

public void ConstructCompositeOutline()
{
    ...
    XmlTextWriter writer = new XmlTextWriter(this.outlineStream, ...);
    writer.WriteStartElement("Root", "http://schemas.microsoft.com/BizTalk/2003/aggschema");
    for (int i = 0; i < this.readerCount; i++)
    {
        writer.WriteStartElement("InputMessagePart_" + i.ToString(), "");
        writer.WriteComment("_");
        writer.WriteEndElement();
    }
    writer.WriteEndElement();
    writer.Flush();
    ...
}

to create an envelope which targetNamespace is equal to 'http://schemas.microsoft.com/BizTalk/2003/aggschema' and that contains as many InputMessagePart elements as the incoming documents to the map.

<ns0:Root xmlns:ns0='http://schemas.microsoft.com/BizTalk/2003/aggschema'>
    <InputMessagePart_0>
        -
    </InputMessagePart_0>
    <InputMessagePart_1>
        -
    </InputMessagePart_1>
</ns0:Root>

Therefore, I decided to extend the code of my classes XslCompiledTransformHelper and XslTransformHelper to handle the case of maps with multiple input messages. In particular, I developed a new class called CompositeStream to wrap an array of Stream objects, one for each input message to the map, an return the above envelope. The implementation of this custom class is quite smart, because instead of copying the bytes of the input streams within a new buffer or a new stream, the Read method just makes up the content of the envelope with the data of the inbound streams to return a composite message with the format expected by the map. When the inbound streams are significantly large, this approach allows saving time and memory for copying data from the inbound streams to a new object, regardless if this latter is a buffer or a stream. The code of the CompositeStream class is shown in the table below:

CompositeStream Class

#region Copyright
//-------------------------------------------------
// Author:  Paolo Salvatori
// Email:   paolos@microsoft.com
// History: 2010-04-07 Created
//-------------------------------------------------
#endregion

#region Using References
using System;
using System.IO;
using System.Text;
using System.Collections.Generic;
using System.Configuration;
using System.Xml;
using System.Xml.Xsl;
using System.Diagnostics;
using Microsoft.XLANGs.BaseTypes;
using Microsoft.XLANGs.Core;
using Microsoft.BizTalk.Streaming;
using Microsoft.BizTalk.CAT.Samples.DynamicTransforms.Helpers.Properties;
#endregion

namespace Microsoft.BizTalk.CAT.Samples.DynamicTransforms.Helpers
{
    public class CompositeStream : Stream
    {
        #region Private Types
        enum State
        {
            Start,
            Overflow,
            Stream,
            End
        }
        #endregion

        #region Private Constants
        private const string DefaultPrefix = "babo";
        private const string StartRoot = "<{0}:Root xmlns:{0}='http://schemas.microsoft.com/BizTalk/2003/aggschema'>";
        private const string EndRoot = "</{0}:Root>";
        private const string StartInputMessagePart = "<InputMessagePart_{0}>";
        private const string EndInputMessagePart = "</InputMessagePart_{0}>";
        #endregion

        #region Private Fields
        private int currentStream = 0;
        private int currentIndex = 0;
        private string prefix;
        private State state;
        private byte[] overflowBuffer;
        private Stream[] streams;
        private bool endOfDocument = false;
        #endregion

        #region Public Constructors
        public CompositeStream(Stream[] streams)
        {
            this.prefix = DefaultPrefix;
            this.streams = streams;
            this.state = State.Start;
        }

        public CompositeStream(Stream[] streams, string prefix)
        {
            this.prefix = prefix;
            this.streams = streams;
            this.state = State.Start;
        }
        #endregion

        #region Public Properties
        public override bool CanRead
        {
            get 
            {
                return true;
            }
        }

        public override bool CanSeek
        {
            get
            {
                return true;
            }
        }

        public override bool CanTimeout
        {
            get
            {
                return false;
            }
        }

        public override bool CanWrite
        {
            get
            {
                return false;
            }
        }

        public override long Length
        {
            get
            {
                int prefixLength = prefix.Length;
                long length = 76 + 3 * prefixLength;
                if (streams != null &&
                    streams.Length > 0)
                {
                    string index;
                    for (int i = 0; i < streams.Length; i++)
                    {
                        if (streams[i].CanSeek)
                        {
                            index = i.ToString();
                            length += streams[i].Length + 39 + 2 * index.Length;
                        }
                        else
                        {
                            throw new NotImplementedException(); 
                        }
                    }
                }
                return length;
            }
        }

        public override long Position
        {
            get 
            { 
                if (state == State.Start)
                {
                    return 0L;
                }
                else
                {
                    throw new NotImplementedException(); 
                }
            }
            set
            {
                if (value == 0L)
                {
                    ResetStream();
                }
                else
                {
                    throw new NotImplementedException();
                } 
            }
        }
        #endregion

        #region Public Methods
        public override int Read(byte[] buffer, int offset, int count)
        {
            int bytesWritten = 0;
            int bytesRead = 0;
            int length = 0;
            byte[] localBuffer;
            StringBuilder builder;

            if (state == State.End)
            {
                return 0;
            }
            while (bytesWritten < count &&
                   state != State.End)
            {
                switch (state)
                {
                    case State.Start:
                        builder = new StringBuilder(128);
                        builder.AppendFormat(StartRoot, prefix);
                        if (streams != null &&
                            streams.Length > 0)
                        {
                            builder.AppendFormat(StartInputMessagePart, currentStream);
                        }
                        localBuffer = Encoding.UTF8.GetBytes(builder.ToString());

                        if (localBuffer.Length <= count)
                        {
                            Array.Copy(localBuffer, 0, buffer, offset, localBuffer.Length);
                            bytesWritten += localBuffer.Length;
                            offset += bytesWritten;
                            state = State.Stream;
                        }
                        else
                        {
                            Array.Copy(localBuffer, 0, buffer, offset, count);
                            overflowBuffer = localBuffer;
                            currentIndex = count;
                            state = State.Overflow;
                            return count;
                        }
                        break;
                    case State.Overflow:
                        length = overflowBuffer.Length - currentIndex;
                        if (length <= count)
                        {
                            Array.Copy(overflowBuffer, currentIndex, buffer, offset, length);
                            bytesWritten += length;
                            offset += length;
                            overflowBuffer = null;
                            currentIndex = 0;
                            if (endOfDocument)
                            {
                                state = State.End;
                            }
                            else
                            {
                                state = State.Stream;
                            }
                        }
                        else
                        {
                            Array.Copy(overflowBuffer, currentIndex, buffer, offset, count);
                            currentIndex += count;
                            return count;
                        }
                        break;
                    case State.Stream:
                        length = count - bytesWritten;
                        bytesRead = streams[currentStream].Read(buffer, offset, length);
                        bytesWritten += bytesRead;
                        offset += bytesRead;
                        if (bytesWritten < count)
                        {
                            builder = new StringBuilder(128);
                            builder.AppendFormat(EndInputMessagePart, currentStream);
                            currentStream++;
                            if (currentStream < streams.Length)
                            {
                                builder.AppendFormat(StartInputMessagePart, currentStream);
                                localBuffer = Encoding.UTF8.GetBytes(builder.ToString());
                            }
                            else
                            {
                                builder.AppendFormat(EndRoot, prefix);
                                localBuffer = Encoding.UTF8.GetBytes(builder.ToString());
                                endOfDocument = true;
                            }
                            if (localBuffer.Length <= count - bytesWritten)
                            {
                                Array.Copy(localBuffer, 0, buffer, offset, localBuffer.Length);
                                bytesWritten += localBuffer.Length;
                                offset += localBuffer.Length;
                                if (endOfDocument)
                                {
                                    if (bytesWritten <= count)
                                    {
                                        state = State.End;
                                    }
                                    return bytesWritten;
                                }
                                break;
                            }
                            else
                            {
                                length = count - bytesWritten;
                                Array.Copy(localBuffer, 0, buffer, offset, length);
                                overflowBuffer = localBuffer;
                                currentIndex = length;
                                state = State.Overflow;
                                return count;
                            }
                        }
                        else
                        {
                            return count;
                        }
                        break;                        
                }
            }
            return bytesWritten;
        }

        public override IAsyncResult BeginRead(byte[] buffer, int offset, int count, AsyncCallback callback, object state)
        {
            return base.BeginRead(buffer, offset, count, callback, state);
        }

        public override IAsyncResult BeginWrite(byte[] buffer, int offset, int count, AsyncCallback callback, object state)
        {
            return base.BeginWrite(buffer, offset, count, callback, state);
        }

        public override void Write(byte[] buffer, int offset, int count)
        {
            throw new NotImplementedException();
        }

        public override void SetLength(long value)
        {
            throw new NotImplementedException();
        }

        public override long Seek(long offset, SeekOrigin origin)
        {
            if (offset == 0 &&
                origin == SeekOrigin.Begin)
            {
                ResetStream();
            }
            else
            {
                throw new NotImplementedException();
            }
            return 0;
        }

        public override void Flush()
        {
            throw new NotImplementedException();
        }
        #endregion

        #region Private Methods
        private void ResetStream()
        {
            for (int i = 0; i < streams.Length; i++)
            {
                if (streams[i].CanSeek)
                {
                    streams[i].Seek(0, SeekOrigin.Begin);
                }
                else
                {
                    throw new NotImplementedException(); 
                }
            }
            state = State.Start;
            endOfDocument = false;
            currentStream = 0;
            currentIndex = 0;
        }
        #endregion
    }
}

Then I extended the XslCompiledTransformHelper class with a set of new methods that accept as parameter an array of objects of type Stream or XLANGMessage and use an instance of the CompositeStream class to apply a transformation map to these latter.

XslCompiledTransformHelper Class

#region Copyright
//-------------------------------------------------
// Author:  Paolo Salvatori
// Email:   paolos@microsoft.com
// History: 2010-01-26 Created
//-------------------------------------------------
#endregion

#region Using References
using System;
using System.IO;
using System.Text;
using System.Collections.Generic;
using System.Configuration;
using System.Xml;
using System.Xml.Xsl;
using System.Xml.XPath;
using System.Diagnostics;
using Microsoft.XLANGs.BaseTypes;
using Microsoft.XLANGs.Core;
using Microsoft.BizTalk.Streaming;
using Microsoft.BizTalk.CAT.Samples.DynamicTransforms.Helpers.Properties;
#endregion

namespace Microsoft.BizTalk.CAT.Samples.DynamicTransforms.Helpers
{
    public class XslCompiledTransformHelper
    {
        ...
        public static XLANGMessage Transform(XLANGMessage[] messageArray,
                                             int[] partIndexArray,
                                             string mapFullyQualifiedName,
                                             string messageName,
                                             string partName,
                                             bool debug,
                                             int bufferSize,
                                             int thresholdSize)
        {

            try
            {
                if (messageArray != null &&
                    messageArray.Length > 0)
                {
                    Stream[] streamArray = new Stream[messageArray.Length];
                    for (int i = 0; i < messageArray.Length; i++)
                    {
                        streamArray[i] = messageArray[i][partIndexArray[i]].RetrieveAs(typeof(Stream)) as Stream;
                    }
                    Stream response = Transform(streamArray, mapFullyQualifiedName, debug, bufferSize, thresholdSize);
                    CustomBTXMessage customBTXMessage = null;
                    customBTXMessage = new CustomBTXMessage(messageName, Service.RootService.XlangStore.OwningContext);
                    customBTXMessage.AddPart(string.Empty, partName);
                    customBTXMessage[0].LoadFrom(response);
                    return customBTXMessage.GetMessageWrapperForUserCode();
                }
            }
            catch (Exception ex)
            {
                ExceptionHelper.HandleException(Resources.XslCompiledTransformHelper, ex);
                TraceHelper.WriteLineIf(debug,
                                        null,
                                        ex.Message,
                                        EventLogEntryType.Error);
                throw;
            }
            finally
            {
                if (messageArray != null &&
                    messageArray.Length > 0)
                {
                    for (int i = 0; i < messageArray.Length; i++)
                    {
                        if (messageArray[i] != null)
                        {
                            messageArray[i].Dispose();
                        }
                    }
                }
            }
            return null;
        }

        
        public static Stream Transform(Stream[] streamArray, 
                                       string mapFullyQualifiedName)
        {
            return Transform(streamArray,
                             mapFullyQualifiedName,
                             false,
                             DefaultBufferSize,
                             DefaultThresholdSize);
        }

        public static Stream Transform(Stream[] streamArray,
                                       string mapFullyQualifiedName,
                                       bool debug)
        {
            return Transform(streamArray,
                             mapFullyQualifiedName,
                             debug,
                             DefaultBufferSize,
                             DefaultThresholdSize);
        }

        public static Stream Transform(Stream[] streamArray,
                                       string mapFullyQualifiedName,
                                       bool debug,
                                       int bufferSize,
                                       int thresholdSize)
        {
            try
            {
                MapInfo mapInfo = GetMapInfo(mapFullyQualifiedName, debug);
                if (mapInfo != null)
                {
                    CompositeStream compositeStream = null;

                    try
                    {
                        VirtualStream virtualStream = new VirtualStream(bufferSize, thresholdSize);
                        compositeStream = new CompositeStream(streamArray);
                        XmlTextReader reader = new XmlTextReader(compositeStream);
                        mapInfo.Xsl.Transform(reader, mapInfo.Arguments, virtualStream);
                        virtualStream.Seek(0, SeekOrigin.Begin);
                        return virtualStream;
                    }
                    finally
                    {
                        if (compositeStream != null)
                        {
                            compositeStream.Close();
                        }
                    }
                }
            }
            catch (Exception ex)
            {
                ExceptionHelper.HandleException(Resources.XslCompiledTransformHelper, ex);
                TraceHelper.WriteLineIf(debug,
                                        null,
                                        ex.Message,
                                        EventLogEntryType.Error);
                throw;
            }
            return null;
        }
        ...
    }
}

In a similar way, I extended the XslTransformHelper class to support maps with multiple input documents. For brevity, I omitted the code of this latter, but you can download the new version of both classes here.

Test

To test the new methods exposed by the classes XslCompiledTransformHelper and XslTransformHelper, I created three XML schemas:

Credentials
Address
Customer

In particular, the Credentials and Address schemas define two complex types used to build the Customer schema. Then I created a transformation map (see the picture below) called AddressAndCredentialsToCustomer that accepts 2 input messages, the first of type Credentials and the second of type Address, and returns a document of type Customer.

Finally I created 2 Unit Test methods called:

TestXslTransformHelperWithMultipleInputs
TestXslCompiledTransformHelperWithMultipleInputs

to test the CompositeStream and the new methods exposed by the the classes XslCompiledTransformHelper and XslTransformHelper. You can change the entries contained in the appSettings section of the App.Config file within the UnitAndLoadTests project to test the 2 classes against your documents and multi-input-message maps.

Conclusions

I updated the original post to reflect the extensions I made. Here you can find a new version of my helper classes and the artifacts (schemas, maps, unit tests) I used to test them.

Note: I spent less than one day to write and test the new code. In particular, I conducted a basic code coverage of the CompositeStream class, but I didn’t test it with more than 2 streams or with large messages. If you find any error, please send me a repro and I’ll do my best, time permitting, to fix the code as soon as possible. Instead, should you find an error in my code and decide to sort it out by yourselves, please let me have the fixed version. ;-)

How to exploit the Text In Row table option to boost BizTalk Server Performance

Paolo Salvatori — Tue, 16 Mar 2010 10:32:13 GMT

Text in Row Table Option in SQL Server

In SQL Server, text, ntext, or image fields are large character or binary strings (up to 2 gigabytes) that by default stored outside a data row. The data row contains only a 16-byte text pointer that points to the root node of a tree built of internal pointers. These pointers map the pages in which the string fragments are stored.

SQL Server provides a table option called text in row to declare that the contents of the fields of type text, ntext, or image whose dimensions are smaller than those of a data page (8Kb) must be stored in the data row. When a large value type or a large object data type column value is stored in the data row, the Database Engine does not have to access a separate page or set of pages to read or write the character or binary string. This makes reading and writing the in-row strings about as fast as reading or writing limited size varchar, nvarchar, or varbinary strings. Similarly, when the values are stored off-row, the Database Engine incurs an additional page read or write.

For more information on this topic, see the following articles:

SQL Server 2005

“In-Row Data” topic in the SQL Server 2005 Books Online.
“sp_tableoption (Transact-SQL)” topic in the SQL Server 2005 Books Online.

SQL Server 2008

“In-Row Data” topic in the SQL Server 2008 Books Online.
“sp_tableoption (Transact-SQL)” topic in the SQL Server 2008 Books Online.

How to exploit the Text in Row table option in BizTalk Server

The following section explains how and when applying the text in row table option to boost BizTalk performance.

Parts Table

When the message size is less than 8k the text in row table option can be applied to the Parts table that has 2 fields of type image:

imgPart: contains a message Part or message part fragment
imgPropBag: contains the message part property bag.

This technique can lead to a decrease of CPU usage up to 20-30% on the SQL and BizTalk nodes and to a significant improvement in terms of latency and throughput, when messages are smaller than 8kb.

Spool Table

Similarly, when the average size of the message context is less than 8 kb, this technique can be extended to the Spool table. This latter contains a column of type image:

imgContext: this field contains the message context.

To successfully apply this technique to the Spool table, you must eliminate unnecessary context properties and distinguished fields to reduce the size of the message context below 8 Kb.

DynamicStateInfo Tables

Finally, the text in row table option can be applied to the DynamicStateInfo_Host tables that contain the serialized state of orchestrations when they encounter a persistence point during their execution. These tables, one for each host, contain a field of type image:

imgData: contains binary-serialized orchestrations

When orchestrations running within a host HostA are so small that their size once serialized is less than 8 kb, the text in row technique can successfully be applied to the DynamicStateInfo_HostA table. Therefore it’s advisable to keep the internal state of orchestrations as small as possible. This technique can significantly reduce the time spent by the XLANG Engine to serialize, persist and restore the internal state of an orchestration in case of persistence point. Moreover, when the size of this latter is smaller than 8 Kb, it allows using the text in row technique with DynamicStateInfo_Host tables. The following techniques can be used to reduce the memory footprint of orchestrations:

· Create variables and messages as late as possible and release them as early as possible for example introducing non transactional scopes inside your orchestrations and declaring variables and messages within these inner scopes instead of declaring them at the top-most level.

· Use static variables and components whenever possible. When a persistence point occurs, these latter will not be serialized along with the orchestration.

Text In Row in Action

During a recent Performance and Stability review with a customer, I applied the text in row technique to the Parts table (see the picture below) of the BizTalkMsgBoxDb as the average size of messages processed by the BizTalk application was less than 2 Kb.

EXEC sp_tableoption N'Parts', 'text in row', '4000'

This led to a decrease of 22% in % Processor Time on BizTalk nodes and to a significant reduction of I/O contention on the SQL Server instance hosting the BizTalkMsgBoxDb.

References

For more information on this topic, see the following articles:

“What you can and can't do with the MessageBox Database server” topic in the BizTalk Core Engine's Weblog.
"BizTalk Server Database Optimization" article on MSDN.

How to Throw Typed Fault Exceptions From Orchestrations Published as WCF Services Revisited

Paolo Salvatori — Fri, 26 Feb 2010 15:18:56 GMT

A couple of you experienced the following error:

“Digest verification failed for Reference '#_0'”

when using my component with the following configuration:

Adapter: WCF-Custom
Binding: WsHttpBinding or NetTcpBinding
Security: Message
ClientCredentialType: Windows

So I decided to reopen the project and look into this problem. I made some researches on MSDN finally I decided to change approach. Hence I made the following changes:

I replaced the custom endpoint behavior with a custom service behavior.
I substituted the MessageInspector component with a custom class which implements the IErrorHandler interface. This implementation is more appropriate to create a custom typed fault.

I successfully tested the new component using 3 different kinds of WCF Receive Locations:

Scenario 1

Adapter: WCF-CustomIsolated
Binding: WsHttpBinding
Security Mode: None
ClientCredentialType: N/A

Scenario 2

Adapter: WCF-CustomIsolated
Binding: WsHttpBinding
Security Mode: Transport
ClientCredentialType: Windows

Scenario 3

Adapter: WCF-Custom
Binding: NetTcpBinding
Security Mode: Message
ClientCredentialType: Windows

Finally, I refreshed the content of the this my post to reflect the new implementation and uploaded the code of the version here. In order to use the new component, you need to accomplish the following steps:

read the article to understand the differences and characteristics of the new implementation.
properly register the FQDN of the new ErrorHandler assembly in the machine.config.
Reconfigure your Receive Location to use the new custom service behavior instead of the previous custom endpoint behavior.

How To Boost Message Transformations Using the XslCompiledTransform class

Paolo Salvatori — Fri, 29 Jan 2010 12:12:00 GMT

Introduction

The BizTalk Runtime still makes an extensive use of the System.Xml.Xsl.XslTransform. When you create and build a BizTalk project, a separate .NET class is generated. for each transformation map. Each of these classes inherits from the Microsoft.XLANGs.BaseTypes.TransformBase class. For convenience, I used Reflector to retrieve and report its code in the table below. As you can easily note, the get accessor of the Transform property returns a XslTransform object.

TransformBase class

[Serializable]
public abstract class TransformBase
{
    // Methods
    protected TransformBase()
    {
    }

    // Properties
    public virtual string[] SourceSchemas
    {
        get
        {
            return null;
        }
    }

    public BTSXslTransform StreamingTransform
    {
        get
        {
            StringReader input = new StringReader(this.XmlContent);
            XmlTextReader stylesheet = new XmlTextReader(input);
            BTSXslTransform transform = new BTSXslTransform();
            transform.Load(stylesheet, null, base.GetType().Assembly.Evidence);
            return transform;
        }
    }

    public virtual string[] TargetSchemas
    {
        get
        {
            return null;
        }
    }

    public XslTransform Transform
    {
        get
        {
            StringReader input = new StringReader(this.XmlContent);
            XmlTextReader stylesheet = new XmlTextReader(input);
            XslTransform transform = new XslTransform();
            transform.Load(stylesheet, null, base.GetType().Assembly.Evidence);
            return transform;
        }
    }

    public XsltArgumentList TransformArgs
    {
        get
        {
            XmlDocument document = new XmlDocument();
            document.PreserveWhitespace = true;
            document.LoadXml(this.XsltArgumentListContent);
            XsltArgumentList list = new XsltArgumentList();
            foreach (XmlNode node in document.SelectNodes("//ExtensionObjects/ExtensionObject"))
            {
                XmlAttributeCollection attributes = node.Attributes;
                XmlNode namedItem = attributes.GetNamedItem("Namespace");
                XmlNode node3 = attributes.GetNamedItem("AssemblyName");
                XmlNode node4 = attributes.GetNamedItem("ClassName");
                object extension = Assembly.Load(node3.Value).CreateInstance(node4.Value);
                list.AddExtensionObject(namedItem.Value, extension);
            }
            return list;
        }
    }

    public abstract string XmlContent { get; }

    public abstract string XsltArgumentListContent { get; }
}

When BizTalk Server 2004 was built, the XslTransform was the only class provided by the Microsoft .NET Framework 1.1 to apply an XSLT to an inbound XML document. When the Microsoft .NET Framework version 2.0. was released, the XslTransform was declared obsolete and thus deprecated. As clearly stated on MSDN, the System.Xml.Xsl.XslCompiledTransform should be used instead. This class is used to compile and execute XSLT transformations. In most cases, the XslCompiledTransform class significantly outperforms the XslTransform class in terms of time need to execute the same XSLT against the same inbound XML document. The article Migrating From the XslTransform Class on MSDN reports as follows:

“The XslCompiledTransform class includes many performance improvements. The new XSLT processor compiles the XSLT style sheet down to a common intermediate format, similar to what the common language runtime (CLR) does for other programming languages. Once the style sheet is compiled, it can be cached and reused.”

The caveat is that because the XSLT is compiled to MSIL, the first time the transform is run there is a performance hit, but subsequent executions are much faster. To avoid paying the extra cost of initial compilation every time a map is executed, this latter could be cached in a static structure (e.g. Dictionary). I’ll show you how to implement this pattern in the second part of the article. For a detailed look at the performance differences between the XslTransform and XslCompiledTransform classes (plus comparisons with other XSLT processors) have a look at following posts.

Although the overall performance of the XslCompiledTransform class is better than the XslTransform class, the Load method of the XslCompiledTransform class might perform more slowly than the Load method of the XslTransform class the first time it is called on a transformation. This is because the XSLT file must be compiled before it is loaded. However, if you cache an XslCompiledTransform object for subsequent calls, its Transform method is incredibly faster than the equivalent Transform method of the XslTransform class. Therefore, from a performance perspective:

The XslTransform class is the best choice in a "Load once, Transform once" scenario as it doesn't require the initial map-compilation.
The XslCompiledTransform class is the best choice in a "Load once, Cache and Transform many times" scenario as it implies the initial cost for the map-compilation, but then this overhead is highly compensated by the fact that subsequent calls are much faster.

As BizTalk is a server application (or, if you prefer an application server), the second scenario is more likely than the first. The only way to take advantage of this class (given that BizTalk does not currently make use of the XslCompiledTransform class) is to write custom components. If this seems a little strange to you, remember that all BizTalk versions since BizTalk Server 2004 have inherited that core engine, based on .NET Framework 1.1. Since the XslCompiledTransform class wasn’t added until .NET Framework 2.0, it wasn’t leveraged in that version of BizTalk. While I’m currently working with the product group to see how best to take advantage of this class in the next version of BizTalk, let’s go ahead and walk through creating a helper class to boost the performance of message transformations in your current BizTalk implementation using the XslCompiledTransform class and let’s compare its performance with another helper component that makes use the old XslTransform class.

BizTalk Application

In order to compare the performance of the XslTransform and XslCompiledTransform classes I created an easy BizTalk application composed of the following projects:

Helpers

This library contains 2 helpers classes called, respectively, XslTransformHelper and XslCompiledTransformHelper. These components share most of the code and expose the same static methods. I minimized the differences between the 2 classes as the final scope was to compare the performance of the XslTransform and XslCompiledTransform classes. As their name suggests, the first helper class uses the XslTransform class, while the second makes use of the XslCompiledTransform class. The Transform static method of both helper classes provides multiple overloads/variants/signatures. This allows the components to be invoked by any orchestration, pipeline component or .NET class in general. Either classes use a static Dictionary to cache maps in-process for later calls. The fully qualified name (FQDN) of a BizTalk map is used as key to retrieve the value of the corresponding instance within the Dictionary. The fully qualified name (FQDN) of a BizTalk map can be easily determined as follows:

Open the BizTalk Administration Console and navigate to the Maps folder within your BizTalk application.
Double click the map in question.
Copy the content of the Name label (see the picture below) and paste it in a text editor.
Append a comma followed by a space (“, “).
Copy the content of the Assembly label (see the picture below) and paste it in a text editor.

Pretty easy, don’t you think?

XslTransformHelper class

#region Copyright
//-------------------------------------------------
// Author:  Paolo Salvatori
// Email:   paolos@microsoft.com
// History: 2010-01-26 Created
//-------------------------------------------------
#endregion

#region Using References
using System;
using System.IO;
using System.Text;
using System.Collections.Generic;
using System.Configuration;
using System.Xml;
using System.Xml.XPath;
using System.Xml.Xsl;
using System.Diagnostics;
using Microsoft.XLANGs.BaseTypes;
using Microsoft.XLANGs.Core;
using Microsoft.BizTalk.Streaming;
using Microsoft.BizTalk.CAT.Samples.DynamicTransforms.Helpers.Properties;
#endregion

namespace Microsoft.BizTalk.CAT.Samples.DynamicTransforms.Helpers
{
    public class XslTransformHelper
    {
        #region Private Constants
        private const int DefaultBufferSize = 10240; //10 KB
        private const int DefaultThresholdSize = 1048576; //1 MB
        private const string DefaultPartName = "Body";
        #endregion

        #region Private Static Fields
        private static Dictionary<string, TransformBase> mapDictionary;
        #endregion

        #region Static Constructor
        static XslTransformHelper()
        {
            mapDictionary = new Dictionary<string, TransformBase>();
        }
        #endregion

        #region Public Static Methods
        public static XLANGMessage Transform(XLANGMessage message,
                                    string mapFullyQualifiedName,
                                    string messageName)
        {
            return Transform(message,
                             0,
                             mapFullyQualifiedName,
                             messageName,
                             DefaultPartName,
                             false,
                             DefaultBufferSize,
                             DefaultThresholdSize);
        }

        public static XLANGMessage Transform(XLANGMessage message,
                                             string mapFullyQualifiedName,
                                             string messageName,
                                             bool debug)
        {
            return Transform(message,
                             0,
                             mapFullyQualifiedName,
                             messageName,
                             DefaultPartName,
                             debug,
                             DefaultBufferSize,
                             DefaultThresholdSize);
        }

        public static XLANGMessage Transform(XLANGMessage message,
                                             int partIndex,
                                             string mapFullyQualifiedName,
                                             string messageName,
                                             string partName,
                                             bool debug,
                                             int bufferSize,
                                             int thresholdSize)
        {

            try
            {
                using (Stream stream = message[partIndex].RetrieveAs(typeof(Stream)) as Stream)
                {
                    Stream response = Transform(stream, mapFullyQualifiedName, debug, bufferSize, thresholdSize);
                    CustomBTXMessage customBTXMessage = null;
                    customBTXMessage = new CustomBTXMessage(messageName, Service.RootService.XlangStore.OwningContext);
                    customBTXMessage.AddPart(string.Empty, partName);
                    customBTXMessage[0].LoadFrom(response);
                    return customBTXMessage.GetMessageWrapperForUserCode();
                }
            }
            catch (Exception ex)
            {
                ExceptionHelper.HandleException(Resources.XslCompiledTransformHelper, ex);
                TraceHelper.WriteLineIf(debug,
                                        null,
                                        ex.Message,
                                        EventLogEntryType.Error);
                throw;
            }
            finally
            {
                if (message != null)
                {
                    message.Dispose();
                }
            }
        }

        public static XLANGMessage Transform(XLANGMessage[] messageArray,
                                             int[] partIndexArray,
                                             string mapFullyQualifiedName,
                                             string messageName,
                                             string partName,
                                             bool debug,
                                             int bufferSize,
                                             int thresholdSize)
        {

            try
            {
                if (messageArray != null &&
                    messageArray.Length > 0)
                {
                    Stream[] streamArray = new Stream[messageArray.Length];
                    for (int i = 0; i < messageArray.Length; i++)
                    {
                        streamArray[i] = messageArray[i][partIndexArray[i]].RetrieveAs(typeof(Stream)) as Stream;
                    }
                    Stream response = Transform(streamArray, mapFullyQualifiedName, debug, bufferSize, thresholdSize);
                    CustomBTXMessage customBTXMessage = null;
                    customBTXMessage = new CustomBTXMessage(messageName, Service.RootService.XlangStore.OwningContext);
                    customBTXMessage.AddPart(string.Empty, partName);
                    customBTXMessage[0].LoadFrom(response);
                    return customBTXMessage.GetMessageWrapperForUserCode();
                }
            }
            catch (Exception ex)
            {
                ExceptionHelper.HandleException(Resources.XslCompiledTransformHelper, ex);
                TraceHelper.WriteLineIf(debug,
                                        null,
                                        ex.Message,
                                        EventLogEntryType.Error);
                throw;
            }
            finally
            {
                if (messageArray != null &&
                    messageArray.Length > 0)
                {
                    for (int i = 0; i < messageArray.Length; i++)
                    {
                        if (messageArray[i] != null)
                        {
                            messageArray[i].Dispose();
                        }
                    }
                }
            }
            return null;
        }

        public static Stream Transform(Stream stream,
                                       string mapFullyQualifiedName)
        {
            return Transform(stream,
                             mapFullyQualifiedName,
                             false,
                             DefaultBufferSize,
                             DefaultThresholdSize);
        }

        public static Stream Transform(Stream stream,
                                       string mapFullyQualifiedName,
                                       bool debug)
        {
            return Transform(stream,
                             mapFullyQualifiedName,
                             debug,
                             DefaultBufferSize,
                             DefaultThresholdSize);
        }

        public static Stream Transform(Stream stream,
                                       string mapFullyQualifiedName,
                                       bool debug,
                                       int bufferSize,
                                       int thresholdSize)
        {
            try
            {
                TransformBase transformBase = GetTransformBase(mapFullyQualifiedName);
                if (transformBase != null)
                {
                    VirtualStream virtualStream = new VirtualStream(bufferSize, thresholdSize);
                    XPathDocument xpathDocument = new XPathDocument(stream);
                    transformBase.Transform.Transform(xpathDocument, transformBase.TransformArgs, virtualStream);
                    virtualStream.Seek(0, SeekOrigin.Begin);
                    return virtualStream;
                }
            }
            catch (Exception ex)
            {
                ExceptionHelper.HandleException(Resources.DynamicTransformsHelper, ex);
                TraceHelper.WriteLineIf(debug,
                                        null,
                                        ex.Message,
                                        EventLogEntryType.Error);
                throw;
            }
            return null;
        }

        public static Stream Transform(Stream[] streamArray, 
                                       string mapFullyQualifiedName)
        {
            return Transform(streamArray,
                             mapFullyQualifiedName,
                             false,
                             DefaultBufferSize,
                             DefaultThresholdSize);
        }

        public static Stream Transform(Stream[] streamArray,
                                       string mapFullyQualifiedName,
                                       bool debug)
        {
            return Transform(streamArray,
                             mapFullyQualifiedName,
                             debug,
                             DefaultBufferSize,
                             DefaultThresholdSize);
        }

        public static Stream Transform(Stream[] streamArray,
                                       string mapFullyQualifiedName,
                                       bool debug,
                                       int bufferSize,
                                       int thresholdSize)
        {
            try
            {
                TransformBase transformBase = GetTransformBase(mapFullyQualifiedName);
                if (transformBase != null)
                {
                    CompositeStream compositeStream = null;

                    try
                    {
                        VirtualStream virtualStream = new VirtualStream(bufferSize, thresholdSize);
                        compositeStream = new CompositeStream(streamArray);
                        XPathDocument xpathDocument = new XPathDocument(compositeStream);
                        transformBase.Transform.Transform(xpathDocument, transformBase.TransformArgs, virtualStream);
                        virtualStream.Seek(0, SeekOrigin.Begin);
                        return virtualStream;
                    }
                    finally
                    {
                        if (compositeStream != null)
                        {
                            compositeStream.Close();
                        }
                    }
                }
            }
            catch (Exception ex)
            {
                ExceptionHelper.HandleException(Resources.DynamicTransformsHelper, ex);
                TraceHelper.WriteLineIf(debug,
                                        null,
                                        ex.Message,
                                        EventLogEntryType.Error);
                throw;
            }
            return null;
        }
        #endregion

        #region Private Static Methods
        private static TransformBase GetTransformBase(string mapFullyQualifiedName)
        {
            TransformBase transformBase = null;
            lock (mapDictionary)
            {
                if (!mapDictionary.ContainsKey(mapFullyQualifiedName))
                {
                    Type type = Type.GetType(mapFullyQualifiedName);
                    transformBase = Activator.CreateInstance(type) as TransformBase;
                    if (transformBase != null)
                    {
                        mapDictionary[mapFullyQualifiedName] = transformBase;
                    }
                }
                else
                {
                    transformBase = mapDictionary[mapFullyQualifiedName];
                }
            }
            return transformBase;
        }
        #endregion
    }
}

XslCompiledTransformHelper class

#region Copyright
//-------------------------------------------------
// Author:  Paolo Salvatori
// Email:   paolos@microsoft.com
// History: 2010-01-26 Created
//-------------------------------------------------
#endregion

#region Using References
using System;
using System.IO;
using System.Text;
using System.Collections.Generic;
using System.Configuration;
using System.Xml;
using System.Xml.Xsl;
using System.Xml.XPath;
using System.Diagnostics;
using Microsoft.XLANGs.BaseTypes;
using Microsoft.XLANGs.Core;
using Microsoft.BizTalk.Streaming;
using Microsoft.BizTalk.CAT.Samples.DynamicTransforms.Helpers.Properties;
#endregion

namespace Microsoft.BizTalk.CAT.Samples.DynamicTransforms.Helpers
{
    public class XslCompiledTransformHelper
    {
        #region Private Constants
        private const int DefaultBufferSize = 10240; //10 KB
        private const int DefaultThresholdSize = 1048576; //1 MB
        private const string DefaultPartName = "Body";
        #endregion

        #region Private Static Fields
        private static Dictionary<string, MapInfo> mapDictionary;
        #endregion

        #region Static Constructor
        static XslCompiledTransformHelper()
        {
            mapDictionary = new Dictionary<string, MapInfo>();
        }
        #endregion

        #region Public Static Methods
        public static XLANGMessage Transform(XLANGMessage message,
                                            string mapFullyQualifiedName,
                                            string messageName)
        {
            return Transform(message,
                             0,
                             mapFullyQualifiedName,
                             messageName,
                             DefaultPartName,
                             false,
                             DefaultBufferSize,
                             DefaultThresholdSize);
        }

        public static XLANGMessage Transform(XLANGMessage message,
                                             string mapFullyQualifiedName,
                                             string messageName,
                                             bool debug)
        {
            return Transform(message,
                             0,
                             mapFullyQualifiedName,
                             messageName,
                             DefaultPartName,
                             debug,
                             DefaultBufferSize,
                             DefaultThresholdSize);
        }

        public static XLANGMessage Transform(XLANGMessage message,
                                             int partIndex,
                                             string mapFullyQualifiedName,
                                             string messageName,
                                             string partName,
                                             bool debug,
                                             int bufferSize,
                                             int thresholdSize)
        {

            try
            {
                using (Stream stream = message[partIndex].RetrieveAs(typeof(Stream)) as Stream)
                {
                    Stream response = Transform(stream, mapFullyQualifiedName, debug, bufferSize, thresholdSize);
                    CustomBTXMessage customBTXMessage = null;
                    customBTXMessage = new CustomBTXMessage(messageName, Service.RootService.XlangStore.OwningContext);
                    customBTXMessage.AddPart(string.Empty, partName);
                    customBTXMessage[0].LoadFrom(response);
                    return customBTXMessage.GetMessageWrapperForUserCode();
                }
            }
            catch (Exception ex)
            {
                ExceptionHelper.HandleException(Resources.XslCompiledTransformHelper, ex);
                TraceHelper.WriteLineIf(debug,
                                        null,
                                        ex.Message,
                                        EventLogEntryType.Error);
                throw;
            }
            finally
            {
                if (message != null)
                {
                    message.Dispose();
                }
            }
        }

        public static XLANGMessage Transform(XLANGMessage[] messageArray,
                                             int[] partIndexArray,
                                             string mapFullyQualifiedName,
                                             string messageName,
                                             string partName,
                                             bool debug,
                                             int bufferSize,
                                             int thresholdSize)
        {

            try
            {
                if (messageArray != null &&
                    messageArray.Length > 0)
                {
                    Stream[] streamArray = new Stream[messageArray.Length];
                    for (int i = 0; i < messageArray.Length; i++)
                    {
                        streamArray[i] = messageArray[i][partIndexArray[i]].RetrieveAs(typeof(Stream)) as Stream;
                    }
                    Stream response = Transform(streamArray, mapFullyQualifiedName, debug, bufferSize, thresholdSize);
                    CustomBTXMessage customBTXMessage = null;
                    customBTXMessage = new CustomBTXMessage(messageName, Service.RootService.XlangStore.OwningContext);
                    customBTXMessage.AddPart(string.Empty, partName);
                    customBTXMessage[0].LoadFrom(response);
                    return customBTXMessage.GetMessageWrapperForUserCode();
                }
            }
            catch (Exception ex)
            {
                ExceptionHelper.HandleException(Resources.XslCompiledTransformHelper, ex);
                TraceHelper.WriteLineIf(debug,
                                        null,
                                        ex.Message,
                                        EventLogEntryType.Error);
                throw;
            }
            finally
            {
                if (messageArray != null &&
                    messageArray.Length > 0)
                {
                    for (int i = 0; i < messageArray.Length; i++)
                    {
                        if (messageArray[i] != null)
                        {
                            messageArray[i].Dispose();
                        }
                    }
                }
            }
            return null;
        }

        public static Stream Transform(Stream stream, 
                                       string mapFullyQualifiedName)
        {
            return Transform(stream, 
                             mapFullyQualifiedName, 
                             false, 
                             DefaultBufferSize, 
                             DefaultThresholdSize);
        }

        public static Stream Transform(Stream stream,
                                       string mapFullyQualifiedName,
                                       bool debug)
        {
            return Transform(stream, 
                             mapFullyQualifiedName, 
                             debug, 
                             DefaultBufferSize, 
                             DefaultThresholdSize);
        }

        public static Stream Transform(Stream stream,
                                       string mapFullyQualifiedName,
                                       bool debug,
                                       int bufferSize,
                                       int thresholdSize)
        {
            try
            {
                MapInfo mapInfo = GetMapInfo(mapFullyQualifiedName, debug);
                if (mapInfo != null)
                {
                    XmlTextReader xmlTextReader = null;

                    try
                    {
                        VirtualStream virtualStream = new VirtualStream(bufferSize, thresholdSize);
                        xmlTextReader = new XmlTextReader(stream);
                        mapInfo.Xsl.Transform(xmlTextReader, mapInfo.Arguments, virtualStream);
                        virtualStream.Seek(0, SeekOrigin.Begin);
                        return virtualStream;
                    }
                    finally
                    {
                        if (xmlTextReader != null)
                        {
                            xmlTextReader.Close();
                        }
                    }
                }
            }
            catch (Exception ex)
            {
                ExceptionHelper.HandleException(Resources.XslCompiledTransformHelper, ex);
                TraceHelper.WriteLineIf(debug,
                                        null,
                                        ex.Message,
                                        EventLogEntryType.Error);
                throw;
            }
            return null;
        }

        public static Stream Transform(Stream[] streamArray, 
                                       string mapFullyQualifiedName)
        {
            return Transform(streamArray,
                             mapFullyQualifiedName,
                             false,
                             DefaultBufferSize,
                             DefaultThresholdSize);
        }

        public static Stream Transform(Stream[] streamArray,
                                       string mapFullyQualifiedName,
                                       bool debug)
        {
            return Transform(streamArray,
                             mapFullyQualifiedName,
                             debug,
                             DefaultBufferSize,
                             DefaultThresholdSize);
        }

        public static Stream Transform(Stream[] streamArray,
                                       string mapFullyQualifiedName,
                                       bool debug,
                                       int bufferSize,
                                       int thresholdSize)
        {
            try
            {
                MapInfo mapInfo = GetMapInfo(mapFullyQualifiedName, debug);
                if (mapInfo != null)
                {
                    CompositeStream compositeStream = null;

                    try
                    {
                        VirtualStream virtualStream = new VirtualStream(bufferSize, thresholdSize);
                        compositeStream = new CompositeStream(streamArray);
                        XmlTextReader reader = new XmlTextReader(compositeStream);
                        mapInfo.Xsl.Transform(reader, mapInfo.Arguments, virtualStream);
                        virtualStream.Seek(0, SeekOrigin.Begin);
                        return virtualStream;
                    }
                    finally
                    {
                        if (compositeStream != null)
                        {
                            compositeStream.Close();
                        }
                    }
                }
            }
            catch (Exception ex)
            {
                ExceptionHelper.HandleException(Resources.XslCompiledTransformHelper, ex);
                TraceHelper.WriteLineIf(debug,
                                        null,
                                        ex.Message,
                                        EventLogEntryType.Error);
                throw;
            }
            return null;
        }
        #endregion

        #region Private Static Methods
        private static MapInfo GetMapInfo(string mapFullyQualifiedName,
                                          bool debug)
        {
            MapInfo mapInfo = null;
            lock (mapDictionary)
            {
                if (!mapDictionary.ContainsKey(mapFullyQualifiedName))
                {
                    Type type = Type.GetType(mapFullyQualifiedName);
                    TransformBase transformBase = Activator.CreateInstance(type) as TransformBase;
                    if (transformBase != null)
                    {
                        XslCompiledTransform map = new XslCompiledTransform(debug);
                        using (StringReader stringReader = new StringReader(transformBase.XmlContent))
                        {
                            XmlTextReader xmlTextReader = null;

                            try
                            {
                                xmlTextReader = new XmlTextReader(stringReader);
                                XsltSettings settings = new XsltSettings(true, true);
                                map.Load(xmlTextReader, settings, new XmlUrlResolver());
                                mapInfo = new MapInfo(map, transformBase.TransformArgs);
                                mapDictionary[mapFullyQualifiedName] = mapInfo;
                            }
                            finally
                            {
                                if (xmlTextReader != null)
                                {
                                    xmlTextReader.Close();
                                }
                            }
                        }
                    }
                }
                else
                {
                    mapInfo = mapDictionary[mapFullyQualifiedName];
                }
            }
            return mapInfo;
        }
        #endregion
    }

    public class MapInfo
    {
        #region Private Fields
        private XslCompiledTransform xsl;
        private XsltArgumentList arguments;
        #endregion

        #region Public Constructors
        public MapInfo()
        {
            this.xsl = null;
            this.arguments = null;
        }

        public MapInfo(XslCompiledTransform xsl,
                       XsltArgumentList arguments)
        {
            this.xsl = xsl;
            this.arguments = arguments;
        }
        #endregion

        #region Public Properties
        public XslCompiledTransform Xsl
        {
            get
            {
                return this.xsl;
            }
            set
            {
                this.xsl = value;
            }
        }

        public XsltArgumentList Arguments
        {
            get
            {
                return this.arguments;
            }
            set
            {
                this.arguments = value;
            }
        }
        #endregion
    }
}

Note: Support for embedded scripts is an optional XSLT setting on the XslCompiledTransform class. Script support is disabled by default. Therefore, to enable script support, it’s necessary to create an XsltSettings object with the EnableScript property set to true and pass the object to the Load method. That’s what I did in my code above.

Schemas

This project contains 2 Xml Schemas, CalculatorRequest and CalculatorResponse, which define, respectively, the request and response message and a PropertySchema that defines the Method promoted property. A CalculatorRequest message can contain zero or multiple Operation elements, as shown in the following picture:

CalculatorRequest message

<CalculatorRequest xmlns="http://microsoft.biztalk.cat/10/dynamictransforms/calculatorrequest">
  <Method>UnitTest</Method>
  <Operations>
    <Operation>
      <Operator>+</Operator>
      <Operand1>82</Operand1>
      <Operand2>18</Operand2>
    </Operation>
    <Operation>
      <Operator>-</Operator>
      <Operand1>30</Operand1>
      <Operand2>12</Operand2>
    </Operation>
    <Operation>
      <Operator>*</Operator>
      <Operand1>25</Operand1>
      <Operand2>8</Operand2>
    </Operation>
    <Operation>
      <Operator>\</Operator>
      <Operand1>100</Operand1>
      <Operand2>25</Operand2>
    </Operation>
  </Operations>
</CalculatorRequest>

A CalculatorResponse message contains a Result element for each Operation element within the corresponding CalculatorRequest message, as shown in the following picture:

CalculatorResponse message

<CalculatorResponse xmlns="http://microsoft.biztalk.cat/10/dynamictransforms/calculatorresponse">
      <Status>Ok</Status>
      <Results>
            <Result>
                  <Value>100</Value>
                  <Error>None</Error>
            </Result>
            <Result>
                  <Value>18</Value>
                  <Error>None</Error>
            </Result>
            <Result>
                  <Value>200</Value>
                  <Error>None</Error>
            </Result>
            <Result>
                  <Value>4</Value>
                  <Error>None</Error>
            </Result>
      </Results>
</CalculatorResponse>

Maps

This project contains the CalculatorRequestToCalculatorResponse map (see the picture below) that transforms an inbound request message into the corresponding response message.

Orchestrations

This project contains the 4 orchestrations.

SingleDynamicTransform Test Case

This flow had been created just to test the XslCompiledTransformHelper class within an orchestration.

The following picture depicts the architecture of the SingleDynamicTransform test case.

Message Flow:

A One-Way FILE Receive Location receives a new CalculatorRequest xml document from the IN folder.
The XML disassembler component within the XMLTransmit pipeline promotes the Method element inside the CalculatorRequest xml document. The Message Agent submits the incoming message to the MessageBox (BizTalkMsgBoxDb).
The inbound request starts a new instance of the SingleDynamicTransform. This latter uses a Direct Bound Port and a Filter to receive only the CalculatorRequest messages with the Method promoted property = “SingleDynamicTransform”.
The SingleDynamicTransform invokes the Transform static method exposed by the XslCompiledTransformHelper class to apply the CalculatorRequestToCalculatorResponse map to the inbound CalculatorRequest message and generate the corresponding CalculatorResponse document.
The CalculatorRequestToCalculatorResponse publishes the CalculatorResponse message to the MessageBox (BizTalkMsgBoxDb).
The response message is retrieved by a One-Way FILE Send Port.
The response message is written to an OUT folder by the One-Way FILE Send Port.

DefaultStaticLoop Test Case

As shown in the picture below, this orchestration receives a CalculatorRequest xml document (80KB) and executes a loop (1000 iterations) in which it uses a Transform Shape to apply the CalculatorRequestToCalculatorResponse map to the inbound message. The orchestration does not produce any response message. The code within the StartStepTrace and EndStepTrace Expression Shapes keeps track of the time spent to execute the map at each iteration, while the code contained in the final Trace Expression Shape writes the total elapsed time on the standard output. The objective of this test case is to measure the time spent by the orchestration to apply the map to the inbound document 1000 times using the Transform Shape.

The following picture depicts the architecture of the DefaultStaticLoop test case.

Message Flow:

A One-Way FILE Receive Location receives a new CalculatorRequest xml document from the IN folder.
The XML disassembler component within the XMLTransmit pipeline promotes the Method element inside the CalculatorRequest xml document. The Message Agent submits the incoming message to the MessageBox (BizTalkMsgBoxDb).
The inbound request starts a new instance of the DefaultStaticLoop . This latter uses a Direct Bound Port and a Filter to receive only the CalculatorRequest messages with the Method promoted property = “DefaultStaticLoop”.
The DefaultStaticLoop executes a loop (1000 iterations) in which it uses a Transform Shape to apply the CalculatorRequestToCalculatorResponse map to the inbound CalculatorRequest message (80KB).

DefaultDynamicLoop Test Case

This component is a variation of the DefaultStaticLoop orchestration. As this latter, it receives a CalculatorRequest xml document (80KB) and executes a loop (1000 iterations), but it doesn’t use a Transform shape to execute the CalculatorRequestToCalculatorResponse map against the inbound message, it rather uses a Message Assignment Shape that contain the following code. See How to Use Expressions to Dynamic Transform Messages for more information on this topic. The objective of this test case is to measure the time spent by the orchestration to apply the map to the inbound document 1000 times using the transform statement provided by the XLANG Runtime.

startTime = System.DateTime.Now;
type = System.Type.GetType("<Map FQDN>");
transform(CalculatorResponse) = type(CalculatorRequest);
stopTime = System.DateTime.Now;
elapsedTime = stopTime.Subtract(startTime);
total = total + elapsedTime.TotalMilliseconds;
i = i + 1;

As the DefaultStaticLoop, the orchestration does not produce any response. The code within the CreateResponse Shape keeps track of the time spent to execute the map at each iteration, while the code contained in the final Trace Expression Shape writes the total elapsed time on the standard output.

The following picture depicts the architecture of the DefaultDynamicLoop test case.

Message Flow:

A One-Way FILE Receive Location receives a new CalculatorRequest xml document from the IN folder.
The XML disassembler component within the XMLTransmit pipeline promotes the Method element inside the CalculatorRequest xml document. The Message Agent submits the incoming message to the MessageBox (BizTalkMsgBoxDb).
The inbound request starts a new instance of the DefaultDynamicLoop . This latter uses a Direct Bound Port and a Filter to receive only the CalculatorRequest messages with the Method promoted property = “DefaultDynamicLoop”.
The DefaultDynamicLoop executes a loop (1000 iterations) in which it uses a Message Assignment Shape to execute the CalculatorRequestToCalculatorResponse map against the inbound CalculatorRequest message (80KB).

CustomDynamicLoop Test Case

As the previous orchestrations, the CustomDynamicLoop receives a CalculatorRequest xml document (80KB) and executes a loop (1000 iterations). However, instead of using a Transform shape or the Dynamic Transformation mechanism provided by BizTalk to apply the map to the inbound document, it uses an Expression Shape (see the code below) to invoke the Transform method exposed by my XslCompiledTransformHelper component. The objective of this test case is to measure the time spent by the orchestration to apply the map to the inbound document 1000 times using the XslCompiledTransformHelper class.

startTime = System.DateTime.Now;
CalculatorResponse = Microsoft.BizTalk.CAT.Samples.DynamicTransforms.Helpers.XslCompiledTransformHelper.Transform(CalculatorRequest, "<Map FQDN>");
stopTime = System.DateTime.Now;
elapsedTime = stopTime.Subtract(startTime);
total = total + elapsedTime.TotalMilliseconds;
i = i + 1;

As the previous orchestrations, the CustomDynamicLoop does not produce any response. The code within the final Trace Expression Shape writes the total elapsed time on the standard output.

The following picture depicts the architecture of the CustomDynamicLoop test case.

Message Flow:

A One-Way FILE Receive Location receives a new CalculatorRequest xml document from the IN folder.
The XML disassembler component within the XMLTransmit pipeline promotes the Method element inside the CalculatorRequest xml document. The Message Agent submits the incoming message to the MessageBox (BizTalkMsgBoxDb).
The inbound request starts a new instance of the CustomDynamicLoop. This latter uses a Direct Bound Port and a Filter to receive only the CalculatorRequest messages with the Method promoted property = “CustomDynamicLoop”.
The CustomDynamicLoop executes a loop (1000 iterations) in which it uses a XslCompiledTransformHelper class to execute the CalculatorRequestToCalculatorResponse map against the inbound CalculatorRequest message (80KB).

Pipeline Components

This project contains 2 custom pipeline components called, respectively, TransformPipelineComponent and LoopbackPipelineComponent.

TransformPipelineComponent

This component can be used within a Receive or a Send custom pipeline to transform the inbound message using the XslCompiledTransformHelper class. For the sake of brevity, we just report the code of the Execute method of in the picture below. Note that the if the loopback property exposed by the component equals true, this latter promotes the RouteDirectToTp context property to true. This way, when the TransformPipelineComponent is used by a Receive Pipeline within a Request-Response Receive Location, when the Message Agent posts the transformed message to the MessageBox, this latter is immediately returned as a response to the Receive Location (Loopback pattern).

Execute method

public IBaseMessage Execute(IPipelineContext context, IBaseMessage message)
{
    try
    {
        if (componentEnabled)
        {
            if (context == null)
            {
                throw new ArgumentException("The pipeline context parameter cannot be null.");
            }
            if (message != null)
            {
                IBaseMessagePart bodyPart = message.BodyPart;
                Stream inboundStream = bodyPart.GetOriginalDataStream();
                Stream outboundStream = XslCompiledTransformHelper.Transform(inboundStream, mapFQDN, traceEnabled, bufferSize, thresholdSize);
                bodyPart.Data = outboundStream;
                context.ResourceTracker.AddResource(inboundStream);
                context.ResourceTracker.AddResource(outboundStream);
                if (loopback)
                {
                    message.Context.Promote("RouteDirectToTP", "http://schemas.microsoft.com/BizTalk/2003/system-properties", true);
                }
            }
        }
    }
    catch (Exception ex)
    {
        ExceptionHelper.HandleException("TransformPipelineComponent", ex);
        TraceHelper.WriteLineIf(traceEnabled,
                                context,
                                ex.Message,
                                EventLogEntryType.Error);
    }
    return message;
}

LoopbackPipelineComponent

This component can be used to set the RouteDirectToTp context property to true to implement the Loopback pattern. When used within a Receive Pipeline, the component allows to promote the MessageType property without the need to use an Xml Disassembler. At runtime, the MessageType is mandatory to determine the map to apply to a given message on a Request or Send Port.

Execute method

public IBaseMessage Execute(IPipelineContext context, IBaseMessage message)
{
    try
    {
        if (loopback)
        {
            message.Context.Promote("RouteDirectToTP", "http://schemas.microsoft.com/BizTalk/2003/system-properties", true);
            if (messageType != null)
            {
                message.Context.Promote("MessageType", "http://schemas.microsoft.com/BizTalk/2003/system-properties", messageType);
            }
        }
    }
    catch (Exception ex)
    {
        ExceptionHelper.HandleException("LoopbackPipelineComponent", ex);
        TraceHelper.WriteLineIf(traceEnabled,
                                context,
                                ex.Message,
                                EventLogEntryType.Error);
    }
    return message;
}

Pipelines

This project contains 2 custom pipelines:

TransformReceivePipeline:this pipeline contains only an instance of the TransformPipelineComponent.
LoopbackReceivePipeline: this pipeline contains only an instance of the LoopbackPipelineComponent.

Then, I created 2 use cases to compare the performance of the default message transformation provided by BizTalk Messaging Engine and the message transformation accomplished using my XslCompiledTransformHelper class.

TransformStaticallyDefined Test Case

The following picture depicts the architecture of the TransformStaticallyDefined test case.

Message Flow:

The DT.TransformStaticallyDefined.WCF-NetTcp.RL WCF-NetTcp Request-Response Receive Location receives a CalculatorRequest xml document submitted running the InvokeStaticMap Unit Test within Visual Studio.
The LoopbackReceivePipeline promotes the RouteDirectToTp property to true and the MessageType property. I could have used the Xml Disassembler component within the Receive Pipeline to find and promote the MessageType, but I preferred to specify the MessageType of the inbound message as part of the configuration of the Receive Location (see the picture below). This way I can avoid the overhead introduced by the Xml Disassembler component and measure just the time spent by the Messaging Engine to apply the CalculatorRequestToCalculatorResponse map statically defined on the Receive Port. Once transformed the CalculatorRequest message into a CalculatorResponse document, the Message Agent posts this latter to the MessageBox.
The transformed message is immediately returned to the Receive Location.
The response message is returned to the InvokeStaticMap Unit Test.

DT.TransformStaticallyDefined.RP Configuration

The screen below shows that the use of the CalculatorRequestToCalculatorResponse map has been statically configured on the DT.TransformStaticallyDefined.RP Receive Port.

DT.TransformStaticallyDefined.WCF-NetTcp.RL Configuration

The following picture shows the configuration of the LoopbackReceivePipeline on the DT.TransformStaticallyDefined.WCF-NetTcp.RL Receive Location.

TransformReceivePipeline Test Case

The following picture depicts the architecture of the TransformReceivePipeline test case.

Message Flow:

The DT.TransformReceivePipeline.WCF-NetTcp.RL WCF-NetTcp Request-Response Receive Location receives a CalculatorRequest xml document submitted running the InvokeDynamicMap Unit Test within Visual Studio.
The TransformPipelineComponent (the following picture shows its configuration) promotes the RouteDirectToTp property to true and transforms the inbound message using the XslCompiledTransformHelper class and the CalculatorRequestToCalculatorResponse map. Then the Message Agent posts the transformed message to the MessageBox.
The transformed message is immediately returned to the Receive Location.
The response message is returned to the InvokeDynamicMap Unit Test.

DT.TransformReceivePipeline.WCF-NetTcp.RL Configuration

The following picture shows the configuration of the TransformReceivePipeline on the DT.TransformReceivePipeline.WCF-NetTcp.RL Receive Location.

UnitAndLoadTests

Finally, I created a Test Project called UnitAndLoadTests that contains a small set of unit and load tests described below:

TestXslTransformHelper: this unit test can be used to measure the time spent to execute loops transformations using the XslTransformHelper class, where loops is defined in the configuration file. The following picture reports the code of the TestXslTransformHelper unit test.

TestXslTransformHelper method

[TestMethod]
public void TestXslTransformHelper()
{
    Assert.AreNotEqual<string>(null, 
                               inputFile, 
                               "The inpuFile key in the configuration file cannot be null.");
    Assert.AreNotEqual<string>(String.Empty, 
                               inputFile, 
                               "The inpuFile key in the configuration file cannot be empty.");
    Assert.AreEqual<bool>(true, 
                          File.Exists(inputFile), 
                          string.Format(CultureInfo.CurrentCulture, "The {0} file does not exist.", inputFile));
    Assert.AreNotEqual<string>(null, 
                               mapFullyQualifiedName, 
                               "The mapFullyQualifiedName key in the configuration file cannot be null.");
    Assert.AreNotEqual<string>(String.Empty, 
                               mapFullyQualifiedName, 
                               "The mapFullyQualifiedName key in the configuration file cannot be empty.");
    if (traceResponses)
    {
        Assert.AreEqual<bool>(true, 
                              Directory.Exists(outputFolder), 
                              string.Format(CultureInfo.CurrentCulture, "The {0} folder does not exist.", outputFolder));
    }
    Type type = null;
    try
    {
        type = Type.GetType(mapFullyQualifiedName);
    }
    catch (Exception ex)
    {
        Assert.Fail(ex.Message);
    }
    MemoryStream stream = null;
    string message;
    using (StreamReader reader = new StreamReader(File.Open(inputFile, FileMode.Open, FileAccess.Read, FileShare.Read)))
    {
        message = reader.ReadToEnd();
    }
    byte[] buffer = Encoding.UTF8.GetBytes(message);
    Stopwatch stopwatch = new Stopwatch();
    Stream output = null;
    TestContext.BeginTimer("TestXslTransformHelper");
    for (int i = 0; i < loops; i++)
    {
        stream = new MemoryStream(buffer);
        stopwatch.Start();
        output = XslTransformHelper.Transform(stream, mapFullyQualifiedName);
        stopwatch.Stop();
        if (output != null && traceResponses)
        {
            using (StreamReader reader = new StreamReader(output))
            {
                message = reader.ReadToEnd();
            }
            using (StreamWriter writer = 
                    new StreamWriter(File.OpenWrite(
                        Path.Combine(outputFolder, 
                                     string.Format(CultureInfo.CurrentCulture, "{{{0}}}.xml", Guid.NewGuid().ToString())))))
            {
                writer.Write(message);
                writer.Flush();
            }
        }
    }
    TestContext.EndTimer("TestXslTransformHelper");
    Trace.WriteLine(String.Format(CultureInfo.CurrentCulture, 
                                  "[TestXslTransformHelper] Loops: {0} Elapsed Time (milliseconds): {1}", loops, stopwatch.ElapsedMilliseconds));
}

TestXslCompiledTransformHelper: this unit test can be used to measure the time spent to execute loops transformations using the XslCompiledTransformHelper class, where loops is defined in the configuration file. For the sake of brevity, I omitted to include the code of the TestXslCompiledTransformHelper unit test as this latter is very similar to one of the previous unit test.
InvokeStaticMap: this unit test can be used to send a single CalculatorRequest xml document to the DT.TransformStaticallyDefined.WCF-NetTcp.RL Receive Location used by the TransformStaticallyDefined Test case.
InvokeDynamicMap: this unit test can be used to send a single CalculatorRequest xml document to the DT.TransformReceivePipeline.WCF-NetTcp.RL Receive Location used by the TransformReceivePipeline Test case.
StaticMapLoadTest: this load test is based on the InvokeStaticMap unit test and can be used to generate traffic against the TransformStaticallyDefined Use Case.
DynamicMapLoadTest: this load test is based on the InvokeDynamicMap unit test and can be used to generate traffic against the TransformReceivePipeline Use Case.

All these tests share the same configuration contained in the App.config configuration file. In particular this latter contains the following information:

The WCF Endpoint used to invoke the DT.TransformStaticallyDefined.WCF-NetTcp.RL and DT.TransformReceivePipeline.WCF-NetTcp.RL Receive Locations.
The appSettings section defines multiple keys that allows to control the runtime behavior of unit and load tests:
- mapFullyQualifiedName: contains the name of the map used by TestXslTransformHelper and TestXslCompiledTransformHelper unit tests.
- inputFile: defines the path of the inbound document used by all unit tests (TestXslTransformHelper , TestXslCompiledTransformHelper, InvokeStaticMap, InvokeDynamicMap).
- outputFolder: indicates the path where to save response messages.
- traceResponses: indicates whether to save response messages.
- loops: allows to control the number of loop iterations performed by the TestXslTransformHelper and TestXslCompiledTransformHelper unit tests.

For the sake of completeness, I include below the App.config I used for my tests.

App.config file

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <system.serviceModel>
    <!-- Bindings used by client endpoints -->
    <bindings>
      <netTcpBinding>
        <binding name="netTcpBinding"
                         closeTimeout="01:10:00"
                         openTimeout="01:10:00"
                         receiveTimeout="01:10:00"
                         sendTimeout="01:10:00"
                         transactionFlow="false"
                         transferMode="Buffered"
                         transactionProtocol="OleTransactions"
                         hostNameComparisonMode="StrongWildcard"
                         listenBacklog="100"
                         maxBufferPoolSize="1048576"
                         maxBufferSize="10485760"
                         maxConnections="200"
                         maxReceivedMessageSize="10485760">
          <readerQuotas maxDepth="32"
                                  maxStringContentLength="8192"
                                  maxArrayLength="16384"
                                  maxBytesPerRead="4096"
                                  maxNameTableCharCount="16384" />
          <reliableSession ordered="true"
                                     inactivityTimeout="00:10:00"
                                     enabled="false" />
          <security mode="None">
            <transport clientCredentialType="Windows" protectionLevel="EncryptAndSign" />
            <message clientCredentialType="Windows" />
          </security>
        </binding>
      </netTcpBinding>
    </bindings>
    <client>
      <!-- Client endpoints used by client excahnge messages with the WCF Receive Locations -->
      <endpoint address="net.tcp://localhost:3816/dynamictransforms"
                      binding="netTcpBinding"
                      bindingConfiguration="netTcpBinding"
                      contract="System.ServiceModel.Channels.IRequestChannel"
                      name="StaticMapEndpoint" />
      <endpoint address="net.tcp://localhost:3817/dynamictransforms"
                      binding="netTcpBinding"
                      bindingConfiguration="netTcpBinding"
                      contract="System.ServiceModel.Channels.IRequestChannel"
                      name="DynamicMapEndpoint" />
    </client>
  </system.serviceModel>
  <appSettings>
    <add key="mapFullyQualifiedName" value="Microsoft.BizTalk.CAT.Samples.DynamicTransforms.Maps.CalculatorRequestToCalculatorResponse,
              Microsoft.BizTalk.CAT.Samples.DynamicTransforms.Maps, Version=1.0.0.0, Culture=neutral, PublicKeyToken=8c83cae5bc47edb0"/>
    <add key="inputFile" value="C:\Projects\DynamicTransforms\Test\UnitTest.xml"/>
    <add key="outputFolder" value="C:\Projects\DynamicTransforms\Test\Out"/>
    <add key="traceResponses" value="false"/>
    <add key="loops" value="1000"/>
  </appSettings>
</configuration>

Results

Let’s start running some of the test cases and unit tests I created. Take into account that the unit tests and that you can find in the code attached to the article are parametric and they can be executed using any xml message and map. Therefore, I strongly encourage you to repeat my tests using your own messages and maps.

TestXslTransformHelper vs TestXslCompiledTransformHelper

I configured both the unit tests to execute the CalculatorRequestToCalculatorResponse map against the the UnitTest.xml file (80KB) 1000 times. Each test method uses an instance of the Stopwatch class to measure the time spent to executing all calls and finally traces a message containing the total elapsed time. The screens below were taken within Visual Studio at the end of the 2 tests.

TestXslTransformHelper

TestXslCompiledTransformHelper

The difference in terms of performance between the 2 unit tests is simply astonishing:

TestXslTransformHelper Unit Test: Total Elapsed Time = ~144 seconds, Average Elapsed Time/Transformation = ~144 milliseconds
TestXslCompiledTransformHelper Unit Test: Total Elapsed Time = ~3.5 seconds, Average Elapsed Time/Transformation = ~3.5 milliseconds

Obviously, I conducted several test runs and they all confirmed that the XslCompiledTransformHelper is class incredibly faster than the XslTransformHelper class and this clearly demonstrates that the XslCompiledTransform class is absolutely much better than the XslTransform class in a Load once, Cache and Transform many times” scenario.

DefaultStaticLoop Test Case vs DefaultDynamicLoop Test Case vs CustomDynamicLoop Test Case

All the orchestrations used in the 3 test cases share the same structure and implement the same behavior using a different technique:

DefaultStaticLoop orchestration: uses a Transform shape to execute the CalculatorRequestToCalculatorResponse against the inbound document.
DefaultDynamicLoop orchestration: uses the transform method within a Message Assignment Shape to accomplish the same task.
CustomDynamicLoop orchestration: uses the XslCompiledTransformHelper.Transform method to invoke the map against against the request message.

Each orchestration contains a loop that executes the message transformation exactly 1000 times and finally reports the total elapsed time. For the test I created 3 separate xml files (they can be found in the Test folder), one for each orchestration. As I explained in the first part of the article, each orchestration receives the request message through a Direct Bound Port. In particular, the following Filter Expression has been defined on the Activate Receive Shape of each orchestration:

http://microsoft.biztalk.cat/10/dynamictransforms/propertyschema.Method == <OrchestrationName>

Therefore, the following files are identical:

DefaultStaticLoop.xml
DefaultDynamicLoop.xml
CustomDynamicLoop.xml

with the exception of the Method element that contains the name of the related orchestration. To execute each test case is sufficient to copy the corresponding file to the Test\IN folder: the DT.FILE.RL FILE Receive Location will than receive the message and activate the intended test case. I used DebugView to keep track of the elapsed time reported by each of the test cases:

Results are quite eloquent and don’t give room to doubts:

DefaultStaticLoop Test Case: Total Elapsed Time = ~57.3 seconds, Average Elapsed Time/Transformation = ~57 milliseconds
DefaultDynamicLoop Test Case: Total Elapsed Time = ~56.2 seconds, Average Elapsed Time/Transformation = ~56 milliseconds
CustomDynamicLoop Test Case: Total Elapsed Time = ~3.6 seconds, Average Elapsed Time/Transformation = ~3.6 milliseconds

Once again, I conducted several test runs to confirm the results obtained and reported above. These latter clearly demonstrated that the XslCompiledTransformHelper class is an order of magnitude faster than the default mechanisms provided by BizTalk for transforming messages.

TransformStaticallyDefined vs TransformReceivePipeline

The objective of this test is to compare the performance of the following test cases:

TransformStaticallyDefined Test Case: as explained in the first part of the article, the inbound CalculatorRequest message is transformed using the CalculatorRequestToCalculatorResponse map declaratively configured on the DT.TransformStaticallyDefined.RP Request Port. Once posted to the MessageBox, the CalculatorResponse transformed message is immediately returned to the DT.TransformStaticallyDefined.WCF-NetTcp.RL Receive Location (Loopback pattern).
TransformReceivePipeline Test Case: the inbound CalculatorRequest message is transformed by the TransformReceivePipeline hosted by the DT.TransformStaticallyDefined.WCF-NetTcp.RL Receive Location. In particular, the TransformPipelineComponent invokes the XslCompiledTransformHelper.Transform static method to apply the CalculatorRequestToCalculatorResponse map to the inbound xml document. The FQDN of the map is declaratively specified in the pipeline configuration. Once posted to the MessageBox, the CalculatorResponse transformed message is immediately returned to the DT.TransformStaticallyDefined.WCF-NetTcp.RL Receive Location (Loopback pattern).

To generate traffic against the 2 test cases and measure performance I used the following Load Tests defined in the UnitAndLoadTests Test Project:

StaticMapLoadTest: this load test is based on the InvokeStaticMap unit test and can be used to generate traffic against the TransformStaticallyDefined Use Case. The test is configured to send 1000 CalculatorRequest messages to the DT.TransformStaticallyDefined.WCF-NetTcp.RL Receive Location using 25 different worker threads.
DynamicMapLoadTest: this load test is based on the InvokeDynamicMap unit test and can be used to generate traffic against the TransformReceivePipeline Use Case. The test is configured to send 1000 CalculatorRequest messages to the DT.TransformStaticallyDefined.WCF-NetTcp.RL Receive Location using 25 different worker threads.

In particular, as shown in the picture below, I created a custom Counter Set called BizTalk composed of the following performance counters:

Inbound Latency (sec): measures the average latency in milliseconds from when the Messaging Engine receives a document from the adapter until the time it is published to Message Box.
Request-Response Latency (sec): measures the average latency in milliseconds from when the Messaging Engine receives a request document from the adapter until the time a response document is given back to the adapter.

Specifically, the average latency measured by the Inbound Latency (sec) counter includes the time spent for transforming the message in both use cases. Obviously it counts also the time spent running other activities like posting the message to the MessageBox, but still it represents a good mechanism to measure to compare the time spent by the 2 test cases for transforming the inbound message.

I conducted several test runs to confirm results obtained. The screens below were taken, respectively, at the end of StaticMapLoadTest and DynamicMapLoadTest:

StaticMapLoadTest Graphs & Summary

DynamicMapLoadTest Graphs & Summary

The following table reports for convenience the results highlighted in the screens above:

Test Case	Inbound Latency (sec)	Request Response Latency (sec)	Avg Test Time (sec)	Tests/sec (Throughput)	Duration (sec)	% CPU Time
StaticMapLoadTest	0.41	0.90	2.55	8.77	114	65.2
DynamicMapLoadTest	0.11	0.49	1.29	17.6	56	43

The difference in terms of latency and throughput between the 2 test cases is quite dramatic and this clearly confirms once again that the XslCompiledTransform class is much faster than the XslTransform class natively used by BizTalk. In our case, the adoption of the custom XslCompiledTransformHelper class allowed to double the throughput and halve the latency. Obviously, the performance gain can vary from case to case as it depends on many factors (inbound message size, map complexity, etc.), but it’s quite evident that the overall performance of a BizTalk application that makes an extensive use of message transformations can greatly be improved using a helper component like the XslCompiledTransformHelper class that exploits the XslCompiledTransform class to compile, invoke and cache maps for later calls.

Conclusions

As I said in the first part of the article, I started to work with the product group to see how best to take advantage of the XslCompiledTransform class in the next version of BizTalk. Nevertheless, you can immediately exploits this class in your custom components to boost the execution of you message transformations. Therefore, I encourage you to download my code here and repeat the tests described in this article using your own messages and maps.

Follow-Up

I wrote another article on this subject and extended my code to support multi-source-document-maps. You can find my post here.

Code

Here you can download the code. Any feedback is highly appreciated. ;-)