WCF LOB Adapter SDK and BizTalk Adapter Pack : WCF LOB Adapter SDK

Things to consider when writing WCF LOB Adapters for consumption through BizTalk

SandeepP — Thu, 12 Mar 2009 08:22:00 GMT

If you are writing a WCF LOB adapter that can be consumed through BizTalk, you need to be aware of certain issues that can manifest because of the way BizTalk interacts with WCF adapters. Some of these can have performance and scalability impact and hence you should consider them when designing/configuring the adapter.

Processing in IConnection/IConnectionFactory instance – When using SSO, for each message, BizTalk will end up creating a new IConnectionFactory and a new IConnection instance. If your adapter is doing a lot of processing in either of those instances, it can cause significant performance impact. So you will need to think of alternatives – possibly doing the processing upfront and caching.
Caching of LOB artifacts on IConnection instance – Typically you will have some LOB artifacts that will comprise the context associated with an IConnection instance. If your adapter is using WCF LOB Adapter SDK’s connection pooling, in the non SSO scenario, the IConnection instances will be closed only when the idle connection timeout expires. So you need to think about freeing up the LOB resources that comprise your connection context. If you make idle connection timeout too small, it would impact performance since LOB connection creation will incur an overhead and if you keep it at a large value then you will be holding on to LOB artifacts for a longer time than you really need to, possible scalability/performance impact.
Throughput stall – You can find more details about the issue on http://blogs.msdn.com/adapters/archive/2008/08/18/throughput-stalls-when-using-adapters-developed-against-the-wcf-lob-adapter-sdk-in-biztalk.aspx. Note that if you don’t limit the number of concurrent connections through your adapter, you won’t run into this problem. The problem happens the contention between Open and Send/Execute for thread pool threads.
Writing/promoting properties to BizTalk message context – When the adapter receives a WCF message from BizTalk, it will have all the properties comprising the BizTalk message context. For messages that originate from the adapter, if you want to have properties either written/promoted to BizTalk message context, you can look at http://technet.microsoft.com/en-us/library/bb246105.aspx on how to go about doing that.
Transaction overhead – BizTalk will by default start a transaction and send the message/process the response in that transaction scope. This can cause a significant overhead particularly if say the LOB doesn’t support transactions. In BizTalk 2009, there is an option on the Send port that lets you configure whether transactions should be enabled or not.

Throughput stalls when using adapters, developed against the WCF LOB Adapter SDK, in BizTalk

mdoctor — Mon, 18 Aug 2008 18:21:17 GMT

We’ve had a few users complaining about the throughput of the adapters (the Adapter Pack ones, and/or custom ones) coming to a standstill during normal operation within BizTalk. In this post, I’ll explain why that happens, and some workarounds.

Let’s say BizTalk has received 100 messages which it needs to send to the adapter on a Send Port. Assume that it queues 50 work items on 50 thread pool threads, with each work item containing 2 messages to be processed.

Now, on each of the above thread pool threads, the logic used is something similar to:

1. for (int i=0; i < numberOfMessages; i++)

2. {

3. IRequestChannel channel = CreateNewIRequestChannel();

4. channel.Open();

5. channel.BeginRequest(message[i], callbackFunction);

6. }

IRequestChannel above maps to an adapter channel – IOutboundHandler in the Adapter SDK.

Now, many of the LOB systems for which the adapters are written are do not have an asynchronous version of the LOB API or SDK, i.e., the LOB API is synchronous – blocks the current thread while the invocation occurs. However, as can be seen above, a call to BeginRequest() is made, which means that the caller wants the adapter channel implementation to return immediately. Therefore, in the Adapter SDK, the implementation is:

7. BeginRequest(Message message, Callback callbackFunction)

8. {

9. ThreadPool.QueueUserWorkItem(channel.Request, message, callbackFunction);

10. return immediately;

11. }

What the Adapter SDK does is, since it needs to return immediately, it queues an additional work item in the thread pool queue. The work item, when run, will call Request(), which is a synchronous call, which maps to IOutboundHandler::Execute() in the adapter, with the message as the parameter. Once the adapter finishes processing the message (synchronously) on the thread pool thread, the Adapter SDK takes care of invoking the callback function to notify BizTalk that the message has completed processing, and to hand the response message to BizTalk.

Also, one more piece of information – the channel.Open() call in line number 4 above, is handled by the Adapter SDK. When the Open() call reaches the Adapter SDK, it tries to obtain a free connection from the connection pool, or create a new connection to the LOB provided the maximum number of connections hasn’t been reached. If it is unsuccessful in both attempts, it blocks until a connection becomes available. Note - The adapters in the Adapter Pack expose a setting (typically named MaxConnectionsPerSystem, or MaxPoolSize, etc); the value of which is passed on to the Adapter SDK via a setting it exposes; custom adapters might expose something similar which the end user can tweak.

Given the two blocks of code above, and the behavior of the Open() call, it is now easy to come up with a situation where you can see a throughput stall.

Assume that the maximum number of threads in the thread pool is 100. Also, let’s say that the maximum number of connections allowed to the LOB system is 10. You receive a large number of messages from your Receive Location (for example, 500?), and BizTalk, in all its enthusiasm, queues up all messages (one message per work item – lets refer to these work items as W1) on threads from the thread pool. Therefore, you have 100 threads (suppose, since its the maximum number of allowable threads in the thread pool), all of them beginning to execute lines 1 to 6 above. Only 10 threads are able to proceed past line number 4, since they were able to open a connection to the LOB (your maximum number of connections is configured to be 10, remember?). The other 90 threads are blocked at line number 4, waiting for a connection to become available.

Of the 10 which passed 4, they proceeded to 5, which means they now enter lines 7 to 11. They all queue up work items (lets call these work items W2) on the the thread pool (courtesy line number 9), and return immediately. These 10 threads are now freed up, which enables them to go and pick more items to work on from the thread pool queue.

Now, the actual processing against the LOB (and the real usage of the connection which was successfully opened in step 4) only happens when W2 gets a chance to do its work. However, the 10 threads which were freed up, they won’t process W2. Why? Because ahead of W2, there are yet 400 more W1 work items (BizTalk had queued 500 items, remember) in the queue. Hence, the 10 threads will pick up 10 more W1 work items. These work items of course can’t progress since they are in the same state as the other 90 – there is no connection available. And a connection won’t become available until a W2 work item will relinquish it, but it can’t since it can only do its work once a thread becomes available, and that will only happen when one of the current 100 threads (which are all stuck on line 4) gets freed up, and …… and you have a thread pool starvation problem.

How do you work around this? A number of ways, and possibly, you need to use a combination of some of them (at least the ones in your control) to work around this:

One point where the threads are stuck is on line number 4. Easy – just increase the number of connections available to the LOB to a really large number, and your problem goes away. Right? Well, you might be prevented from doing that based on server resources, licenses, etc.
Lines 7 to 11 are responsible for queuing additional work items in the thread pool queue. You could avoid that if there was no need to queue a work item – for example, if your LOB exposed an asynchronous API, you could directly invoke that, passing it the callback routine pointer to “call back” on once it completed, and you could return immediately after kicking off that asynchronous API. Of course, this is not always possible. For example, LOBs like SAP and Siebel – the SDKs which we use did not offer an asynchronous API which we could use in the adapters.
One of the limitations which led to this situation, was that the maximum number of threads in the thread pool was limited. If you could increase the number of threads in the pool to a sufficiently large number, you wouldn’t see this problem. BizTalk exposes a setting for increasing the number of threads – see http://msdn.microsoft.com/en-us/library/cc296779.aspx. Note that you should try this out in your environment, since increasing it to a large value could possibly hurt performance.
In my example above, I’ve assumed that if there are 100 threads in the thread pool (maximum), all threads are being used during the processing of messages for my adapter. This is not always the case – the BizTalk runtime (and possibly other artifacts in the same process) will compete for threads from the thread pool, leaving lesser threads for you to work with. One way around this is to configure the send handler for your adapter to run in its own host instance (which means a separate process), leading to lesser contention for the threads from the pool in your process.
If you look at the code again, you notice that the main reason you got into this situation, was BizTalk itself – it queued up a huge number of messages (lines 1 to 6). The link mentioned above contains the information on settings which control the number of messages BizTalk processes simultaneously; tweaking something here can possibly help.

The safest way would be to ensure that the number of messages BizTalk hands to the adapter (lines 1 to 6) is the same as the maximum number of connections allowed to the LOB. I’m not 100% sure, but I think the In-Process Messages Per CPU setting can be used for this purpose; I’ll have to go through all the documentation for the throttling settings a little more thoroughly – meanwhile, you can do the same (the link available in point number 3 above seems to be the place to start).

Changes to the WCF LOB Adapter SDK in SP2

mdoctor — Mon, 21 Jul 2008 21:15:00 GMT

If you’re developing your own adapters against the WCF LOB Adapter SDK, then it might be worth your while to try out the June CTP of the SP2 of the WCF LOB ASDK. There are a few minor changes, but we might make more based on your feedback.

I’m mentioning the changes from SP1 to the June CTP of SP2 in brief. For more details, you should refer to the documentation.

Support for VS 2008 and .NET 3.5.
Pre SP2, the ASDK allowed all input channel shapes – IInputChannel, IInputSessionChannel, IReplyChannel, and IReplySessionChannel. In SP2, there is a setting exposed to the adapter, which allows the adapter to control which of the above four shapes should be allowed. This is important when the adapter is used with BizTalk, and you want to integrate the LOB transaction (assuming it understands System.Transaction) with the BizTalk MessageBox transaction – in such a situation, you only want to allow one-way channels to be used. In other cases (SAP for example), where a reply always needs to be sent back to the LOB, you might want to only allow two-way channels.
Pre SP2, the WSDL generated (based on the operations selected in the Metadata Search Browse (MSB) UI) was always compiled before being returned to the caller (or being converted to a WCF proxy). In some situations (e.g., where the adapter is overriding the XSD generation to return metadata for – let’s say a System.Data.DataSet) – the XSD/WSDL might not compile (even though svcutil.exe can successfully convert it to a WCF proxy). In order to allow this, a setting is exposed to the adapter which determines whether the WSDL should be compiled or not, before being converted to a WCF proxy. The advantage of compilation is that it can catch “real” errors in the wsdl.
There are two new QualifiedTypes added for System.Data.DataSet. (For “why two” and “why not just one”, refer to the documentation).
Pre SP2, when using the “Consume Adapter Service” tool in a BizTalk project, the XSD files generated had names like “MyBinding1.xsd”, “MyBinding2.xsd”, etc. Even though a “filename prefix” textbox was present, it had a couple of drawbacks - (a) the user had to provide the prefix, which might not clearly indicate what the XSDs contained, and (b) the same prefix applied to all XSDs generated during the metadata resolution of all operations selected in the UI. In SP2, it is now possible for the adapter to specify a “filename hint” for each XSD generated.

Connection Pooling in the WCF LOB ASDK based adapters.

mdoctor — Sun, 13 Apr 2008 18:08:00 GMT

I was recently contacted by a customer using the SAP Adapter. They were using the Adapter directly via a WCF proxy (i.e., without using BizTalk) from within an ASP.NET application. Their complaint was that the throughput was very low. On investigating further, I found that the reason was due to connections being opened and closed, as well as metadata being retrieved from SAP, for every call. All this in spite of the binding property "EnableConnectionPooling" being true. That's strange you might think, but not once you know the details of how the ASDK functions ...

In the WCF LOB ASDK, there was a need for maintaining a pool of open connections to the LOB, since opening connections to LOB systems is usually expensive. When a connection is required by a channel to a particular LOB server (as determined by the URI), with a specific set of credentials, then the appropriate connection is picked up from the pool and used; when the channel is closed, the connection is returned back to the pool.

In the current ASDK design, the connection pools are maintained at the channel factory instance. (The exact reasons of why this design was chosen - topic of another blog post). What this means is that if you have two channel factory instances, then each channel factory has its own set of connection pools. i.e., for a server A, each channel factory will have one connection pool for this server. Connections from the pools in one channel factory cannot be used by another channel factory.

Another set of objects which are cached are metadata objects. The adapters, at runtime, when receiving a message (corresponding to some LOB operation), first connect to the LOB to obtain metadata for the operation. This is required in order to understand how to parse the request message (for example - how many parameters does the function have? What are their names? etc). Once metadata is obtained from the LOB, this metadata is cached within the ASDK. There are a number of settings (exposed by the ASDK to the adapter) which determine for how long is the metadata cached, where is it cached, etc. The adapter has control over whether the ASDK should cache the metadata at an AppDomain level (i.e., "static"), or per channel factory instance. The SAP Adapter instructs the ASDK to cache the metadata per channel factory instance. (Why? Another blog post).

At this point, here's what we know - in the SAP Adapter, both, the connection pools, as well as the metadata caches, are maintained within each channel factory object. If you have two channel factories, then you will have a separate set of connection pools and metadata objects. The connection pools and metadata objects are freed up only when the channel factory object is closed.

Now, when you use the Metadata Search Browse tool (using "Add Adapter Service Reference") in your application, we generate an interface (which is the ServiceContract), as well as a client proxy. For example, if you generated the client proxy for RFC_CUSTOMER_GET, you would see a proxy class generated named RfcClient, as well as an interface named Rfc. To use the generated proxy, you would have code similar to:

RfcClient client = new RfcClient(binding, endpointAddress);
client.Open();
client.RFC_CUSTOMER_GET();
//do work here
client.Close();

Now, here's something which you might not know - every time you create an instance of RfcClient, this causes a new channel factory to be created, and then, from that channel factory, a channel is created. When you close the RfcClient object, the channel is closed, as well as the channel factory. This leads to the connection pool being closed (i.e., all connections in the pool are closed), and the metadata objects are freed up. When you create a new instance of RfcClient, another channel factory is created, which comes with its own pool and metadata cache. Therefore, effectively, connections are not being pooled, (and neither is the metadata cache acting as a cache), since they are freed up the moment the RfcClient object is closed.

So, how do you get around this? One way would be to cache the RfcClient object, and then use that from multiple threads (for example). However, this really doesn't help. The reason being, that the RfcClient object not only represents one channel factory, but also just one channel (which ultimately maps to one LOB connection). Therefore, if you have, say, 50 threads accessing the same RfcClient object, they will all ultimately end up using the same LOB connection, which will definitely not be good for performance.

What you really want to do, is open one channel factory, cache that, and then, open and close channels at will from that channel factory. When you open a channel, a connection will be obtained from the connection pool; and when you close the channel, the connection will be released back to the pool. Plus, the metadata cache will be shared across the channels, since they are part of the same channel factory.

So, how do you this? You can't use the RfcClient proxy. Instead, you need to use the ChannelFactory<> class, and use the generated ServiceContract (which will be named Rfc if you added only RFCs in the "Add Adapter Service Reference" wizard). (You can always figure out the name of the interface, by looking up the definition of the generated proxy, and seeing which interface it implements). Thus, you need to do something like:

//do this only once
ChannelFactory<Rfc> factory = new ChannelFactory<Rfc>(binding);
factory.Open();
//cache the "factory" object somewhere.

//When you need to make a call to SAP:
Rfc channel = factory.CreateChannel(endpointAddress);
channel.RFC_CUSTOMER_GET();
((IChannel)channel).Close();
//that's all.

//When you ultimately want to free up
//all connections in the pool, and free
//up all the stored metadata,
factory.Close();

Announcing the Microsoft BizTalk Adapter Pack – Office Developer Program

sreeramn — Sun, 30 Mar 2008 18:48:00 GMT

As was already announced on this forum, the BizTalk Adapter Pack (BAP) was announced more than a month ago during the Office Developer Conference and became available starting March 1st. The BAP provides a robust and comprehensive out-of-the-box connectivity infrastructure to three major line-of-business (LOB) systems - SAP, Siebel, and Oracle Databases. The technology is based on the Windows Communication Foundation (WCF) LOB Adapter SDK and inherently relies on the core WCF concepts, principles, and classes implemented in the .NET Framework 3.5.

Along with the BAP, we also launched the BizTalk Adapter Pack – Office Developer Program, designed to validate the use of the BizTalk Adapter Pack as a connectivity solution in direct integration scenarios with Microsoft Office SharePoint Server (MOSS) 2007 and the Office Business Applications such as Microsoft Office Excel and Microsoft Office Word. The scope of the program is validation of six key scenarios, as depicted in the following diagram.

Customer Participation

The BAP - Office Developer Program is now open for new customer nominations. Joining this program will provide you with access to conference calls with members of the BAP development team, access to private discussion forums, and the ability to discuss and report bugs and issues directly back to the team at Microsoft.

If you would like to work closely with the product team and implement and test any of the six scenarios in your solution, please fill out the short survey and be specific in answering the questions to help us accurately evaluate your application. We will contact you shortly thereafter. If you have any questions, you can contact us via e-mail by clicking here.

You can also get additional information about this program on the SharePoint Product Team blog site.

The BizTalk SOA Customer Programs Team

Transactional behavior in a receive location using an adapter

mdoctor — Sun, 23 Mar 2008 14:23:00 GMT

Quite a few customers developing custom adapters built on the WCF LOB Adapter SDK have come back to us asking for details on how to have transactional behavior while receiving messages from the LOB and submitting them to BizTalk (receive location) - for example, assume in your receive location:

You read some data from the LOB
You delete the data from the LOB (since you don't want to read it again)
In IInboundHandler, you construct a message containing this data
You return this message via the out parameter in the TryReceive function (in other words, you are submitting this message into BizTalk).

If an error occurs after step 4, you have lost the data! (since it has already been deleted from the LOB).

If your LOB supports transactions, then you can prevent this from happening - the same transaction can be used to read + delete the data from the LOB, as well as submit the message into BizTalk, and if anything fails at any point, everything is rolled back.

More information on how you can do this - read this post by Sonu Arora at http://blogs.msdn.com/sonuarora/archive/2007/06/19/transaction-support-in-inbound-message-exchange-handler.aspx.

How To: Obtain a list of receive actions when an ASDK-based Adapter is used with BizTalk in a Receive Location

mdoctor — Fri, 18 Jan 2008 08:46:00 GMT

The WCF LOB Adapter SDK (ASDK) contains a behavior named "InboundActionEndpointBehavior" which you can add to your ServiceHost in inbound scenarios. What this behavior does is, at runtime, analyzes the contract deployed, determines the Actions on the Contract, and passes those to the adapter in the StartListener() call (on the IInboundHandler interface) via the "string[] actions" parameter. This allows the adapter to determine what actions the user is listening for, and send only messages of those type.

More information on this can be found at: http://blogs.msdn.com/sonuarora/archive/2007/06/11/passing-soap-actions-to-adapter-inbound-handler-for-filtering-type-of-listeners.aspx

What do you do when the adapter is used in a Receive Location from BizTalk? The WCF Adapter allows you specify endpoint behaviors for the ServiceHost which it creates, and that's how you can plug in the InboundActionEndpointBehavior. However, this doesnt work as expected. Reason - the WCF Adapter uses an "untyped" contract for the Service, i.e., the Action/ReplyAction is specified as "*" / "*". Hence, even if you add the endpoint behavior, the "string[] actions" array just contains "*". How would a user let the adapter know what operations he is interested in?

Attached is a sample endpoint behavior which you can use. The primary difference between this behavior and the one which ships with ASDK is this - the ASDK behavior determines the list of actions from the deployed contract, and passes those to the adapter. On the other hand, the sample endpoint behavior (attached) gets the list of actions via user input. That is, while configuring your receive location, add this endpoint behavior (via the behaviors tab), and you'll see two properties: "receiveActions" and "delimiter". You need to concatenate all the actions into a single string and set it on the "receiveActions" property. Also, specify the delimiter you used in the "delimiter" property. The behavior will then split the concatenated string into a list of actions, store them in an ASDK specific class, and add this object to the Binding Parameter Collection. ASDK will then pick out the actions from there and pass them to the adapter in the StartListener call.

The attached .cs file contains comments towards the top of the file detailing how it needs to be compiled, and what entries need to be made in machine.config.

Getting LOB metadata from SAP, OracleDB and Siebel adapter using svcutil

Jayanthi S — Tue, 23 Oct 2007 11:56:00 GMT

The WCF based adapters that ship in the Biztalk Adapter Pack are WCF bindings. The svcutil tool can be used to get metadata for a LOB method using these bindings. The scheme in the URI determines which adapter binding is loaded by svcutil. However when using the svcutil tool, there is no easy way to specify the LOB credentials that should be used to get metadata from the LOB. One way to specify the LOB credentials to our adapters is to set it in the URI. However, by default (for security reasons) the adapters do not accept credentials in the URI.

To get around this, you need to set the binding property acceptCredentialsInUri to true and then specify the LOB credentials in the URI. To be able to set binding properties or other custom settings when using svcutil.exe, you should use the svcutil.exe.config file. This article describes how to use it.

Attached is a sample svcutil.exe.config which has the endpoint and binding configurations for all the three adapter bindings. Place this in the same folder as svcutil.exe.

The following are sample command lines to use to get metadata from our adapters.

SvcUtil.exe "sap://user=youruser;passwd=yourpassword;Client=800;lang=EN@A/adapsap47/00?op=http://Microsoft.LobServices.Sap/2007/03/Rfc/RFC_GET_FUNCTION_INTERFACE"

SvcUtil.exe "oracledb://User=MYUSER;Password=MYPASSWORD@ADAPTER/?op=http://Microsoft.LobServices.OracleDB/2007/03/SCOTT/Table/AEMP/Insert"

SvcUtil.exe "siebel://Username=MYUSER;Password=MYPASSWORD@mssebldemo:2321?SiebelObjectManager=SSEObjMgr_enu&SiebelEnterpriseServer=ent771&Language=enu&op=http://Microsoft.LobServices.Siebel/2007/03/BusinessObjects/Account/Agreement/Query"

Note the quotation mark around the URI. In the configuration for Siebel (in the attached config file) also note the idleConnectionTimeout without which it will take the default 1 minute for svcutil to terminate. Look for more details on the idleConnectionTimeout in a separate post on Siebel.

Note also that if you use the Add Adapter Service Reference tool (that gets installed with the WCF LOB Adapter SDK) to get metadata for a LOB method for use in a C# or VB project, the AcceptCredentialsInUri binding property will not be visible or settable. You can set the LOB credentials using the credentials tab in the tool.

Sample - SQL Adapter built on the ASDK.

mdoctor — Sat, 06 Oct 2007 00:00:00 GMT

Attached is a sample SQL Adapter built on the ASDK. This adapter (being a sample), is rather simple:

(a) Only stored procedure execution is supported.

(b) For obtaining metadata about the result set returned by the stored procedure, the stored procedure is executed (GetSchemaOnly) with default values for each input parameter.

The adapter requires the Adventure Works sample database, which can be obtained from here: http://www.codeplex.com/MSFTDBProdSamples/Release/ProjectReleases.aspx?ReleaseId=4004. (I used AdventureWorksDB.msi).

There is a test project included which should help you out in figuring how the connection uri should look like. The test project browses, searches, and gets the metadata for a stored procedure. Then, it executes a procedure to get the ManagerId for a particular employee.