Keith Lim's Biztalk BAM Blog

BAM continuation between WF/WCF application and BizTalk application

2007-07-05T22:36:00Z

Awhile back a user asked me this question on BAM continuation from a WF/WCF application to a BizTalk application: I have a process which starts a workflow and then an orchestration. I want to create an activity cross the workflow and the orchestration. So I create a ContinuationToken in the WF IC and create a ContinuationID in the Orchestration TPE. However, it does not seem work. The data from WF and the data from Orch are written into two records in the Completed table. My questions are: Is it...(read more)

Steps to facilitate the creation of WCF Interceptor Configuration file.

2007-06-29T04:56:00Z

The BAM Interceptor for WCF does not come with a UI tool to create the Interceptor Configuration file, and from the Beta II customer feedback thus far, this is the main pain point as well. I will try to provide some steps and guidelines to help ease this concern. A common scenario is when the WCF application configuration file is updated and the IC is created and deployed, there is not data intercepted and persisted into the BAM PITDB (Primary Import Tables Database). When the event log is checked...(read more)

BAM Interceptor for WCF

2007-06-29T02:01:00Z

Many of you guys are eagerly anticipating the release of the Microsoft BizTalk Server 2006 R2… and the only definite answer to that is it will be available in Q3 of 2007. In the meantime, the Beta 2 release of it can be downloaded from this link: http://connect.microsoft.com/ In this series of blogs I will again as usual, delve on the BAM side of BizTalk offering and answer any questions from you. In R2, there are two new interceptors added, namely the BAM Interceptor for WCF (Windows Communication...(read more)

How to programatically retrieve data from BAMPrimaryImport using supportable API

2006-03-08T04:53:00Z

Eventhough there are Portal UI and Excel that would allow you to query and access BAM activity data, it seems that there are some of you who would like to programmatically access/query this data from a custom application. In BizTalk Server Documentation, currently in its 2006 pre-released version, you will find information on the BAMManagementService API. (Under Technical Reference | Developers Reference | BizTalk Server 2006 .Net Class Reference | Microsoft.BizTalk.Bam.Webservices.Management )....(read more)

Soliciting Biztalk BAM scenarios and issues

2006-02-10T03:32:00Z

How great it would be if every features we put in to the software is what customers need/want. Anyway, I am creating this entry to solicit your input on how you use or plan to use BizTalk BAM. Please response as a comment with your scnenario. Do also tell us what feature you like and dislike. Thanks, Keith...(read more)

BAM API: Using OrchestrationEventStream

2006-02-09T02:32:00Z

Since there is not much documentation* on this yet on how to use OrchestrationEventStream (an EventStream that is in synch with Orchestration), here is a quick info on the howto:

This new feature is found in Microsoft.BizTalk.Bam.XLANGs assembly. From you BizTalk project, you can write the following in your orchestration, e.g. Expression Editor (click on the appropriate shape in orchestration to launch this):

// Sample code for using OrchestrationEventStream from an orchestration, no need to new it:
Microsoft.BizTalk.Bam.EventObservation.OrchestrationEventStream.BeginActivity( "PurchaseOrder", "PO123");

Note: this only applies to Biztalk 2006.

* To use this in Biztalk 2004 SP1, you will need a hot fix, its kb article can be found here: http://www.kbalertz.com/Feedback_900083.aspx,

Sample code to retrieve server and database info of Biztalk MessageBox and PIT

2006-02-09T02:08:00Z

Since there is a request for this, here is an example to retrieve server name and database for Biztalk MessageBox and BAM Primary Import database. Since this is accessing internal implementation, this code is not supportable (use at your own risk) and users will be required to maintain this code on their own when doing upgrade.

Here are the steps:

Create a console appplication project in Visual Studio,

Cut and paste the code below and run

Sample output:

InfoFound Management Server KW0081 Database BizTalkMgmtDb
InfoFound MsgBox Server KW0081 Database BizTalkMsgBoxDb
InfoFound PIT Server KW0081 Database BAMPrimaryImport
InfoFound DTA Server KW0081 Database BizTalkDTADb

Sample Code:

using

System;

using

System.Data.SqlClient;

using

System.Globalization;

using

Microsoft.Win32;

namespace KeithLimBizTalkBAMUtility

{

class InfrastructureInfo

{

[STAThread]

static void Main(string[] args)

{

InfrastructureInfo ii =

new InfrastructureInfo();

ii.GetInfo();

}

internal const string BtsAdminRegistryKey = @"Software\\Microsoft\\BizTalk Server\\3.0\\Administration";

internal const string BtsServerRegistryKey = @"MgmtDBServer";

internal const string BtsNameRegistryValue = @"MgmtDBName";

internal const string BtsMgmtSprocRetrieveMsgBoxPITandDTA = @"SELECT SubscriptionDBServerName, SubscriptionDBName, BamDBServerName, BamDBName, TrackingDBServerName, TrackingDBName FROM adm_Group";

internal const string BtsMigrateError = @"Error";

internal const string BtsMigrateInfo = @"Info";

internal const string BtsMigrateWarning = @"Warning";

private string managementServer;

private string managementDatabase;

private string dtaServer;

private string dtaDatabase;

private string pitServer;

private string pitDatabase;

private string msgBoxServer;

private string msgBoxDatabase;

internal static string BuildConnectionString(string server, string database)

{

const string SqlConnectionString = @"Data Source=""{0}"";Initial Catalog=""{1}"";Integrated Security=SSPI;Application Name=""BamManager"";Connect Timeout=60";

return String.Format(CultureInfo.InvariantCulture,

SqlConnectionString, server, database);

}

private void GetInfo()

{

RegistryKey key = Registry.LocalMachine.OpenSubKey(BtsAdminRegistryKey,

false);

SqlDataReader reader =

null;

SqlConnection btmConnection =

null;

if (key != null)

{

this.managementServer = (string)key.GetValue(BtsServerRegistryKey);

this.managementDatabase = (string)key.GetValue(BtsNameRegistryValue);

}

else

{

Console.WriteLine(BtsMigrateError + "Failed to retrieve Management database connection string from " + BtsAdminRegistryKey);

return;

}

Console.WriteLine(BtsMigrateInfo +

String.Format("Found Management Server {0} Database {1}",

this.managementServer, this.managementDatabase));

try

{

// Open up the Management Database

string btmConnectionString = BuildConnectionString(this.managementServer, this.managementDatabase);

btmConnection =

new SqlConnection(btmConnectionString);

btmConnection.Open();

// Query the Primary Import Database

SqlCommand cmd =

new SqlCommand(BtsMgmtSprocRetrieveMsgBoxPITandDTA, btmConnection);

reader = cmd.ExecuteReader();

if (!reader.Read())

{

Console.WriteLine(BtsMigrateError + "Failed to retrieve PIT and DTA databases from management DB (is group deployed?)");

}

this.msgBoxServer = reader.GetString(0);

this.msgBoxDatabase = reader.GetString(1);

this.pitServer = reader.GetString(2);

this.pitDatabase = reader.GetString(3);

this.dtaServer = reader.GetString(4);

this.dtaDatabase = reader.GetString(5);

Console.WriteLine(BtsMigrateInfo +

String.Format("Found MsgBox Server {0} Database {1}",

this.msgBoxServer, this.msgBoxDatabase));

Console.WriteLine(BtsMigrateInfo +

String.Format("Found PIT Server {0} Database {1}",

this.pitServer, this.pitDatabase));

Console.WriteLine(BtsMigrateInfo +

String.Format("Found DTA Server {0} Database {1}",

this.dtaServer, this.dtaDatabase));

}

finally

{

if (reader != null)

{

reader.Close();

}

btmConnection.Close();

}

Continuation scenarios with mixed of TPE-enabled and non-TPE-Enabled tracking:

2006-02-08T05:43:00Z

It has been a while since I last blogged on this. Let's continue from where I left off.

Let's pick two continuation scenarios: the first, a TPE segment continues to a non-TPE segment, and the reverse being the second.

1. TPE-enabled segment continues to a non-TPE-enabled segment.

In this scenario, you would need only a Continuation folder. The naming of this folder is important since the concatenation of the folder name and its mapped value will form the continuation TokenID used in non-TPE-enabled segment. For example, if the Continuation folder name is "TPE2OES" and the mapped value has "PO123" for instance, the continuation TokenID would be "TPE2OESPO123". Once you have mapped other data items and milestones, you will deploy* this.

In the related orchestration, in the process somewhere downstream, you will need to insert the following:

// Continuation segment using OrchestrationEventstream**

// Where ContinuationTokenID based on this instance contains the value ""TPE2OESPO123".***

Microsoft.BizTalk.Bam.EventObservation.OrchestrationEventStream.UpdateActivity("PurchaseOrder", ContinuationTokenID,

"Column1", Value1,

"Column2", Value2);

Microsoft.BizTalk.Bam.EventObservation.OrchestrationEventStream.EndActivity("PurchaseOrder", ContinuationTokenID);

2. Non-TPE-enabled segment continues to a TPE-enabled segment.

In the reverse scenario where an activity begins from an non-TPE segment, the following code would be required:

// Sample main segment of continuation using OrchestrationEventStream.***

// Activity in this instance contains: "PO124"

Microsoft.BizTalk.Bam.EventObservation.OrchestrationEventStream.BeginActivity("PurchaseOrder", ActivityID);

Microsoft.BizTalk.Bam.EventObservation.OrchestrationEventStream.UpdateActivity("PurchaseOrder", ActivityID,

"Column1", Value1,

"Column2", Value2);

// CONTINUATION LINE HERE:

Microsoft.BizTalk.Bam.EventObservation.OrchestrationEventStream.EnableContinuation("PurchaseOrder", ActivityID, "OES2TPE" + ActivityID);

Microsoft.BizTalk.Bam.EventObservation.OrchestrationEventStream.EndActivity("PurchaseOrder", ActivityID);

To create the continue-to segment, in your tracking profile, you would need to add a ContinuationID folder, name this folder as "OES2TPE" and map the relevant item (one that will give you the above "PO124" for this particular instance of message. Deploy* this and drop a message and see the beauty flows :)

* You will get a warning that the tracking profile is missing the ContinuationID or Continuation folder, but since we know what we are doing :), just ignore and proceed.

** Note: there is no BeginActivity call since this is a continued-to segment.

***The above can easilly be changed to used DirectEventStream or BufferedEventStream. If you do so, make sure you call the Flush method explicity or set the flush threshold to 1 or greater in the constructor.

DirectEventStream vs BufferedEventStream

2006-02-02T03:35:00Z

Which EventStream to use:

DirectEventStream (DES) vs BufferedEventStream (BES)

Use DES when you don't want any latency of data appearing in BAMPrimaryImport database. Since this is synchronous call, any error will be thrown back to the calling application.

Use BES if your concern is throughput. This is particularly suited for scenarios with spurts of large chuck of events. A small penalty for using this is that there will be small latency between when the event is sent and the event appearing in BAMPrimaryImport database.

For more info on EventStream: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/sdk/htm/frlrfmicrosoftbiztalkbameventobservationeventstreamclasstopic.asp

How to write BAM Events from pipeline components.

2006-01-30T22:44:00Z

In BizTalk 2004, to write any BAM event from the BizTalk pipeline, you will need to write some code :) To simplify the learning, I will illustrate by adapting an SDK sample.

Adapt the SDK CustomComponent (found in .../SDK/Samples/Pipelines/CustomComponent) sample, you will find two directories:

FixMsg - the custom pipeline component (you will need to splice BAM code here)
PipelineComponentSample - custom pipeline and will use the above custom pipeline component.

Modify the FixMsg to include the below sample (Search for Execute method):

public IBaseMessage Execute(IPipelineContext pc, IBaseMessage inmsg)
{

IBaseMessagePart bodyPart = inmsg.BodyPart;
if (bodyPart!=null)
{

...
...
//Splice BAM Code here:
WriteBAMEvent(pc, "Execute", inmsg);

}
return inmsg;

}

// Add a new method to write to BAM

private void WriteBAMEvent(IPipelineContext context, string StageDescription, IBaseMessage inmsg)
{

EventStream _BAMEventStr = null;
// Create an EventStream instance
try
{

// Use can use BES/DES or MES
// DES - DirectEventStream - writes direct to BAMPrimaryImport (Synchronous call, no latency)
//_BAMEventStr = new DirectEventStream("Integrated Security=SSPI;Data Source=.;Initial Catalog=BAMPrimaryImport",1);

// BES - BufferedEventStream - writes to MessageBox (Asynchronous call, no wait in call)
// _BAMEventStr = new BufferedEventStream("Integrated Security=SSPI;Data Source=.;Initial Catalog=BizTalkMsgBoxDb",1);

// MES - MessagingEventStream - Async and participates in Pipeline transaction context).
// - Note: Available in BizTalk 2004 with SP1 patch and Biztalk 2006
_BAMEventStr = context.GetEventStream();

}

catch(Exception e)
{

throw (e);

}

// Sample BAM Event Code here:
string ActivityID;
//Create a new, unique activity identifier to use as the ActivityID in BAM
ActivityID="BAM_" + Guid.NewGuid().ToString() + "_" + DateTime.Now + "_" + StageDescription;

//Start the activity record identified by ActivityID
_BAMEventStr.BeginActivity("PurchaseOrder",ActivityID);

_BAMEventStr.UpdateActivity("PurchaseOrder",ActivityID, "ProductName","QFE1117");

_BAMEventStr.EndActivity("PurchaseOrder",ActivityID);

// Optional Flush
//_BAMEventStr.Flush();

}

Reference on MessagingEventStream:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/sdkmref[bts06]/html/M_Microsoft_BizTalk_Component_Interop_IPipelineContext_GetEventStream.asp

Reference on EventStream:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/sdkmref[bts06]/html/T_Microsoft_BizTalk_Bam_EventObservation_EventStream.asp

To understand the above and details on how to fetch message specific data, please refer to sectio on Developing Pipeline Components in the help or use this link:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/sdk/htm/ebiz_prog_pipe_qdri.asp
You would at least need to implement these interfaces.
IBaseComponent, IComponent, IComponentUI.

The above is tedious and is the only means for writing to BAM in Biztalk 2004. In Biztalk 2006, you can do some of this easily via TPE drag and drop.

Means to BAM-Tracked “BAMified” Business Process Data:

2005-10-25T05:04:00Z

Means to BAM-Tracked “BAMified” Business Process Data:

There are several means to enable business process data to be tracked in BAM. In Voyager (BizTalk 2004), programmatically there are the DirectEventStream and BufferedEventStream classes, and non-programmatically there is the Tracking Profile Editor (TPE) drag-and-drop mapping mechanism.

In Pathfinder (BizTalk 2006), both of these areas are further enhanced. In brief, programmatically, there are the addition of MessagingEventStream and OrchestrationEventStream which are EventStreams that provide transaction consistent processing in the context of Biztalk pipelines and orchestrations respectively.

In the drag-and-drop TPE, in additional to the tracking from orchestrations, tracking can now be done from pipelines as well.

TPE, a shift from orchestration to activity centric view:

There is a fundamental shift in tracking profile view; what used to be an orchestration centric view has now changed to an activity centric view. Data that you want to be tracked thru TPE for a single activity definition must be mapped in a single tracking profile. For example, if you have a continuation scenario from one orchestration to orchestration, or a pipeline to an orchestration to a pipeline, all of the information must be presented in one single tracking profile. In other words, one tracked activity must have only one tracking profile (*.btt) file.

So, what are these Continuation and ContinuationID folders?

A continuation folder is needed for every TPE-enabled tracking that is a source* segment of a continuation. (Note: I am saying a source and not a main segment since in a multi- continuation scenario, there would be more than one source segments but only one main segment.)

On the other hand, a continuationID folder is needed for every TPE-enabled segment is continued to or the target.

Example:

For a simple scenario where you would continue from Orhestration1 to Orchestration2, employing the non-programmatic mechanism via TPE drag-and-drop, you would need to create a pair of Continuation and ContinuationID folder. The name of the folder must be the same, a good naming convention would be to go according to what it does such as “Cont_Orch1_2_Orch2”. You would then need map the content of each of this folder with a schema item from each orchestration that would contain the same data as it passes from one orchestration to another. Typically, this piece of data is from the message pay load schema. This data must be unique from one instance message to another to avoid data from one activity instance polluting another.

You must be curious as to why, such a lengthy explanation. This is because there are cases where you would not need Continuation and ContinationID folder in a pair. In cases where you want have a mixed of TPE and non-TPE (DES/BES/MES/OES), you would then need to know exactly when to use Continuation and when to use ContinuationID folders.

Explain it in another way, if you have TPE continues to non-TPE segment, you would need a Continuation folder, and if you have non-TPE continues to TPE segment, you would need a ContinuationID folder.

Now when you do this, what would be the TokenID? To be continue…

Biztalk BAM Blogging Introduction

2005-10-12T04:16:00Z

As 2005 is coming to a close, I want to take this opportunity to wish you a happy 2006; this also means we are inching closer towards the release of BizTalk 2006 :)

What is this blog good for...you may ask? This blog is for you the users. :) The topic that it focuses on is BizTalk in general and BizTalk BAM specifically. I will use this as a central depository (referenced from my forum replies) as well as direct replies from which to disseminate insights and personal experience from working with this product on a daily basis.

You, as the user can help me better help you. Please use the comment (on this posting) to ask for anything you may want me to delve into... of course, this got to be BizTalk and BAM related :)

Happy BAMifying,

Keith Lim