Welcome to MSDN Blogs Sign in | Join | Help

BizTalk vs. Windows Workflow Foundation (WF)

I have seen the questions about the differences between BizTalk and WF and "when to use what" have been raised multiple time.  Based on my experiences with both technologies and my readings (some listed below), I summarized the usage scenario as follows:

- Use WF for application hosted workflow and human workflow
- Use BizTalk for server-hosted system workflow, BAM, B2B and EAI (enterprise application integration).
 
In case you're interested to know why?  Not surprisingly, this is very much driven by the product features.
For WF:
  • WF is hosted inside application. 
  • Also, WF supports state machine workflows and dynamic update on workflows, which make it ideal for impelmenting human workflows or other dynamic workflows.
 
For BizTalk:
  • BizTalk workflow is hosted inside BizTalk process and depends on BizTalk server runtime.
  • BizTalk has extensive line of adapters and pipeline components that are often a must-have for cross-platform Enterprise Application Integration (EAI).
  • BizTalk provides tools for working with trading partners such as Business Activity Services (BAS), accelerators for industry standards (RosettaNet, SWIFT etc.).  These features make BizTalk more suitable for B2B scenarios.
  • Other features that BizTalk has but WF doesn't (or has to be implemented by developer)
    •     Tracking: natively integrated wtih the Business Activity Monitoring (BAM)
    •     Transaction: supports both atomic transaction and long-running transaction
    •     Extensive set of tools for admin, management, migration and scaling

Some other posts related to this topic I found quite helpful and highly recommend are:

Microsoft's offical comparison on these two technologies at MSDN Developer Center.
Brian Loesgen's blog http://blogs.ineta.org/bloesgen/archive/2005/10/09/56481.aspx
Lamont Harrington's blog http://blogs.msdn.com/lamonth/archive/2005/09/14/466222.aspx
Darren Jefford's blog http://blogs.msdn.com/darrenj/archive/2005/09/15/467838.aspx

Posted by chunyu | 5 Comments

Scope expansion of my blog

In the past four years, together with other fellows at BAM team, we helped build the BAM feature from ground up.  The groundbreaking BAM v1 was first shipped with BizTalk 2004 and the much improved, highly anticipated v2 is to be shipped with BizTalk 2006.  I felt I'm very fortunate to be a part of a talented and fun team building a highly successful v1 feature for a major Microsoft enterprise offering.  It's an incredibly fun, exciting and rewarding experience.  Recently I moved on to the application architect role, looking to bridge the gap between IT and business.  Because of this change, I decided to expand the scope of my blog to other BPM related technologies in addition to BAM and BizTalk.
Posted by chunyu | 0 Comments

Get Microsoft BizTalk 2006 Beta1 today!

BizTalk Server 2006 Beta was just released a few days ago. 

Click here to learn about the improvements and new features in BizTalk 2006.  At the bottom of the page, there is a link that will bring you to the page where you can sign up to get a copy of the Beta1 product.

Specifically in BAM area, the new features incldue support for messaging tracking, a public managent web service, portal and alerting etc.  There're some improvements in the infrastruture as well.  One of them is that you can add new activity items to an existing activity w/o the need to first undeploy everything.

Note that you can deploy BAM tracking infrastruture to Yukon SQL database in Beta1, but not to Yukon Olap analysis database yet.  The support will be added in Beta2.

Posted by chunyu | 0 Comments

Indexes in the BAM star-schema database

Some customers have asked why some tables in the star-schema database have indexes and others don't.

As suggested by SQL Books Online “Creating and Using Data Warehouses”, all the dimension tables have the index automatically created on the primary key column “<dimension name>_ID”. (BTW these implicit indexes don’t show up in the Query Analyzer, but you can see them in the Enterprise Manager.)  In addition, for the hierarchical dimensions (data dimensions and time dimensions) composite indexes are created on all dimension levels.

One place that doesn’t have index is the fact table.  Theoretically, the fact table should be indexed on the composite primary key made up of the foreign keys of the dimension tables.  A few special considerations were taken into account before it was decided not to create index on the BAM fact table:

1. Index occurs overhead on data insert (as well as delete & update)

2. Unlike the BAM primary import table which needs to support heavy instance query and real-time aggregation queries, fact table is only queried during the execution of the star-schema transformation stored proc.  

3. Unlike dimension table, the entire fact table is truncated before next DTS run.  The table is expected to stay relatively small (of course, table size depends on the incoming data volume, the scheduling of cubing DTS and archiving DTS).  Indexing small tables may not be optimal because it can take SQL Server longer to traverse the index searching for data than to perform a simple table scan.

4. The star-schema is dynamically created. Customers may know best what additional index they may need on the fact table based on their business data characteristics and DTS scheduling etc.

Posted by chunyu | 2 Comments

BAM real-time aggregation vs. scheduled aggregation

BAM supports two types of aggregations: real-time aggregation (RTA) and scheduled aggregation.  Some customers have asked what are the differences between them and when to choose what.

The biggest difference is the underling storage.  The storage of RTA is a SQL table and the aggregation is updated and maintained by SQL trigger.  The event importing and aggregation update are completed in the same transaction, therefore its data latency is negligible and almost real-time.  The scheduled aggregation is saved in Olap cubes which need to be processed periodically by the cubing DTS package (btw the Olap cubes and the cubing DTS package are all dynamically created by the BAM Manager command line utility bm.exe).

The biggest advantages of RTA are its zero-latency and low maintenance.  Once your new instance data is in the BAM database, the aggregation instantly reflects the new data.  And there's no need to run any DTS package, the sql trigger takes care of everything for you automatically.  Compared to scheduled aggregation though, RTA doesn't support as many dimensions and measures, and it can only keep relatively short period of data to keep satisfactory query performance.  Scheduled aggregation, on the other hand, have little problem handling years of years of enterprise data, and can support much more dimensions and measures.  Scheduled aggregation's advantanges don't come without a price -- it needs Analysis Service licence, a star-schema database and scheduling of the cubing DTS.  And the new instance data won't make into the cubes until the next cubing DTS run.

After you understand the pros and cons of both aggregations, it's easy to understand their usage scenarios: use RTA for time-critical, small set of Key Performance Indicators (KPI) type of tracking and use scheduled aggregation for historcail/trend analysis which normally involves large volume of data and requires complex cubes.

Posted by chunyu | 0 Comments

Three steps to monitor your business in real-time using BizTalk and BAM

With some arduous efforts, you finally got to see BizTalk server automates, orchestrates and runs your business process day in and day out. Congratulations! You may see messages/documents come and go, but no one really has a grasp how your overall business is doing. So, you may be wondering "Gee, I wish I can 'see' instantly how my business is doing without even talking to the grumpy IT guys". Bingo, BAM is exactly what you need -- it gives business users visibilities into the running business process without even writing a single line of code!

They're many excellent documents out there detailing how to use BAM (see the documentation links to the right). However, most contents were broken down by roles (admin, developer and business users) and some customers have complained they were somewhat lost when trying to develop their BAM solutions end-to-end. In this post I try to give a simple conceptual overview about how you can leverage BAM to track your business processes.

 

(Note: left column: files created by or for business users; middle column: BAM tools shipped with BizTalk 2004; right column: BizTalk and BAM artifacts.  The blue dash line shows data flow in run-time.)

As illustrated in the diagram above, the first step is to define activities and views using BAM Excel workbook. You make a copy of the BAM workbook template and give the copy a new name, say, "MyWorkbook.xls". You will use this copy to define activities, which mandate what data are to be collected, and views, which describe how the data are to be aggregated. The result workbook captures business users' "interest" in data -- what to collect and how to aggregate.

For example, there may be hundreds of, or even thousands of, data items in a purchase order Xml document sent to you by one of your suppliers. But as a business manager, you may care about only a few of those, such as Purchase Order Amount, Product Category, Customer Location (country, state, city) etc. and you want to organize the data based on location. These payload data will be extracted from the received message in run-time. In addition, you may also want to know when the purchase order is received, approved and shipped by your company. These timestamps are to be collected by the BizTalk server and referred as "Business Milestone" in BAM.

After you go through the wizard in the BAM workbook and define the activities and views, you save the workbook and export the BAM Definition Xml. The exported BAM Definition Xml will be needed in step 3. Please refer to the "BizTalk Information Worker Product Documentation" regarding how to use the BAM workbook wizard.

The second step is to use the supplied command line utility bm.exe to deploy the workbook from the first step. bm.exe does many things for you. In a nutshell, it takes a BAM Definition file (either a BAM workbook or a Xml file) and a BAM Configuration Xml File as input, and dynamically generates the entire tracking infrastructure for you. Simply put, BAM Definition file defines what data to collect and how to aggregate them, whereas BAM Configuration Xml specifies where to deploy your dynamic infrastructure. Generated infrastructure artifacts include SQL tables, views, triggers, stored procedures, Olap cubes and cubing DTS package (if you define measures and dimensions in your view) and archiving DTS package (if you specify the archiving database in your BAM configuration Xml). The tracking infrastructure serves as the data store and the aggregation engine. Note that after deployment, bm.exe also spits out a live data workbook <workbook name>_LiveData.xls, which you use to browse the both scheduled and/or real-time aggregation data.

Up to this point, you have both the tracking infrastructure and UI tool for browsing the data, but you don't have any tracking data yet. This is what step 3 about -- it instructs the BizTalk server to extract the data of interest at runtime and import them into the tracking infrastructure. How do you accomplish this? By the supplied Windows UI tool called Tracking Profile Editor (TPE). You first import the BAM Definition Xml from step 1 and your deployed orchestration assembly into TPE. Then you map each data item in the BAM Definition Xml to the message part in the assembly. The mapping result is called "tracking profile" in BizTalk and can be saved as .btt file. After you're done with the mapping, you deploy the tracking profile back to the BizTalk configuration database. Click here for TPE online help.

Going through the three steps above, now you're ready to monitor your business in real-time. When your orchestration assembly processes next message, it will write tracking data to the tracking infrastructure and you'll be able to see the aggregated data using the live data workbook from step 2. And, did I say you don't need to write any code?

Enjoy your new BAM solution!

Posted by chunyu | 6 Comments

bm.exe deployment common errors and resolutions

Admin uses BAM Manager command line utility bm.exe to manage the dynamic SQL and OLAP tracking infrastructure.  bm.exe does many things for you during the deployment of a Xml or XLS file -- creating SQL tables, indexes, triggers, views, stored procs, OLAP cubes, DTS packages, SQL and OLAP security roles etc.  I compiled some of the common errors into this thread.  Let us know if you encountered other errors and need help.  We'll try to resolve that for you.

1. Many "fnGetString" or "Unable to get resource string" popup error dialogs or "Old Format or Invalid Type Library" error in the command line output.
Cause & Resolution: This is a known issue and has been addressed in BizTalk Online Documentation . To work around this problem, you can either install the Office Multilingual User Interface (MUI) pack which can be found on www.microsoft.com or change your locale to US-EN just for the duration of the deployment.

2. "Unable to connect to the registry on the server <servername>, or you are not a member of the OLAP Administrators group on this server."
Cause & Resolution: first, make sure you have SQL2000 SP4 installed on your Analysis Server. Second, you need to have the admin priviledge to deploy OLAP cubes to your Analysis Server.  Check out this KB article http://support.microsoft.com/?id=231951 to make sure you have all the necessary permissions.

3. "Queryinterface for interface DTS.CustomTask failed".
Cause & Resolution: You're missing either SQL SP3 or SQL client tools or both.  Install both should make this error go away.

4. "Error compiling hidden module: Utility".
Cause & Resolution: You're missing some dependency files.  Make sure your files meet the following minimum version requirement.  The requirement is also documented at BizTalk Admin Guide.

VBE6.dll  6.3.91.8
Excel.exe  10.0.4524.0
stdole2.tlb  3.50.5014.0
mso.dll  10.0.4219.0
fm20.dll 2.1.7017.1
scrrun.dll  5.6.0.6626
MSCOMCTL.OCX  6.1.95.45
msado15.dll  2.71.9030.0
msxml4.dll  4.0.9406.0 (SP1)

Some user reported they have multiple versions of ADO on their machines and don't know which one is actually being referenced by Bam.xls.  In that case, close bam.xls, go to the directory and run Regsvr32 msado15.dll,  then reopen Bam.xls.

5. "Invalid object name 'dbo.bam_<...>_ActiveAliasView' "

This is a known issue.  It's due to the naming confliction among Alias, Duration, measures, dimensions or dimension levels.  Check your view, make sure all Alias, Duration, Dimension, dimension levels and Measures names are unique.  In BizTalk 2006, bm.exe will spit out more descriptive error messages.

Posted by chunyu | 13 Comments

Welcome to the BAM Online Lounge!

2004 is the first year BAM, a powerful business user-oriented feature of BizTalk Server, was released to the public.  I figured the best way to conclude year 2004 is to start off a blog dedicated to BAM. 

Happy blogging!

-Chun
BAM Developer
Posted by chunyu | 2 Comments
 
Page view tracker