The parallel shape in BizTalk Server (v2004 to v2009) is widely misinterpreted to be multi-threaded.
What the parallel shape does well is that it ensures that all branches complete their execution before the next process takes place. The parallel shape does a context switch on a thread wait (busy waiting, such as waiting for a message to come in from a receive port) between branches.
However – suppose that the number of branches of executions are not known at design time, then the parallel shape would not be suitable to use. A different approach will be required.
A classic example of parallel execution is de-batching a large message file into smaller files, and then aggregating all the results into a single large result. Each subset (small file) represents an independent unit of work that does not rely or relate to other subsets, except that it belongs to the same batch (large message). Eventually each child process (the independent unit of work) will return a result message that will be collected with other results, into a collated response message to be leveraged for further processing.
0. The initial step consists of de-batching the large message in the receive pipeline into smaller child messages.
1. The receive pipeline publishes the child messages to the message box database.
2. Child business process orchestrations (or 2-way send ports) subscribe against the child messages to accommodate business requirements. Note that individual business processing does not explicitly require orchestrations.
3. Responses from Line-of-Business (LOB) applications are submitted back to the child orchestrations (or to 2-way send ports).
4. These responses are aggregated back by an aggregator (can be accommodated by an orchestration as depicted, or by a SQL stored procedure).
5. The aggregator will submit the final aggregated result.
6. The next process (which may or may not use the final aggregated result) will proceed.
De-batching is conducted within the receive pipeline. To learn how to split a large file into smaller files, please refer to the BizTalk Server 2006 samples – Split File Pipeline example. Even though the sample is based on BizTalk Server 2006 (R1) – it remains relevant for BizTalk Server 2006 R2, and BizTalk Server 2009.
The next step is a bit trickier. The smaller messages need to be correlated somehow, so that their results can be aggregated into a singular large response message (for steps 4 and 5 to be successful). Correlation is typically done in orchestration, but the proposed method here avoids the use of orchestration altogether. In order to have correlation in messaging, please refer to Paolo Salvatori’s blog on Synchronous To Asynchronous Flows Without An Orchestration.
In it, he describes a method that requires a 2-way receive port (such as a web service) , because he leverages a couple of message context properties that are unique to a 2-way receive. These are
In addition to
It is strongly suggested to learn how correlation can be achieved in messaging (without orchestration) as described by Paolo Salvatori before proceeding to the next step.
Once that method has been mastered, a slight adjustment is needed. Since the de-batching method is leveraged in the pipeline, correlation in messaging can be achieved with a 1-way receive port (such as picking up the large message from a file directory), by using message context properties that are unique to the de-batch method.
Each child message (which is now in the messagebox database) should instantiate a business process (either with an orchestration workflow or a 2-way send port) that will eventually receive a result message.
Finally an aggregator is required to collect and collate the results from the child business processes. This aggregator can be an orchestration, or the preferred way would be from a SQL stored procedure.
*Image courtesy of Paolo Salvatori
This method was also communicated in http://geekswithblogs.net/asmith/archive/2005/06/06/42281.aspx
Note that the XSLT can be eventually replaced by a custom pipeline component or directly by an envelope (in this case the send pipeline needs to contain the Xml Assembler component).
Please note the standard disclaimer that abdicates any form of responsibility:
The sample code is the property of Microsoft Corporation ®.
THIS CODE AND INFORMATION IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY AND/OR FITNESS FOR A PARTICULAR PURPOSE.
The sample code is not supported under any Microsoft standard support program or service. These samples are provided AS IS without warranty of any kind. Microsoft further disclaims all implied warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The entire risk arising out of the use or performance of the samples and documentation remains with you. In no event shall Microsoft, its authors, or anyone else involved in the creation, production, or delivery of the samples, code, and documentation, be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the code, samples, or documentation, even if Microsoft has been advised of the possibility of such damages.
For more information concerning the parallel shape, see “How to Configure the Parallel Actions Shape” topic in the BizTalk Server documentation.
Here are some related blogs regarding the parallel shape: