Welcome to MSDN Blogs Sign in | Join | Help

Max Akbar

For the hardcore Commerce Server Developer... and demystify the product

Syndication

News

These postings are provided "AS IS" with no warranties, and confer no rights. Use of included script and code samples are subject to the terms specified here.
Everything you ever wanted to know about pipelines but were afraid to ask (Part I)

What are Pipelines

Commerce Server pipeline is a framework that links one or more components and executes them in sequence. The Pipeline framework provides stages to abstract one or more categories of work. The Pipeline Stage also determine the sequence in which each category of work is performed. Each Pipeline Component is COM object that is put inserted into a stage and is executed in sequence by the Pipeline Object. Pipelines are linked and it’s configuration stored in files Pipeline configuration files (.pcf).

Pipeline Execution

The following diagram displays the Basket Pipeline and the execution starts with the Product Info Stage and continues down to the last Stage Order Subtotal. You can add or delete Stages as well as move the sequence of a particular Stage.

 

Figure 1 Commerce Server Pipeline

Pipeline Properties

Use the Pipeline Properties dialog box to change the name of the pipeline, to specify its transaction compatibility, to specify its currency mode, or to save notes about the pipeline.

Figure 2 Pipeline Property

The following is a description of each property for the UI above.

Use this

To do this 

Label

Type the name that the Pipeline Editor displays for the pipeline.

Transaction

Select which pipeline object that you can use the pipeline configuration file (.pcf) with. The valid options are as follows:

·         Any pipeline. You can use the configuration file together with an OrderPipeline object.

·         Requires transacted pipeline. You can use the configuration file together with transacted pipeline objects (PooledTxPipeline, MtsTxPipeline).

·         Requires non-transacted pipeline. You can use the configuration file together with non-transacted pipeline objects (PooledPipeline, MtsPipeline).

Supply a currency LCID

Specify a locale ID (LCID) for currency handling. A value of zero (0) indicates to use the default locale ID of the server. The default currency LCID is zero (0).

Supply the number of decimal places used for currency rounding

Specify how many digits of precision to use for the currency. For example, you use two decimal places for United States pennies. Commerce Server uses this value for rounding purposes, and to determine the scaling factor to use in legacy currency keys.

Description

Type descriptive comments about the pipeline.

Pipeline Stages

The following diagram displays the Product Info Stage which consists of three Pipeline components. You can add or delete components using the Pipeline Editor. You can also move the individual components sequence.

Figure 3 Product Info Stage

Stage Properties

Use the Stage Properties dialog box to change the properties assigned to a pipeline stage.

Figure 4 Stage Properties                                                                                                                                                    

The following is a description of each property for the UI above.

Use this

To do this 

Label

Specify the name that the Pipeline Editor displays for the stage.

Description

Type descriptive comments about the stage.

GUID

View the GUID that identifies the component on any computer. This ID never changes.

Mode

A value of zero (0) prevents the stage from running.

Error level

Specify the error level for the stage.

Note

When any component in the stage generates an error above the level specified in the Error level box, the stage skips the remaining components in the stage, and logs the event in the log file.

 

The pipeline maintains the maximum error level encountered by any component in the pipeline. It uses the Error Level setting on the stage to compare this maximum error level and only runs components in the stage if the current error level is less than or equal to the error level limit specified for the component.

Pipeline Component

A component is a Component Object Model (COM) object designed to perform some operation on a dictionary -- a collection of name/value pairs -- that is a different representation of a runtime object. This can be a dictionary that represents an OrderForm object or a dictionary that represents another type of object.

 

Figure 5 Query Catalog Info Pipeline Component

Components Properties

Use the Component Properties dialog box to change the name of a pipeline component, to view the class ID or Program ID of a component, and to save notes about a component. Use the Component Properties dialog box, Values Read and Written tab, to view the values that the component reads from and writes to the order form and the values it reads from the Context dictionary object. For descriptions of each value read and written, access the corresponding component from the Pipeline Component Reference. These two “Component Properties” and “Values Read and Written” are common to all Pipeline components.

Figure 6 Components Properties

The following is a description of each property for the UI above.

Use this

To do this 

Label

Specify the name that the Pipeline Editor displays for the component.

Class ID

View the GUID that identifies the component on any computer. This ID never changes.

Program ID

View the easily recognizable ID for the component. This ID never changes.

Required

Check to make sure that the pipeline component is not removed from the pipeline.

Description

Type descriptive comments about the component.

 

Figure 7 Component Values Read and Written

The following is a description of each property for the UI above.

Note: The Pipeline Editor cannot display values read and written for components that do not support the IPipelineComponent Interface. We will discuss the Interfaces that the Pipeline components implement later on.

Use this

To do this 

Values Read

View the values that the component reads from the order form before it performs its function.

Values Written

View the values that the component writes to the order form or other dictionaries after it completes its function.

Context Values Read

View the values that the component reads from the Context dictionary object. The Context dictionary object is passed as a parameter to the Execute method of the pipeline.

 

There are however some out of the box components that need additional configuration or other information, this data is displayed on additional property pages.

Properties Dialog Box Tab Name

Description

Eval Expressions Tab

Determines how the CSFEvalTargetGroups component evaluates targeting expressions for advertising, content selection, and discounts.

Execution Mode Tab

Determines how the PaymentMethodRouter component processes credit cards and whether to validate payment totals.

History Penalty Tab

Determines how the CSFHistoryPenalty component applies penalties to content items based on how recently they have been selected.

Load History Tab

Determines where the CSFLoadHistory component retrieves the history string.

Order Discount Tab

Determines how the OrderDiscount component applies discounts scheduled by the Marketing Manager to the shopping basket.

Order Level Discount Tab

Determines how the OrderLevelDiscountApply component applies order level subtotal discounts.

Payment Router Tab

Determine the behavior of the PaymentMethodRouter pipeline component when processing credit cards and validating payment totals.

PO to File Tab

Determines how the POtoFile component retrieves and sends a purchase order or other textual data to a file.

Record History Tab

Determines how many items the CSFRecordHistory component saves in the history list string, which is used to contain the identifiers of winning content items.

Save History Tab

Determines where the CSFSaveHistory component saves the history string.

Scriptor Tab

Stores the script to be run by the Scriptor component.

Select Winners Tab

Determines how the CSFSelectWinners component displays house ads.

Send SMTP Tab

Determines where the SendSMTP component sends an e-mail message and the information sent.

Splitter Tab

Determines how the Splitter component divides an order into groups based on a list of item keys.

Test SendSMTP Tab

Tests the configuration of the SendSMTP component by sending a test e-mail message.

 

Commerce Server provides five objects to execute pipelines:

1.       MtsPipeline

Use this object to run a non-transacted Microsoft Commerce Server pipeline.

2.       MtsTxPipeline

Use this object to run a transacted Microsoft Commerce Server pipeline configuration.

3.       PooledPipeline

Use this object to run a Microsoft Commerce Server pipeline configuration file. The PooledPipeline object uses COM+ object pooling to greatly increase the loading speed of pipelines, where the contained pipeline components support object pooling.

4.       PooledTxPipeline

Use this object to execute a Microsoft Commerce Server pipeline. Use in place of the PooledPipeline object where you need transactions. The PooledTxPipeline object uses COM+ object pooling to increase the loading speed of pipelines.

5.       OrderPipeline

Use this object to run a Microsoft Commerce Server Content Selection pipeline. To create an Order Processing pipeline (OPP), use a PooledPipeline, PooledTxPipeline, MtxPipeline, or MtsTxPipeline object.

The MtsPipeline and MtxTxPipeline is recommended for development only. All other Pipelines are recommended for production. The reason for this is that the MtsPipeline and MtxTxPipeline objects do not load the configuration into memory and makes it easier to develop rather than stopping and starting IIS every time a change occurs.

Note: MicroPipe is no longer supported in Commerce Server 2007. In earlier version of the Commerce Server you could only execute the Pipeline component.

By default Commerce Server has the following  Pipeline files:

1.       Advertising.pcf

2.       Basket.pcf

3.       Chckout.pcf

4.       Discounts.pcf

5.       DMLPipe.pcf

6.       Product.pcf

7.       RecordEvent.pcf

8.       Total.pcf

When unpacking the CSharp site you will also get the above Pipeline files. These Pipeline files can be used as template to start a new Commerce Site.

Note: Commerce Server 2007 does not have Pipeline templates.

Pipelines in a nutshell

So to summarize everything, Pipeline framework consists of COM objects that are linked and stored in pcf file format. You can further categorize pipelines into stages. Pipelines are executed by the Pipeline object and configured by Pipeline Editor. Pipelines can be executed transactional and be pooled for performance. Commerce Server has a dozen or more Pipeline components and several default pcf files containing one or more Pipeline components. You can also create your own Pipeline components.

How are Pipelines used in Commerce Server

Commerce Server has three types of Pipelines:

1.       Order Processing

The Order Processing pipelines (OPP) are software infrastructures that link several components and run them in sequence on an OrderForm or Dictionary object. The Order processing Pipeline implement aspects of business-to-consumer and business-to-business checkout transactions such as discounts, Inventory checking, Product Catalog checking etc…

2.       Content Selection Framework

The Content Selection Framework (CSF) is a platform-level framework for building targeted content delivery applications on the Internet. The targeted advertising and discount features of Microsoft Commerce Server are built by using the CSF. The framework provides the means for building new types of content delivery applications and makes it easy to customize and extend the advertising and discount delivery systems.

3.       Direct Mailer

The Direct Mailer pipeline runs components that screen recipients and personalize e-mail, and then sends the personalized e-mail.

You can add third-party or custom components to the Commerce Server pipeline. Those components need implement only the standard pipeline component interfaces. They also, however, need to understand the dictionary keys that components in the pipeline use.

How Pipelines are executed

To execute a pipeline, you add the name of the pipe to the pipelines node for your site in the site Web.config file, see example bellow.

<pipelines siteName="CSharpSite">

  <pipeline name="basket" path="pipelines\basket.pcf" transacted="true" type="OrderPipeline" loggingEnabled="false"/>

</pipelines>

 

Each pipeline node must have the following attributes:

·         Name - The name that you will reference in your application when executing the pipeline.

·         Path - The relative path of the pipeline configuration file (.pcf).

·         Transacted - Set this attribute to true or false to indicate the pipeline’s transacted state.

·         Type - You can use either OrderPipeline or CSFPipeline for this attribute. You use an OrderPipeline to process orders and a CSFPipeline to display advertisements.

·         LoggingEnabled - Set to true to enable logging. When you enable logging, the log file is created in the pipelines\log directory. You must also set appropriate permissions for the current w3wp.exe process to be able to create log files.

Sample code to execute a pipeline.

PipelineInfo info1 = new PipelineInfo(pipelineName, OrderPipelineType.Basket)

Basket.RunPipeline(info1);

 

Pipeline Types available.

public enum OrderPipelineType

{

      Custom,

      Product,

      Basket,

      Total,

      Checkout,

      AcceptBasket

}

 

This enumerator provides pipeline type information to PipelineInfo to allow an object to tailor its behavior to the requirements of the particular pipeline in use, without relying on the name of the pipeline.

The AcceptBasket type is used by the LOB Adapter.

When using the Custom type, the PipelineInfo constructor will not attempt to set up the pipeline context.

Member name

Description

AcceptBasket

Indicates an AcceptBasket pipeline (e.g. AcceptBasket.pcf).

Basket

Indicates a Basket pipeline (e.g. Basket.pcf).

Checkout

Indicates a Checkout pipeline (e.g. Checkout.pcf).

Custom

Indicates a custom pipeline type.

Product

Indicates a Product pipeline (e.g. Product.pcf).

Total

Indicates a Total pipeline (e.g. Total.pcf).

 

Summary

At this point you should have a good context of what Pipelines are and in the next part we will get a deeper understanding of Pipelines by developing and understanding Pipeline internals. Keep an eye out for part II of the “Everything you ever wanted to know about pipelines but were afraid to ask”.

Published Thursday, August 17, 2006 11:54 PM by Max Akbar

Filed under: ,

Comment Notification

If you would like to receive an email when updates are made to this post, please register here

Subscribe to this post's comments using RSS

Comments

# Commerce Server 2007: Max Starts Blogging! @ Wednesday, August 23, 2006 9:34 PM

In my last post which listed a number of great new Commerce Server posts I forgot to mention that Max Akbar [MSFT] - Product Manager (Maxter Chief) has started blogging with a vengeance!

Jeff Lynch [MVP]

# re: Everything you ever wanted to know about pipelines but were afraid to ask (Part I) @ Tuesday, August 29, 2006 3:47 PM

Hi Max,

I'm creating a new CheckInventory Stage, to resolve some custom needs of my client and I have a doubt.

I need to access the Invetory Catalog of the product item but I can't find a way to use a Current Context. The workaround I create every-time a new Context for the product, but the performance is going down.

Do you know some way to do this?

Thank you,
Willian

wleite

# re: Everything you ever wanted to know about pipelines but were afraid to ask (Part I) @ Wednesday, August 30, 2006 2:33 PM

The IPipelineComponent interface has two methods Execute and EnableDesign.

The Execute method has three parameters one of them is a context (pdispContext) value. See the interface implementation below:

// IPipelineComponent

[ComImport, Guid("CA215520-9A2F-11D0-B86C-00C04FD7A0F9"), InterfaceType((short) 1)]
public interface IPipelineComponent
{
     [MethodImpl(MethodImplOptions.InternalCall, MethodCodeType=MethodCodeType.Runtime)]
     int Execute([In, MarshalAs(UnmanagedType.IDispatch)] object pdispOrder, [In, MarshalAs(UnmanagedType.IDispatch)] object pdispContext, [In] int lFlags);
     [MethodImpl(MethodImplOptions.InternalCall, MethodCodeType=MethodCodeType.Runtime)]
     void EnableDesign([In] int fEnable);
}

// IPipelineComponent

You can pass the context using the pipelineinfo object in your Execute Method get the value from pdispContext.

Good luck,
-Max

Max Akbar

# CS 2007: StarterSite CTP and assorted links @ Thursday, August 31, 2006 4:34 PM

The StarterSite CTP is now available for download. This should help and get everyone started on a solid...

Nihit Kaul's World of Commerce and Testing

# re: Everything you ever wanted to know about pipelines but were afraid to ask (Part I) @ Thursday, September 14, 2006 5:57 PM

Hi Max,

I'm trying what you suggest but I still can't get the Catalog Context (and Inventory Context) inside the Pipeline.

<Code>
public int Execute(object pdispOrder, object pDispContext, int lFlags)
{
 ...
 
 Context = (IDictionary)pdispContext;
 
 ...
</Code>  

For this pDispContext Object I found this Keys : Evaluator, Language, CatalogConnectionString, MessageManager, CacheManager, CatalogCacheConfiguration, ProfileService, CommerceResources and CacheName.

None of them have the Catalog Context, so I tried to found where can I pass, in RunPipeline, the Catalog Context but I can't find it.

To run my pipelines I'm using the BasketHelper from TestStore.
 
<Code>
public PipelineExecutionResult RunBasketPipeline(string pipelineName)
{
   if (String.IsNullOrEmpty(pipelineName))
   {
       pipelineName = BasketHelper.DefaultBasketPipelineName;
   }

   using (PipelineInfo pipeline = new PipelineInfo(pipelineName, OrderPipelineType.Basket))
   {
       return this.RunPipeline(pipelineName, OrderPipelineType.Basket);
   }
}

...

public virtual PipelineExecutionResult RunPipeline(PipelineInfo pipelineInfo);

</Code>

As you can see I not have any other parameters to put the Catalog Context. In the last Commerce Server Architecture Deep Dive WebCast they tell me that the Default CheckInventory and UpdateInventory call your own StoredProcedures in DataBase to do the Check and Update Inventory.

For Workaround the problem I'm using this function:

<Code>
private InventoryContext CreateInventoryContext()
{
   // Create a CatalogSiteAgent to connect to the database.
   CatalogSiteAgent catalogSiteAgent = new CatalogSiteAgent();
   catalogSiteAgent.SiteName = MyConstants.ConfigSiteName;
   catalogSiteAgent.IgnoreInventorySystem = false;

   // Create a CatalogContext object.
   CatalogContext catalogContext = CatalogContext.Create(catalogSiteAgent);
   
   // Get the InventoryContext property.
   InventoryContext inventoryContext = catalogContext.InventoryContext;

   return inventoryContext;
}
</Code>

Do you have any ideas?

Thank you,

Willian Leite

wleite

# re: Everything you ever wanted to know about pipelines but were afraid to ask (Part I) @ Sunday, September 17, 2006 3:31 PM

Hi Willian,

You are so very close good job. I am about to post part III of this ongoing post and has the sample code you are looking for :).

-Max

Max Akbar

# Commerce Server 2007: Great Pipeline Posts by Max! @ Sunday, September 17, 2006 8:19 PM

Max Akbar [MSFT] has recently finished a series of really good posts about the Commerce Server Pipelines system.

Jeff Lynch [MVP]

# re: Everything you ever wanted to know about pipelines but were afraid to ask (Part I) @ Monday, September 18, 2006 7:47 PM

runnig the checkout pipeline I get this error:
error Component Execution failed for component[0x1] hr: 0x80004002 ProgID: Commerce.UpdateInventory Unable to cast object of type 'System.DBNull' to type 'Microsoft.CommerceServer.Runtime.Configuration.CommerceResourceCollection'

How can I debug this?Is is configuration or registery problem or is it sending bad data to pileline?

Thanks,

May

May

# re: Everything you ever wanted to know about pipelines but were afraid to ask (Part I) @ Monday, September 18, 2006 7:53 PM

Check to make sure you have an inventory resource? If you are not using the Inventory system remove the Inventory pipelines.

Also check the eventlog to see if you have anything there.

Good luck,
-Max

Max Akbar

# Pipeline development in Commerce Server 2007 as prescribed by Max @ Sunday, October 08, 2006 10:26 PM

Max Akbar [MSFT] has recently finished a series of really good posts about the Commerce Server...

Biztalk 2006

# Developing Your First Order Pipeline Component @ Saturday, November 11, 2006 7:53 PM

As previously discussed the pipeline system enables sequential workflow processing. It is a COM-based...

using Colin.Bowern;

# re: Everything you ever wanted to know about pipelines but were afraid to ask (Part I) @ Monday, March 12, 2007 9:22 AM

Great blog entry.  

You state "The MtsPipeline and MtxTxPipeline is recommended for development only."  However, nowhere can I find how to configure/control what object is used to execute a pipeline.  In fact I cannot find information about a Commerce Server install relates to COM+ at all.  From reading the product documentation, I gather that the installation wizard creates a COM+ application for the CS instance, but no further information is detailed.  Can you elaborate on this and/or provide references to material that explain this.

Chuck