Streamlining Custom Pipeline Component Development Using Base Class Library

Streamlining Custom Pipeline Component Development Using Base Class Library

Rate This
  • Comments 7

It is widely known that custom pipeline components are highly popular extensibility points in the BizTalk architecture. The topic has been extensively discussed for many years and many useful content is available on the Web. Through observations in recent customer engagements, we noticed that there is still a couple of recommendations which could be made to help BizTalk developers save time and streamline the development of custom pipeline components. The following blog post is based off a successful customer engagement during which the concepts highlighted below have been validated and proven in a real-world BizTalk solution. It’s time for the community to benefit from our lessons learned, hence this post.

As things stand at the moment, a substantial amount of repetitive coding may be involved when building custom pipeline components. Every custom pipeline component requires repetitive implementation of a minimum of 3 standard interfaces (IBaseComponent, IPersistPropertyBag, IComponentUI) and one extra interface depending on the component type (IComponent, IAssemblerComponent, IDisassemblerComponent).

One of the challenges is that implementation logic of the 3 standard interfaces is often duplicated leading to extra code to be maintained. The amount of duplicate logic can be greatly minimized by introducing a set of base classes from which BizTalk developers can derive their implementations of custom pipeline components. It will help eliminate the majority of repetitive tasks typically involved in the development of pipeline components such as writing plumbing code for persisting and reading configuration properties, returning user-friendly property names, descriptions and design-time metadata elements.

Moreover, by just adding a bit of intellect into the base class library, it will be possible to enable the developers to apply declarative attributes in order to rapidly customize their pipeline components with localizable titles, descriptions, design-time icons and friendly property names without writing a single line of plumbing code.

In order to demonstrate the benefits of the reusable base class library, we have built a prototype which quickly turned itself into a first-class member of the solution framework in the customer project. The new base classes helped the developers ensure that all custom pipeline components share common capabilities such as instrumentation, exception handling, loading and saving component configuration, BAM tracking and more.

We have provided 3 abstract base classes (CustomComponentBase, AssemblerComponentBase and DisassemblerComponentBase) from which the developers derive their end implementations. The class diagram can be depicted as follows:
aaaaa 
The key benefit of this design was that the amount of plumbing code required for building a custom pipeline component was greatly reduced. The BizTalk developers were able to take advantage of the declarative programming model and add support for fully localizable UI elements such as property names, descriptions and toolbox icons without writing a single line of code. They were also able to stay focused on the core logic whereas the responsibility for exception handling, instrumentation and toolbox interoperability was offloaded to the base class library. This reduced the overall implementation time and delivered better quality code.

 

The example below illustrates the difference between traditional approach and the new concepts in the pipeline component development:

//
// Pipeline component configuration property management using the traditional approach (equal to 46 lines of code).
//
private string customProperty1 = "default value";

[Description("Static description for customProperty1 embedded into code")]
public string СustomProperty1
{
      get { return this.customProperty1; }
      set { this.customProperty1 = value; }
}

private bool customProperty2 = false;

[Description("Static description for customProperty2 embedded into code")]
public bool СustomProperty2
{
    get { return this.customProperty2; }
    set { this.customProperty2 = value; }
}

private int customProperty3 = true;

[Description("Static description for customProperty3 embedded into code")]
public int СustomProperty3
{
    get { return this.customProperty3; }
    set { this.customProperty3 = value; }
}

public virtual void Load(IPropertyBag pb, int errlog)
{
      object val = null;

      val = this.ReadPropertyBag(pb, "СustomProperty1");

      if (val != null)
      {
            this.СustomProperty1 = (String)(val);
      }

      val = this.ReadPropertyBag(pb, "СustomProperty2");

      if (val != null)
      {
          this.СustomProperty2 = (bool)(val);
      }

      val = this.ReadPropertyBag(pb, "СustomProperty3");

      if (val != null)
      {
          this.СustomProperty3 = (int)(val);
      }
}

public virtual void Save(IPropertyBag pb, bool fClearDirty, bool fSaveAllProperties)
{
      this.WritePropertyBag(pb, "СustomProperty1", this.СustomProperty1);
      this.WritePropertyBag(pb, "СustomProperty2", this.СustomProperty2);
      this.WritePropertyBag(pb, "СustomProperty3", this.СustomProperty3);
}

By contrast, the new base class library delivered greater simplicity which enabled replacing the above code fragment with the following:

//
// Pipeline component configuration property management using the new approach (equal to 12 lines of code).
//
// PropNameСustomProperty[n] is a resource key, the actual name will be loaded from a locale-specific resource file.
// PropDescСustomProperty[n] is a resource key, the actual description will be loaded from a locale-specific resource file.
//
[Browsable(true)] [BtsPropertyName("PropNameСustomProperty1")]
[BtsDescription("PropDescСustomProperty1")]
public string СustomProperty1 { get; set; } [Browsable(true)] [BtsPropertyName("PropNameСustomProperty2")] [BtsDescription("PropDescСustomProperty2")] public bool СustomProperty2 { get; set; } [Browsable(true)] [BtsPropertyName("PropNameСustomProperty3")] [BtsDescription("PropDescСustomProperty3")] public int СustomProperty3 { get; set; }

In summary, the base class library helped encapsulate most common tasks typically associated with custom pipeline component development such as configuration property management. In the above example, the amount of plumbing code was reduced from 46 lines to 12 lines (that’s 75% reduction). The new codebase is easier to maintain and it helps avoid common pitfalls such as forgetting to persist a property value into the property bag.

The reduction in the volume of redundant plumbing code is of course not the only benefit. The real power of a base class library for custom pipeline components comes with the ability to centralize all other common functionality which needs to be shared across all derived implementations:

  • Common configuration properties – any common configuration properties which are required for all child implementations of pipeline components can be defined in the base class. This is very handy when there is a need to enrich the entire suite of pipeline components with a common set of properties and avoid duplicating them in each individual component.
  • Tracing and instrumentation – all derived implementations of pipeline components can be transparently instrumented with tracing performed by the base classes. This will enable all derived components to output key telemetry events into the trace log, for example, to indicate when the Execute method has been invoked, when it has finished and now long it has taken to execute.
  • Exception handling and reporting – the base class is a good place to consolidate the exception management logic and provide a common mechanism for catching, handling and reporting on run-time exceptions.

Now, let’s put everything that was being said about base class library for pipeline components into a visual context. For sake of simplicity, we will title all further sections as “This is how…”.

This is how class inheritance helps derive the end implementations based on particular type of pipeline component:

SamplePipelineComponentsClassDiagram 
This is how class inheritance helps derive the end implementations based on particular type of pipeline component:
p1 
This is how a base class for disassembling pipeline components implements the IDisassemblerComponent interface:
p2  
This is how the explicit implementation of the Disassemble method allows wrapping its user-defined counterpart into a rich instrumented context:
p3 

It is worth sharing some key notes and observations made during the development phase:

  • When compiling a BizTalk pipeline containing a custom pipeline component implemented using the base class library, you will get the following warning message:

    Warning BTP0006: Component 'FullComponentTypeName' does not implement IComponentUI interface.

    This warning is the result of a missing interface implementation directly by the pipeline component. Since the IComponentUI interface is implemented by the PipelineComponentBase class, the compiler is not currently able to intelligently treat such a case. The warning message can be ignored (we found no way of suppressing this warning in the current release of BizTalk project system). We will see if this can be improved when we ship an update or service pack for the current release.
  • The derived implementation of a custom pipeline component will still have to be decorated with a ComponentCategory attribute, otherwise the component will not show up on the Choose Items dialog in Visual Studio IDE. This is due to the fact that we are not currently reflecting the custom .NET attributes across the entire inheritance chain. Again, this appears to be an unfair limitation on which the Product Group was advised and asked to consider improving this behavior.

Show me the money code! You can find the source code in the attachment below. Please share your feedback to help improve the base class library.

Streamlining Custom Pipeline Component Development Using Base Class Library.zip

For more information on the related topic, please visit the following resources:

  • The download is no longer available. Any chance to get this up again?

  • Link to your code is either deleted or wrong.

  • Thank you, folks, for reporting the problem with our sample code location. I have now corrected the URL. Please nativagate to code.msdn.microsoft.com/.../FileDownload.aspx

  • I can't get the component to show up in the toolbox. This is most likely related to namespace/resource manager. I've had these problems before. Here is sample of my code which is attemping to use the framework. Am I missing anything here? I do have resouce file added, I do have my strings defined there, project compiles OK, just when I want to use the component is throws me classic "Pipeline component is invalid". Samples which were included in the ZIP file work fine, they show up in the toolbox. So, I"m close but lost:

       [Guid("593641E3-105D-4A80-90CF-9DCF048435F1")]

       [BtsComponentName("TestName")]

       [BtsDescription("TestDescription")]

       [ComponentCategory(CategoryTypes.CATID_PipelineComponent)]

       [ComponentCategory(CategoryTypes.CATID_Validate)]

       public class TestPc : ValidatorComponentBase

       {

           public TestPc() : base(Resources.ResourceManager)

           {

           }

           public override IBaseMessage Execute(IPipelineContext pContext, IBaseMessage pInMsg)

           {

               throw new NotImplementedException();

           }

       }

  • Artur, have you checked whether your custom pipeline component's assembly is copied to "%ProgramFiles(x86)%\Microsoft BizTalk Server XXXX\Pipeline Components\" where XXXX is the version number of BizTalk Server installed on your machine?

    You will also need to make sure that the base class library assembly (containing ValidatorComponentBase) is also copied to the above folder. In addition, both assemblies must be GACed.

  • here was some sort of mixup with GUIDs. Some other things were done to the project before I installed and started using your library. After I started with a clean, fresh project everything worked beautifully. After that it was just WOW, pretty cool library. Previously I used pipeline component wizard, which was pretty good, but this is gold :)

    I would like to dig deeper into the library and use other features. I especially interested in error handling and BAM. I've looked at the samples included with the ZIP file but it's not very clear to me how to proceed. For example, I'm assuming that setting EnableActivityTracking = true enables BAM and EnableDetailedExceptions = true will be verbose about errors in my pipeline. I don't know much about BAM and want else I'd have to set up in order for this to, for example, collect business data on how fast my component is performing. What are the error handling and tracing options? Would this work with, for example, event tracing for windows? Is there something that I should pay more attention in the samples to figure out how things work with BAM or tracing? Perhaps HL7D disassembler sample has already BAM set up and I just need to depoy it.

  • Artur,

    The two properties you are referring to were included in the sample to demonstrate how developers could take advantage of our base class library and share common configuration properties in the BizTalk custom pipeline components. These properties are functional out-of-the-box.

    The EnableDetailedExceptions property doesn’t require any pre-requisites. When enabled, it instructs the exception message formatter to include the full stack information. As a result, you will see verbose error details, exactly as you have already guessed.

    The second property (EnableActivityTracking) enables a custom pipeline component to track its performance metrics using the BAM infrastructure. It requires a special BAM activity to be deployed as a pre-requisite. The BAM activity definition is included into the source code under name of PipelineComponentBenchmarkActivity.xml. In order to deploy this activity, you will need to execute the following command:

    bm.exe deploy-all -DefinitionFile:PipelineComponentBenchmarkActivity.xml

    The bm.exe utility can be found in %ProgramFiles(x86)%\Microsoft BizTalk Server [XXXX]\Tracking where [XXXX] identifies the version of BizTalk Server installed on your machine.

    With respect to the tracing capability, our base class library is already fully integrated with the ETW-based instrumentation library which I separately blogged about here:

    blogs.msdn.com/.../best-practices-for-instrumenting-high-performance-biztalk-solutions.aspx

    To learn about how to enable and work with ETW tracing in the BizTalk solutions (including custom pipeline component base library), please refer to the following write-ups:

    blogs.msdn.com/.../management-tasks-for-biztalk-cat-instrumentation-framework-fully-explained.aspx

    blogs.msdn.com/.../biztalk-application-tracing-made-easy-with-biztalk-cat-instrumentation-framework-controller.aspx

    Enjoy! And please do share your feedback.

Page 1 of 1 (7 items)
Leave a Comment
  • Please add 3 and 3 and type the answer here:
  • Post