Introduction

Recently my team have worked with some customers around hybrid integration projects involving exposing a number of on-premise applications outside of the organization. The Windows Azure Service Bus has been one of the key technologies which has allowed us to deliver these projects successfully.

During the various projects we have worked on we found that while the Azure Service Bus offers a superb platform in the cloud to use for your messaging solution, some customers struggle to effectively write the necessary code to integrate with the Windows Azure Service Bus that would be both simple to use but also deliver some of the powerful features you might need for a robust implementation. Some examples of this are mainly around things like error handling patterns and complex message exchange patterns.

This common challenge lead us to create the AppFx.ServiceBus open source project which is on codeplex. http://appfxservicebus.codeplex.com/.

The aim of the AppFx.ServiceBus framework is to allow development teams to use a framework on top of the Azure Service Bus API which will abstract them from some of the complexities you may require in real world implementations. This framework will make it simpler to implement the various message exchange patterns through a mainly configuration based experience and also take care of things like error handling, retry and dead lettering. The underlying goal is to make it easier for you to adopt the Windows Azure Service Bus and to allow you to focus more of your time on functional code and less on the plumbing and integration code.

The Sample

In this article I want to talk through a sample scenario which I have seen a few customers implement where they have an existing application which exposes a web service interface and we want to expose some of the methods from this interface out via the Azure Service Bus. AppFx.ServiceBus is a framework which allows you to implement many of the message exchange patterns using the Queue & Topic features of the Azure Service Bus.

The scenario covers two companies called Fabrikam and Contoso. Fabrikam is an insurance company. Contoso is a business partner of Fabrikam and they need to submit insurance claims for members. Fabrikam wants to implement the Windows Azure Service Bus Brokered Messaging (also referred to as “queues and topics”) so that Contoso or any other business partners can submit large volumes of claims to Fabrikam and they will be loaded into Fabrikam systems at a pace that their applications can handle.

Fabrikam has a Claims processing application which exposes web services and Fabrikam will use the AppFx.ServiceBus host to act as a bridge which will listen for messages on the service bus and load them into their Claims processing application.

The main reasons Fabrikam have chosen to implement this solution with Windows Azure Service Bus Brokered Messaging feature are:

  1. Windows Azure Service Bus will make it easy to expose messaging to many different business partners
  2. Windows Azure Service Bus will support durable messaging in the cloud
  3. Windows Azure Service Bus will support pub/sub via a topic so if they were to acquire other insurance companies it would be feasible to send claims to different organisations based on routing rules
  4. Windows Azure Service Bus supports integration with many different technologies making it easy for business partners to integrate with them

The main reasons Fabrikam chose to use AppFx.ServiceBus for part of this implementation are:

  1. Fabrikam does not have an enterprise integration server which can easily handle the on-premise connectivity to Windows Azure Service Bus
  2. AppFx.ServiceBus makes it easy for Fabrikam to listen to the Windows Azure Service Bus and to focus on the functional code they need to write
  3. AppFx.ServiceBus encourages some standards which will help to make it easier to develop messaging solutions on Windows Azure Service Bus
  4. AppFx.ServiceBus does not restrict the partners in their choice of technology which they use to send messages to Fabrikam

Architecture

The following diagram shows the high level architecture we are trying to implement in this scenario.

image

Walk-through

In the walk through we will talk about how the partners could go about developing this solution which could then easily be extended to cover other scenarios based on the integration pieces put in place by these partners.

The Sample Solution

For this walk-through I have provides a sample solution which is available for download from the following location:

http://appfxservicebus.codeplex.com/releases/view/102544

In the sample you will see the below solution structure which will implement the scenario.

image

In the picture above you can see the solution with the following projects:

  • Contoso.Application = A windows forms application to simulate Contoso sending messages
  • Fabrikam.Claims = A console application hosting a WCF service to simulate the Claims Processing Application
  • Fabrikam.ServiceBus.Bridge = A class library which will be hosted by AppFx.ServiceBus to process messages from the Windows Azure Service Bus
  • Contracts = A class library containing an XSD file defining the submit claim message

Note:

If your looking to get up and running with the sample you will need to replace a few configuration settings with your own Azure Service Bus Namespace and the key for your Service Identity. To make this easy if you search in the solution for the string “****” you will easily find all of the places where you need to add a setting.

The Claims Processing Application

To simulate the Claims Processing Application which Fabrikam owns we will use a WCF service which will look similar to the web service exposed by the Claims Processing Application. The Fabrikam.Claims project in the sample will implement this stub.

The project is a console application and will come online and host a WCF service which will be able to save the claim. The project contains a file called IClaimsService.cs which contains the definition for the claims service which is shown below.

image

The ClaimsService.cs file contains the implementation of the Claims Service which is shown below.

image

You can see the service is very simple and will just write to the console that a claim has been received and then return an ID in the response.

The following picture will show you how the console application will initiate the WCF Service host and then be listening for inbound calls to the Claims Service.

image

At this point the console application can be started and the WCF service will be available.

Step 1: Developing Message Definitions

As with most integration solutions it is common to separate the application specific message formats from those that you will flow around your integration/API layer. This is particularly important in this scenario as earlier in the article we discussed how Fabrikam want to be able to use this solution to extend to other businesses they may acquire in the future.

In the solution there is a contracts application which contains an XSD schema called SubmitInsuranceClaim.xsd. This schema contains the definition for a submit claim message which is shown below.

image

You can see in this case it’s a pretty simple message for demonstration purposes but the principles are the same. We will define a message using XSD and we can then create messages which can be serialized using the XML or JSON serialization formats. In our solution for this sample we will generate classes from this schema to make it easy to produce messages meeting the right format. The XSD approach also allows us a nice versioning strategy for our messages.

Step 2: Fabrikam configures Windows Azure Service Bus

Fabrikam will need to configure the Windows Azure Service Bus to support the messaging scenario we want to implement. This scenario will be implemented by using the following tasks:

  1. Create a service bus namespace for the messaging scenario
  2. Create a topic for messaging
  3. Create a subscription for Fabrikam
  4. Create a service identity for Fabrikam in the Access Control Service
    1. Give Fabrikam the listen permission for the subscription
  5. Create a service identity for Contoso in the Access Control Service
    1. Give Contoso the send permission for the topic

To reduce the overall length of this article I am going to assume you already know how to do the above actions through the Windows Azure Management Portal and the Windows Azure Access Control Service.

In case you are not familiar with the above please refer to the Windows Azure Documentation.

To make the sample simpler you can use the owner credential from the access control service in your configuration (not recommended for real world solutions).

Note:

In real world solutions you would be implementing specific identities in the Access Control Service for the different parties involved in your scenario. You could also be implementing different access control permissions for access to different queues or topics. If you are looking into this I’d recommend that you take a look at the SBAzTool tool which can help you do this.

You could also make it easier to setup the topics and subscriptions by using the Service Bus Explorer Tool to import the Msdnflash-Sample_Entities.Xml file which is in the solution and contains all of the messaging entity definitions.

Once configured your messaging entities should look like the following picture:

image

Step 3: Fabrikam develops message handlers

The next step is that developers in Fabrikam will create some message handlers to be used by the AppFx.ServiceBus framework to handle the claims message and to push it on to the Claims Processing Application.

If you refer to the Fabrikam.ServiceBus.Bridge project you will see the below items in the project.

image

Step 3.1 Adding WCF Service Reference

To begin we need to create a proxy for the WCF Claims Service. If you remember from the Fabrikam.Claims project the address for the service was http://localhost:8733/ClaimsService.

Using the normal WCF Add Service Reference wizard we will add a reference to that service. In the sample I have called this reference ClaimsServiceAgent.

I assume you already know how to add a WCF Service Reference, but incase not please refer to the MSDN Documentation for this.

Step 3.2 Adding Claims Message Class

If you look in the contracts folder you will see a class called com.fabrikam.contracts.claims.v1.0.cs. This class is generated from the XSD schema we discussed earlier. To generate this class I have modified the build process for this Visual Studio project to include some extra MsBuild tasks before the project is compiled. If you refer to the OverrideBuild.targets file you will see the below MSBuild elements.

image

We then modify the Visual Studio project file to import the OverrideBuild.targets file so that these extra targets are executed at compile time.

This target is used to execute svcutil.exe to generate a class from the schema. This class can then be used in the project.

Step 3.3 Adding Message Handler

The next step is to create a message handler for the Submit Claim Message. If you refer to the SubmitClaimHandler.cs file you will see the code to implement a message handler. This class implements the IHandleMessage interface from the AppFx.ServiceBus framework. This class is also decorated with attributes from the Managed Extensibility Framework (MEF) which means that at runtime it is loaded by the framework and can be used to handle messages.

The below picture shows the declaration of the class with its MEF attributes.

image

The key thing in this declaration is the Message Type. This is in the format of [XML Namespace]#[RootElementName] as described in the AppFx.ServiceBus documentation which talks about the message type formats. This declaration basically tells the framework that this class is a message handler and it handled Submit Claim Messages.

In the message handler the developer will need to implement the rest of the IHandleMessage interface.

Handle Method

This method is the key method to implement for this message handler. Handle is the method for handling one way messages. The below picture shows the implementation of the Submit Claims Message Handler.

image

You can see that the message handler will do some basic mapping to create the request message which will be sent to the Claims Processing Application WCF Service. It will then instantiate a proxy object and make a call to the Claims Service.

If the response is successful (i.e.: a Claim ID is returned) then the handler assumes everything is fine. If the claim id is not valid then the handler will throw an error which the framework will use to determine that the framework can retry the message.

If an error is thrown by the Claims Service then the handler will let this bubble up to the AppFx.ServiceBus framework which will initiate a process where the message will be retried a number of times depending on the Max Delivery Count setting on the subscription in Windows Azure Service Bus.

Deserialize Request Message

This property tells the framework if the message handler wants the framework to deserialize the message before handing it to the message handler. In most cases the developer will choose true for this option unless they are implementing a generic handler.

image

Input CLR Message Type

This property tells the framework what CLR type the message should be serialized to before passing it to the handler.

image

Other Methods & Properties

There are a couple of other properties and methods in the message handler but they don’t really apply much to this scenario so I will let you look at them in the source code for the sample for more info.

Step 3.4 Adding AppFx.ServiceBus Configuration

The AppFx.ServiceBus host needs to be configured with some settings to tell it how to connect to the Windows Azure Service Bus and also any configuration settings which your message handlers may need. In the sample solution you will see a file called AppFx.ServiceBus.Hosts.Console.exe.config. This file contains all of the settings we need for the sample and it will be placed with the AppFx.ServiceBus host at runtime.

Inside the file you will see the below settings which are used by the message handler to connect to the Claims Processing Applications WCF Service which look like in the below picture.

image

You will also see the AppFx.ServiceBus custom configuration section in the configuration file which is used to tell the AppFx.ServiceBus host to listen to our subscription we created earlier. The configuration for this looks like the below:

image

You can see we have a connection string for the Windows Azure Service Bus Namespace and then the AppFx.ServiceBus section defines which queues or subscriptions we will listen to.

Note that AppFx.ServiceBus also uses log4net so you can configure any diagnostics and logging settings also.

The configuration file is set to copy to the output directory at compile time.

Step 3.5 Modifying Build Output

In the message handler project we want to copy the various items from the AppFx.ServiceBus console host to the bin directory of the message handler project. To do this we have modified the MsBuild process like we did earlier to generate classes from the schema. The below picture shows how we did this.

image

Step 4: Fabrikam deploys and configures AppFx.ServiceBus host

In a real world scenario Fabrikam would use the AppFx.ServiceBus windows service host to be their runtime host listening to the service bus. For the purposes of this sample we will use the console application host instead as its easier to show you what is happening. At compile time the AppFx.ServiceBus console host is copied to the bin directory of the message handler project and you will be able to start it by double clicking the AppFx.ServiceBus.Hosts.Console.exe file.

Step 5: Contoso Develops Client to Send Messages

We said earlier that Contoso may use different technology to Fabrikam so we want to ensure that using the AppFx.ServiceBus framework doesn’t restrict what technology Contoso may use. To demonstrate this principle we will use a sample application which will use REST to send messages to the Windows Azure Service Bus from Contoso and Fabrikam will process them using the AppFx.ServiceBus framework.

In the Constoso.Application project we have developed a simple windows forms application with a form which will capture some basic details and then make a call to the Windows Azure Service Bus. The form looks like the following picture.

image

Behind the submit button we will have code which will do the following:

  1. Make a call to the Windows Azure Access Control Service to get a token to send messages to the topic
  2. Create a message in the JSON format to represent the submit claim message
  3. Send the message to the topic

The below picture shows this code.

image

Step 5.1 Getting a Token from ACS

When you wish to send a message to the Windows Azure Service Bus you need to have a token which has been issues by the Windows Azure Access Control Service which will grant you the “Send” permission for the queue or topic which you would like to send a message to. With a .net client using the Windows Azure Service Bus SDK this can be abstracted from you in a number of different ways such as WCF bindings. In this scenario we are trying to demonstrate a basic client which doesn’t have access to the AppFx.ServiceBus or Windows Azure Service Bus SDK’s. In this case the client will need to make its own call to the Access Control Service so it can be issued a token which it can use to POST a message to the Windows Azure Service Bus topic.

The below picture shows how to make a call to the ACS using the WebClient object and an HTTP POST to retrieve a token to access the Service Bus endpoint.

image

Step 5.2 Creating the JSON Message

In order to create the message in JSON I have created a class from the schema in exactly the same way as earlier in the article and then used the JsonSerializer from Newtonsoft to serialize this to produce a JSON string representing the submit claims message. The below code shows how this is done.

image

Note that there are a number of ways that I could have produced this JSON string and this was just a nice way to show how easy it can be using the XSD schema as the message definition.

Note:

If you would like to learn more about JSON or Newtonsoft’s JSON.net components please refer to the following link:

http://james.newtonking.com/projects/json-net.aspx

Step 5.3 Sending the message

The final step is to send the message, the below picture shows how the message is sent. In the below picture you will see the code to achieve this.

image

Using REST we have again used the WebClient object to do an HTTP POST. The JSON string is the body of the message but we have also set a couple of header properties which are important for message processing:

  1. The Authorization Header contains the token from the Access Control Service to grant access to the topic
  2. The Content-Type is set to text/json to tell the AppFx.ServiceBus framework that the message is a JSON message. It could also be set to text/xml if the message was xml serialized
  3. The BrokerProperties header is set to a string containing properties for Windows Azure Service Bus. One of the key properties we need to set for AppFx.ServiceBus is the Label property. We use this property to communicate what the message type is. This is then used to process the message and find a handler for it.

As you can see the content type and label are the only properties which AppFx.ServiceBus needs you to set in this scenario. Note that there are other properties Windows Azure Service Bus may need you to set in similar scenarios for things like routing.

Step 6: Running the Sample

Once you have hit F5 in Visual Studio you will see the Claims Processing Application WCF service will be online and also the Contoso Windows Application will start up. You will also need to ensure you double click the AppFx.ServiceBus.Hosts.Console.exe file to start listening to the Windows Azure Service Bus.

When you click the send button you will get a message box prompting that the message has been sent to the Azure Service Bus Topic.

image

You will then see the console window for the AppFx.ServiceBus host will update to indicate it has processed the message like in the below picture.

image

You will also see that the back end Claims Processing Application service was executed and the console window will show you that the claim was processed.

image

How might the partners extend this integration Pattern

As I mentioned earlier in the article this pattern is one Fabrikam were looking at strategically and there are a few ways which they may extend the use of this pattern in the future. Some of the ones they could consider include:

  1. They could include more message types for different processes and messaging scenarios
  2. They could allow other business partners to send messages to Fabrikam
  3. They could use routing and pub/sub on Windows Azure Service Bus to send messages to any companies they acquire
  4. Any companies they acquire could also use AppFx.ServiceBus as a bridge component between their on-premise services and Windows Azure Service Bus

Summary

In this article we have shown you how combining Windows Azure Service Bus and AppFx.ServiceBus made it easy for Fabrikam to expose some of their existing on premise web services to the cloud using a hybrid integration pattern. AppFx.ServiceBus provided a solid framework which Fabrikam could install, configure and then extend through the message handler patterns to implement an on premise bridge connecting the web services to the service bus in the cloud.

By using some of the development techniques encouraged by the AppFx.ServiceBus approach Fabrikam was able to ensure that their on-premise implementation was interoperable with client applications who could use a whole set of different technologies ranging from simple REST clients through to those using the Windows Azure Service Bus SDK or any of the SDK’s for other languages (or even using the AppFx.ServiceBus client side framework).

While this example covered a single messaging pattern and customer scenario there are many other patterns and features available in the AppFx.ServiceBus framework which could help you to accelerate your real-world projects with Windows Azure Service Bus brokered messaging. To find out more about these features please refer to the projects Codeplex site on the following link:

http://appfxservicebus.codeplex.com/

About the Author

MSDN Flash PicMichael Stephenson is a UK based Integration Specialist who has worked with many consultancies and customers architecting and developing integration solutions based on Microsoft technologies including BizTalk Server and Windows Azure. Michael is heavily involved in community activities such as the UK Connected Systems User Group and speaking at events in Europe. Michael has been a Microsoft MVP for the past 5 years.

Acknowledgements

Thanks to the following people for reviewing this article

Kent Weare

Kent Weare is a Senior Enterprise Architect for a large Power Generation company in Canada.  For the past 9 years he has been very involved in the integration of disparate systems using technologies such as BizTalk Server and Windows Azure Service Bus.  Kent remains active within the integration community by blogging, writing technical books and speaking at events in both Canada and Europe.  For these efforts, Microsoft has recognized him a Microsoft MVP for the past 6 years.

Steef-Jan Wiggers

Steef-Jan Wiggers Microsoft Integration MVP is an IT architect/Consultant at motion10 with over 14 years of experience as a technical lead developer and application architect, specializing in custom applications, enterprise application integration (BizTalk), Web services and Windows Azure. He has experience in architecting, designing, developing, and supporting sophisticated and innovative software using many different Microsoft technologies and products.

Steef-Jan is very active in the BizTalk community as a blogger, Wiki author/editor, forums moderator, and public speaker. He is a published author (BizTalk Server 2010 Cookbook) and Microsoft Integration MVP. He has received the Microsoft Most Valuable Professional (MVP) award for his contributions to the world-wide BizTalk Server community in 2010. He was re-awarded his MVP status in July 2011 and 2012.