Keep track of all the latest news and events on developer tools and technologies you care about
Recently my team have worked with some customers around hybrid integration projects involving exposing a number of on-premise applications outside of the organization. The Windows Azure Service Bus has been one of the key technologies which has allowed us to deliver these projects successfully.
During the various projects we have worked on we found that while the Azure Service Bus offers a superb platform in the cloud to use for your messaging solution, some customers struggle to effectively write the necessary code to integrate with the Windows Azure Service Bus that would be both simple to use but also deliver some of the powerful features you might need for a robust implementation. Some examples of this are mainly around things like error handling patterns and complex message exchange patterns.
This common challenge lead us to create the AppFx.ServiceBus open source project which is on codeplex. http://appfxservicebus.codeplex.com/.
The aim of the AppFx.ServiceBus framework is to allow development teams to use a framework on top of the Azure Service Bus API which will abstract them from some of the complexities you may require in real world implementations. This framework will make it simpler to implement the various message exchange patterns through a mainly configuration based experience and also take care of things like error handling, retry and dead lettering. The underlying goal is to make it easier for you to adopt the Windows Azure Service Bus and to allow you to focus more of your time on functional code and less on the plumbing and integration code.
In this article I want to talk through a sample scenario which I have seen a few customers implement where they have an existing application which exposes a web service interface and we want to expose some of the methods from this interface out via the Azure Service Bus. AppFx.ServiceBus is a framework which allows you to implement many of the message exchange patterns using the Queue & Topic features of the Azure Service Bus.
The scenario covers two companies called Fabrikam and Contoso. Fabrikam is an insurance company. Contoso is a business partner of Fabrikam and they need to submit insurance claims for members. Fabrikam wants to implement the Windows Azure Service Bus Brokered Messaging (also referred to as “queues and topics”) so that Contoso or any other business partners can submit large volumes of claims to Fabrikam and they will be loaded into Fabrikam systems at a pace that their applications can handle.
Fabrikam has a Claims processing application which exposes web services and Fabrikam will use the AppFx.ServiceBus host to act as a bridge which will listen for messages on the service bus and load them into their Claims processing application.
The main reasons Fabrikam have chosen to implement this solution with Windows Azure Service Bus Brokered Messaging feature are:
The main reasons Fabrikam chose to use AppFx.ServiceBus for part of this implementation are:
The following diagram shows the high level architecture we are trying to implement in this scenario.
In the walk through we will talk about how the partners could go about developing this solution which could then easily be extended to cover other scenarios based on the integration pieces put in place by these partners.
For this walk-through I have provides a sample solution which is available for download from the following location:
In the sample you will see the below solution structure which will implement the scenario.
In the picture above you can see the solution with the following projects:
If your looking to get up and running with the sample you will need to replace a few configuration settings with your own Azure Service Bus Namespace and the key for your Service Identity. To make this easy if you search in the solution for the string “****” you will easily find all of the places where you need to add a setting.
To simulate the Claims Processing Application which Fabrikam owns we will use a WCF service which will look similar to the web service exposed by the Claims Processing Application. The Fabrikam.Claims project in the sample will implement this stub.
The project is a console application and will come online and host a WCF service which will be able to save the claim. The project contains a file called IClaimsService.cs which contains the definition for the claims service which is shown below.
The ClaimsService.cs file contains the implementation of the Claims Service which is shown below.
You can see the service is very simple and will just write to the console that a claim has been received and then return an ID in the response.
The following picture will show you how the console application will initiate the WCF Service host and then be listening for inbound calls to the Claims Service.
At this point the console application can be started and the WCF service will be available.
As with most integration solutions it is common to separate the application specific message formats from those that you will flow around your integration/API layer. This is particularly important in this scenario as earlier in the article we discussed how Fabrikam want to be able to use this solution to extend to other businesses they may acquire in the future.
In the solution there is a contracts application which contains an XSD schema called SubmitInsuranceClaim.xsd. This schema contains the definition for a submit claim message which is shown below.
You can see in this case it’s a pretty simple message for demonstration purposes but the principles are the same. We will define a message using XSD and we can then create messages which can be serialized using the XML or JSON serialization formats. In our solution for this sample we will generate classes from this schema to make it easy to produce messages meeting the right format. The XSD approach also allows us a nice versioning strategy for our messages.
Fabrikam will need to configure the Windows Azure Service Bus to support the messaging scenario we want to implement. This scenario will be implemented by using the following tasks:
To reduce the overall length of this article I am going to assume you already know how to do the above actions through the Windows Azure Management Portal and the Windows Azure Access Control Service.
In case you are not familiar with the above please refer to the Windows Azure Documentation.
To make the sample simpler you can use the owner credential from the access control service in your configuration (not recommended for real world solutions).
In real world solutions you would be implementing specific identities in the Access Control Service for the different parties involved in your scenario. You could also be implementing different access control permissions for access to different queues or topics. If you are looking into this I’d recommend that you take a look at the SBAzTool tool which can help you do this.
You could also make it easier to setup the topics and subscriptions by using the Service Bus Explorer Tool to import the Msdnflash-Sample_Entities.Xml file which is in the solution and contains all of the messaging entity definitions.
Once configured your messaging entities should look like the following picture:
The next step is that developers in Fabrikam will create some message handlers to be used by the AppFx.ServiceBus framework to handle the claims message and to push it on to the Claims Processing Application.
If you refer to the Fabrikam.ServiceBus.Bridge project you will see the below items in the project.
To begin we need to create a proxy for the WCF Claims Service. If you remember from the Fabrikam.Claims project the address for the service was http://localhost:8733/ClaimsService.
Using the normal WCF Add Service Reference wizard we will add a reference to that service. In the sample I have called this reference ClaimsServiceAgent.
I assume you already know how to add a WCF Service Reference, but incase not please refer to the MSDN Documentation for this.
If you look in the contracts folder you will see a class called com.fabrikam.contracts.claims.v1.0.cs. This class is generated from the XSD schema we discussed earlier. To generate this class I have modified the build process for this Visual Studio project to include some extra MsBuild tasks before the project is compiled. If you refer to the OverrideBuild.targets file you will see the below MSBuild elements.
We then modify the Visual Studio project file to import the OverrideBuild.targets file so that these extra targets are executed at compile time.
This target is used to execute svcutil.exe to generate a class from the schema. This class can then be used in the project.
The next step is to create a message handler for the Submit Claim Message. If you refer to the SubmitClaimHandler.cs file you will see the code to implement a message handler. This class implements the IHandleMessage interface from the AppFx.ServiceBus framework. This class is also decorated with attributes from the Managed Extensibility Framework (MEF) which means that at runtime it is loaded by the framework and can be used to handle messages.
The below picture shows the declaration of the class with its MEF attributes.
The key thing in this declaration is the Message Type. This is in the format of [XML Namespace]#[RootElementName] as described in the AppFx.ServiceBus documentation which talks about the message type formats. This declaration basically tells the framework that this class is a message handler and it handled Submit Claim Messages.
In the message handler the developer will need to implement the rest of the IHandleMessage interface.
This method is the key method to implement for this message handler. Handle is the method for handling one way messages. The below picture shows the implementation of the Submit Claims Message Handler.
You can see that the message handler will do some basic mapping to create the request message which will be sent to the Claims Processing Application WCF Service. It will then instantiate a proxy object and make a call to the Claims Service.
If the response is successful (i.e.: a Claim ID is returned) then the handler assumes everything is fine. If the claim id is not valid then the handler will throw an error which the framework will use to determine that the framework can retry the message.
If an error is thrown by the Claims Service then the handler will let this bubble up to the AppFx.ServiceBus framework which will initiate a process where the message will be retried a number of times depending on the Max Delivery Count setting on the subscription in Windows Azure Service Bus.
This property tells the framework if the message handler wants the framework to deserialize the message before handing it to the message handler. In most cases the developer will choose true for this option unless they are implementing a generic handler.
This property tells the framework what CLR type the message should be serialized to before passing it to the handler.
There are a couple of other properties and methods in the message handler but they don’t really apply much to this scenario so I will let you look at them in the source code for the sample for more info.
The AppFx.ServiceBus host needs to be configured with some settings to tell it how to connect to the Windows Azure Service Bus and also any configuration settings which your message handlers may need. In the sample solution you will see a file called AppFx.ServiceBus.Hosts.Console.exe.config. This file contains all of the settings we need for the sample and it will be placed with the AppFx.ServiceBus host at runtime.
Inside the file you will see the below settings which are used by the message handler to connect to the Claims Processing Applications WCF Service which look like in the below picture.
You will also see the AppFx.ServiceBus custom configuration section in the configuration file which is used to tell the AppFx.ServiceBus host to listen to our subscription we created earlier. The configuration for this looks like the below:
You can see we have a connection string for the Windows Azure Service Bus Namespace and then the AppFx.ServiceBus section defines which queues or subscriptions we will listen to.
Note that AppFx.ServiceBus also uses log4net so you can configure any diagnostics and logging settings also.
The configuration file is set to copy to the output directory at compile time.
In the message handler project we want to copy the various items from the AppFx.ServiceBus console host to the bin directory of the message handler project. To do this we have modified the MsBuild process like we did earlier to generate classes from the schema. The below picture shows how we did this.
In a real world scenario Fabrikam would use the AppFx.ServiceBus windows service host to be their runtime host listening to the service bus. For the purposes of this sample we will use the console application host instead as its easier to show you what is happening. At compile time the AppFx.ServiceBus console host is copied to the bin directory of the message handler project and you will be able to start it by double clicking the AppFx.ServiceBus.Hosts.Console.exe file.
We said earlier that Contoso may use different technology to Fabrikam so we want to ensure that using the AppFx.ServiceBus framework doesn’t restrict what technology Contoso may use. To demonstrate this principle we will use a sample application which will use REST to send messages to the Windows Azure Service Bus from Contoso and Fabrikam will process them using the AppFx.ServiceBus framework.
In the Constoso.Application project we have developed a simple windows forms application with a form which will capture some basic details and then make a call to the Windows Azure Service Bus. The form looks like the following picture.
Behind the submit button we will have code which will do the following:
The below picture shows this code.
When you wish to send a message to the Windows Azure Service Bus you need to have a token which has been issues by the Windows Azure Access Control Service which will grant you the “Send” permission for the queue or topic which you would like to send a message to. With a .net client using the Windows Azure Service Bus SDK this can be abstracted from you in a number of different ways such as WCF bindings. In this scenario we are trying to demonstrate a basic client which doesn’t have access to the AppFx.ServiceBus or Windows Azure Service Bus SDK’s. In this case the client will need to make its own call to the Access Control Service so it can be issued a token which it can use to POST a message to the Windows Azure Service Bus topic.
The below picture shows how to make a call to the ACS using the WebClient object and an HTTP POST to retrieve a token to access the Service Bus endpoint.
In order to create the message in JSON I have created a class from the schema in exactly the same way as earlier in the article and then used the JsonSerializer from Newtonsoft to serialize this to produce a JSON string representing the submit claims message. The below code shows how this is done.
Note that there are a number of ways that I could have produced this JSON string and this was just a nice way to show how easy it can be using the XSD schema as the message definition.
If you would like to learn more about JSON or Newtonsoft’s JSON.net components please refer to the following link:
The final step is to send the message, the below picture shows how the message is sent. In the below picture you will see the code to achieve this.
Using REST we have again used the WebClient object to do an HTTP POST. The JSON string is the body of the message but we have also set a couple of header properties which are important for message processing:
As you can see the content type and label are the only properties which AppFx.ServiceBus needs you to set in this scenario. Note that there are other properties Windows Azure Service Bus may need you to set in similar scenarios for things like routing.
Once you have hit F5 in Visual Studio you will see the Claims Processing Application WCF service will be online and also the Contoso Windows Application will start up. You will also need to ensure you double click the AppFx.ServiceBus.Hosts.Console.exe file to start listening to the Windows Azure Service Bus.
When you click the send button you will get a message box prompting that the message has been sent to the Azure Service Bus Topic.
You will then see the console window for the AppFx.ServiceBus host will update to indicate it has processed the message like in the below picture.
You will also see that the back end Claims Processing Application service was executed and the console window will show you that the claim was processed.
As I mentioned earlier in the article this pattern is one Fabrikam were looking at strategically and there are a few ways which they may extend the use of this pattern in the future. Some of the ones they could consider include:
In this article we have shown you how combining Windows Azure Service Bus and AppFx.ServiceBus made it easy for Fabrikam to expose some of their existing on premise web services to the cloud using a hybrid integration pattern. AppFx.ServiceBus provided a solid framework which Fabrikam could install, configure and then extend through the message handler patterns to implement an on premise bridge connecting the web services to the service bus in the cloud.
By using some of the development techniques encouraged by the AppFx.ServiceBus approach Fabrikam was able to ensure that their on-premise implementation was interoperable with client applications who could use a whole set of different technologies ranging from simple REST clients through to those using the Windows Azure Service Bus SDK or any of the SDK’s for other languages (or even using the AppFx.ServiceBus client side framework).
While this example covered a single messaging pattern and customer scenario there are many other patterns and features available in the AppFx.ServiceBus framework which could help you to accelerate your real-world projects with Windows Azure Service Bus brokered messaging. To find out more about these features please refer to the projects Codeplex site on the following link:
Michael Stephenson is a UK based Integration Specialist who has worked with many consultancies and customers architecting and developing integration solutions based on Microsoft technologies including BizTalk Server and Windows Azure. Michael is heavily involved in community activities such as the UK Connected Systems User Group and speaking at events in Europe. Michael has been a Microsoft MVP for the past 5 years.
Thanks to the following people for reviewing this article
Kent Weare is a Senior Enterprise Architect for a large Power Generation company in Canada. For the past 9 years he has been very involved in the integration of disparate systems using technologies such as BizTalk Server and Windows Azure Service Bus. Kent remains active within the integration community by blogging, writing technical books and speaking at events in both Canada and Europe. For these efforts, Microsoft has recognized him a Microsoft MVP for the past 6 years.
Steef-Jan Wiggers Microsoft Integration MVP is an IT architect/Consultant at motion10 with over 14 years of experience as a technical lead developer and application architect, specializing in custom applications, enterprise application integration (BizTalk), Web services and Windows Azure. He has experience in architecting, designing, developing, and supporting sophisticated and innovative software using many different Microsoft technologies and products.
Steef-Jan is very active in the BizTalk community as a blogger, Wiki author/editor, forums moderator, and public speaker. He is a published author (BizTalk Server 2010 Cookbook) and Microsoft Integration MVP. He has received the Microsoft Most Valuable Professional (MVP) award for his contributions to the world-wide BizTalk Server community in 2010. He was re-awarded his MVP status in July 2011 and 2012.