Muhammed Ismail's Blog

Why?

Generally when using MSMQ within your organization or within your network it's quite common use the default native transport (TCP over port 1801).  When going across various network or between multiple organizations, one may not always have control over routers, firewalls, or other devices which could block 1801, and in many cases it's just not a good idea to open additional ports.

MSMQ v3.0 (XP and 2003) and later have the ability to send messages over HTTP, in short the MSMQ message is sent over port 80 (or 443 for SSL).  The message is picked up on the receiving end (via IIS) and directed to the appropriate queue.

 What Needs to be Installed?

In order to only send messages, no additional components for MSMQ need to be installed except for "Common".  To receive MSMQ messages over HTTP though, the MSMQ/HTTP option must be selected.  This installs and configures the required MSMQ components in IIS, and should allow messages to be received.

Keep in mind that when sending transactional messages, that you are both sending and receiving ... the original message gets sent, and the acknowledgement is then returned.  So in the case of transactional messages, you will most likely want the MSMQ/HTTP component installed on both the sender and receiver.

How to Send Messages?

Sending messages over HTTP is simple, generally no coding changes are required except to specify a format name using HTTP/HTTPS.  An format name for an MSMQ over HTTP message may look something like DIRECT=HTTP://www.mydestination.com/msmq/myQueueName.

There is also the ability to use virtual queues, which then map to a real queue, so the real queue name doesn't have to be exposed.   This is done through the config files located in the %windir%\system32\msmq\Mapping directory.  Information on modifying these files is contained in the Word documents linked through the Resources section below.

What about Acknowledgements?

Regular MSMQ acknowledgements are returned to the IP of the sender.  In the case of HTTP messaging this would be difficult as the message is often sent from inside a private network using an internal IP.   MSMQ over HTTP uses Stream Receipts to address this issue, in short a URL can be specified where the acknowledgements messages should be sent.  So long as the receiver can connect to the specified Stream Receipt URL, we should be able to return the acknowledgement messages successfully (also over HTTP).

The Stream Receipt configuration also allows transactional messages to be sent across an NLB or similar device, which is NOT the case with regular MSMQ.  In these scenarios, using MSMQ/HTTP may provide a significant advantage if transactional messages are required.

Security?

Security is a concern, and even if the data within the message body is encrypted, the queue name and other information relating to the message are still viewable in the HTTP packets (you should be able to see this with a simple network capture).  To address this, HTTPS can be used, or for additional security, client certificates.

It's easy to misplace the certificates when using the additional security featured.  The Trusted CA certificate needs to be placed in the Certificate store for the computer.  If using client certificates, then the certificates need to be placed in the store for the MSMQ service.

Downsides?

In a single word, performance.  MSMQ over HTTP is slower than regular MSMQ simply because of the overhead involved.  The message must first get sent to IIS then gets passed to MSMQ, instead of being sent natively.  In many cases where volume is not terribly high or high-performance is not a top requirement, MSMQ over HTTP(S) provides an excellent solution to transporting application data between multiple networks. 

Additional Resources:

The following documents are excellent resources on understanding how to get started with MSMQ/HTTP(S):

MSMQ HTTP Deployment Scenarios for Windows Server 2003 and Windows XP Professional

Configuring HTTPS Messaging for MSMQ 3.0

What is MSMQT?

The simplest way to explain MSMQT, would be to say that it's an MSMQ emulator for BizTalk.  MSMQT was introduced with BizTalk Server 2004, although there is also a regular MSMQ Adapter available, enabling BizTalk to use regular "full-blown" MSMQ.

Why MSMQT?

As you probably know, regular MSMQ keeps data in its storage directory, using it's own storage format.  MSMQT stores data directly in the BizTalk MessageBox, eliminating the need for an MSMQ store.  Here are a few of the advantages from using MSMQT:

• No hard storage limit on Windows 2000 as the data is stored directly in the BizTalk MessageBox
• Data is stored centrally in BTS, so you don't need to worry about backing/maintaining two separate data stores.
• Multiple instances of the MSMQT Service share the same database (or data store in this case)

However, there are some disadvantages to MSMQT as well, these include:

• Poor performance, MSMQT is single threaded. In high volume scenarios, this could cause messages to be processed quite slowly.
• Not all MSMQ features are implemented in MSMQT (remember that MSMQT emulates MSMQ, so it's not identical).  In particular most of the MSMQ v3.0 features (such as MSMQ over HTTP) are not available
• No published APIs for programtic use
• MSMQT is only used by BizTalk.  MSMQ on a BizTalk computer could also be shared with other applications.

Fault Tolerance

On BizTalk 2004, the recommended way to achieve fault-tolerace was to arrange BizTalk hosts in an NLB type of arrangement.  Those of you familiar with MSMQ will know that sending transactional MSMQ messages across an NLB is typically not supported.  MSMQT provided a way to use BizTalk along with transactional MSMQ messages and still have fault tolerance.  Since all of the data is kept in the MessageBox, acks can be correctly issued so long as all of the receiving endpoints are part of the same BizTalk group.

Regular MSMQ typically achieves fault-tolerance through a cluster, and this configuration wasn't introduced in BizTalk until the 2006 version was released.  If you require transactional messages along with fault-tolerance on BizTalk 2004, then MSMQT might be the right choice for you.  Otherwise, Clustering BizTalk 2006 along with clustering MSMQ will most likely provide better performance.

Issuing Acks

Since MSMQT only emulates MSMQ, it's worth noting that the acknowledgement messages are not issued at exactly the same times as they're issued with MSMQ.  MSMQ will issue an acknowledgement message (ack) for transactional messaging when the messages arrives at the destination queue on the receiver.  MSMQT behaves slightly differently, and only issues the ack after the message has been picked up by a subscriber (so if you don't have a valid subscriber active to pick up the message, no ack gets issued).

Running both MSMQ and MSMQT?

I should probalby point out that since MSMQT pretty much copies everything that MSMQ does, it also listens on the same ports.  It's easy to install one or the other, but if you'd like to have both running side-by-side, you can do that too.  Take a look at the following article for more information:

895867 How to install the BizTalk Adapter for MSMQ in BizTalk Server 2004 when MSMQT is installed or when the MSMQC Adapter sample is installed
http://support.microsoft.com/default.aspx?scid=kb;EN-US;895867

Summary

If you require fault-tolerance and transactional messaging, and have a compelling reason to stay on BizTalk 2004, then MSMQT may be the right choice.  Otherwise in most other cases, clustering BizTalk 2006 along with clustering MSMQ would provide a better overall solution. 

Why?

Why bother clustering MSMQ at all?  Fault Tolerance.  As you probably know, Microsoft Message Queuing (MSMQ) is a queuing system which is typically used as middle-ware between applications.  MSMQ can store almost any type of data within MSMQ messages, and route them to another MSMQ queue manager where they can be retrieved by the receiving application.  MSMQ already provides an extra layer of reliability in the case where something on the back-end may not be available, the front-end can continue to receive requests and store them as MSMQ messages.  So what happens if the computer running MSMQ goes down, and the entire machine becomes unavailable?  That's where a cluster helps us, as it adds provides fault-tolerance for those types of scenarios.  Node #1 fails, and Node #2 automatically takes it place.  An extremely simple and relatively inexpensive (given today's hardware costs) solution. 

How to Cluster MSMQ?

All you need to cluster MSMQ in addition to the hardware and basic cluster setup, is a network name resource, and a disk-resource.  Step-by-step instructions can be found in the white-paper titled Deploying MSMQ 3.0 in a Server Cluster.

How Many Instances?  How does my Application Distinguish Between Local and Clustered MSMQ?

After you've successfully clustered MSMQ, it's important to understand the difference between local and clustered MSMQ.  By default the local instance of the Message Queuing Service is set to Manual after MSMQ has been clustered, but it's still there, and can easily be started at any time.  If you try running an MSMQ Application from the local command prompt, it's either going to use the local MSMQ Service, or if that service isn't started, it's probably going to generate an error.

For most people in a typical two-node cluster, there are three instances of MSMQ.  Two local instances (one on each node) and the clustered instance which runs on the Active Node.  The clustered instance an be failed over between nodes from Active to Passive and vice-versa.  There aren't always three instances though, a better way of stating that you can have (# of cluster nodes) + (# of clustered instances of MSMQ) nodes.

So, if there are so many available instances of MSMQ, how do you get your application to use a particular instance?  The answer depends on the the computer name that the application sees as its native computer name.  If you launch a regular command prompt and type in " set COMPUTERNAME" it should display the current computer name.  If this is the name of the local node, then the application will try to use the local instance of MSMQ.  In order to get something to run within the context of the cluster, it needs to be using the same Network Name resource as clustered MSMQ.  This can be accomplished in one of two ways:

-Cluster your application in cluster administrator, ensuring it has a dependency on the same Network Name Resource being used by clustered MSMQ.  Then select the "Use Network Name as Computer Name" option

-Create a clustered command prompt.  Basically create a clustered resource for cmd.exe, and ensure that you follow the rules above and create a dependency on the same Network Name Resource being used by MSMQ, and ensure that the "Use Network Name as Computer Name" option is selected.  When you bring the resource online, you should see the command prompt.  From this clustered command prompt, try the same "set COMPUTERNAME" test I mentioned above.  This time you should see the virtual cluster name being displayed instead of the local node name.  If you run your application from this clustered command prompt, it should see clustered MSMQ as it's native instance.  You can also run Computer Management (compmgmt.msc) from this prompt to administer clustered MSMQ.  Be aware that the prompt will not be displayed if you aren't either physically at the node, or if you're not logged in using the /console switch through mstsc.

Administering Clustered MSMQ

In order to properly administer clustered MSMQ, it's recommended to either run Computer Management from a clustered command prompt (see above), or otherwise you can use the MMCV utility.  MMVC will allow you to launch Computer Management using a particular computer name, so you can easily specify the name of your virtual clustered server.  The syntax to use is "mmcv.exe -s <myVirtualServerName>", it's an extremely useful tool, and I'd strongly suggest including a copy of it on all of your MSMQ cluster nodes.  You can find out more about MMCV and find a link with a download in the following article: How to use the Mmcv.exe utility to manage clustered Message Queuing resources

Helpful Tips

Most clusters have multiple IPs, use the BindInterfaceIP registry key with clustered MSMQ to ensure that MSMQ binds to the correct IP: A cluster node with two network cards does not receive messages

Ensure that the local instances of MSMQ are set to run using the Local System account, even if they're stopped and set to manual.  Having them set to use a particular user in an environment running clustered MSMQ will most likely cause issues.

If you're going to be using Public Queues, or will have clustered MSMQ installed with the Active Directory Integration option, then ensure that Kerberos is enabled on your Network Name Resource in Cluster Administrator.  This will create an object for the virtual machine in Active Directory which will hold MSMQ's public information.

Keep in mind that clustered MSMQ inherits the installation settings from MSMQ on the nodes, so only select the features that you need in order to minimize overhead.

Failover over the cluster is equivalent to restarting the MSMQ service.  In short, express messages will be lost, so if you need to preserve your data, then ensure that the message are either set to "recoverable" or use transactional queues.

I hope that you found this brief explanation of clustering MSMQ to be useful, and please don't hesitate to post any comments, questions, or suggestions.