In this series I would like to introduce you to MSMQ history, key concepts, dwell into painpoints, suggest workarounds, and give an insigth into to future.
What is MSMQ?
MSMQ enables Asychronous communicaiton between two different applications. Application A(Sender) sends a message to a queue hosted in MSMQ, Application B(receiver) receives and processes the message from the queue MSMQ is usually used in the following scenarios a. Where there is an impedence mismatch between the sending and receiving applicaiton b. Load-levelling scenarios c. Sending and Receiving applications need to be running at different times d. Sending and Receiving applications need to be running across network or even across the internet.
MSMQ enables you to send messages over a variety of protocols such as TCP, IPX (if you care anymore :-) ) , HTTP, PGM (Pragmatic Multicasting) MSMQ provides out of the box security and routing of messages.
MSMQ was designed and developed by a Microsoft developement team in Haifa, Israel.
MSMQ version 1.0 was shipped with the NT Option pack. In this version MSMQ was integrated with SQL Server. The programming model was easy but the Installation and management was cumbersome.
MSMQ version 2.0 shipped as an optional windows component with Windows 2000. Active Directory was born, MSMQ was quick to grab the opportunity to replace SQl-server dependency with AD.
MSMQ vesrion 3.0 shipped as an optional windows component with WinXP and Windows 2003. Major new features like HTTP protocol support, Multicasting support, Distribution lists, etc were introduced. Security and reliability of the product was greatly enhanced in version 3.0
MSMQ can be installed in two modes
You can program MSMQ using 4 APIs
Queues are nothing but First-In-First-Out(FIFO) datastructures.Queues can be created using the MSMQ management tool inside Computer management console MMC or programatically. Queues can be created and used dynamically. However you need to be aware of issues that might be caued by replication delays in the case of public queues. Private queues does not have such issues There is no physical limit on the number of queues that you can create on a given computer
Queues hold messages. Messages contain several properties and body. Some properties such as priority, correlationID, messageID etc are used to specify the contract of the message. Some properties like Label, body etc can contain the payload or data. The total size of the message including properties and body is limited to 4MB. The body of the message can contain pretty much anyting. You can ship COM objects that supports IPersistsStream interface, .NET Data types that can be serialized through binary or XML serializer. You need to be aware of the fact that message occupy both virtual memory and page pool. A fixed number of page pool bytes are consumed per message. If you have a lot of messages (like 1 million) on the disk you will start putting pressure on the page pool. In normal situaltions where sender and receiver applications are working smoothly you don't end up with a lot of messages in the queue. However in scenarios where there is a huge impedence mismatch between the sender and receiver or the receiver applicatio is down you might end up queuing a lot of messages.
MSMQ supports specifying storage quotas at the machine level and queue level. This ensures that you get warnings in the form of Negative acknowledgements when you hit the quota. This can help you prevent situations you have too many messages in the queue which will put pressure on system memory and page pool.
MSMQ support two types of messages
You can send messages to both local and remote queues. You can send messages using several protocols such as TCP, IPX, HTTP, PGM (Multicasting). Receive is always done over RPC. You can receive messages from both local and remote queues.
Messages can be send and received as a part of a transaction.
MSMQ supports two types of transactions
You can send and receive transactional messages to both local and remote queues.
High Availability and Scalability
MSMQ provides high availability by the use of Microsoft Clustering service. MSMQ supports both Active-Active and Active-Passive clusering. MSMQ provides scalability in two scenarios.
In this scenario you can have a farm of MSMQ servers behind a hardware or software load balancer. Clients can send messages to the MSMQ servers through the load balancer.
If you are using the TCP protocol to send messages you are limited to only non-transactional sends.
If you are using the HTTP protocol you can send messages both transactionaly and non-transactionlly.
In this scenario the queues are hosted on a central server (clustered for high availability). Client applications send messages to the queues. The reveiver applications are hosted on a farm of MSMQ servers. They receive and process messages from the central queues. Messages in the central queues are levelled across the server farm.
MSMQ supports authentication and encryption of messages. When you send a message you set certain properties of the message to specify whether you want authentication and encryption or not. You can also select the has algorithm and encryption algorithm that msmq uses to sign and encrypt the message.
You can set the "requires authentication" property on the queue thereby resticting the applications to be able to send authenticated messages to the queue.
You can also set the encryption property of the queue to accept only encrypted messages.
Note: Messages are encrypted only on the wire, once they reach the destination queue messages are decrypted and stored as plain text.