Authors: Jonathan Doyon (Genetec), Konstantin Dotchkoff (Microsoft)
Running an app in a cloud environment is all about scalability and using the economics of the Cloud. Scaling out (or scaling back) provides a great flexibility to adapt to workload changes. Windows Azure provides load-balancing capabilities that can transparently distribute workload across multiple instances of an app. However, what should you do if your application is not completely stateless and requires session affinity (a.k.a. "sticky" sessions). Well, there are several solutions to this problem. In this blog post we describe a design pattern used to combine the requirements for load-balancing of a client workload with the need for sticky connections.
This pattern was implemented in the Stratocast solution by Genetec based on the specific application requirements. However, the pattern itself is a common one and applicable to a broader range of scenarios that call for sticky sessions.
Stratocast is a security solution that allows the user to view live and recorded video that is safely stored in Windows Azure from your laptop, tablet or smartphone. The solution allows the user to monitor multiple live or recorded video streams. Once a camera is connected to the cloud service it will keep streaming video. If for some reason a camera gets disconnected then it should reconnect to the same server component.
Even if we don't think about cameras and video streaming, you will find the same type of requirements in a lot of different scenarios. For the remaining part of the blog we will use "client" to refer to a client application or device (such as a camera) that accesses a server-side component running on multiple compute instances in Windows Azure.
So let's drill down into the details. Initially, a client will need to establish a connection to a server component running in Azure (in the case of Stratocast to the Azure-based video recorder). It will initiate an HTTPS call to the public endpoint of the Azure Cloud service. The endpoint is load balanced by the Azure load balancer and the call will reach one of the running instances. The server component that receives the request will perform a custom load balancing based on the application specific business logic. In the case of Stratocast, for example, there are three important requirements:
While the first requirement is about load distribution, the second and third ones are related to the stateful nature of the communication.
After performing the custom routing logic, the server instance will redirect (using HTTP redirect) the client to the selected server instance by using its direct port endpoint. Once redirected the client will establish a connection that is stateful in nature (hence, all the requirements around session affinity). In the case of Stratocast, the connection will stay open for video streaming.
Let us expand a bit more on the details behind this solution. The Azure compute role hosting the server, (in the case of Stratocast- responsible to record video) has two public endpoints: an input endpoint and an instance input endpoint.
The input endpoint is a regular public endpoint automatically load balanced by Azure. Cameras always initiate communication on this port and then the system will issue an HTTP redirect to the appropriate instance input endpoint as determined by the custom load balancing logic explained earlier.
The instance input endpoint (a.k.a. direct port endpoint) is used to communicate with load balanced role instances. It requires a port range in the configuration file instead of just one public port (see Define Direct Port Endpoints for a Role for more details). Azure will automatically assign one port in the range to each instance. This endpoint can be used for direct communication with the instance - Azure does not load balance requests coming on the instance input port. When a role instance starts up, some code was added in the role to read the public port of the instance using the Azure API and to record it in an Azure storage table. This information is used later by the custom load balancing algorithm to redirecting calls to a specific server instance. The solution also monitors the health of each instance and keeps the list of server instances in the table storage up-to-date.
The following graphic shows the conceptual flow of the pattern:
Figure 1 – Custom load balancing pattern overview
For sure, this is not the only way to solve this challenge, but after reviewing and trying out some different approaches this pattern was found to be very useful for the described requirements. Using IIS Application Request Routing (ARR) is one alternative that can be considered. In comparison, ARR would provide the following:
For a detailed list of ARR features you can take a look at Using the Application Request Routing Module.
In the case of Stratocast, due to the very dynamic nature of the routing information, an ARR-based solution seemed to be more complex with respect to the deployment automation, configuration and management.
In summary, the described pattern includes the following elements:
In this blog post, using Stratocast as an example we described a pattern for implementing custom load-balancing in Windows Azure. We also discussed which requirements led to this design and what should be considered when evaluating potential alternatives such as IIS ARR.