Essential Windows Azure (Microsoft Cloud) Knowledge : Part 1: Web roles, Worker Roles

Globally distributed data centers
The Windows Azure Platform is big, very big. It is comprehensive and perhaps you could argue it is complex, as all large systems invariably become. I want to use a series of posts to remind me what I “must” bring up during my Azure one-day, in person workshops.

In all seriousness, this post is directed to developers, architects and technical decision makers. Maybe in a future post I'll lower the technical barriers and explain things even more simply. I would argue this post covers the spectrum - from basic to fairly sophisticated.

I assume that you understand the Windows Azure Platform is a cloud-based computing technology from Microsoft, built upon a highly evolved programming environment and hosted in mega-data centers throughout the world.

This post is very visual. I want to convey as much as I can with as many diagrams as possible. You obviously can't pull up PowerPoint during Thanksgiving, but if someone asks you for an explanation, having a visual in your head really helps.

I've been doing lectures about cloud computing for a few years now. Along the way I have constructed 100s of slides that explain the Microsoft cloud, the Windows Azure platform. I want to present some of them to you here. It should help you understanding the massive capabilities of the platform as well as explain how some things work.

The basics - hosting web sites and web services

The point of the diagram below is to think about hosting your web-based content and services. It also addresses running background processes.

You can think of Compute as being a container for web roles and worker roles.
Compute enables you to run application code in the cloud and allows you to quickly scale your applications. Each Compute instance is a virtual machine that isolates you from other customers
- Compute runs a Virtual Machine (VM) role
- Compute automatically includes network load balancing and failover to provide continuous availability.
  - Windows Azure provides a 99.95% monthly SLA for Compute services
Web roles are simply front-end web applications and content hosted inside of IIS in a Microsoft data center.
- What is IIS?
  - Internet Information Services (IIS) is a web server application and set of feature extension modules that support HTTP, HTTPS, FTP, FTPS, SMTP and NNTP.
  - IIS can host ASP.NET, PHP, HTML5, and Node.js.
    
    Note that you are not limited to ASP.NET, or MVC. You can also use PHP, Node.js, and HTML5.
- You can quickly and easily deploy web applications to Web Roles and then scale your Compute capabilities up or down to meet demand.
Web roles can host WCF Services.
- The Windows Communication Foundation (or WCF), is an application programming interface (API) in the .NET Framework for building connected, service-oriented applications.
- WCF unifies most distributed systems technologies that developers have successfully used to build distributed applications on the Windows platform over the past decade.
- WCF supports sending messages using not only HTTP, but also TCP and other network protocols.
- WCF has built-in support for the latest Web services standards (SOAP 1.2 and WS-*) and the ability to easily support new ones.
- WCF supports security, transactions and reliability.
- WCF supports sending messages using formats other than SOAP, such as Representational State Transfer (REST).

ASP.NET Web Forms versus MVC

ASP.NET Web Forms has been around for a while and is a mature technology that runs small and large scale websites alike. MVC is the newer technology that promises many advantages.

Web Forms is built around the Windows Form construction model
- Web Forms have a declarative syntax with an event driven model.
- Web Forms allow visual designers can use a drag and drop, WYSIWYG, interface.
- Web Forms make it possible for you drop controls onto the ASP.NET page and then wire up the events
  - Microsoft basically extended the Visual Basic programming model to the Web
Web Form disadvantages include:
- Display logic coupled with code, through code-behind files
- Difficult unit testing because of coupling
- ViewState and PostBack model
- State management of controls leads to very large and often unnecessary page sizes

MVC
The ASP.NET MVC Framework is a web application framework that implements the model-view-controller (MVC) pattern.

At the expense of drag and drop, MVC gives you a very granular control over the output of the HTML that is generated.
MVC supports a ‘closer to the metal’ experience to the developers that program with it, by providing full control and testability over the output that is returned to the browser
Clear separation of concerns
- Results in strong support for unit testing
MVC easily integrates with JavaScript frameworks like jQuery or Yahoo UI frameworks
MVC allows you to map URLs logically and dynamically, depending on your use
MVC provides RESTful interfaces are used by default (this helps out with SEO)

Worker roles are part of compute but are not hosted in IIS.

Applications hosted within Worker roles can run asynchronous, long-running or perpetual tasks independent of user interaction or input.

Worker roles let you host any type of application, including Apache Tomcat and Java Virtual Machines (JVM).
Applications are commonly composed of both Web and Worker roles.
- A common implementation in Windows Azure takes input from a Web role, sends those requests through a Queue to a Worker role, then processes the requests and stores the output.

Download for Azure SDK

Sample Implementation
Imagine that you are Microsoft and that you want to offer video encoding services to customers. That means that someone like me can take my home videos, upload them to the Microsoft Cloud, specifically Windows Azure Media Services. Next, I can use a management API that Microsoft provides, and programmatically encode my videos so they can run well on other devices. This simply means I want to take my vacation.mpg video and convert it to a native QuickTime format, like .mov files. Many of you blog readers know that there are many video formats, such as WMV, AVI, MP4, MOV - just to name a few.

The diagram below illustrates how such an offering might exist. Let's walk through it.

A sample scenario

Imagine the user wants to upload their video so they can get it encoded in multiple formats, so the video will look good across a spectrum of devices.

Let's walk through a scenario.

Step 1	The user would visit http://azureinjestmedia.com.
Step 2	The user would upload their videos (vacation1.mov, and vacation2.mov). The user wants to get vacation1.wmv and vacation2.wmv.
Step 3	The user uses the portal to indicate how they wish to process the video. The user indicates they want wmv files. The user kicks the encoding process off and waits a few minutes.
Step 4	The user now wants to get the wmv files. So the user simply downloads them, courtesy of the portal interface.

The portal that user's interact with is a web role

Note that the web role is the portal. It interacts with the user who wants to user Microsoft's video services.
- Microsoft could have built the portal using ASP.NET Web Forms, MVC, PHP, HTML5, Node.js. Microsoft probably would choose MVC because of it's testability, and fine-grained control over the rendered HTML to the user.
The portal runs inside of IIS and inside a VM that is running Windows Server 2008 R2.
- You may have multiple instances running that Azure will automatically load balance requests for.
The web role can interact with the worker role using queues.
The web role takes the user's video and stores inside of Azure Storage, it sends the worker role some instructions about where the .mov files are and what the desired
- It does so using the Windows Azure Queues.

Background Process - Worker Role

Like a Windows Service
- The Worker Role is similar to a windows service.
Long Running
- It starts up and is running all the time.
No timer
- Instead of a timer, it uses a simple while(true) loop and a sleep statement.
Background processing
- This is great for background processing.
Data Required
- Worker roles usually need some data to work with.
The Queue is the data bridge
- You can communicate between a worker and a web role via the use of a queue.
Worker role simple reads from queue
- The worker role doesn’t care how stuff got into the queue
First in First out
- The worker role processes items in the queue using FIFO.
The user interacts with the web role, not the worker role
- Generally speaking it is the web role that is user driven and causes data to go into the queue.
The worker role interacts with storage.
The worker role knows there is two types of storage containers
- There are 3 main categories of storage - 2 Azure Blob Containers and one Azure Table
  - BlobContainer = Movies to Encode
    - Movies that still need to be processed and encoded.
  - BlobContainer = Encoded movies
    - The finished product, multiple movie formats, one for each device type
  - Azure Tables
    - Stores the meta data about the Azure blobs.
    - It records the location of the Azure blobs so the worker role knows where to read and write video content
      - It knows because of the two types of Azure blob containers

Notes for the diagram above

Here is some details about he diagram above.

The web role interacts with the user
The user may download or upload files.
The user may upload a video because they want it encoded
- The web role would be the portal where the user does that
- But the user may also wish to download the finished product (the encoded video performed by the worker role)
  - The portal must allow downloads from BlobContainer = EncodedMovies
The web role could read/write Azure Tables. But we may choose to let the worker role do that.
- The web role writes Azure blob locations as text strings to queues and forgets about them.

Notes for the diagram above.

Notice many worker and web roles in many racks.
There are several instances of Fabric Controller instances running in various racks in data centers.

One is elected to act as the primary controller.
- If it fails, another picks up the slack.
- There fabric controllers are redundant.
- If you start a service on Azure, the FC can fall over entirely and your service is not shut down.
The Fabric Controller uses the Preboot eXecution Environment
- PXE, also known as Pre-Execution Environment; sometimes pronounced "pixie"
  - PXE is an environment to boot computers using a network interface independently of data storage devices (like hard disks) or installed operating systems
  - PXE leverages the Internet Protocol (IP), User Datagram Protocol (UDP), Dynamic Host Configuration Protocol (DHCP) and Trivial File Transfer Protocol (TFTP) to support boostrapping a computer
The Fabric Controller runs Sysprep, the system is rebooted as a unique machine

Understanding the Fabric Controller

Modified Windows Server 2008	The Fabric Controller is a modified Windows Server 2008 OS, as are the host OS and the standard pre-configured Web and Worker Role instances.
The search for free nodes	The fabric controller looks for available nodes and looks for (in the standard case) two nodes that do not share a Fault Domain. This provides yet another degree of fault-tolerance.
A VM and various Virtual Hard Drives	A virtual machine is created and multiple hard drives are mounted, one for your role type, one for temp files, and others.
Hyper-V in the house	A complex series of steps allow for upgrades, leveraging a differencing VHD and a Microsoft-customized Hyper-V instance, built specifically for the data center hardware.