Tom Hollander's blog

Windows Azure in Australia: Facts and Rumours

Tom Hollander — Mon, 20 May 2013 20:58:42 GMT

Today, Microsoft has made the long-awaited announcement that Windows Azure is coming to Australia! Of course it’s been possible for Aussies to deploy and use applications in Windows Azure for some time, but this has always required deploying to overseas data centres. This announcement is about the upcoming availability of two new Windows Azure sub-regions in New South Wales and Victoria, complete with data geo-replication between the sub-regions. This is big news for many Australian customers, as it will mean lower latency when accessing Windows Azure infrastructure, as well as increased flexibility for customers with specific compliance and data sovereignty concerns.

While we’re very excited to be able to finally confirm this news, there are a number of rumours surrounding this announcement which we are not able to confirm or deny at this time:

We have no comment on whether these new sub-regions will be deployed in the Sydney Opera House and on a Melbourne tram
We have no comment on whether we will be employing a “Star Wars” defence mechanism to protect the infrastructure against attacks from crocodiles, funnel web spiders and drop bears
We have no comment on whether the fans in the servers will spin in the opposite direction to those in our northern hemisphere servers
We have no comment on whether the new servers will play a “Crikey!” sound when they boot
We have no comment about whether the backup generators will be powered by beer (however, if this were true, we can confirm that it would not be Fosters).

I’m sorry I can’t put any of these rumours to bed just now, but in the meantime I hope the news that Windows Azure is coming to town is exciting enough to keep you going for a while. If you’re already using Windows Azure, we look forward to seeing your apps deployed to the Aussie regions soon! And if you’re not… well what are you waiting for?

Keep your Windows Azure applications running with custom health checks

Tom Hollander — Fri, 17 May 2013 13:37:30 GMT

Summary: Even though Windows Azure does a great job of keeping your VMs running, only you know exactly what it means for your own apps to be healthy. This post and sample code shows a pattern for implementing custom health checks that can report the health of your application, recover from failures if possible, and bring an instance offline if not.

The big selling point of Platform as a Service (PaaS) is that managing things like the hardware, network and operating system is someone else’s problem. So if a Windows Azure server spontaneously combusts in the middle of the night, the fabric will automatically detect this and move any affected VMs onto servers not currently engulfed by flames. However keeping your applications healthy is still your responsibility. While a simple web application is almost certain to keep running if the underlying OS and hardware are healthy, this isn’t necessarily true for a complex enterprise application. A complex app could depend on various Windows Services running, files and registry keys being in a certain state, and connectivity to local and remote services being in place. If any of these things break it could be disastrous for your application, yet as far as the Windows Azure fabric is concerned, everything is running perfectly.

Fortunately, Windows Azure provides you with a couple of hooks that let you implement health checks into your deployments. The first of these is Load Balancer Probes. These can be specified in your ServiceDefinition.csdef file, and instruct the Windows Azure load balancer to regularly ping URLs of your choice to check that your service is still alive. If the response is anything but a 200 OK, your VM is deemed unhealthy and it’s removed from the load balancer. This mechanism is useful as it’s completely declarative, works on both PaaS (cloud services) and IaaS (virtual machines), and can be hooked into existing production or health check pages built into a web app. On the downside, when a Load Balancer Probe detects your instance is unhealthy it doesn’t apparently change the VM state or otherwise indicate to operations staff that something is wrong. Also, web pages are not always the best launching point for checking health of things like Windows Services.

The other option is to subscribe to the RoleEnvironment.StatusCheck event, which gets fired every time the Windows Azure Guest Agent needs to report its status to the fabric. In the event handler you can implement any logic you wish, and if you deem the instance unhealthy you can request the instance’s status be changed to Busy until the next status check, resulting in it being removed from the load balancer. Since this approach leverages the RoleEnvironment it’s PaaS only, but I found it more useful than Load Balancer Probes so it’s the approach I’ll use in this post and accompanying sample.

First, the mandatory disclaimer. I’ve made some source code available that shows the solution described in the post, but this is sample, not production quality code. It hasn’t been extensively tested and may well contain bugs. For enterprise use you should look at incorporating dependency injection and possibly a plug-in framework like MEF to provide more flexibility and testability. And the two health check classes I’ve supplied are designed as examples of what kind of thing is possible, rather than being complete implementations that will solve real problems. However I’m confident the pattern is sound, so feel free to build upon this (with appropriate testing) in your own solutions.

With that out of the way, let’s talk about the solution. My code contains a project called AzureHealthCheck which contains both the framework classes and a couple of sample health checks. This can be wired up into any web or worker roles in your PaaS cloud services. You do this by creating an instance of HealthCheckController, starting it when the role starts and stopping it when the role stops:

public class WebRole : RoleEntryPoint
{
    private HealthCheckController _healthCheckController = new HealthCheckController();

    public override bool OnStart()
    {
        _healthCheckController.Start();
        return base.OnStart();
    }

    public override void OnStop()
    {
        _healthCheckController.Stop();
        base.OnStop();
    }       
}

I won’t show all the code for HealthCheckController here, but here’s what it does:

Retrieves the list of health checks and their settings from the ServiceConfiguration setting called “HealthCheckers”, instantiates the classes and stores them in a list
Subscribes to the RoleEnvironment.Changed and RoleEnvironment.StatusCheck events
In the RoleEnvironment_Changed event handler, reloads the configuration and refreshes the list of health checkers
In the RoleEnvironment_StatusCheck event handler:

Iterates through all configured health checks

Calls the CheckHealth() method
If the instance is deemed unhealthy, and the health checker wasn’t configured to keep the instance running anyway, calls SetBusy() to remove the instance from the load balancer
For any unhealthy or error conditions, logs the results to Windows Azure Diagnostics (which could be read by System Center Operations Manager or similar tools to perform further actions such as rebooting or reimaging the instance)

The health checkers are custom classes that implement the IHealthChecker interface:

public interface IHealthChecker
{
    void Initialize(string initData);
    HealthCheckResult CheckHealth();
}

public class HealthCheckResult
{
    public bool IsHealthy { get; set; }
    public bool RemediationAttempted { get; set; }
    public string StatusMessage { get; set; }
}

As you can see, this is a pretty simple contract. Each health checker can be initialised with a custom string, it can check health, and it can report the result: whether the check showed the instance as healthy, whether remediation was performed, and any status messages that should be logged.

In my code I have two sample health checkers. The first, PingUrlHealthChecker, is a lot like a Load Balancer Probe in that it calls a URL in your main web site and reports unhealthy if the response is anything but 200 OK. This health checker does not attempt any remediation:

public class PingUrlHealthChecker : IHealthChecker
{
    private Uri _pingUrl;

    private string GetWebUrlBase()
    {
        var serverManager = new ServerManager();
        var siteName = RoleEnvironment.CurrentRoleInstance.Id + "_Web"; // TODO: allow user to specify which site to use
        var site = serverManager.Sites[siteName];
        var binding = site.Bindings[0]; // TODO: allow user to specify which binding to use
        return  String.Format("{0}://{1}:{2}/", binding.Protocol, binding.EndPoint.Address, binding.EndPoint.Port);
    }

    public void Initialize(string initData)
    {
        // initData is a relative URL to ping, e.g. "/foo/bar.aspx"
        var urlBase = new Uri(GetWebUrlBase());
        _pingUrl =  new Uri(urlBase, initData);
    }

    public HealthCheckResult CheckHealth()
    {
        var result = new HealthCheckResult
        {
            IsHealthy = true,
        };

        HttpWebResponse response; 
        try
        {
            var client = WebRequest.Create(_pingUrl) as HttpWebRequest;
            response = client.GetResponse() as HttpWebResponse;
        }
        catch (WebException wex)
        {
            response = (HttpWebResponse) wex.Response;
        }
        if (response.StatusCode != HttpStatusCode.OK)
        {
            result.IsHealthy = false;
            result.StatusMessage += String.Format("HTTP Response '{0} {1}' returned from URL {2}.", (int)response.StatusCode, response.StatusDescription, _pingUrl.ToString());
        }
        
        return result;
    }
}

The second sample health checker, WindowsServiceHealthChecker does just what it says on the box: it checks if a specified Windows Service is running. However this one supports remediation, in that it will attempt to restart the service if it isn’t running. If it’s successful, the instance will still be reported as healthy (although a warning is still logged). If not, the instance will be reported as unhealthy.

public class WindowsServiceHealthChecker : IHealthChecker
{
    private string _serviceName;

    public void Initialize(string initData)
    {
        _serviceName = initData;
    }

    public HealthCheckResult CheckHealth()
    {
        var result = new HealthCheckResult();
        var sc = new ServiceController(_serviceName);

        if (sc.Status == ServiceControllerStatus.Stopped)
        {
            result.RemediationAttempted = true;
            result.StatusMessage = String.Format("Service '{0}' was restarted.", _serviceName);
            try
            {
                sc.Start();
                DateTime start = DateTime.Now;
                while (sc.Status != ServiceControllerStatus.Running && (DateTime.Now - start).TotalSeconds < 10)
                {
                    sc.Refresh();
                }
            }
            catch (Exception ex)
            {
                result.StatusMessage = String.Format("Service '{0}' could not be restarted: '{1}'. ", _serviceName, ex.Message);
            }

        }

        if (sc.Status == ServiceControllerStatus.Running)
        {
            result.IsHealthy = true;
        }
        else
        {
            result.IsHealthy = false;
            result.StatusMessage += String.Format("Service '{0}' is in state {1}.", _serviceName, sc.Status);
        }

        return result;
    }
}

Hopefully these two sample health checkers will give you an idea of what’s possible, so you can come up with more interesting and sophisticated ones for your own application. Once your health checkers are defined, the last step is to configure your cloud service with a setting called “HealthCheckers” in ServiceConfiguration.cscfg. The value is a JSON-formatted array of health checkers, each of which you must specify the typeName (assembly-qualified), the initData, and optionally a Boolean value keepRunningIfUnhealthy (default is false). An example setting is shown below, although keep in mind that if you edit it using Visual Studio’s role configuration dialog you don’t need to escape the quotes yourself.

<Setting name="HealthCheckers" value="[{&quot;typeName&quot;: &quot;AzureHealthCheck.PingUrlHealthChecker, AzureHealthCheck&quot;, &quot;initData&quot;: &quot;FileProbe.txt&quot;},
{&quot;typeName&quot;: &quot;AzureHealthCheck.WindowsServiceHealthChecker, AzureHealthCheck&quot;, &quot;initData&quot;: &quot;W3SVC&quot;},
{&quot;typeName&quot;: &quot;AzureHealthCheck.WindowsServiceHealthChecker, AzureHealthCheck&quot;, &quot;initData&quot;: &quot;W32Time&quot;, &quot;keepRunningIfUnhealthy&quot; : &quot;true&quot;}]" />

Note that since this setting is in CloudConfiguration.cscfg, you can change it after deployment, although of course any health check assemblies you use need to be deployed with your solution.

Running the Sample

If you like the look of this and would like to try it out, please download the sample code. Here’s what you need to do to get it running and test it out:

If you don’t already have a Windows Azure subscription, sign up now and download the .NET SDK for Visual Studio 2012
Open the sample code solution in Visual Studio 2012
Update the diagnostics connection string for your cloud storage account
Publish the solution to the cloud, ensuring you enable remote desktop and configure your credentials
Once the solution is up and running in the cloud, remote desktop into any one of the instances and try:

Browsing to E:\sitesroot\0 and deleting/renaming the file called ProbeFile.txt. You should see the instance status change to Busy and removed from the load balancer, and error logs written to the WADLogsTable. Once you’re done, restore the file to make the instance healthy again.
Opening the Services console and stopping the “World Wide Web Publishing Service”. You should see the service automatically restart, and a warning log written to the WADLogsTable, but the instance will remain healthy as the issue was remediated.
Opening the Services console, stopping and disabling the “Windows Time” service (disabling it prevents the remediation from succeeding). You should see logs in the WADLogsTable saying the instance is unhealthy, however it will not be set to Busy since this health checker has keepRunningIfUnhealthy set to true.

As I’ve said before, this is just a sample—but I hope that it will serve as a useful staring point that will help you build rock-solid PaaS solutions on top of Windows Azure.

Running scripts from a Windows Azure role’s OnStart method

Tom Hollander — Wed, 15 May 2013 05:36:30 GMT

Summary: Startup scripts declared in ServiceDefinition.csdef work well in most cases, but if you need to modify IIS configuration you’ll need to run your scripts from your role’s OnStart method. This post includes some sample code that can do this using a configuration-driven declarative approach.

In a Windows Azure cloud service, you can specify scripts that should run at role startup time using the <Startup> element in ServiceDefinition.csdef. This works well in almost all cases, and as it’s declarative you don’t need to write any code (other than the scripts themselves, obviously). However there is one common scenario where startup scripts won’t cut it: anytime you need to query or modify your IIS configuration. Why? As Kevin Williamson depicts nicely in this post, the IISConfigurator does not create IIS websites (step 8 in Kevin’s post) until after the startup tasks have already finished (step 7). So if you want to do anything to IIS that can’t be defined declaratively in ServiceDefinition.csdef, you’ll need to do it at some point after the IISConfigurator has done its bit.

The logical place do this is in your role’s OnStart method (triggered in step 9). Normally you’ll do this via a call to Process.Start and associated code, and this is a fine approach – however compared to the declarative simplicity of “real” startup tasks, hard-coding the details into C# code makes me feel a little dirty, especially if you want to call more than one script or you’re doing this in multiple applications.

So I built a simple solution that lets you use a declarative approach to start processes and scripts from your role’s OnStart method. The full code can be downloaded here; the highlights are described below.

This solution lets you specify the scripts you want to run using a configuration setting defined in ServiceConfiguration.cscfg, for example:

<ConfigurationSettings>
  <Setting name="RoleStartupScripts" value="startup\test1.cmd!;startup\test2.cmd;Powershell.exe startup\test3.ps1" />
    ....
</ConfigurationSettings>

The setting value contains a semicolon-separated list of scripts to run. Note that the first script has an exclamation mark after its filename – this is my way of saying that the code needs to wait for that process to finish before starting the next script. For scripts without the exclamation mark, it won’t wait. There is no way to specify that a script needs to run as elevated; if you need this (which will generally be the case for IIS manipulation), be sure to set <Runtime executionContext="elevated" /> in your ServiceDefinition.csdef file to force the role startup code (and all processes it launches) to be elevated.

Next, I built an abstract class StartupScriptRoleEntryPoint that extends RoleEntryPoint to automatically launch the processes specified in configuration. My class is directly in the WebRole project, but you’d probably want to move this into a reusable library.

public abstract class StartupScriptRoleEntryPoint : RoleEntryPoint
{
    public override bool OnStart()
    {
        var startupScripts = CloudConfigurationManager.GetSetting("RoleStartupScripts");
        if (!String.IsNullOrEmpty(startupScripts))
        {
            var scriptList = startupScripts.Split(';');
            foreach (var script in scriptList)
            {
                bool waitForExit = false;

                string scriptCommandLine = script;
                if (script.EndsWith("!"))
                {
                    scriptCommandLine = script.Substring(0, script.Length - 1);
                    waitForExit = true;
                }

                var args = CmdLineToArgvW.SplitArgs(scriptCommandLine);

                var processStartInfo = new ProcessStartInfo()
                {
                    FileName = args[0],
                    Arguments = string.Join(" ", args.Skip(1).ToArray()),
                    WorkingDirectory = AppDomain.CurrentDomain.BaseDirectory,
                    UseShellExecute = true,
                };
                var process = Process.Start(processStartInfo);
                if (waitForExit)
                {
                    process.WaitForExit();
                }
            }
        }
        return base.OnStart();
    }
}

This code should be fairly self-explanatory – it loops through each specified script and launches it as a new process. The one unusual bit is the CmdLineToArgvW.SplitArgs call; this uses P/Invoke to call a Win32 API to parse the command line arguments out of a string which is necessary when launching a process from .NET.

Now all that’s left is to update the WebRole class to derive from StartupScriptRoleEntryPoint instead of the default RoleEntryPoint – no other code changes are needed. You could do this for a worker role too I guess, although I’m not sure if there’s a scenario where this is useful.

public class WebRole : StartupScriptRoleEntryPoint
{
    public override bool OnStart()
    {
        // For information on handling configuration changes
        // see the MSDN topic at http://go.microsoft.com/fwlink/?LinkId=166357.

        return base.OnStart();
    }
}

Since this approach uses configuration settings from ServiceConfiguration.cscfg, one interesting consequence is that you can change which scripts run in which environments – this may be particularly useful when running under the emulator on your local PC where you don’t want scripts doing weird things to your box. It’s also possible to change the configuration after deployment, but remember the settings are only used when the role starts (and any scripts you refer to need to already exist on the Windows Azure VM).

Using XPath to set environment variables in ServiceDefinition.csdef

Tom Hollander — Tue, 14 May 2013 02:04:52 GMT

In my last post on zipping IIS log files in Windows Azure, my ServiceDefinition.csdef file included the following code to set an environment variable to the value of a local resource path:

<Task commandLine="Startup\ScheduleLogFileZipAndDeleteTask.cmd" executionContext="elevated" taskType="simple">
  <Environment>
    <Variable name="ZippedLogFilesPath">
      <RoleInstanceValue xpath="/RoleEnvironment/CurrentInstance/LocalResources/LocalResource[@name='ZippedLogFiles']/@path" />
    </Variable>
  </Environment>
</Task>

As you can see, an XPath query is used to retrieve the resource path. Someone asked me what this XPath query refers to. I didn’t know, so I looked into it.

There is a page in the Windows Azure documentation which talks about this feature, and gives a whole bunch of examples of values you can pull out using different XPath queries. You can use this to set environment variables either for a specific startup task (specifying it under the <Startup> element), or for the role entry point code (specifying it under the <Runtime> element). However the documentation still doesn’t explain what XML document is being queried, so you can’t tell what other information may be available to you.

After a bit of poking around, I found the mysterious document. It only seems to be created if you include an xpath element somewhere in your ServiceDefinition.csdef file, so you can add a dummy environment variable like this to ensure the file is created:

<Runtime>
  <Environment>
    <Variable name="TestIsEmulated">
      <RoleInstanceValue xpath="/RoleEnvironment/Deployment/@emulated"/>
    </Variable>
  </Environment>
</Runtime>

Now when you deploy the solution (either to Windows Azure or to the local compute emulator), you’ll get a file with a filename similar to DeploymentId.DeploymentId.RoleInstanceName.RoleEnvironment.1.xml. On a real Windows Azure VM, this file is in C:\Config; in the local compute emulator it is more hidden, living at C:\Users\username\AppData\Local\dftmp\Config. (Don’t confuse this file with another one in the same folder that doesn’t end with RoleEnvironment.1, as this file has a different schema and data, and as far as I know can’t be queried in the same way). In both Windows Azure and the emulator, the role environment XML file should have the same format – here’s an example from my IIS log zipping app:

<?xml version="1.0" encoding="utf-8"?>
<RoleEnvironment xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <Deployment id="046cfa59b8894be3b0ca5f0c6aaeb237" emulated="false" />
  <CurrentInstance id="WebRole1_IN_0" roleName="WebRole1" faultDomain="0" updateDomain="0">
    <ConfigurationSettings>
      <ConfigurationSetting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString" value="DefaultEndpointsProtocol=https;AccountName=xxxx;AccountKey=xxxxx" />
      <ConfigurationSetting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.AccountEncryptedPassword" value="xxxx" />
      <ConfigurationSetting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.AccountExpiration" value="2014-05-08T23:59:59.0000000+10:00" />
      <ConfigurationSetting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.AccountUsername" value="remoteuser" />
      <ConfigurationSetting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.Enabled" value="true" />
      <ConfigurationSetting name="Microsoft.WindowsAzure.Plugins.RemoteForwarder.Enabled" value="true" />
    </ConfigurationSettings>
    <LocalResources>
      <LocalResource name="DiagnosticStore" path="C:\Resources\directory\046cfa59b8894be3b0ca5f0c6aaeb237.WebRole1.DiagnosticStore\" sizeInMB="4096" />
      <LocalResource name="ZippedLogFiles" path="C:\Resources\directory\046cfa59b8894be3b0ca5f0c6aaeb237.WebRole1.ZippedLogFiles\" sizeInMB="1000" />
    </LocalResources>
    <Endpoints>
      <Endpoint name="Endpoint1" address="10.78.196.62" port="80" publicPort="80" protocol="http" />
      <Endpoint name="Microsoft.WindowsAzure.Plugins.RemoteAccess.Rdp" address="10.78.196.62" port="3389" publicPort="0" protocol="tcp" />
      <Endpoint name="Microsoft.WindowsAzure.Plugins.RemoteForwarder.RdpInput" address="10.78.196.62" port="20000" publicPort="3389" protocol="tcp" />
    </Endpoints>
  </CurrentInstance>
  <Roles />
</RoleEnvironment>

There’s not a whole lot in there that isn’t alluded to in the documentation, but it’s always nice to know exactly what it is that you’re dealing with.

Zip your IIS log files before transferring with Windows Azure Diagnostics

Tom Hollander — Thu, 09 May 2013 01:04:00 GMT

Summary: If your Windows Azure-hosted website is really popular, the IIS log files can start to get pretty big. To prevent them from filling up your local VM’s quota and to minimise storage size and transfer bandwidth, you can zip the log files before Windows Azure Diagnostics transfers them to blob storage.

One of the many nice things about Windows Azure Diagnostics is that it can automatically transfer your IIS log files to blob storage, read for you to pull down and analyse using your favourite log analysis tool. This capability is turned on by default in the diagnostics.wadcfg file you get when you create a new Windows Azure Cloud Service using the .NET SDK, and for most sites the default behaviour is just fine. But if your website is really popular, those log files can start to get very large. This could have a few implications:

IIS log files on Windows Azure Cloud Service VMs are written to your “local resources” area on drive C:. This area has a quota. If you go beyond the quota, your site should keep running, but you may lose log files if they are cleaned up by the diagnostics agent to keep you under the quota.
Once transferred to a blob container in Windows Azure Storage, you pay per megabyte stored. Even with today’s crazy cheap storage prices, the more you store, the more you pay.
Similarly, you pay to transfer files from Windows Azure Storage to your local environment – both in dollars (Windows Azure bandwidth costs) and time.

Because IIS log files are plain text with lots of repetition, they compress very well (generally over 90% reduction). However unfortunately there’s no built-in way to compress IIS log files. But with the awesome power of PowerShell, Windows Azure startup tasks and the Windows Task Scheduler, rolling your own solution isn’t too hard. In fact it’s super easy as I’ve already written some sample code which you can download here. Disclaimer: This code is provided as-is and should be modified and tested to ensure it meets your requirements. Also, I’m actually pretty rubbish at PowerShell so I know the code could be improved a lot.

I won’t show all of the code in this blog post, but I will summarise how it works so you can customise it as needed.

First, we’ll need to declare a few things in our ServiceDefinition.csdef file:

Allocate some space in our local resource areas to hold our zipped log files
Run a startup task which will schedule a PowerShell script to run every hour to zip the log files
Pass an environment variable to the above script that contains the path to our local resources area

Here are the relevant parts of the file:

<ServiceDefinition name="ZipLogFileTest" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceDefinition" schemaVersion="2013-03.2.0">
  <WebRole name="WebRole1" vmsize="Small">
     ...
    <LocalResources>
      <LocalStorage name="ZippedLogFiles" sizeInMB="1000" />
    </LocalResources>
    <Startup>
      <Task commandLine="Startup\ScheduleLogFileZipAndDeleteTask.cmd" executionContext="elevated" taskType="simple">
        <Environment>
          <Variable name="ZippedLogFilesPath">
            <RoleInstanceValue xpath="/RoleEnvironment/CurrentInstance/LocalResources/LocalResource[@name='ZippedLogFiles']/@path" />
          </Variable>
        </Environment>
      </Task>
    </Startup>
  </WebRole>
</ServiceDefinition>

Next, we need a batch file which creates the scheduled task. Depending on your needs you may want to tweak the parameters, but in my example the task is run hourly. Note the weird quoting in the parameters to schtasks.exe which it seems to insist on to prevent weird errors. Also note that I am passing the path to the ZippedLogFilesPath as a parameter to the scheduled PowerShell script, as the environment variable isn’t accessible at the time the script runs.

rem ScheduleLogFileZipAndDeleteTask.cmd
powershell.exe Set-ExecutionPolicy RemoteSigned -force
schtasks.exe /create /sc HOURLY /tn ZipAndDeleteLogFiles /tr "\"powershell.exe\" -file \"%~dp0IISLogZipAndDelete.ps1\" %ZippedLogFilesPath%." /RL highest /F /RU SYSTEM >> ScheduleLogFileZipAndDeleteTask.out.log 2>> ScheduleLogFileZipAndDeleteTask.err.log

Now we need our PowerShell script. It’s a little long so if you want to go through it, download the full sample. But here’s the gist of what it does:

Uses the WebAdministration PowerShell module to look up the IIS log path
Iterate through each folder under this path (as IIS creates a second level of folders under the configured log path)

Create a corresponding folder under the ZippedLogFiles resource folder (passed as a parameter to the script)
Iterate through each file in the IIS log path

Check if the last-modified-date of the file is more than 60 minutes old
If so:

Zip the individual log file
Copy the zipped file to the corresponding subfolder under ZippedLogFiles
Delete the original unzipped log file

Iterate through each folder and file under ZippedLogFiles

Check if the last-modified-date of the file is more than 1 day (1440 minutes) old
If so, delete the file.

The PowerShell script depends on the System.IO.Compression.ZipFile class which comes with .NET Framework 4.5. Therefore you’ll need Windows Server 2012 (Windows Azure OS Family version 3) VMs to run this script. If you’re using an older OS, you may need to use a different approach to compress the file.

Finally, you need to configure Windows Azure Diagnostics to transfer files from our ZippedLogFiles folder, but ignore the standard IIS log files folder. You can do this by modifying diagnostics.wadcfg by removing the IISLogs element and adding DirectoryConfiguration element pointing to the local resource, for example:

<DiagnosticMonitorConfiguration configurationChangePollInterval="PT1M" overallQuotaInMB="4096" xmlns="http://schemas.microsoft.com/ServiceHosting/2010/10/DiagnosticsConfiguration">
  <DiagnosticInfrastructureLogs />
  <Directories bufferQuotaInMB="1024"  scheduledTransferPeriod="PT30M">
    <DataSources>
      <DirectoryConfiguration container="wad-iis-zippedlogs" directoryQuotaInMB="1024">
        <LocalResource name="ZippedLogFiles" relativePath="."/>
      </DirectoryConfiguration>
    </DataSources>
    <CrashDumps container="wad-crash-dumps" />
  </Directories>
  ...
</DiagnosticMonitorConfiguration>

Now it’s simply a matter of using your app and waiting for the various timers to go off, and you should see your IIS log files beautifully gift-wrapped as zip files in the wad-iis-zippedlogs container in your configured Windows Azure Diagnostics storage account. Please let me know if you find this useful.

Using Windows Firewall to restrict access to Windows Azure instances

Tom Hollander — Tue, 07 May 2013 11:06:00 GMT

Summary: If you ever have a need to restrict access to your Windows Azure deployment to known IP address ranges, you can do this by programmatically modifying the Windows Firewall. You’ll need to do this both at startup, and whenever your role topology changes, as the Windows Azure guest agent also likes to modify firewall rules.

When you deploy your PaaS cloud service to Windows Azure, it’s on the internet. This means it can be accessed by anyone who is also on the internet, subject to your application’s authorisation logic. Normally this is a good thing—we build and deploy apps so people can use them. However sometimes you may want to restrict access to certain known client machines. Some of the reasons I’ve seen for this include:

You’re working on an unreleased, super-secret application that you don’t want anyone to stumble upon until you’re good and ready.
You have multiple deployments for different environments (e.g. development, test, UAT) which should each only be accessed by certain people but you don’t want to change the app’s authorisation logic.
Your application is only designed for internal users and you don’t want to allow remote access. In this case you should also use a federated identity solution to authorise individual users, but if you’re really paranoid you may want to prevent these users from accessing the app when away from the corporate network.
Even though you’re happy for anyone to access the application’s public UI, you want some additional security (beyond standard user authentication) for admin services, such as RDP or an administrative UI running on a different port.

If you’re only interested in blocking traffic to IIS-hosted services, one option is to use IIS’s “IP and Domain Restrictions” feature to restrict access as described in Liam Cavanagh's Blog. This is a good option as the IP restrictions are configuration-based, and Windows Azure won’t do anything that interferes with your settings. However as Liam points out, this capability depends on a Windows feature that isn’t installed by default, so you’ll need to write a startup script to install the “IP and Domain Restrictions” feature.

However this solution will not work if you want to restrict access to something that’s not IIS hosted, such as RDP. In this case you need to step it up a level by modifying the Windows Firewall which runs locally on each Windows Azure VM. Normally it’s pretty easy to configure Windows Firewall to only allow access to a given port to specific IP ranges: you set up a single rule for that port and set the scope to your IP ranges; any traffic from other IP addresses will be automatically blocked as there is no matching firewall rule. Unfortunately this is a bit more complex in Windows Azure, as the fabric will automatically configure a whole bunch of firewall rules (helpfully named after random GUIDs) that open certain ports to all IP addresses. You can still add additional rules if you want, but due to the way that firewall rules combine, you can’t easily block access once another rule has already granted it.

The most practical option I found was to modify the existing firewall rules to add the IP range restrictions. There are a few different APIs to do this, but the easiest way is to use the Network Security PowerShell cmdlets. These run on Windows Server 2012, so make sure you set your Windows Azure osFamily attribute to “3”. Here’s a simple PowerShell script that modifies all existing rules for two ports (80 and 3389) to restrict them to specific IP addresses and ranges:

# SetFirewallRestrictions.ps1
$date = Get-Date
Write-Host "Updating firewall rule restrictions at " $date "`r`n"
$allowedRanges = ('3.2.1.0/24', '1.2.3.4') 
Get-NetFirewallPortFilter | ? LocalPort -eq '80' | Get-NetFirewallRule | Set-NetFirewallRule -RemoteAddress $allowedRanges
Get-NetFirewallPortFilter | ? LocalPort -eq '3389' | Get-NetFirewallRule | Set-NetFirewallRule -RemoteAddress $allowedRanges

To make it easier to call the script and log the output, let’s also create a .cmd file to wrap it:

rem SetFirewallRestrictions.cmd
cd /d %~dp0
Powershell set-executionpolicy remotesigned -force
Powershell .\SetFirewallRestrictions.ps1 >> SetFirewallRestrictions.out.log 2>> SetFirewallRestrictions.err.log

So now we have our scripts, it’s just a matter of including them into our Windows Azure cloud service role and calling them at the appropriate time. The obvious place to do this is at role startup, so let’s add a startup task to the ServiceDefinition.csdef file. Remember to run it as elevated since we’re playing with security settings. Also note the <Runtime> element which I’ll get to in a moment.

<ServiceDefinition name="FirewallTest" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceDefinition" schemaVersion="2013-03.2.0">
  <WebRole name="WebRole1" vmsize="Small">
   ...  
    <Startup>
      <Task commandLine="Startup\SetFirewallRestrictions.cmd" executionContext="elevated" taskType="simple" />
    </Startup>
    <Runtime executionContext="elevated" />
  </WebRole>
</ServiceDefinition>

With the startup script in place, Windows Azure will automatically update the firewall settings with your IP range restrictions just as the instance starts. So we’re done, right? Actually not quite. After some testing, we found that our custom firewall rules were occasionally overwritten after the instance has been running for a while. After further investigation we discovered that the Windows Azure Guest Agent (an app which the fabric installs on your VM to carry out various cloudy tasks) recreates the firewall rules whenever the role is scaled – yes, even on the instances that were already running. So in order to keep our IP address restrictions, we need to reapply our firewall script when the role’s topology changes. Important: This solution is based on experimentation, not insider knowledge of exactly how this part of the platform works. It’s possible that there are other times when the firewall rules are overwritten. Use this solution at your own risk and test thoroughly.

In order to reapply the firewall rules when the app is scaled, it’s necessary to subscribe to the RoleEnvironment.Changed event in your RoleEntryPoint-derived class. In the event handler we can launch the same script again. The <Runtime executionContext=”elevated” /> entry we put in above ensures that this code runs as admin.

public class WebRole : RoleEntryPoint
{
    public override bool OnStart()
    {
        RoleEnvironment.Changed += RoleEnvironment_Changed;
        return base.OnStart();
    }

    void RoleEnvironment_Changed(object sender, RoleEnvironmentChangedEventArgs e)
    {
        if (e.Changes.Any(ch => ch is RoleEnvironmentTopologyChange))
        {
            var processStartInfo = new ProcessStartInfo(@"Startup\SetFirewallRestrictions.cmd");
            var process = Process.Start(processStartInfo);
        }
    }
}

So far it looks like this is a solid option for restricting access to your Windows Azure applications to specific IP addresses. If you find any issues or have any suggestions please let me know, and I’ll promise to do the same!

Keeping diagnostics in sync across Windows Azure instances

Tom Hollander — Sun, 05 May 2013 08:36:00 GMT

Windows Azure Diagnostics provides a great way for operations staff, developers and testers to find out what’s going on within a Windows Azure PaaS (Cloud Service) deployment. In a nutshell, it lets you specify what types of logs you are interested in (application logs, event logs, performance counters, or log files), and at specified intervals these logs are transferred from your Windows Azure VMs to blobs and tables in your Windows Azure Storage Account.

To specify what diagnostics you’re interested in, you normally include a file called diagnostics.wadcfg within each role of your Cloud Service (or, you can also do it programmatically). This file specifies your “factory default” diagnostics settings and gets baked into your deployment package, so you can’t modify it without redeploying. However, as soon as your role instances start, the diagnostics settings are copied into a blob in the wad-control-container, which is what Windows Azure Diagnostics uses at runtime. There’s a separate blob for each instance of each role of each deployment, and this can be changed after deployment. In fact, if you have the new Windows Azure SDK 2.0 for .NET or third party tools like Cerebrata’s Azure Diagnostics Manager it’s very easy to update the live diagnostics from a GUI so you don’t need to hand-edit the control container blobs.

Even though it’s possible to set different diagnostics configuration for each instance of a role, in practice it’s pretty unlikely that you’ll want this. The people who built the aforementioned tools must agree because they also provide easy ways to update the diagnostics configuration of all instances of a role at the same time. However, consider what happens in the following situation:

2 instances of WebRole1 are deployed, and Windows Azure uses the “factory default” diagnostics configuration loaded from diagnostics.wadcfg.
To improve operations processes, the diagnostics configuration for both instances of WebRole1 are modified to “new improved” diagnostics configuration using a tool
Based on high user demand, the deployment is scaled (either automatically or manually) by adding a 3rd instance of WebRole1.

So what diagnostics configuration would you expect the new 3rd instance to have? Well, the result which you’d almost certainly want is for the 3rd instance’s diagnostics configuration to match the other two. However, when the new instance is deployed it will come with the “factory default” settings, and won’t get “new improved” settings unless someone explicitly applies them.

If this is a concern to you, the good news it’s very easy to change the behaviour with a bit of code, and the better news is that I’ve already written the code for you. Hopefully the code is pretty self-explanatory, but basically whenever a role instance starts, if that instance isn’t the first one (instance 0), the diagnostics configuration is copied from instance 0’s current version. You may wonder what happens if instance 0 is ever reimaged or fails over – it’s actually OK as its wad-control-container blob remains intact even when the new VM starts.

My code is below; if you need to do this across multiple roles or applications, of course you should put this into a reusable library.

public class WebRole : RoleEntryPoint
{
    public override bool OnStart()
    {
        // For information on handling configuration changes
        // see the MSDN topic at http://go.microsoft.com/fwlink/?LinkId=166357.

        CopyDiagnosticsSettingsFromExistingInstance();

        return base.OnStart();
    }

    private static void CopyDiagnosticsSettingsFromExistingInstance()
    {
        var trace = new DiagnosticMonitorTraceListener();
        try
        {
            var sourceInstance = RoleEnvironment.CurrentRoleInstance.Role.Instances.OrderBy(i => i.Id).First();
            if (RoleEnvironment.CurrentRoleInstance.Id != sourceInstance.Id)
            {
                trace.WriteLine(String.Format("Copying live diagnostics settings from instance {0}.", sourceInstance.Id));
                var diagnosticsConnectionString = CloudConfigurationManager.GetSetting("Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString");
                var deploymentId = RoleEnvironment.DeploymentId;
                var deploymentDiagnosticsManager = new DeploymentDiagnosticManager(diagnosticsConnectionString, deploymentId);

                var sourceDiagnosticsManager = deploymentDiagnosticsManager.GetRoleInstanceDiagnosticManager(sourceInstance.Role.Name, sourceInstance.Id);
                var sourceConfig = sourceDiagnosticsManager.GetCurrentConfiguration();

                if (sourceConfig != null) // May happen during deployment when all instances are coming online
                {
                    var thisDiagnosticsManager = deploymentDiagnosticsManager.GetRoleInstanceDiagnosticManager(RoleEnvironment.CurrentRoleInstance.Role.Name, RoleEnvironment.CurrentRoleInstance.Id);
                    thisDiagnosticsManager.SetCurrentConfiguration(sourceConfig);
                }
            }
        }
        catch (Exception ex)
        {
            trace.WriteLine(String.Format("Error while copying diagnostics settings: {0}", ex.ToString()));
        }
    }
}

It’s critical that developers “design for operations” to ensure applications they develop can be kept running smoothly over time. Diagnostics and scalability are two aspects of designing for operations, and while Windows Azure provides great platform services for both, with this code in place these two concerns will work even better together. Please let me know if you find this useful or have any suggestions or questions.

Integrate Lync into your intranet sites using the NameCtrl plug-in

Tom Hollander — Sat, 02 Mar 2013 10:57:00 GMT

Introduction

If you’re a regular user of both SharePoint and Lync, you’ve probably seen that SharePoint integrates with Lync by showing users’ presence on the page, and giving you a nice hover dialog allowing you to collaborate with them. But did you know that you can add the same capabilities to your own intranet sites? And did you know it can work on Firefox and Chrome as well as in Internet Explorer? Even if you knew it was possible, chances are you haven’t found it easy, since the documentation is frankly pretty awful.

Now before I go any further, yes this approach relies on plug-ins, and yes we all know that isn’t the way of the future. Very soon it will become much easier to integrate Lync presence into web apps using the new Unified Communications Web API (UCWA) which will ship as an update to Lync Server 2013. However even with that coming, there is still value in client-side plug-ins, and the integration with the Lync client you get with the plug-in approach does provide a very nice experience.

You can download a working sample showing everything I discuss in the post here. Important: I’ve made this available as a downloadable HTML file instead of hosting it on a web server as it won’t work in the Internet zone anyway. However after downloading you’ll need to right-click the file, choose Properties and Unblock before your browser will launch the plug-ins and scripts.

It’s not the most beautiful web code you’ll come across (nor has it been extensively tested) but I wanted to show all of the key concepts in one file without any unnecessary complexity.

Prerequisites

There are a few things required for this approach to work. If you can’t rely on all of these, make sure you do defensive coding to fall back gracefully when the plug-in isn’t available.

Your users will need Lync installed and be signed in
The NameCtrl plug-in must be properly registered. This should happen automatically when Lync is installed.
Your users will need to be using a supported web browser. I don’t know the official support matrix, but with Lync 2013 installed and the site coded correctly it works with Internet Explorer, Firefox and Chrome on Windows. I don’t know if it works on non-IE browsers with versions prior to Lync 2013 (let me know!).
Your web site must be in Internet Explorer’s Intranet or Trusted Sites security zone (even if being accessed from a different browser). This may sound like a big restriction, but in practice you’ll probably only want to do this on intranet sites anyway.

Initialising the Plug-In

The NameCtrl plug-in exposes the same API to all browsers, but the way you initialise it is different on different browsers.

On Internet Explorer, you need to create an ActiveX object of the type NameCtrl of the class “Name.NameCtrl”:

nameCtrl = new ActiveXObject("Name.NameCtrl");

On Chrome and Firefox you need an <object> tag to initialise the plug-in:

<object id="application/x-sharepoint-uc" type="application/x-sharepoint-uc" width="0" height="0" style="visibility: hidden;"></object>

My demo code (based on a simplified version of what ships with SharePoint 2013) to initialise using the correct approach based on the browser is as follows:

var nameCtrl = null;

$(document).ready(function () {

    try {
        if (window.ActiveXObject) {
            nameCtrl = new ActiveXObject("Name.NameCtrl");

        } else {
            nameCtrl = CreateNPApiOnWindowsPlugin("application/x-sharepoint-uc");
        }
        attachLyncPresenceChangeEvent();
    }
    catch (ex) { }
});


function IsSupportedNPApiBrowserOnWin() {
    return true; // SharePoint does this: IsSupportedChromeOnWin() || IsSupportedFirefoxOnWin()
}

function IsNPAPIOnWinPluginInstalled(a) {
    return Boolean(navigator.mimeTypes) && navigator.mimeTypes[a] && navigator.mimeTypes[a].enabledPlugin
}

function CreateNPApiOnWindowsPlugin(b) {
    var c = null;
    if (IsSupportedNPApiBrowserOnWin())
        try {
            c = document.getElementById(b);
            if (!Boolean(c) && IsNPAPIOnWinPluginInstalled(b)) {
                var a = document.createElement("object");
                a.id = b;
                a.type = b;
                a.width = "0";
                a.height = "0";
                a.style.setProperty("visibility", "hidden", "");
                document.body.appendChild(a);
                c = document.getElementById(b)
            }
        } catch (d) {
            c = null
        }
    return c
}

Subscribing to Presence Updates

Once you’ve initialised your plug-in, you can use it to check the presence for any number of users. Doing this properly takes a few steps:

Subscribe to the plug-in’s OnStatusChange event
Call the plug-in’s GetStatus method for each user whose presence you are interested in
In the OnStatusChange event, update your UI (for example to change the presence icon or colour) based on the reported user’s new presence.

There are a few tricks here which I figured out the hard way:

When you call GetStatus the first time, you probably won’t get the user’s actual presence back. Really this method is used to tell the plug-in you want to subscribe to a user. The correct presence value will be returned asynchronously in the OnStatusChange event handler.
The GetStatus method takes two parameters. The first is the SIP address (typically the same as their email address) of the user you are subscribing to. The documentation says that the second parameter is “a string that identifies the HTML <DIV> tag to update with the correct pawn icon”. I’ve never got this to work, and in any event I prefer to control my own UI rather than have the plug-in decide how it should look. In my tests, you can use any value for this string in Internet Explorer however it won’t work in Chrome or Firefox if you use an empty string (but other random strings worked fine).
The OnStatusChange event handler gives you three parameters. The docs say (accurately enough): “The first parameter is the sign-in name of the user, the second parameter is the new status, and the third parameter is the same ID that is passed to the GetStatus method.” The presence value returned in the second parameter is numeric; the docs have a partial list but see my code snippet below for some other values that may be returned.

So, with that out of the way, here’s my code to subscribe to users’ presence and update the UI (using CSS classes) when their presence changes:

function addUser() {
    var userName = $('#userName').val();
    var userElementId = getId(userName);
    $('<div/>', {
        text: userName,
        id: userElementId,
        class: 'user',
        onmouseover: "showLyncPresencePopup('" + userName + "', this)",
        onmouseout: "hideLyncPresencePopup()",
    }).appendTo('#users');

    if (nameCtrl) {
        nameCtrl.GetStatus(userName, 'users');
    }
    $('#userName').val('');
}

function getLyncPresenceString(status) {

    switch (status) {
        case 0:
            return 'available';
            break;
        case 1:
            return 'offline';
            break;
        case 2:
        case 4:
        case 16:
            return 'away';
            break;
        case 3:
        case 5:
            return 'inacall';
            break;
        case 6:
        case 7:
        case 8:
        case 10:
            return 'busy';
            break;
        case 9:
        case 15:
            return 'donotdisturb';
            break;
        default:
            return '';
    }
}

function attachLyncPresenceChangeEvent() {
    if (!nameCtrl) {
        return;
    }
    nameCtrl.OnStatusChange = onLyncPresenceStatusChange;
}

function onLyncPresenceStatusChange(userName, status, id) {
    var presenceClass = getLyncPresenceString(status);

    var userElementId = getId(userName);
    var userElement = $('#' + userElementId);
    removePresenceClasses(userElement);
    userElement.addClass(presenceClass);

}

function removePresenceClasses(jqueryObj) {
    jqueryObj.removeClass('available');
    jqueryObj.removeClass('offline');
    jqueryObj.removeClass('away');
    jqueryObj.removeClass('busy');
    jqueryObj.removeClass('donotdisturb');
    jqueryObj.removeClass('inacall');
}

function getId(userName) {
    return userName.replace('@', '_').replace('.', '_');
}

Showing and Hiding the Lync contact card

One final thing you can do with the plug-in is have it show a Lync contact card for a given user. This is useful as the contact card shows additional information about a user (full name, location, phone number, org chart, etc), and also allows you to interact them using any Lync modality (IM, phone, video, etc).

Showing and hiding the contact card is pretty straightforward: the ShowOOUI method shows it, and the HideOOUI method hides it. You just need to wire it up to an appropriate event (typically onmouseover and onmouseout respectively).

The trick is getting it to display in the right place. Without going into all of the details, here is some code which I’ve found to work pretty well:

function showLyncPresencePopup(userName, target) {
    if (!nameCtrl) {
        return;
    }

    var eLeft = $(target).offset().left;
    var x = eLeft - $(window).scrollLeft();

    var eTop = $(target).offset().top;
    var y = eTop - $(window).scrollTop();

    nameCtrl.ShowOOUI(userName, 0, x, y);
}

function hideLyncPresencePopup() {
    if (!nameCtrl) {
        return;
    }
    nameCtrl.HideOOUI();
}

Conclusion

So there you have it. If your organisation uses Lync, hopefully you’ve noticed how awesome it is. But it’s even awesomer (yes, I know that isn’t a word) if you can launch it exactly when you need it, which is often when using some intranet site or line-of-business application which tells you which people you need to communicate with. Even as we slowly move toward a plug-in-free world, it’s important to keep in mind what’s possible with the technology you already have deployed, and use it to create the best experiences you can.

New ABC iView plug-in for TunerFreeMCE

Tom Hollander — Tue, 06 Nov 2012 10:35:00 GMT

Most Australians will recognise ABC iView as the country's best online TV service. In recent years they have done a good job of expanding to multiple platforms including PS3, Xbox 360, iOS and many smart TVs. Unfortunately one platform that is close to my heart but has not received any love from the ABC is Windows Media Center. (Windows 8 is also missing out but it's early days there so hopefully things will change soon!).

The only way I've found to get ABC iView through Media Center is to use the TunerFreeMCE application. This is an excellent app that provides a remote control friendly interface to many popular TV services. But the app was built primarily for UK services, and while its plug-in model made it possible to support international services like ABC, the original plug-in was somewhat limited and unreliable.

Recently TunerFreeMCE author Martin Millmore added some enhancements to the app which made it possible to build much more reliable plug-ins. Building on top of this and some richer metadata from the ABC, I'm pleased to announce that I've rewritten the plug-in. Users of the old version will notice more programs, grouping of episodes under series, and improved reliability when starting or ending programs. If you haven't used it before I encourage you to download TunerFreeMCE and give it a go. There's no need to download my plug-in separately as it can be accessed from the app's Settings page. There's also a plug-in available for SBS Australia that I wrote a while back but haven't yet migrated to the improved plug-in model.

Rumours of Windows Media Center's demise are greatly exaggerated. After a recent hardware failure of my Media Center PC I needed to decide whether to fix it or try some other path, and indeed I did experiment with a non-PC PVR. However I still think there's no better solution for combining broadcast TV with my digital media library. With TunerFreeMCE and support for services I care about, the solution is even more complete.

Automated Build and Deployment with Windows Azure SDK 1.6

Tom Hollander — Tue, 06 Dec 2011 00:36:37 GMT

A few months ago I posted on how to automate deployment of Windows Azure projects using MSBuild. While the approach documented in that post continues to work, Windows Azure SDK 1.6 has introduced some new capabilities for managing Windows Azure credentials and publishing settings which I wanted to leverage and build upon. With this new approach, you’ll no longer need to manually manage details such as Subscription IDs, hosted service names and certificates. Because this approach relies on a few tools that are too big to share in a blog post, I’ve also created the Windows Azure Build & Deployment Sample on MSDN which contains all of the tools and sample projects described in this post.

Before we go into details on how the build and deployment process works, let’s look at how Windows Azure SDK 1.6 manages credentials and publishing profiles:

The Visual Studio “Publish Windows Azure Application” dialog contains a link to a special page on the Windows Azure Portal that allows you to download a .publishsettings file. This file contains all of the details of your subscription(s), including subscription IDs and certificates.
The same “Publish Windows Azure Application” dialog allows you to import the .publishsettings file, which results in the certificate being installed on your local machine, and the subscription details imported into a Visual Studio file called Windows Azure Connections.xml (this lives in %UserProfile%\Documents\Visual Studio 2010\Settings). Note that after you import the .publishsettings file you should delete it (or at least protect it) as it contains the full certificate and private key that grants access to your Windows Azure subscription.
When you are ready to publish your Windows Azure application, you can create a new "publish profile” or use an existing one. A publish profile is saved in your Windows Azure project with a .azurePubxml extension, and contains various details such as your hosted service name, storage account name and deployment slot. The .azurePubxml file doesn’t contain your subscription details, but it does list the subscription name that must correspond to an entry in your Windows Azure Connections.xml file.

In updating my scripts for automated build and deployment on a build server, I wanted to leverage as much as this as possible, but I needed to build some tools that mirror some of the steps done by Visual Studio’s “Publish Windows Azure Application” dialog, since you may not have Visual Studio installed on your build server.

The build and deployment solution contains the following components:

The AzureDeploy.targets file, which is installed on your build server to tell MSBuild to package and deploy your solution to Windows Azure
The ImportPublishSettings tool, to import a .publishsettings file onto a build server
The AzureDeploy.ps1 PowerShell script, which also depends on a helper library called AzurePublishHelpers
A TFS Build Definition that passes properties to MSBuild to initiate the build and deployment process.

The following diagram shows how all the components and files come together, and I’ll describe the details of each below.

The AzureDeploy.targets file

In my previous post on this topic, I showed you how you can edit your .ccproj project file to define additional targets used in the build process. This approach is still an option, but this time I’ve change my approach by creating a custom MSBuild .targets file which is installed on the build server. This is generally a better option as you don’t need to hand-edit .ccproj files, the custom targets run only on the build server (not on development workstations), and the targets can be reused for multiple Windows Azure projects.

The code included in my AzureDeploy.targets file is shown below. This file needs to be copied to your build server to C:\Program Files\MSBuild\Microsoft\VisualStudio\v10.0\Windows Azure Tools\1.6\ImportAfter, and it will be automatically referenced by the main Windows Azure targets file.

<Project ToolsVersion="4.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <PropertyGroup>
    <PackageName>$(AssemblyName).cspkg</PackageName>
    <PackageForComputeEmulator>true</PackageForComputeEmulator>
  </PropertyGroup>
  <Target Name="AzureDeploy" AfterTargets="Build" DependsOnTargets="Publish" Condition="$(AzurePublishProfile)!=''">
    <Message Text="Executing target AzureDeploy from AzureDeploy.targets file"/>
    <Exec WorkingDirectory="$(MSBuildProjectDirectory)" 
         Command="$(windir)\system32\WindowsPowerShell\v1.0\powershell.exe -f c:\builds\AzureDeploy.ps1 $(PublishDir) $(PackageName) &quot;Profiles\$(AzurePublishProfile)&quot;" />
  </Target>
</Project>

The purpose of this code is to tell MSBuild to package the project for Windows Azure (achieved with the dependency on the Windows Azure SDK’s Publish target) and then call a PowerShell script (you need to change the path depending on how you set up your build server). Note that this target only runs when the AzurePublishProfile MSBuild property is set, which we’ll do later on when we set up the TFS build definition.

Note that you may want to make some other customisations in a .targets or .ccproj file, for example to transform configuration files. I haven’t described this in this post, but there is some information on this in my previous post on this topic.

The ImportPublishSettings tool

The ImportPublishSettings tool (available from the Windows Azure Build & Deployment Sample) can be used to import Windows Azure credentials from a .publishsettings file into your build server. It has been designed to operate in the exact same way as the Visual Studio “Publish Windows Azure Application” dialog, so if you have Visual Studio installed on your build server you can use that instead of this tool. Whichever tool you use, this is a one-time process that is completed when you first set up your build process.

This is a simple command line tool which takes 3 parameters (only one of which is required):

publishSetingsFilename: the .publishsettings file to import
certStoreLocation (optional): the certificate store to which the certificate should be imported. Possible values are CurrentUser (the default) or LocalMachine. You should use CurrentUser if your build process is running as a user account, or LocalMachine if you are running under a system account such as NETWORK SERVICE.
connectionsFileName (optional): the filename in which the imported settings should be stored. This defaults to “%UserProfile%\Documents\Visual Studio 2010\Settings\Windows Azure Connections.xml”. You may want to change this if your build process is running under a system account such as NETWORK SERVICE.

If you run the tool under the same account used for your build process, you shouldn’t need to anything more. However if you run it as a different user (for example, you run the script as yourself but your build process runs under NETWORK SERVICE), you will need to open the MMC Certificates snap-in and grant permissions for the certificate private key to the build process’s account.

The AzureDeploy.ps1 PowerShell Script

The AzureDeploy.ps1 PowerShell script is responsible for taking a packaged Windows Azure application and deploying it to the cloud. The sample implementation included in the sample project is pretty simple, and you may want to extend it to perform additional steps such as installing certificates, creating storage accounts, running build verification tests or swapping staging and production slots. There are also a couple of things which you will need to customise depending on if you’re running as a normal user account or a system account. Still, hopefully this script is a useful starting point for your build and deployment process.

The script takes three parameters, which will normally be passed to it by the TFS build process (but you can pass them in yourself for test purposes):

BuildPath: The folder containing the Windows Azure project on the build server, for example “C:\Builds\1\ProjectName\BuildDefinitionName\Sources\AzureProjectName”.
PackageName: The unqualified name for the Windows Azure .cspkg name, for example AzureProjectName.cspkg
PublishProfile: The path to the .azurepubxml file that should be used for deploying the solution. Note that only some of the properties in this file are used for deployment, such as the subscription name, hosted service name, storage account name and deployment slot. Other settings in this file, such as EnableIntelliTrace, are not currently used by the sample scripts.

The deployment script depends on a helper library called AzurePublishHelpers.dll (which is also used by the ImportPublishSettings tool), which knows how to read and write from the various files used in the solution. In order to make this library available to PowerShell you will need to install it as a PowerShell module. To this, first open the folder for PowerShell modules, which is “C:\Windows\System32\WindowsPowerShell\v1.0\Modules\AzurePublishHelpers” (replace System32 with SysWOW64 if you’re running your build as 32-bit on a 64-bit system). Then create a folder called AzurePublishHelpers and copy in the AzurePublishHelpers.dll file.

The TFS Build Definition

The final piece of the puzzle is setting up one or more Build Definitions. I’m using Team Foundation Server for this, but if you’re using a different ALM toolset you should be able to accomplish something similar.

You can configure your Build Definitions however you want, for example to use different XAML workflows, different settings for running tests, versioning assemblies, etc. You should have at least one Build Definition for each Windows Azure environment you want to deploy to, for example Test, UAT or Production. To configure a Build Definition to deploy to Windows Azure, you’ll need to choose the Process tab and enter the name of your chosen Publish Profile (.azurepubxml file) in the MSBuild Arguments property as shown in the screenshot below:

Conclusion

I hope this post and the accompanying tools and samples help you automate most (if not all) of your Windows Azure build and deployment process. I’ll try to keep the post up-to-date as the platform continues to evolve. If you have questions or comments, please feel free to post here or on the Windows Azure Build & Deployment Sample page.