I got a note today from Rob Tiffany, another one of our brilliant Microsoft Architects. (I saw another one, because I like to think I am as brilliant… [Note: I do work for Microsoft too.])

Here’s Rob’s note:

Sync Framework is an amazingly effective, provider-based framework for syncing just about anything, and in fact, is the basis for SQL Azure Data Sync.
This release to Open Source will really benefit the community at large, and gives an opportunity for deep .NET developers to be involved with some great bits.

The profound effects of the Consumerization of IT (CoIT) is blurring the lines between consumers and the enterprise. The fact that virtually every type of mobile device is now a candidate to make employees productive means that cross-platform, enabling technologies are a must. Luckily, Microsoft has brought the power to synchronize data with either SQL Server on-premise or SQL Azure in the cloud to the world of mobility. If you’ve ever synched the music on your iPhone with iTunes, the calendar on your Android device with Gmail, or the Outlook email on your Windows Phone with Exchange, then you understand the importance of sync. In my experience architecting and building enterprise mobile apps for the world’s largest organizations over the last decade, data sync has always been a critical ingredient.

The new Sync Framework Toolkit found on MSDN builds on the existing Sync Framework 2.1′s ability to create disconnected applications, making it easier to expose data for synchronization to apps running on any client platform. Where Sync Framework 2.1 required clients to be based on Windows, this free toolkit allows other Microsoft platforms to be used for offline clients such as Silverlight, Windows Phone 7, Windows Mobile, Windows Embedded Handheld, and new Windows Slates. Additionally, non-Microsoft platforms such as iPhones, iPads, Android phones and tablets, Blackberries and browsers supporting HTML5 are all first-class sync citizens. The secret is that we no longer require the installation of the Sync Framework runtime on client devices. When coupled with use of an open protocol like OData for data transport, no platform or programming language is prevented from synchronizing data with our on-premise and cloud databases. When the data arrives on your device, you can serialize it as JSON, or insert it into SQL Server Compact or SQLite depending on your platform preferences.

The Sync Framework Toolkit provides all the features enabled by the Sync Framework 4.0 October 2010 CTP. We are releasing the toolkit as source code samples on MSDN with the source code utilizing Sync Framework 2.1. Source code provides the flexibility to customize or extend the capabilities we have provided to suit your specific requirements. The client-side source code in the package is released under the Apache 2.0 license and the server-side source code under the MS-LPL license. The Sync Framework 2.1 is fully supported by Microsoft and the mobile-enabling source code is yours to use, build upon, and support for the apps you create.

Now some of you might be wondering why you would use a sync technology to move data rather than SOAP or REST web services. The reason has to do with performance and bandwidth efficiency. Using SOA, one would retrieve all the data needed to the device in order to see what has changed in SQL Server. The same goes for uploading data. Using the Sync Framework Toolkit, only the changes, or deltas, are transmitted over the air. The boosts performance and reduces bandwidth usage which saves time and money in a world of congested mobile data networks with capped mobile data plans. You also get a feature called batching, which breaks up the data sent over wireless networks into manageable pieces. This not only prevents you from blowing out your limited bandwidth, but it also keeps you from using too much RAM memory both on the server and your memory-constrained mobile device. When combined with conflict resolution and advanced filtering, I’m sold!

I think you’ll find the Sync Framework Toolkit to be an immensely valuable component of your MEAP solutions for the enterprise as well as the ones you build for consumers.

Enjoy!

Eric

By now you should know that yesterday we went live with the Commercial Availability and release of Windows Azure. (WOO_HOOT!)

One of the great things is that MSDN subscribers get to utilize Windows Azure for their projects with some great offers.
(The offers are outlined elsewhere on my blog, or you can get up to speed here.)

For those of you with an MSDN subscription who’d like to sign up now,

grab your credit card, and follow along with this play-by-play on how to get running:

Step 1:

Login into your MSDN account, and go to the “My Account” tab.

Then click on the link for “Windows Azure Platform” to signup on the MOCP Portal. 

(You can’t repeat this step once you are signed up, so you most likely won’t look back.)

Step 2:

You should find your details on the MOCP portal.

(If you haven’t ever been here, you may need to setup your account address and details for the MOCP portal.)

You should see your shopping cart filled with your Windows Azure Platform MSDN offering.

Acknowledge your rate plan and purchase, and then click “Checkout >”.

Step 3:

You now will need your credit card, or a PO if you qualify.  I’m sticking with the card route.

Why a credit card?  Well, if you go over your MSDN allotment, you will need to ante-up and pay for the services.

Step 4:

Enter your card details, and upon successful card authorization, you’ll be able to continue.

(Note: I’ve changed details throughout the screenshots here, so this credit card number isn’t going to help you.)

Step 5:

Read and comply with the Online Subscription Agreement for your particular account.

You will need to check the box, enter name and title, and click “I Accept”

Step 6:

Review your order, and confirm accordingly.

When your complete, click “Confirm Order”

Step 7:

Jot down your confirmation number in case anything bad happens.

Add the email to your safe senders if necessary.

Click “Proceed to activation…”

Step  8:

Enter a subscription name.  This will be used as your Project Name on the Windows Azure Portal, so choose wisely.

When you're finished, click “Next”.

Step 9:

Enter details for your Service Administrator.  Ensure that the details have been updated successfully.

When your completed, click “Next”.

Step 10:

Review your Service Activation Summary.  Note your Service Provisioning Info.

When your satisfied, click “Finish” and your service will begin the provisioning process.

Step 11:

THANK YOU, for choosing to use the Windows Azure Platform! 

You can now click on “Close” and you will be redirected to the MOCP Subscription Management screen.

Step 12:

You should now see your Windows Azure Platform MSDN subscription, with your service active.

You can elect to edit service details, opt-out of auto renew, or edit billing information from the MOCP site.

Please bookmark the mocp.microsoftonline.com site for future use.

Step 13:

You can now sign into the Developer Portal and you should see your created project amongst your project lists.

Your ready to go, and can click on that project and begin to create services, use storage accounts, etc.

Step 14:

Tell your friends how easy, manageable, extensible and scalable the Windows Azure Platform really is.

Build awesome applications with the Windows Azure, and let me know about them!

Enjoy!

Happy Holidays!

Well, my vacation time is quickly coming to an end, and soon it will be the New Year!

I'm consistently asked what the default date/time sttings are when deploying to Windows Azure.  It may not be obvious to developers new to the platform. 
All Windows Azure servers utilize UTC time, and 'en-US' locale settings out-of-box.  It is recommended to maintain applications that can convert local time (UTC) into the desired time/date formats.

You can see this by using remote desktop to "peer into and inspect" settings on your Windows Azure deployments. You will find that the server localization settings (Control Panel) are indeed set to "United States" for location, and "English (United States)" for Language Settings.
For date/time settings, "GMT Greenwich Mean Time : Dublin, Edinburgh, Lisbon, London" is the default selection, no matter which data center you are deployed to as well.

The Fabric Controller maintains operating system time synchronization for the system when roles are first booted. You could, with the latest SDKs. utilize administrative access startup scripts to change the localization settings (including local server time), but it is not recommended you do so.  Doing so will introduce instability and the fabric controller may determine your role is out of sync.  You will likely end up cycling your roles as fabric attempts to bring them to goal state in such a case.  A better strategy is to write our applications to be aware that they actually run with UTC and default US CultureInfo settings.

For more info on manipulating Date/Time settings with .NET Framework 4.0, refer here:
http://msdn.microsoft.com/en-us/library/5hh873ya.aspx

Another huge shortcut tip:

Using Remote Desktop is extremely helpful to diagnose issues with Windows Azure, including ensuring an application you might want to script during startup, will actually install and run on the virtualized role images.   You can Edit your Remote Desktop connection files (.rdp) by right-clicking the file saved to your desktop, and selecting Edit.  This will enable you to map your local drive to the VM, so you can seamlessly transfer files to and from the role instance you are connected to.

I'm consistently using PowerShell, or Snippet Compiler (created by Jeff Key, and available at http://www.sliver.com), to output settings, or programatically manupulate objects.
PowerShell is already on the Windows Azure role instances for your enjoyment, and Snippet Compiler take a few seconds to copy & paste into the C:\Applications folder on the VM to "play with".  

Season's Greetings! 
Eric

o Data centers we have built from the ground up: Those are Quincy, Washington and San Antonio, Texas.

o Data centers currently under construction: Those are Dublin, Ireland and Chicago, Illinois.

· How we choose future data center locations: When choosing future data center sites, we take into account over 35 weighted criteria in our “global heatmap,” including close proximity to an ample power source and fiber optic networks, a large pool of skilled labor, affordable energy rates, tax, and to determine the long-term viability of each site.

· Broad financial figures on data centers: We have only announced our expenditures on the data centers listed above that we have built from the ground up, and those figures aren’t necessarily indicative of all of our data center investments.

· The Microsoft data center strategy: Our strategy is focused on smart growth, measurable efficiency and global trust.

o Smart Growth: Smart growth means judiciously adding data centers and capacity within those data centers when and where we need them based on the demands of the customers of our online services. One of the ways we do this is by using new technology that enables us to scale quickly as needed. The technology we’re focused on for flexible growth right now is containers.

o Measurable Efficiency: When you’re managing an infrastructure of hundreds of thousands of servers it is essential that you run it efficiently. The first thing you need to do in this regard is to constantly measure, and then with that data make sure you make the right power and performance tradeoffs as you focus on uptime, availability, performance, and power usage. Everything has to be manageable – from the supply chain to the servers to the network -- and it has to be built to operate with great processes every day. Furthermore, you need to apply technology for efficiency both in architecting services to run efficiently and in applying breakthrough technologies such as virtualization.

o Global Trust: When customers think about trust, they tend to think security and privacy. But Microsoft thinks of trust more broadly: Our goal is to keep our customers’ data and our sites secure from attack and also safeguard our customers’ personally identifiable information. We must be available and reliable when customers need us, providing performance they can count on. And we need to deliver our products in the language they want, with special market considerations when appropriate – (e.g. mobile).

· Sustainability efforts: We are focused on integrating the latest innovations in technology and building techniques to maximize efficiencies and utilize alternative energy sources when and where possible to improve our Power Usage Effectiveness (PUE) and overall impact on the environment. As updated technology and techniques are developed, we make constant adjustments to our existing and new facilities, as well as our processes, to continue to improve our effectiveness.

· Amount of power we use: When asked, Microsoft’s power consumption is between 30 and 50 percent better than the global industry average for traditional facilities. At Microsoft, we believe that it is important to track and monitor an average PUE across all of our data centers (no matter how small or how old) in order to truly understand how well our data center operations are under control, and to allow us to make the right business decisions.  Our current annual global average PUE is 1.60.

  · Azure Services Platform in our data centers: Windows Azure is an Internet-scale cloud services platform (with an integrated development, service hosting, and management environment) maintained in Microsoft data centers. This environment includes a robust and efficient core of compute and simple storage capabilities and support for a rich variety of development tools and protocols. Windows Azure provides developers with on-demand compute and storage to host, scale, and to manage web applications in Microsoft data centers.

· Windows Server 2008 in our data centers: Windows Server 2008 is the operating system of choice for both on-premises data center applications and hosting applications in the cloud. Windows Azure is built on Windows Server 2008.

· Windows relevancy to the new Azure Services Platform for cloud computing: Windows Azure is built on Windows Server 2008. For years, Windows has connected thousands of peripherals and partners to the PC ecosystem. Now, as the power of devices increases and the ubiquity of the web unfolds, Microsoft is delivering on its Software + Services strategy by building platforms, tools, infrastructure, and solutions to enable new kinds of applications and services that extend from the server, to the data center, to the cloud – and from the browser, to the PC, to the phone and beyond. Windows Server 2008 is the operating system of choice for both on-premises data center applications and hosting applications in the cloud.

· SQL Data Services (SDS) in our data centers: Customers can use SQL Data Services to store virtually any amount of data in the cloud. SDS is stored in large storage clusters in Microsoft data centers located across North America. Microsoft also plans to offer the service from international locations such as Europe and Asia. Users can group their data into authorities, which are affiliated with specific data centers and therefore provide control over the geographic location of the data for reduced latency.

· Microsoft Online Services in our data centers: Delivers enterprise-class software as subscription services to businesses of all sizes hosted by Microsoft and sold with partners. This includes Microsoft Exchange Online, Microsoft Office SharePoint Online, Microsoft Office Live Meeting, Exchange Hosted Services, Microsoft Dynamics CRM Online, and soon – Microsoft Office Communications Online.

· Generation 4 Modular Data Center Vision: The concept behind the modular data center builds on the innovation deployed at our Chicago data center, which will house shipping containers packed with up to 2,500 servers each.  Container facilities help ensure that server capacity meets customer demands, reduce the likelihood of overcapacity, and reduce the time to build a data center from 24 to 12 months. Our “Gen 4” modular data centers will take the flexibility of containerized servers and apply it across the entire facility, which will be composed of modular “building blocks” of prefabricated mechanical, electrical and security components, in addition to containerized servers. These facilities can be built incrementally as capacity grows and deployed in only three to six months, reducing capital costs by 20% to 40%. Modular data centers will be scalable and sustainable, ensuring that our global footprint is only as big as capacity demands that it be. All information that can be shared publicly is posted on the GFS data center team blog posted here that outlines Microsoft’s vision of the foundation of our cloud data center  infrastructure in the next five years. A cool video is posted here.

· Security and Privacy: Microsoft endeavors to protect the privacy and security of our customers.  We comply with all applicable privacy laws, and we follow the privacy practices outlined in the Microsoft Online Privacy Statement.  Microsoft is committed to protecting the security of your personal information. We use a variety of security technologies and procedures to help protect your personal information from unauthorized access, use, or disclosure. For example, we store customer information on computer systems with limited access, which are located in controlled facilities. When we transmit highly confidential information over the Internet, we protect it through the use of encryption. All information that can be shared is posted on the GFS web site’s security page here.

So, as you can see, it’s all about Data.  :)

Very recently I had a customer who approached me with a concern, and their problem resembles some common architectural guidance we have been talking with other customers about.  I am hopeful this post fills a void in one or two areas for many of you using Windows Azure.

The scenarios was that this customer has content that that cache from a third-party CDN service. The customer is not using the Windows Azure Storage CDN capabilities yet.  The content cached in the CDN is used by their ASP.NET web application on Windows Azure, and is held within the role’s ASP.NET Cache via the System.Web.Caching.CacheDependency class, which usually picks up static content from the root of the website.  Periodically, the customer would like to replace the file, invalidating the current ASP.NET cache content, and having it pickup the new (non-stale) content.  Remember, caching works great for content that is static, and does not change often, but there are times you might need to change the content.

Normally, with a standard ASP.NET server, you would just perform a file replacement at the root website file system location and be done, as the CacheDependency class would detect the changes and perform the needful cache update.   However, with Azure, you don’t have write access to the directory, and you would need to re-deploy your service to get the same effect, which could mean service degradation even with the proper upgrade domain configuration.  Remember by default for a subscription, you are limited to 20 weighted role instances throughout all your subscriptions services, unless you have called support to increase your subscription quota. You also by default have an upgradeDomainCount of 5, unless you modify this attribute on your ServiceDefinition  element in your ServiceDefinition.csdef file.  So if you had 20 instances of small VM’s running, across 5 UpgradeDomains, doing a in-place upgrade of a service will leave you with 4 small VMs unavailable while the upgrade process (Fabric Controller) does it thing.

So to solve the issue, the idea is two-fold.  First, move the file that the CacheDependency class depends upon to provisioned LocalStorage on the VM itself, which will give you the ability to update the file without performing a redeployment or upgrade. (LocalStorage is writeable, and makes for an excellent location for scratch files, temporary caches, configuration files and other non-durable (transient) storage needs.  Second, provision the file that is in that LocalStorage from a durable storage location, in our case, Windows Azure Blob Storage, and implement a timer that periodically replaces that file.  This has the benefit of allowing you to update one location, and have multiple instances pickup the changes, without having to change files manually on every instance, or redeploy your application packages.  There is some “buyer beware” here, in that you do pay for storage of the file, just like you would for deployment packages stored in Azure Blob Storage, but the cost is minimal compared with the benefits.  Additionally, you should ensure your storage account and roles are within the same datacenter geographic locations, or add calculations to include the bandwidth costs across different datacenters for the file updates.  Again, this is generally a minimal amount compared to the costs to geo-replicate files on your own.

Since I just returned from a (much needed and enjoyed) vacation to Canada, I didn’t have time to write a sample for this.  However, I turned to my good buddy and co-worker at Microsoft, Balakrishna Mishro (Wipro), who was able to drum up a quick code sample while I was in yet another meeting.

The code sample, which you can download (LocalStorageSample.zip - 166kb) and use at your own risk, was created by simply creating a new ASP.NET web role, with a new BackgroundWorker class that is instantiated on Application_Start, via the Global.asax codebehind, passing the Cache property of the HTTPApplication Context in.

The BackgroundWorker class does all the “heavy” lifting, and we use the StorageManagement API to retrieve the file via the CloudBlob.DownloadToFile method on a simple timer. (Note: Added some line breaks here for readability.) :

   1:  using System;

   2:  using System.Collections.Generic;

   3:  using System.Linq;

   4:  using System.Web;

   5:  using System.Threading;

   6:  using System.Timers;

   7:  using System.Configuration;

   8:  using Microsoft.WindowsAzure; 

   9:  using Microsoft.WindowsAzure.StorageClient;

  10:  using Microsoft.WindowsAzure.ServiceRuntime;

  11:  using System.Web.Caching;

  12:  using System.IO;

  13:   

  14:  namespace LocalStorageWebRole

  15:  {

  16:      public class BackgroundWorker

  17:      {

  18:          System.Timers.Timer timer = null;

  19:          Cache cache = null;

  20:   

  21:          public BackgroundWorker(Cache application)

  22:          {

  23:              timer = new System.Timers.Timer(1);

  24:              timer.Interval = 1;

  25:              timer.Elapsed += new ElapsedEventHandler(timer_Elapsed);

  26:              timer.Enabled = true;

  27:              this.cache = application;

  28:          }

  29:   

  30:          void timer_Elapsed(object sender, ElapsedEventArgs e)

  31:          {

  32:              timer.Enabled = false;

  33:   

  34:              timer.Interval = double.Parse(

RoleEnvironment.GetConfigurationSettingValue("TimeInterval"));

  35:   

  36:              string blobUrl = ConfigurationManager.AppSettings["BlobUrl"];

  37:   

  38:   

  39:              CloudStorageAccount cloudStorageAccount = 
CloudStorageAccount.FromConfigurationSetting("DataConnectionString");

  40:              CloudBlobClient cloudBlobClient = cloudStorageAccount.CreateCloudBlobClient();

  41:   

  42:              try

  43:              {

  44:                  CloudBlockBlob blob = new CloudBlockBlob(blobUrl, cloudBlobClient);

  45:   

  46:   

  47:                  int startIndex = blobUrl.LastIndexOf("/".ToCharArray()[0]);

  48:                  string fileName = string.Format("{0}{1}", 
RoleEnvironment.GetLocalResource("InstanceDriveCache").RootPath, blobUrl.Substring(startIndex));

  49:                  string content = string.Empty;

  50:   

  51:                  if (cache.Get("ETag") == null)

  52:                  {

  53:                      blob.FetchAttributes();

  54:                      blob.DownloadToFile(fileName);

  55:                      cache["ETag"] = blob.Properties.ETag;

  56:   

  57:                      using (TextReader tr = new StreamReader(fileName))

  58:                      {

  59:                          content = tr.ReadToEnd();

  60:                      }

  61:   

  62:                      //cache.Insert("Content", 
content, 
new System.Web.Caching.CacheDependency(fileName));

  63:                      cache.Add("Content", 
content, 
new System.Web.Caching.CacheDependency(fileName),

Cache.NoAbsoluteExpiration, 
Cache.NoSlidingExpiration, 
CacheItemPriority.High, 
OnRemove);

  65:                  }

  66:                  else

  67:                  {

  68:                      blob.FetchAttributes();

  69:                      if (!cache["ETag"].ToString().Equals(blob.Properties.ETag))

  70:                      {

  71:                          blob.DownloadToFile(string.Format("{0}{1}", 
RoleEnvironment.GetLocalResource("InstanceDriveCache").RootPath, blobUrl.Substring(startIndex)));

  72:                          cache["ETag"] = blob.Properties.ETag;

  73:                      }

  74:                  }

  75:              }

  76:              catch (StorageClientException ex)

  77:              {

  78:                  //Log the error

  79:              }

  80:              timer.Enabled = true;

  81:          }

  82:   

  83:          private void OnRemove(string key, object value, CacheItemRemovedReason reason)

  84:          {

  85:              string blobUrl = ConfigurationManager.AppSettings["BlobUrl"];

  86:              int startIndex = blobUrl.LastIndexOf("/".ToCharArray()[0]);

  87:              string fileName = string.Format("{0}{1}", 
RoleEnvironment.GetLocalResource("InstanceDriveCache").RootPath, blobUrl.Substring(startIndex));

  88:   

  89:              using (TextReader tr = new StreamReader(fileName))

  90:              {

  91:                  string content = tr.ReadToEnd();

  92:                  cache.Add("Content", 
content, 
new System.Web.Caching.CacheDependency(fileName),

Cache.NoAbsoluteExpiration, 
Cache.NoSlidingExpiration, 
CacheItemPriority.High, 
OnRemove);

  94:              }

  95:   

  96:              

  97:          }

  98:   

  99:      }

 100:  }

This code can be leveraged as a starting point to accomplish many scenarios with your Windows Azure worker roles, for example dynamically replacing PHP configuration files to change PHP server configurations dynamically, reconfigure Tomcat Apache servers, add/remove website directories, etc.  Your creativity is the limit. 

Let us know via comments how you have approached such situations. I’d love to hear some more successful scenarios.

-Eric