Supporting Billions of entities/rows for Mobile – Android Series - Part 6–Reading and Writing to Windows Azure (Cloud-based) Tables using standard HTTP and Fiddler

 

Supporting Billions of entities/rows for Mobile – Android Series - Part 6–Reading and Writing to Windows Azure (Cloud-based) Tables using standard HTTP and Fiddler

  • Comments 1
 
Past Posts in this series
 
Post Topic Link
Part 1 - Why Scale Matters http://blogs.msdn.com/b/brunoterkaly/archive/2011/09/27/supporting-billions-of-entities-rows-for-mobile-android-series-part-1-why-scale-matters.aspx
Part 2 - What are some high level cloud offerings? http://blogs.msdn.com/b/brunoterkaly/archive/2011/09/27/supporting-billions-of-entities-rows-for-mobile-android-series-part-2-what-are-some-high-level-cloud-offerings.aspx
Part 3–Architecture and Data Options http://blogs.msdn.com/b/brunoterkaly/archive/2011/09/28/supporting-billions-of-entities-rows-for-mobile-android-series-part-3-architecture-and-data-options.aspx
Part 4–Building a Cloud-based RESTful service for our Android, iOS, and Windows Phone 7 Clients http://blogs.msdn.com/b/brunoterkaly/archive/2011/09/28/supporting-billions-of-entities-rows-for-mobile-android-series-part-4-building-a-cloud-based-restful-service-for-our-android-ios-and-windows-phone-7-clients.aspx
Part 5–Using the Portal and Setting up your Azure Account (Microsoft Cloud) http://blogs.msdn.com/b/brunoterkaly/archive/2011/10/05/supporting-billions-of-entities-rows-for-mobile-android-series-part-5-using-the-portal-and-setting-up-your-azure-account-microsoft-cloud.aspx
You will need to download the free Azure trial.                     
   
 
The last post was about …
  In the last post I illustrated how to create a RESTful service using .NET technologies. Here is the diagram from the last post:

5tw420z0
 
This post is to illustrate that you can go directly to the Storage Service with HTTP bypassing .NET
  Although it is not recommended for this particular application, it is possible communicate directly with the Storage Service. Direct communication with Azure tables requires some advanced techniques, working with request headers and request bodies. Later in this post you will learn what makes this technique somewhat challenging.


gl3nhvsu

We need to able to:
1. Create a table
2. Insert into a table
3. Query a table

We need do this using all standard http without any .NET whatsoever. We will use Fiddler to get this done. We can also use the Azure Storage Explorer.
 
Azure Storage Explorer
ase4_blobs.png
 

Azure Storage Explorer is a useful GUI tool for inspecting and altering the data in your Windows Azure Storage storage projects including the logs of your cloud-hosted applications.

All 3 types of cloud storage can be viewed and edited: blobs, queues, and tables.

Description Link
Home page on CodePlex for Azure Storage Explorer http://azurestorageexplorer.codeplex.com/
   

Login to http://windows.azure.portal and choose Hosted Services, Storage Accounts & CDN.

0vf2k02l

Now you need your storage access keys.

4f5px5yl

Copy the storage access keys to the clipboard.

zmpvyhy3

Now are are ready to configure the Azure Storage Explorer.

bp0vkom1

As expected, there are no tables and no data. That is the next step.

image
 
Adding a Table Object with Storage Explorer
  Follow these steps to add a table object:
1. Click Tables
2. Click New
3. Enter a table name


usk0gj0q
 
Adding Entities
  Each table can have many Entities. You can think of an entity as a row in a table. But in reality an entity is a collection of name-value pairs.  Notice under the Storage Type panel you can see the Tables button. Click tables before following the rest of the steps.

ucqytfub
Notice that a PartitionKey and RowKey are required columns.
 
Querying the table
  You can also hit the query button to see the entities in the table. Later, when we create the Android client, you will be able to use LINQ to query the entities.
image
 
Using http to interact with Azure Tables
  Http is the most generic way to interact with Azure tables. I will be using Fiddler add new records and to query the data in Azure tables.
3szv5gtp

You can download Fiddler here:

Description Link
Download Fiddler http://www.fiddler2.com/fiddler2/
   

Authentication

The Table service requires that each request be authenticated. Both Shared Key and Shared Key Lite authentication are supported.

More information can be found here
Description Link
Authentication Information at Microsoft http://msdn.microsoft.com/en-us/library/windowsazure/dd179428.aspx
   

 
Entering an HTTP Request in Fiddler
  Step 1 -  Click on Request Builder
Step 2 -  Enter the URL and select GET
Step 3 -  Enter the Request Header (you cannot do this without my tool explained below. See “Generating the Request Header” section below).
Step 4 -  Hit Execute

gz5fsztr 

The Request Header is going to require you to generate a Shared Key Lite.
 
Viewing the Result in Fiddler
  After hitting Execute in Step 4 above, results will appear.

sgaijwcz
 
Viewing the results (see above)
  Step 1 – Double click the result item in the upper left window. This brings up results window in the lower right pane of Fiddler

Step 2 – Click on TextView to see the results in AtomPub format.

Step 3 – View the data. Notice you can see Bruno and Haybusa that we added previously with the Azure Storage Explorer.
 
Generating the Request Header
lbtlwmog
  If you look carefully, you will notice the SharedKeyLite authorization signature. This isn’t trivial to generate, since you need to compute a Hash-based Message Authentication Code (HMAC) using the SHA256 hash function.

Luckily I’ve done that for you.
 
A Simple Windows Forms Application
mhp4avet
  You are looking at a custom application that I wrote. I am providing the code below in case you want to do this yourself. You typically would use the ADO.NET Data Services library in C# or VB.

But if you want to interact with Azure Tables from non-.NET clients, you will need to master the techniques I am illustrating here. You can learn more at the link below.

Description Link
Azure REST-API http://msdn.microsoft.com/en-us/library/windowsazure/dd179423.aspx
   

The trick is understanding the code behind the Make Request button, which fills in all the boxes you see above. The ones to paste into fiddler are:

1. URI
2. HttpHeader
public partial class Form1 : Form
{
    const string uriTemplate = "http://{0}.table.core.windows.net/{1}";
    const string uriTemplate2 = "http://{0}.table.core.windows.net/{1}/{2}";
    string _sharedKey = "== your shared key from the portal goes here==";
    string _accountName = "fastmotorcycle";
    string _tableName = "fastbikes";

    public Form1()
    {
        InitializeComponent();
    }


    private void cmdMakeHeader_Click(object sender, EventArgs e)
    {
        var account = _accountName;
        var sharedKey = Convert.FromBase64String(_sharedKey);


        this.txtAccountName.Text = account;
        this.txtSharedKey.Text = _sharedKey;
        this.txtTableName.Text = _tableName;


        // Fill datetime field
        string datetime = DateTime.UtcNow.ToString("R", CultureInfo.InvariantCulture);
        this.txtDate.Text = datetime;
            
        // Fill uri field
        string uri = string.Format(uriTemplate, _accountName, _tableName);
        this.txtURI.Text = uri;
            
        // Create request object
        WebRequest request = WebRequest.Create(uri);

        // Add date header
        request.ContentLength = 0;
        request.Headers.Add("x-ms-date", datetime);

            
        // Get resource
        var resource = request.RequestUri.PathAndQuery;
    
        // Sign date, account and uri
        string stringToSign = string.Format("{0}\n/{1}{2}",
                request.Headers["x-ms-date"],
                account,
                resource
            );

        this.txtMessageToSign.Text = stringToSign;


        // Sign the string using shared key
        var hasher = new HMACSHA256(sharedKey);
        string signedSignature = Convert.ToBase64String(hasher.ComputeHash(Encoding.UTF8.GetBytes(stringToSign)));
        this.txtSignedMessage.Text = signedSignature;

        // Add authorization header
        string authorizationHeader = string.Format("{0} {1}:{2}", "SharedKeyLite", account, signedSignature);
        request.Headers.Add("Authorization", authorizationHeader);


        // Show the signed message
        this.txtSignedMessage.Text = request.Headers.Get(0).ToString() + "\r\n";
        this.txtSignedMessage.Text += request.Headers.Get(1).ToString() + "\r\n";

        // Show the entire header
        this.txtHttpHeader.Text = "x-ms-date:" + request.Headers.Get(0).ToString() + "\r\n";
        this.txtHttpHeader.Text += "Authorization:" + request.Headers.Get(1).ToString() + "\r\n";
        this.txtHttpHeader.Text += "Content-Type:" + "application/atom+xml" + "\r\n";

        if (this.txtPartitionKey.Text == String.Empty ||
            this.txtRowKey.Text == String.Empty)
            return;

        this.txtResponseBody.Text = string.Format(Strings.response_header,
                                                    this.txtPartitionKey.Text,
                                                    this.txtRowKey.Text);




    }

    private void cmdClip2_Click(object sender, EventArgs e)
    {
        Clipboard.SetData(DataFormats.Text, (Object)this.txtResponseBody.Text);
    }

    private void cmdClip1_Click(object sender, EventArgs e)
    {
        Clipboard.SetData(DataFormats.Text, (Object)this.txtHttpHeader.Text);
    }



}
 
You can even add data using this technique
  Notice that this time I added a PartitionKey and RowKey in Steps 1 and 2 below.

cur33vw5

Now we need to copy the Response Body in addition to the previous fields. The XML below was generated by my tool. Notice the partitionkey and rowkey from the form above.

 

<?xml version="1.0" encoding="utf-8"?>
<entry xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" xmlns="http://www.w3.org/2005/Atom">
  <title />
  <author>
    <name />
  </author>
  <updated>2011-10-04T15:28:55.7259928Z</updated>
  <id />
  <content type="application/xml">
    <m:properties>
      <d:PartitionKey>Bruno</d:PartitionKey>
      <d:RowKey>GSXR1000</d:RowKey>
      <d:Timestamp m:type="Edm.DateTime">0001-01-01T00:00:00</d:Timestamp>
    </m:properties>
  </content>
</entry>


Notice the Make Header tool creates the 3 core pieces of information you will need to paste into Fiddler’s Request Builder:

1. URI
2. Request Header
3. Request Body

When adding date, you will need to do a POST. Here is a table that explains the HTTP verbs and how they relate to operations:

HTTP Verb Operation
GET Query data
POST Insert data
DELETE Delete data
PUT Update data
MERGE Merge data
   



1s3xztlh


 
In Fiddler do the following:
  Step 1 – Change from GET to POST
Step 2 – Paste in the new Request headers from the tool above
Step 3 – Paste in the Request Body from the tool above
Step 4 – Hit execute
Step 5 – Realize that you just added data to your table using a PURE http protocol

waslub2k

Notice that we have added the data. You can see the GSXR1000 data in Azure Storage Explorer.

image
 
Conclusions
  This post was important for several reasons. First, it introduced the Windows Azure Portal and addressed the Hosted Service and Storage Account and how they related to your RESTful service and to your infinitely scaled Azure Table Service Storage.

Windows Azure Portal Section Description
Hosted Service Where your RESTful service will reside and be available to mobile clients. This is the next blog post. We will migrate our RESTful project in Visual Studio to the Microsoft Cloud. Remember, not only is our data scalable, but so is our RESTful service.

Building the RESTful Service The blog link
Part 4–Building a Cloud-based RESTful service for our Android, iOS, and Windows Phone 7 Clients http://blogs.msdn.com/b/brunoterkaly/archive/2011/09/28/supporting-billions-of-entities-rows-for-mobile-android-series-part-4-building-a-cloud-based-restful-service-for-our-android-ios-and-windows-phone-7-clients.aspx
   

Storage Account Where our data is stored. Explored deeply in this post, using the Azure Storage Explorer and Fiddler with standard HTTP requests. In the next post we will show how this data is consumed by the RESTful service. Mobile clients will not access Azure tables directly. They will go through the RESTful Hosted Service.
   

Typically, you will access Azure Table Storage through the ADO.NET Data Services API, found inside the Azure SDK Download. But there may be situations where you want to access Azure table storage outside of a Windows environment, such as from Java or PHP or any other non-MS platform. Remember, Azure tables are incredibly scalable, cost-effective, and performant. And Azure tables are also available from a purely http world.

What Where
Free download link for the 90-day Azure trial. Click the photo to the right. lahxjo2d
   
        

 

  • Thanks for all the articles. I am new to Win Azure and Mobile domains. This was very helpful. Is there a link to download the Make Header win form app? Please share. I can be reached at prashant_ nerkar@hotmail.com

Page 1 of 1 (1 items)
Leave a Comment
  • Please add 5 and 6 and type the answer here:
  • Post