Walkthrough: Windows Azure Blob Storage (Nov 2009 and later)

Walkthrough: Windows Azure Blob Storage (Nov 2009 and later)

Rate This
  • Comments 27

Similar to the table storage walkthrough I posted last week, I updated this blog post for the Nov 2009/v1.0 and later release of the Windows Azure Tools.

This walkthrough covers what I found to be the simplest way to get a sample up and running on Windows Azure that uses the Blob Storage service. It is not trying to be comprehensive or trying to dive deep in the technology, it just serves as an introduction to how the Windows Azure Blob Storage Service works.

Please take the Quick Lap Around the Tools before doing this walkthrough.

Note: The code for this walkthrough is attached to this blog post.

After you have completed this walkthrough, you will have a Web Role that is a simple ASP.NET Web Application that shows a list of files that are stored and can be downloaded from Blob Storage. You can use the Web Role to add files to Blob storage and make them available in the list.

image

Blob Concepts

Each storage account has access to blob storage. For each account there can be 0..n containers. Each container contains the actual blobs, which is a raw byte array. Containers can be public or private. In public containers, the URLs to the blobs can be accessed over the internet while in a private container, only the account holder can access those blob URLs.

Each Blob can have a set of metadata set as a NameValueCollection of strings.

image

Creating the Cloud Service Project

1. Start Visual Studio as an administrator (Screen shots below are from VS 2008, VS 2010 is also supported)

2. Create a new project: File | New Project

3. Under the Visual C# node (VB is also supported), select the “Cloud Service” project type then select the “Windows Azure Cloud Service” project template. Set the name to be “SimpleBlobSample”. Hit OK to continue.

image

This will bring up a dialog to add Roles to the Cloud Service.

4. Add an ASP.NET Web Role to the Cloud Service, we’ll use the default name of “WebRole1”.  Hit OK.

image

Solution explorer should look as follows:

image

We’ll now cover the implementation, which can be broken up into 5 different parts:

  1. Implementing the UI
  2. Connecting to Windows Azure storage
  3. Adding blobs
  4. Enumerating existing blobs
  5. Deleting blobs

Implementing the UI

5. Next open up Default.aspx and add the code for the UI. The UI consists of:

  • GridView at the top
  • Label and FileUpload control
  • 2 Label and TextBox pairs (File Name and Submitter)
  • Field validators to ensure that all of the fields are filled out before the file is uploaded.

Add the following between the template generated <div></div> elements:

<asp:GridView ID="fileView" AutoGenerateColumns="false" DataKeyNames="FileUri" runat="server"
    OnRowCommand="RowCommandHandler">
    <Columns>
        <asp:ButtonField Text="Delete" CommandName="DeleteItem" />
        <asp:HyperLinkField HeaderText="Link" DataTextField="FileName" DataNavigateUrlFields="FileUri" />
        <asp:BoundField DataField="Submitter" HeaderText="Submitted by" />
    </Columns>
</asp:GridView>
<br />
<asp:Label ID="filePathLabel" Text="File Path:" AssociatedControlID="fileUploadControl"
    runat="server" />
<asp:FileUpload ID="fileUploadControl" runat="server" />
<asp:RequiredFieldValidator ID="filUploadValidator" ControlToValidate="fileUploadControl"
    ValidationGroup="fileInfoGroup" ErrorMessage="Select a File" runat="Server">
</asp:RequiredFieldValidator>
<br />
<asp:Label ID="fileNameLabel" Text="File Name:" AssociatedControlID="fileNameBox"
    runat="server" />
<asp:TextBox ID="fileNameBox" runat="server" />
<asp:RequiredFieldValidator ID="fileNameValidator" ControlToValidate="fileNameBox"
    ValidationGroup="fileInfoGroup" ErrorMessage="Enter the File Name" runat="Server">
</asp:RequiredFieldValidator>
<br />
<asp:Label ID="submitterLabel" Text="Submitter:" AssociatedControlID="submitterBox"
    runat="server" />
<asp:TextBox ID="submitterBox" runat="server" />
<asp:RequiredFieldValidator ID="submitterValidator" ControlToValidate="submitterBox"
    ValidationGroup="fileInfoGroup" ErrorMessage="Enter the Submitter Name" runat="Server">
</asp:RequiredFieldValidator>
<br />
<asp:Button ID="insertButton" Text="Submit" CausesValidation="true" ValidationGroup="fileInfoGroup"
    runat="server" OnClick="insertButton_Click" />
<br />
<br />
<asp:Label ID="statusMessage" runat="server" />

6. If you now switch to design view, you will see:

image

You’ll also notice from that aspx that there is an event handler for OnRowCommand on the GridView to handle the DeleteItem command, IDs for the TextBoxes and an event handler for the OnClick event on the Submit button.

The code for these will be filled out further down in the walkthrough.

Connecting to Windows Azure storage

7. Open Default.aspx.cs and add the code to connect to the Blob Storage Service to the Page_Load() method.

using Microsoft.WindowsAzure;
using Microsoft.WindowsAzure.StorageClient;

namespace WebRole1
{
    public partial class _Default : System.Web.UI.Page
    {
        private CloudBlobClient _BlobClient = null;
        private CloudBlobContainer _BlobContainer = null;

        protected void Page_Load(object sender, EventArgs e)
        {
            // Setup the connection to Windows Azure Storage
            var storageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");
            _BlobClient = storageAccount.CreateCloudBlobClient();

8.  Setup the “DataConnectionString” setting by opening up the configuration UI for WebRole1.  Right click on the WebRole1 node under the Roles folder in the SimpleBlobStorage cloud service project and select “Properties”.

image

9. Switch to the Settings tab and click “Add Setting”.  Name it DataConnectionString, set the type to ConnectionString and click on the “…” button on the far right.

image

Hit “OK” to set the credentials to use Development Storage.  We’ll first get this sample working on development storage then convert it to use cloud storage later.

10. If you actually tried to connect to Blob Storage at this point, you would find that the CloudStorageAccount.FromConfigurationSetting() call would fail with the following message:

ConfigurationSettingSubscriber needs to be set before FromConfigurationSetting can be used

This message is in fact incorrect – I have a bug filed to get this fixed, to say “Configuration Setting Publisher” and not “ConfigurationSettingSubscriber”.

To resolve this, we need to add a bit of template code to the WebRole.cs file in WebRole1.  Add the following to the WebRole.OnStart() method in WebRole.cs in the WebRole1 project.

using Microsoft.WindowsAzure;

public override bool OnStart()
{
    DiagnosticMonitor.Start("DiagnosticsConnectionString");

    #region Setup CloudStorageAccount Configuration Setting Publisher

    // This code sets up a handler to update CloudStorageAccount instances when their corresponding
    // configuration settings change in the service configuration file.
    CloudStorageAccount.SetConfigurationSettingPublisher((configName, configSetter) =>
    {
        // Provide the configSetter with the initial value
        configSetter(RoleEnvironment.GetConfigurationSettingValue(configName));

        RoleEnvironment.Changed += (sender, arg) =>
        {
            if (arg.Changes.OfType<RoleEnvironmentConfigurationSettingChange>()
                .Any((change) => (change.ConfigurationSettingName == configName)))
            {
                // The corresponding configuration setting has changed, propagate the value
                if (!configSetter(RoleEnvironment.GetConfigurationSettingValue(configName)))
                {
                    // In this case, the change to the storage account credentials in the
                    // service configuration is significant enough that the role needs to be
                    // recycled in order to use the latest settings. (for example, the 
                    // endpoint has changed)
                    RoleEnvironment.RequestRecycle();
                }
            }
        };
    });
    #endregion

    // For information on handling configuration changes
    // see the MSDN topic at http://go.microsoft.com/fwlink/?LinkId=166357.
    RoleEnvironment.Changing += RoleEnvironmentChanging;

    return base.OnStart();
}

The comments (which I wrote and is included in the samples) should explain what is going on if you care to know.  In a nutshell, this code bridges the gap between the Microsoft.WindowsAzure.StorageClient assembly and the Microsoft.WindowsAzure.ServiceRuntime library – the Storage Client library is agnostic to the Windows Azure runtime as it can be used in non Windows Azure applications.

This code essentially says how to get a setting value given a setting name and sets up an event handler to handle setting changes while running in the cloud (because the ServiceConfiguration.cscfg file was updated in the cloud).

The key point is that with this snippet of code in place, you now have everything in place to connect to Windows Azure storage and create a CloudBlobClient instance.

Adding Blobs

11. In order to add blobs to a container, you first need to setup a container.  Let’s add this code to the Page_Load() method in Default.aspx.cs.  For a production application, you will want to optimize this code to avoid doing all this work on every page load.

Page_Load() should now look as follows.

protected void Page_Load(object sender, EventArgs e)
{
    // Setup the connection to Windows Azure Storage
    var storageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");
    _BlobClient = storageAccount.CreateCloudBlobClient();

    // Get and create the container
    _BlobContainer = _BlobClient.GetContainerReference("publicfiles");
    _BlobContainer.CreateIfNotExist();

    // Setup the permissions on the container to be public
    var permissions = new BlobContainerPermissions();
    permissions.PublicAccess = BlobContainerPublicAccessType.Container;
    _BlobContainer.SetPermissions(permissions);

    // Show the current list.
    UpdateFileList();
}

Note: The container is named with DNS naming restrictions (i.e. all lower case) and is created if it does not exist.  Additionally, the container is set to be a public container – i.e. the URIs to the blobs are accessible by anyone over the internet. 

Had this been a private container, the blobs in that container could only be read by code that has the access key and account name.

12.  Let’s now add the code to Default.aspx.cs to add a blob when the “Submit” button on the UI is clicked (remember the event handler was defined in the aspx).

A GUID is created for the file name to ensure a unique blob name is used.  The file name and submitter are gotten from the TextBoxes in the UI.

Blob Metadata, or user defined key/value pairs, is used to store the file name and submitter along with the blob. 

protected void insertButton_Click(object sender, EventArgs e)
{
    // Make a unique blob name
    string extension = System.IO.Path.GetExtension(fileUploadControl.FileName);

    // Create the Blob and upload the file
    var blob = _BlobContainer.GetBlobReference(Guid.NewGuid().ToString() + extension);
    blob.UploadFromStream(fileUploadControl.FileContent);

    // Set the metadata into the blob
    blob.Metadata["FileName"] = fileNameBox.Text;
    blob.Metadata["Submitter"] = submitterBox.Text;
    blob.SetMetadata();

    // Set the properties
    blob.Properties.ContentType = fileUploadControl.PostedFile.ContentType;
    blob.SetProperties();

    // Update the UI
    UpdateFileList();
    fileNameBox.Text = "";
    statusMessage.Text = "";
}

Enumerating Existing Blobs

13. In order to simplify the databinding to the UI, lets add a FileEntry class that contains the data we want to show in the UI for each blob.  One instance of a FileEntry corresponds to a blob. 

Right click on WebRole1 and select Add | Class…

image

Name the class FileEntry.cs and hit OK.

14. Fill out FileEntry.cs with the following code:

public class FileEntry
{
    public Uri FileUri { get; set; }
    public string FileName { get; set; }
    public string Submitter { get; set; }
}

15. Back to Default.aspx.cs, let’s add code to populate the GridView by getting the collection of blobs from ListBlobs() and creating a FileEntry for each item.

FetchAttributes() is used to retrieve the blob metadata.

using System.Collections.Generic;
using System.Collections.Specialized;

private void UpdateFileList()
{
    // Get a list of the blobs
    var blobs = _BlobContainer.ListBlobs();
    var filesList = new List<FileEntry>();

    // For each item, create a FileEntry which will populate the grid
    foreach (var blobItem in blobs)
    {
        var cloudBlob = _BlobContainer.GetBlobReference(blobItem.Uri.ToString());
        cloudBlob.FetchAttributes();

        filesList.Add(new FileEntry() { 
            FileUri = blobItem.Uri,
            FileName = cloudBlob.Metadata["FileName"],
            Submitter = cloudBlob.Metadata["Submitter"]
        });    
    }
    
    // Bind the grid
    fileView.DataSource = filesList;
    fileView.DataBind();
}

Deleting Blobs

16. Add code to delete the blob in the row command handler that was setup in the aspx. This is as simple as calling CloudBlobContainer.DeleteIfExists() for the blob where the CloudBlob instance is gotten from the Guid + file extension generated during the upload.

protected void RowCommandHandler(object sender, GridViewCommandEventArgs e)
{
    if (e.CommandName == "DeleteItem")
    {
        var index = Convert.ToInt32(e.CommandArgument);
        var blobName = (string)fileView.DataKeys[index].Value;
        var blobContainer = _BlobClient.GetContainerReference("publicFiles");
        var blob = blobContainer.GetBlobReference(blobName);
        blob.DeleteIfExists();
    }

    // Update the UI
    UpdateFileList();
}

Testing the Application

17. Build and hit F5 to run the application.

Note: The FileUpload control has a file size limit. You can modify it by changing the httpRuntime maxRequestLength attribute in web.config.

image

Moving from Development Storage to Cloud Storage

18. Now I want to switch this to use Windows Azure Storage, not the development storage.  The first step is to go to the Windows Azure Developer Portal and create a storage account.

Login and click on “New Service”, and select “Storage Account”:

image

Fill out the service name, the public name and optionally choose a region.  You will be brought to a page that contains the following (note I rubbed out the access keys):

image

19. You will use the first part of the endpoint, (jnakstorageaccount) and the one of the access keys to fill out your connection string.

20. Open the WebRole1 config again, bring up Settings | DataConnectionString and fill out the account name and the account key and hit OK.

image

21. Hit F5 to run the application again. 

This time you will be running against cloud storage – note that the data you entered when running against the development storage is no longer there.

Important Note: Before deploying to the cloud, the DiagnosticsConnectionString also needs to use storage account credentials and not the development storage account.

Please see the Deploying a Cloud Service walkthrough to learn how to deploy this to the Windows Azure cloud.

For more information, please see the Blob Service API documentation and the Programming Blob Storage white paper.

I know that there are a number of different concepts that have to be pieced together, hopefully this walkthrough has been helpful in understand how everything fits together.

Attachment: SimpleBlobSample.zip
Leave a Comment
  • Please add 8 and 6 and type the answer here:
  • Post
  • This would've been of great help a few days ago back when I was doing my Storage Explorer. (http://storageexplorer.cloudapp.net)

    Great post though :)

  • Great walkthrough, thanks!

    First one I've found that accurately applied to the latest APIs and was well written enough to follow- I'm finally creating blobs and containers, seems so easy now :)

  • seab.gomez, rabelBodhi -- thanks so much, glad it was a help for you.

  • Once again the best tutorial on the given subject (Azure Storage Blobs).  

    Good developers also make good documentation writers - since we know what is important and we care about the technology.

    Microsoft should have real developers in its MSDN staff. ;-)

  • Thanks ehuna - much appreciated.  

  • I have a question about the work-around you have regarding this error:

    ConfigurationSettingSubscriber needs to be set before FromConfigurationSetting can be used

    I used this trick to successfully make a cloud worker role project to upload/download stuff from my cloud storage blob.

    However, I tried to put a silverlight web service onto the cloud, using the instructions here:

    http://davidburela.wordpress.com/2008/12/10/silverlight-hoste-on-windows-azure/

    I'm having trouble debugging this application inside the Azure sandbox environment (but with a data connection string pointing at my blob storage account in azure). I get exactly the error you mention, even though I have the code that you told us to put, inserted into the webrole.cs file of the web service that my silverlight web app calls.

    Any ideas on why I might still be seeing this error?

  • Dipankar,

    Can you set a breakpoint on SetConfigurationSettingPublisher and the call to FromConfigurationSetting?  

    The only reason for the issue you are seeing is if that code isn't getting run before the call to FromConfigurationSetting(), so please double check that.

    You can also use CloudStorageAccount.Parse() which is a simpler way to get things working but you lose the ability to rotate the storage account key by updating the service configuration file.

  • This is a great walkthrough.I have just started learning Azure and this has been of great help to me.Thanks!!

    Now this walkthrough is with a public container.Can you tell me how to create a private container in the development storage? I have tried by omitting the "permissions.PublicAccess = BlobContainerPublicAccessType.Container;"

    line,since by default a container is private and then setting the account key and name(arbitrary) in settings I also tried to access the blob in the container.But this approach is not working.Any ideas where I went wrong?

  • Sunanda, you can explicitly set the container to be private by setting the BlobContainerPublicAccessType to Off. What this will do will make it so that you can only access blobs in that container via the API and not by URL.  

  • This no longer works with SDK 1.3 and VS 2010, maybe you could reply here if you know the problem?

    social.msdn.microsoft.com/.../fb381f8f-19ce-4a90-8004-bc2c87c8d4c8

    Thank you!

  • Nice work Jnak,

    But instead of doing STEP 9 (9. Switch to the Settings tab and click “Add Setting”.  Name it DataConnectionString,...) and doing a work around for the webrole's onStart(), we can simple do a

    CloudStorageAccount myStorageAccount = CloudStorageAccount.DevelopmentStorageAccount;

    and continue as it is...?

    I use this method, what are the merits and demerits of using this over yours...?

  • Naveen - the problem with using dev storage programmatically is that you'll have to change your code to run in the cloud where there is no dev storage.

    Also, as other people have mentioned, with the release of 1.3, it's better (i.e. it'll work) to use CloudStorageAccount.Parse(RoleEnvironment.GetConfigurationSetting(mysetting)) instead of FromConfigurationSetting.

  • Hi,

    I'm having trouble uploading files to our website that runs on multiple instances on azure.

    I get file not found when the code tries to process the file i just uploaded..

    any ideas to fix that ?

  • SvovelSyre - uploading files where?  To storage?  You can use the azure storage viewer in VS to see the files once you upload them which will ensure that you uploaded them correctly.  If you mean you uploaded them to the temp storage space on the VM, make sure you are using the local storage APIs.

Page 1 of 2 (27 items) 12