In our continuous endeavor to enrich the development experience, we are extremely pleased to announce the new Storage Emulator, which has much improved parity with the Windows Azure Storage cloud service.
Storage Emulator emulates the Windows Azure Storage blob, table and queue cloud services on local machine which helps developers in getting started and basic testing of their storage applications locally without incurring the cost associated with cloud service. This version of Windows Azure Storage emulator supports Blob, Tables and Queues up until REST version 2012-02-12.
How it works?
Storage Emulator exposes different http end points (port numbers: 10000 for blob, 10001 for queue and 10002 for table services) on local host to receive and serve storage requests. Upon receiving a request, the emulator validates the request for its correctness, authenticates it, authorizes (if necessary) it, works with the data in SQL tables and file system and finally sends a response to the client.
Delving deeper into the internals, Storage Emulator efficiently stores the data associated with queues and tables in SQL tables. However, for blobs, it stores the metadata in SQL tables and actual data on local disk one file for each blob, for better performance. When deleting blobs, the Storage Emulator does not synchronously clean up unreferenced blob data on disk while performing blob operations. Instead it compacts and garbage collects such data in the background for better scalability and concurrency.
Installing Storage Emulator
Storage Emulator can work with LocalDB, SQL express or even a full blown SQL server as its SQL store.
The following steps would help in getting started with emulator using LocalDB.
Alternatively, if you have storage emulator 1.7 installed, you can do an in place update to the existing emulator. Please note that storage emulator 1.8 uses a new SQL schema and hence a DB reset is required for doing an in place update, which would result in loss of your existing data.
The following steps would help in performing an in place update.
Storage emulator 1.8 supports the REST version 2012-02-12, along with earlier versions. Below are the service specific enhancements.
In 2012-02-12 REST version, Windows Azure Storage cloud service introduced support for container leases, improved blob leases and asynchronous copy blob across different storage accounts. Also, there were enhancements for blob shared access signatures and blob leases in the 2012-02-12 version. All those new features are supported in Storage Emulator 1.8.
Since the emulator has just one built in account, one can initiate cross account copy blob by providing a valid cloud based URL. Emulator serves such cross account copy blob requests, asynchronously, by downloading the blob data, in chunks of 4MB, and updating the copy status.
To know more about the new features in general, the following links would be helpful:
Storage Emulator 1.8 also garbage collects the unreferenced page blob files which may be produced as a result of delete blob requests, failed copy blob requests etc.
In 2012-02-12 REST version, Windows Azure Storage cloud service introduced support for Queue shared access signatures (SAS). Storage Emulator 1.8 supports Queue SAS.
In 2012-02-12 REST version, Windows Azure Storage cloud service introduced support for table shared access signatures (SAS). Storage Emulator 1.8 supports Table SAS.
In order to achieve full parity with Windows Azure Storage table service APIs, the table service in emulator is completely rewritten from scratch to support truly schema less tables and expose data for querying and updating using ODATA protocol. As a result, Storage Emulator 1.8 fully supports the below table operations which were not supported in Emulator 1.7.
Storage Emulator 1.8 has a great extent of parity with the Windows Azure Storage cloud service in terms of API support and usability and we will continue to improve it. We hope you all like it and please share your feedback with us to make it better.
Nagarjun Guraja
Windows Azure Storage
This is extremely useful. Thank you !!!
Congrats on your new release!
After upgrading to 1.8, my app crashed while writing to the emulated table storage. I've tracked this down to _A TRAILING WHITESPACE_.
Have a look at this simple LINQPad query:
----------------------
void Main()
{
var account = CloudStorageAccount.DevelopmentStorageAccount;
var client = account.CreateCloudTableClient();
var table = client.GetTableReference("test");
table.DeleteIfExists();
table.CreateIfNotExists();
var foo = new Foo { Str = "foo ", PartitionKey = "A", RowKey = "A" };
table.Execute(TableOperation.Insert(foo));
}
class Foo : TableEntity
public string Str { get; set; }
// results in: 400 Bad Request - One of the request inputs is not valid
New or legacy (context stuff) API does not matter. Same result.
Str = "foo " will crash
Str = "foo" will not
Please look into this.
Cheers
Jabe
@Jabe, this is a known issue and we are looking at fixing this. Sorry for the inconvenience this has caused.
Storage Emulator 1.8 does not support request URL with more than 300 characters, DataServiceRequestException will be thrown in this case: (Azure SDK 1.7)
System.Data.Services.Client.DataServiceRequestException: An error occurred while processing this request. ---> System.Data.Services.Client.DataServiceClientException:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""www.w3.org/.../strict.dtd">
<HTML><HEAD>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Bad Request - Invalid URL</h2>
<hr><p>HTTP Error 400. The request URL is invalid.</p>
</BODY></HTML>
at System.Data.Services.Client.DataServiceContext.SaveResult.<HandleBatchResponse>d__1e.MoveNext()
--- End of inner exception stack trace ---
at System.Data.Services.Client.DataServiceContext.SaveResult.HandleBatchResponse()
at System.Data.Services.Client.DataServiceContext.SaveResult.EndRequest()
at System.Data.Services.Client.DataServiceContext.SaveChanges(SaveChangesOptions options)
Test code:
public class TestEntity : TableServiceEntity
public TestEntity(string pk, string rk) : base(pk, rk) { }
static void TestUrlLimit(int urlLength)
var myAccount = CloudStorageAccount.DevelopmentStorageAccount;
var tableClient = myAccount.CreateCloudTableClient();
string table = "TestTable";
tableClient.CreateTableIfNotExist(table);
var context = tableClient.GetDataServiceContext();
string pk = "0";
string rk = new string('0', urlLength - 77); // Set the length of row key to build a url with "urlLength" characters
var entity = new TestEntityX(pk, rk);
context.AttachTo(table, entity);
context.UpdateObject(entity);
Uri uri;
context.TryGetUri(entity, out uri);
Console.WriteLine("URL length: {0}", uri.AbsoluteUri.Length);
try
context.SaveChanges();
Console.WriteLine("Success");
catch (Exception e)
Console.WriteLine("Error: {0}", e);
static void Test()
TestUrlLimit(300); // Success
TestUrlLimit(301); // Error
I'm not sure whether it is a known issue/limitation of Azure Storage Emualtor. Currently, I just use "context.SaveChanges(SaveChangesOptions.Batch)" to avoid the exception in this case.
Is there any way to solve this problem?
Thanks,
Dong
Hi Dong,
You need to set http.sys regkeys in order to allow larger URLs.
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\HTTP\Parameters]
"MaxFieldLength"=dword:00010000
"MaxRequestBytes"=dword:00010000
The above maps to 64KB.
Please refer to support.microsoft.com/.../820129 for more details.
I hope this helps.
Hi Jean,
I have checked the values under HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\HTTP\Parameters and found the root cause of URL length issue:
The default value of UrlSegmentMaxLength is 260 (support.microsoft.com/.../820129), so the max length of Azure request url to emulator is 40 ("127.0.0.1/.../") + 260 (segment of pk & rk) = 300.
I changed value of UrlSegmentMaxLength to 0 (length that is bounded by the maximum value of a ULONG) and there's no URL issue now.
Many thanks for your help!