SQL Azure Team Blog

SQL Server Management Studio Support for SQL Azure!!!

2009-11-10T14:45:48Z

Hi Folks,

Just a quick post to let you know that the long awaited build of SQL Server Management Studio with support for SQL Azure is available today for MSDN and TechNet subscribers and will be generally available for the rest of the world as of tomorrow. The download page can be found here. I will post a link to the tools only install when it becomes available tomorrow.

This support is included as part of the November CTP release of SQL Server 2008 R2. This release not only includes SQL Azure support from within SSMS but a wealth of other SQL Server 2008 specific features. Check out this blog entry for more.

Thanks,

Dave

SQL Azure Portal issue this week

2009-11-06T17:05:49Z

Hi Everyone,

We received an inquiry earlier this week about a user who was trying to access the SQL Azure Portal and was unable to log in. I want to take a quick moment to respond broadly just incase anyone else had the same experience.

There were no issues with the SQL Azure Database service this week. There was, however, an availability issue with the SQL Azure Portal while some configuration changes were being made in preparation for the launch of SQL Azure Database at PDC on November 17th. The issue we experienced was with our portal only – not the actual service. No one trying to connect to the service or their database(s) were impacted at all.

The portal is different than the service end point. The current portal is a placeholder tool (CTP only) used for signing up for the SQL Azure service prior to go-live. It also offers capabilities for creating and deleting databases within a provisioned SQL Azure Database server but, it is only one of several tools that are used to do this. Other tools, such as SSMS and SQLCMD, are routinely used for these create/delete operations. The use of these tools was not impacted at all due to the portal issue. Your data was, and is, safe, sound and accessible in our highly available service.

I also want to take a moment to thank all of our tremendous users. Your feedback has shaped the service and will continue to as we enhance SQL Azure Database and add new features. Keep the feedback coming. We listen and act upon it.

The next couple weeks are going to be exciting. There is much more to come as we continue to extend the SQL Server Platform to the cloud.

See you all at PDC,

Dave

SQL Azure at PASS Summit 2009

2009-11-01T22:59:43Z

This upcoming week, PASS (Professional Association for SQL Server) will be holding its annual summit in Seattle, WA. The summit runs from November 2nd through the 5th and, as described on the website, provides:

In-depth technical sessions all focused on SQL Server
Unparalleled access to the industry’s top SQL Server experts and the Microsoft SQL Server development team
Unique opportunity to network with your peers, share challenges, and get answers and advice
Return to work with new skills and knowledge to do your job better, faster, easier – right away.

This year the SQL Azure team will be presenting the following sessions:

What’s new in SQL Azure - Patric McElroy
Come and learn how SQL Azure has evolved over the past year based on your feedback. In this session you'll learn how SQL Azure delivers on promise of Database as a Service. You'll see how easy it is to take an existing class of SQL Server applications and extend them to the service using existing SQL Server knowledge, protocols, client libraries and tools. With minimal changes, your application will be running in a highly available and scalable service. Finally we’ll touch on the business model, terms of use and present a roadmap for the service.

Building Applications with SQL Azure and Windows Azure – David Robinson and Liam Cavanagh
Are you looking to reduce the costs of building and maintaining enterprise applications? Do you want to extend the reach of your applications across multiple devices, locations and partners? SQL Azure and Windows Azure provides you a friction free, highly scalable platform for building applications. The scale and reach of the cloud lights up a new class of application scenarios. Come see how easy it is to consume SQL Azure from within Windows Azure. In addition, we will dive into Microsoft’s new Data Hub for businesses and see how this SQL Azure powered synchronization service allows for data aggregation within the Hub to provide straight-forward data sharing between on-premises databases, business partners, remote offices and mobile users.

Roles and Responsibilities Managing a Microsoft SQL Azure Database - Nino Bice
Come and learn about database administration in the new world of SQL Azure. Starting from the creation of an SQL Azure account, learn about tools and techniques for provisioning accounts, servers, and databases. See how billing is managed. Learn how to collaborate with your peers to build an Azure project that uses SQL Azure. We demonstrate the mechanisms for database management such as managing logins and permissions. See how SQL Azure automates the tedious and complex administrative tasks (e.g., machine failures) and allows you to spend more time on the important design functions of a DBA.

Windows Azure Platform Launch Update

2009-10-29T18:59:00Z

In October 2008 at our Professional Developer Conference we opened the Community Technology Preview (CTP) for the Windows Azure platform (Windows Azure, SQL Azure and .NET Services). The CTP has been invaluable in terms of shaping many aspects of the Windows Azure platform.

PDC 2009 is an exciting event for all of us who are working on the Windows Azure platform as it’s our opportunity to show you the work that we’ve done over the past year, including some brand new features. Concurrent with the conference we will begin the Windows Azure platform’s transition from a CTP to a business. For your planning purposes, this post maps out the key stages of that transition that you should be aware of.

At PDC 2009, on November 17th, 2009, a number of new features in Windows Azure will be made available for the first time. The CTP will remain open through December 31st, allowing you to experiment with the full feature platform and to give us any final feedback.
Beginning January 2010, new customers will have to sign up for an offer to access services on the Windows Azure platform. Usage during the month of January will be at no charge, so you can see your exact usage while still enjoying free service.
On February 1, 2010, we will begin charging customers for using the Windows Azure platform.

Making the transition in these three steps accomplishes a few goals. First, it gives you a chance to explore our full feature set for free. Second, it allows our team time to get your feedback on the new features and address any issues that arise. Finally, it lets you preview exactly how billing will work before you need to start paying.

We’d like to thank everyone who has participated in our Community Technology Preview so far. Your feedback has really shaped the platform, and we’re looking forward to sharing our launch with you next month at PDC 2009!

Updated 10/30/2009: Clarified that January is free of charge.

Updated CTP for SQL Azure Database includes complete feature set for PDC 2009!

2009-10-14T16:58:47Z

A few short weeks ago we announced the SQL Azure Database August CTP. Since the announcement, tens of thousands in the community have signed up for the service and have provided us with a tremendous amount of positive feedback. Today, the SQL Azure Database October CTP (CTP 2) update is another major milestone as we get ready for the Professional Developers Conference (PDC) on November 17th - 20th, 2009. We are also excited to announce that this CTP represents the complete feature set that will be available in the SQL Azure Database at PDC.

“We see SQL Azure as the perfect fit for information that needs to be kept in one safe, well-structured, accessible spot. Partitioning the data helps ensure scalability as more and more users interact with the site, and it also isolates tracking resources, helping avoid single points of failure,” said Luigi Rosso, Chief Technology Officer at leading interactive firm, Archetype. “To establish the kind of default redundant installation we get with SQL Azure, we’d have to purchase at least two dedicated servers and hire IT staff to administer them. A reliable system in place that is managed, efficient, and fast is critical for us. With SQL Azure, we don’t have to buy servers or manage and monitor them for capacity to ensure availability for our customers.”

The October CTP has been deployed to one of our go-live production clusters. This production cluster is significantly larger and more powerful than the machine cluster that is supporting the August CTP but is a completely separate machine cluster serviced by a dedicated developer portal (https://sql.azure.com). Accounts for all existing users of the current CTP (August CTP) have been automatically provisioned for access to the new October CTP and environment. Simply go to the developer portal (https://sql.azure.com) to activate your account and create servers on the new environment. Servers you create on this new environment will be reachable through a new address(<servername>.database.windows.net – we’ve dropped the ‘ctp’ moniker from the base address name).

When SQL Azure Database becomes generally available, this environment will automatically roll over into a fully supported production environment and all your databases and data in this environment will be converted into an active subscription to the SQL Azure Database service based on the subscription offer you choose.

Note: Existing servers and databases that you created in the August CTP environment will still be fully accessible and available through the current service URL (<servername>.ctp.database.windows.net), in parallel with the October CTP environment. The developer portal for the August CTP environment will also continue to be available, although through a new URL (https://ctpportal.database.windows.net ). As noted above, the previous URL will be re-used to support the new production environment.

As the August CTP is running on hardware below our production standard, that environment will not roll over into a production environment as part of our go-live plans. That environment will be decommissioned by the end of the year (Dec ’09).

We invite you to begin to use the new cluster to take advantage of all the cool new relational features available as part of the updated service!

The key new features included in this October CTP are listed below.

Firewall Support – The new firewall features allows a customer to specify an allow list of IP address that can access their SQL Azure Server. Security is a concern for companies looking at storing data in the cloud and with this new feature you can rest assured that only hosts you specify will be allowed to connect. Please be aware that your firewall will deny all connections by default, so please go to the SQL Azure Portal and configure your allow list so that existing clients can continue to connect.
Support for Bulk Insert – One of the pain points we heard from customers was around the speed at which they were able to load data into the system. We have taken this feedback and have enabled support for Bulk Insert. This will improve the rate at which you are able to load data into the system by a few orders of magnitude. This change also enables you to use the SqlBulkCopy class from within ADO.Net
Database Edition Selection – This allows you to select which SQL Azure Database edition (Web Edition (up to 1GB relational database) or Business Edition (up to 10GB relational database)) is created during the database provisioning process. This is surfaced both in the SQL Azure Portal and in the T-SQL Create Database statement. For example, to create a Business Edition database the T-SQL command would be as follows:

CREATE DATABASE foo (MAXSIZE = 10GB)

Updates to SQL Azure Portal – We have made numerous functionality changes and bug fixes in the portal. These include database edition selection when creating a database in the portal, viewing the size of your database and also the ability to configure your firewall settings.
Additional T-SQL Support – Due to customer demand, we have enabled support for additional T-SQL statements. The complete list can be found on MSDN and it includes support for items like Synonyms, Types, Table Value Parameters, additional systems views and more.
Updated SQL Azure Server Properties – We have finalized the internal SQL Server engine properties so you can reliably tell if you are connecting to an instance of SQL Server on premises or connecting to SQL Azure in the cloud. The major changes include a new value for Edition of “SQL Azure” and a new Engine Edition value of “5”. For further details please refer to the SQL Azure documentation on MSDN.
Updated SQL Azure Error Messages – We have enhanced the error messages returned should an error occur. Our goal to ensure that should you experience an error, the message received is meaningful and actionable by you.

Consuming data stored in SQL Azure Database is also critical, so as a reminder, the updated SQL Server Driver for PHP 1.1 with support for SQL Azure Database was recently released and can be found here. In addition to SQL Azure Database support, this release includes significant enhancements for PHP development including support for PHP version 5.3, performance improvements, and new features such as scrollable results sets, row count, and support for UTF-8 encoding. If you would like to find out more, you can check out the documentation on MSDN. Also, you should download the refreshed Windows Azure Platform Training Kit that reflects the changes in this October CTP.

A good portion of the new features and enhancements for this CTP came to us as requests from the user community. We listen to and appreciate your feedback as it helps us to build a better platform that will provide you critical capabilities for your database workloads. Please keep the feedback coming.

When we started this journey our goal was to extend the SQL Server Platform to the cloud and to provide our customers with relational database features and availability with cost efficiencies. We are on track to deliver that goal at PDC. There is much more to come as we continue to light up new scenarios and experiences. There will be more on this at PDC ‘09 and the SQL Azure team hopes to see everyone there.

SQL Azure at SQL PASS Summit

2009-10-05T19:02:14Z

Hi Folks,

Its almost conference season. Starting in November, we have SQL PASS, followed by Tech-Ed Europe and culminating with the launch of SQL Azure at PDC. Its going to be an exciting few weeks.

Liam recorded a Channel 9 video along with Buck Woody and Michael Rys on the upcoming SQL PASS Summit. You can view the video here

Here is the synopsis of the video:

Join thousands of SQL Server & BI professionals at the Washington State Convention & Trade Center in Seattle from Nov. 2-5 for the largest SQL Server conference in the world! PASS Summit offers 168 technical sessions presented by SQL Server experts, one-on-one time with Microsoft’s SQL Server engineering team, free troubleshooting & design architecture guidance from Microsoft consultants, keynotes from Microsoft’s senior executives and much more!

Senior Data Platform Technology Specialist Buck Woody stopped by the Channel 9 studio to talk to Michael Rys, Principal Program Manager in SQL Server Engineering Team, and Liam Cavanagh, Senior Program Manager, about upcoming sessions you won’t want to miss this year’s PASS Summit conference.

Hope to see some of you there,

Dave

Short blip in service availability today

2009-09-18T00:50:00Z

We experienced a short blip in the availability of SQL Azure today and I wanted to post a quick note to let you know what happened.

Inside of SQL Azure we have these automated monitoring tools called watchdogs. These watchdogs monitor every aspect of the system and if one of them notices that something is not functioning correctly, it raises an alert to our operations staff and will, depending on the issue type, start some sort of corrective maintenance.

Shortly before noon (PDT) an alert was raised that the SQL Azure gateways were unable to communicate with the backend data nodes. Once the alert was raised, the operations team and certain members of the product team were automatically notified. Due to the granularity of the watchdogs, we were quickly able to identify the issue. The issue stemmed from a configuration change that was made as we were getting the cluster ready for the upcoming refresh of our CTP deployment. The configuration error was remedied and the communications between the gateways and the backend data nodes were restored thus restoring access to the service. The elapsed time from when the service became unavailable to the time it was restored was just under an hour. We have started implementing safeguards to ensure such configuration issues don't occur in the future.

Our goal is and always has been to be as transparent with the user community as possible. With that, as soon as we noticed that there was a service disruption; notification was sent to the MSDN forums. Once we go live, customers would have received an email notifying them. We posted an additional update a few minutes later once we identified what the issue was. If we had not identified the issue so quickly, our incident response plan, or "playbook" as we call it, requires us to notify our users every hour until the issue is resolved. Our goal is to ensure that if an incident should arise, our customers are never questioning what is going on and are always kept in the loop. We believe that by combing our best of breed data platform service with clear, frequent communications, we will only strengthen the rock solid relationship we have with our customers.

You will notice above I mention a refresh to our CTP bits. Yes, a refresh is coming. I'll be sending out more information on that soon.

Thanks,

Dave

So… What do you think?

2009-08-24T05:40:42Z

One of the most important pieces of information a product team can get is an understanding of what people think about their product, and how they are using it. My question for you is…What do you think? How has your experience been thus far? We would love to know – the good and the bad.

So please feel free to either post a comment or drop me an email (david.robinsonATmicrosoft.com).

Also, don’t forget the SQL Azure documentation can be found here and the SQL Azure Getting Started forum is located here.

Dave

Try SQL Azure Database CTP Today

2009-08-18T18:01:00Z

Check out the free trial of Microsoft® SQL Azure Database Community Technology Preview (CTP). SQL Azure Database, a part of the Windows Azure platform, is a cloud-based database service built on Microsoft SQL Server technology. With SQL Azure organizations will benefit from a pay-as-you grow model with enterprise-class availability, data protection, scalability, and security.

Also available as a CTP is the SQL Server Driver for PHP 1.1, which provides new capabilities for building PHP applications and support for SQL Azure, enabling developers to build PHP apps with relational database capabilities using SQL Server or SQL Azure Database. The CTP for the SQL Server Driver for PHP is available for download today.

With the explosion of data-driven Web 2.0 applications, along with the need to extend existing applications and build innovative data solutions in the cloud – these are exciting milestones. With SQL Azure, developers building Web 2.0, ASP.NET and PHP applications can use familiar tools and data models to develop on a pay-as-you-grow, secure, scalable and highly available database service at minimal infrastructure cost. In fact, there are really no comparable solutions available today --- we are leading the industry by offering a relational database service. When hosting your application in Windows Azure and with SQL Azure as your cloud database, building credible, reliable cloud-based applications becomes very easy.

The SQL Azure Database CTP includes:

· Relational data model supporting Transact-SQL (T-SQL), including T-SQL stored procedures. Access SQL Azure with familiar data access APIs such as ODBC, ADO.Net, PHP, etc. The high degree of compatibility with SQL Server enables easy migration of existing Line of Business (LOB) or Web applications to the cloud.

· Self-provisioning, auto-healing and disaster recovery, with high availability and no physical database administration. Self service provisioning means you can provision any number of databases and not have to worry about machines, disks, or server configuration.

Pay-as-you-grow multi-tenant scalable service model.
Efficiencies from an enterprise class data center without the administrative overhead.
Support for familiar tools so developers can leverage existing skills to speed time to solution

SQL Server Driver for PHP CTP will include:

Support for SQL Azure, PHP version 5.3, and UTF-8
Support for Scrollable results and row count
Migration to the SQL Server 2008 Native Client framework with enhanced performance

SQL Azure Database will be available as a free trial until the service goes live in November. SQL Azure Database will be available in two editions: Web Edition, which will include up to 1 GB of relational data, priced at $9.99 per month, and Business Edition, which will include up to 10 GB of data, priced at $99.99 per month.

Other Resources:

· Register for SQL Azure Database CTP (Note: due to high demand there may be delays in issuing invitation codes as we onboard new participants.)

· Additional information on SQL Azure

· Windows Azure Platform Training Kit

· Download SQL Server Driver for PHP 1.1

· Additional information on SQL Server Driver for PHP.

SQL Azure MSDN Documentation

2009-08-11T17:27:48Z

Just a quick post. As I am sure some of you have noticed, the SQL Azure MSDN documentation is live and can be found here. This is an important milestone as we get ready for the upcoming CTP and go live later this year. The documentation should answer any questions you have, and if not feel free to let us know or post a question to the Getting Started forum located here.

Speaking of the CTP, it will be here before you know it so go here and sign up to be notified when it is available and to get an invitation code.

More good stuff coming soon so stay tuned.

-Dave

Start with SQL Express?

2009-07-13T04:21:00Z

I have gotten a bunch of questions from people asking what exactly do we mean when we say start with SQL Express and then migrate the app to SQL Azure. I think this is best summarized by a quick customer example…

A couple of weeks ago we had a group of early adopters on campus to take some existing applications and move them to SQL Azure and Windows Azure. The customer simply opened up SQL Server Management Studio and used the Generate Script Wizard to create the DDL and DML for the database. They then connected to their SQL Azure Database and ran the script.

So when we say start with SQL Express, we mean you can develop your SQL Azure application locally using one of the on premises versions of SQL Server (SQL Express is free download) and then easily move it to SQL Azure when the CTP starts in August. So get started :)

Any questions or comments, let me know…

-Dave

SQL Azure - Same great platform, just a better aligned name

2009-07-10T04:18:00Z

Yesterday we announced a branding update to SQL Azure (formerly SQL Services). I think you will agree that our new name is a better fit. If you look across the Windows platform, you will see a consistent branding theme that runs the gamut from Windows Mobile on mobile devices, to Windows Client on the desktop, Windows Server as the leading server operating system and more recently Windows Azure as our soon to be released operating system in the cloud. SQL Data Services is now known as SQL Azure Database. Same SQL Platform, same team, same innovation. SQL Server is the data platform of Windows Server so it made sense to align the name of the relational database platform of Windows Azure, SQL Azure. SQL Azure is built on the SQL Server technology foundation which delivers relational database capabilities in the cloud.

Regardless of what platform your solution targets, your existing SQL Server skills and experience don’t change. That was the goal of the team when we decided to accelerate our relational capabilities and add TDS support. A goal we will be delivering upon when we release SQL Azure later this year.

The next few months are going to be exciting. On the technology front, we are getting ready for our next CTP in August. You can, however, get started today with SQL Express. The team has been working closely with some early adopters and the feedback has been overwhelming positive. Thank you for being so patient, the wait is almost over. Over the next day or two, we will be pushing additional details regarding SQL Azure, including posting our first round of SQL Azure Database documentation to MSDN (once it’s posted, I will post a link to it here).

Stay Tuned,

Dave

SQL Server - Your Data, Any Place, Any Time...and now in the cloud with SQL Azure Database

TSQL Support in SQL Data Services

2009-07-07T22:56:00Z

This is a high-level overview of TSQL support in SDS. We are also working on publishing a more detailed documentation.

Overview

In its first release SQL Data Services will support a subset of TSQL language. Each TSQL construct can be categorized as supported, partially supported or unsupported in an SDS session. “Supported” in this context means that there is no difference in the way SDS treats a given statement or function compared to the on-premise SQL; “partially supported” means that SDS supports a subset of the functionality; “unsupported” means that a given feature is not supported by SDS. At this point we do not plan to add any new TSQL constructs to the language or change the way existing functionality works.

In many respects SDS is very similar to an on-premise instance of SQL Server. However, there are some differences:

· SDS is a multi-tenant system

· The hardware resources are owned, hosted and maintained by Microsoft

· SDS is a service

These differences impose certain requirements on the system, which, in turn, translate into TSQL restrictions. These restrictions fit into three buckets:

· Resource manipulation. We will block statements and options that try to directly manipulate physical resources. This category includes RESOURCE GOVERNER, file group references and some physical server DDL statements.

· Things that are either not applicable in the services world or are taken care of by us. Replication is a good example – we take care of it, so you don’t have to.

· Finally, there will be a few things that we just won’t be able to enable in time for our initial release. Each feature that we add to SDS requires additional work to ensure that the service remains performant, scalable and secure. This extra work takes time, which means some of the features will have to wait until v2. Support for distributed query and CLR-related functionality are a few examples of the features in this category.

What’s in/what’s out

So let’s take a look at what we plan to support in SDS v1. For clarity, we will break everything into three categories and we’ll look into each category separately: DDL, DML and Manageability.

DDL

SDS will support or partially support creation, manipulation and deletion of the following objects:

· FUNCTION

· INDEX

· PROCEDURE

· ROLE

· SCHEMA

· STATISTICS

· SYNONIM

· TABLE

· TRIGGER

· VIEW

Most of the CREATE and ALTER statements for the above objects will be partially supported. This partial support will manifest itself in the restriction on some of the parameters you can specify when creating or altering the objects. For example, the filegroup option of the CREATE TABLE statement will not be supported.

Users will not be able to create objects not listed above.

DML

The majority of DML will be either supported or partially supported. This includes

· SELECT/INSERT/UPDATE/DELETE functionality

· DML triggers

· JOINs

· Transactions

· A large number of built-in functions (aggregates, math, date and time, ranking, etc.)

Manageability

Manageability is the area which will be restricted the most. The following list highlights some of the details:

· Things that will work

· Query tuning via SET SHOWPLAN and SET STATISTICS

· Index tuning via create and drop index

· Statistic management via UPDATE STATISTICS

· Information schema views and system catalog views (e.g., sys.databases)

· Local HA is automatically provided as part of the service.

· Upcoming version of SSMS and VS tools will work against SDS

· Things that won’t work

· Backup command is not available due to multi-tenant nature of the service. We recommend using BCP or SSIS instead.

· Server options (sp_configure)

· SQL Profiler

· SQL traceflag

Exchange Hosted Archive - A True Testament of Scalability

2009-06-08T18:50:00Z

Hi everyone, this is Shankar Pal. I am a Principal Program Manager on the SQL Data Services (SDS) team. I spend my time working on the backend design for large, enterprise applications.

I wanted to share with you some of my experiences on the scalability of SQL Data Services and how this is best exemplified by one of our online services, the Microsoft Exchange Hosted Archive (EHA). This is a very rich service for e-mail archive, e-Discovery and regulatory compliance for corporate customers and large organizations. The next generation EHA uses the same relational database service infrastructure as SDS. I will focus on the section of the service pertaining to the scale aspects of the workload, and discuss how the relational database service addresses the scale requirements of EHA.

First, a brief introduction to the characteristics of the workload. Archived messages accumulate in the system and are governed by the retention policies of the customers. The message lifecycle goes from archival of messages in the system, retention based on retention policies (e.g. 3 years) and purging the messages at the end of the retention period. Inserted messages are full-text indexed on the header, subject line, message body and a variety of common business attachments such as Word documents. E-discovery consists of structured and full-text query of the messages. Examples are searches based on various properties such as the send time, the sender, or full-text search of the message body.

EHA looked for a long-term solution in a relational database service which would scale to a much higher archiving limit per customer than the current system, be easy to administer, provide the required availability and keep pace with the rapid growth of the service. The result is the next generation EHA which is powered by the SDS relational database service platform. The service allows the seat limit per customer to become many fold larger; this is achieved by distributing the archived emails from each customer to a large number of servers rather than to a specific server. Performance enhancements are seen during message insertions, as well as in structured and full-text queries across the system. For more information about the backend architecture, you can view a presentation from Gopal Kakivaya, a Distinguished Engineer in the SDS team, from last year’s PDC. That video can be found at http://mschnlnine.vo.llnwd.net/d1/pdc08/WMV-HQ/BB03.wmv.

Email is a very prolific form of communication. The high volume of incoming data is automatically replicated within the database service to provide fault-tolerance against various types of failures. The platform provides high availability whether storing gigabytes, terabytes, or petabytes of data. Each cluster of machines has a capacity to store hundreds of terabytes for email archive. Together with the replication and the backup requirements, the total capacity of the EHA cluster is petabytes of data, a testament to the scalability of the SDS relational database service platform.

The enormous scale is achieved with surprising simple design principles. Mail messages can be partitioned in several ways, the most obvious being by customer or user. Such segments can grow quite large, so for more parallelism, each customer or user’s messages can be partitioned further, most notably by the send time. A variation of this partitioning scheme is used for the EHA application. The partitions for each customer are scattered over many servers. This increases the throughput of the system for message insertion by distributing the write operations over a large number of physical servers. The net result is much higher insertion rate compared to the current EHA system.

Queries benefit from the physical distribution of the data as well by executing on multiple partitions scattered over many server machines. The process of running queries in parallel, sometimes referred to as fan out, and aggregating the responses pays greater rewards the more complex the query and the bigger and more distributed the overall data set. Customers, especially in heavily regulated industries, frequently perform full-text searches using date ranges and other qualifiers. The more structured the query the more relevant the results. Our measurements with real customer messages show that queries with a high degree of fan-out often execute an order of magnitude faster compared to a single instance of a server.

The new generation archive will be available later this summer. It is very exciting to build a system which uses physical distribution of data to meet the scale and performance requirements of a large enterprise application. The self-managing system simplifies a host of administrative functions and makes those more reliable.

One more TechEd Video - The New Face of Microsoft SQL Data Services

2009-06-08T18:43:00Z

Here is one last video we filmed a few weeks back during TechEd. In this one, we discuss the new features of SDS including the development model, tools support, and how the services interoperate with the other building block services of the Azure Services Platform

The video can be found here