SQL Azure Team Blog : Scalability

SQL Azure at PASS Summit 2009

davidrob — Sun, 01 Nov 2009 22:59:43 GMT

This upcoming week, PASS (Professional Association for SQL Server) will be holding its annual summit in Seattle, WA. The summit runs from November 2nd through the 5th and, as described on the website, provides:

In-depth technical sessions all focused on SQL Server
Unparalleled access to the industry’s top SQL Server experts and the Microsoft SQL Server development team
Unique opportunity to network with your peers, share challenges, and get answers and advice
Return to work with new skills and knowledge to do your job better, faster, easier – right away.

This year the SQL Azure team will be presenting the following sessions:

What’s new in SQL Azure - Patric McElroy
Come and learn how SQL Azure has evolved over the past year based on your feedback. In this session you'll learn how SQL Azure delivers on promise of Database as a Service. You'll see how easy it is to take an existing class of SQL Server applications and extend them to the service using existing SQL Server knowledge, protocols, client libraries and tools. With minimal changes, your application will be running in a highly available and scalable service. Finally we’ll touch on the business model, terms of use and present a roadmap for the service.

Building Applications with SQL Azure and Windows Azure – David Robinson and Liam Cavanagh
Are you looking to reduce the costs of building and maintaining enterprise applications? Do you want to extend the reach of your applications across multiple devices, locations and partners? SQL Azure and Windows Azure provides you a friction free, highly scalable platform for building applications. The scale and reach of the cloud lights up a new class of application scenarios. Come see how easy it is to consume SQL Azure from within Windows Azure. In addition, we will dive into Microsoft’s new Data Hub for businesses and see how this SQL Azure powered synchronization service allows for data aggregation within the Hub to provide straight-forward data sharing between on-premises databases, business partners, remote offices and mobile users.

Roles and Responsibilities Managing a Microsoft SQL Azure Database - Nino Bice
Come and learn about database administration in the new world of SQL Azure. Starting from the creation of an SQL Azure account, learn about tools and techniques for provisioning accounts, servers, and databases. See how billing is managed. Learn how to collaborate with your peers to build an Azure project that uses SQL Azure. We demonstrate the mechanisms for database management such as managing logins and permissions. See how SQL Azure automates the tedious and complex administrative tasks (e.g., machine failures) and allows you to spend more time on the important design functions of a DBA.

Try SQL Azure Database CTP Today

davidrob — Tue, 18 Aug 2009 18:01:00 GMT

Check out the free trial of Microsoft® SQL Azure Database Community Technology Preview (CTP). SQL Azure Database, a part of the Windows Azure platform, is a cloud-based database service built on Microsoft SQL Server technology. With SQL Azure organizations will benefit from a pay-as-you grow model with enterprise-class availability, data protection, scalability, and security.

Also available as a CTP is the SQL Server Driver for PHP 1.1, which provides new capabilities for building PHP applications and support for SQL Azure, enabling developers to build PHP apps with relational database capabilities using SQL Server or SQL Azure Database. The CTP for the SQL Server Driver for PHP is available for download today.

With the explosion of data-driven Web 2.0 applications, along with the need to extend existing applications and build innovative data solutions in the cloud – these are exciting milestones. With SQL Azure, developers building Web 2.0, ASP.NET and PHP applications can use familiar tools and data models to develop on a pay-as-you-grow, secure, scalable and highly available database service at minimal infrastructure cost. In fact, there are really no comparable solutions available today --- we are leading the industry by offering a relational database service. When hosting your application in Windows Azure and with SQL Azure as your cloud database, building credible, reliable cloud-based applications becomes very easy.

The SQL Azure Database CTP includes:

· Relational data model supporting Transact-SQL (T-SQL), including T-SQL stored procedures. Access SQL Azure with familiar data access APIs such as ODBC, ADO.Net, PHP, etc. The high degree of compatibility with SQL Server enables easy migration of existing Line of Business (LOB) or Web applications to the cloud.

· Self-provisioning, auto-healing and disaster recovery, with high availability and no physical database administration. Self service provisioning means you can provision any number of databases and not have to worry about machines, disks, or server configuration.

Pay-as-you-grow multi-tenant scalable service model.
Efficiencies from an enterprise class data center without the administrative overhead.
Support for familiar tools so developers can leverage existing skills to speed time to solution

SQL Server Driver for PHP CTP will include:

Support for SQL Azure, PHP version 5.3, and UTF-8
Support for Scrollable results and row count
Migration to the SQL Server 2008 Native Client framework with enhanced performance

SQL Azure Database will be available as a free trial until the service goes live in November. SQL Azure Database will be available in two editions: Web Edition, which will include up to 1 GB of relational data, priced at $9.99 per month, and Business Edition, which will include up to 10 GB of data, priced at $99.99 per month.

Other Resources:

· Register for SQL Azure Database CTP (Note: due to high demand there may be delays in issuing invitation codes as we onboard new participants.)

· Additional information on SQL Azure

· Windows Azure Platform Training Kit

· Download SQL Server Driver for PHP 1.1

· Additional information on SQL Server Driver for PHP.

Exchange Hosted Archive - A True Testament of Scalability

davidrob — Mon, 08 Jun 2009 18:50:00 GMT

Hi everyone, this is Shankar Pal. I am a Principal Program Manager on the SQL Data Services (SDS) team. I spend my time working on the backend design for large, enterprise applications.

I wanted to share with you some of my experiences on the scalability of SQL Data Services and how this is best exemplified by one of our online services, the Microsoft Exchange Hosted Archive (EHA). This is a very rich service for e-mail archive, e-Discovery and regulatory compliance for corporate customers and large organizations. The next generation EHA uses the same relational database service infrastructure as SDS. I will focus on the section of the service pertaining to the scale aspects of the workload, and discuss how the relational database service addresses the scale requirements of EHA.

First, a brief introduction to the characteristics of the workload. Archived messages accumulate in the system and are governed by the retention policies of the customers. The message lifecycle goes from archival of messages in the system, retention based on retention policies (e.g. 3 years) and purging the messages at the end of the retention period. Inserted messages are full-text indexed on the header, subject line, message body and a variety of common business attachments such as Word documents. E-discovery consists of structured and full-text query of the messages. Examples are searches based on various properties such as the send time, the sender, or full-text search of the message body.

EHA looked for a long-term solution in a relational database service which would scale to a much higher archiving limit per customer than the current system, be easy to administer, provide the required availability and keep pace with the rapid growth of the service. The result is the next generation EHA which is powered by the SDS relational database service platform. The service allows the seat limit per customer to become many fold larger; this is achieved by distributing the archived emails from each customer to a large number of servers rather than to a specific server. Performance enhancements are seen during message insertions, as well as in structured and full-text queries across the system. For more information about the backend architecture, you can view a presentation from Gopal Kakivaya, a Distinguished Engineer in the SDS team, from last year’s PDC. That video can be found at http://mschnlnine.vo.llnwd.net/d1/pdc08/WMV-HQ/BB03.wmv.

Email is a very prolific form of communication. The high volume of incoming data is automatically replicated within the database service to provide fault-tolerance against various types of failures. The platform provides high availability whether storing gigabytes, terabytes, or petabytes of data. Each cluster of machines has a capacity to store hundreds of terabytes for email archive. Together with the replication and the backup requirements, the total capacity of the EHA cluster is petabytes of data, a testament to the scalability of the SDS relational database service platform.

The enormous scale is achieved with surprising simple design principles. Mail messages can be partitioned in several ways, the most obvious being by customer or user. Such segments can grow quite large, so for more parallelism, each customer or user’s messages can be partitioned further, most notably by the send time. A variation of this partitioning scheme is used for the EHA application. The partitions for each customer are scattered over many servers. This increases the throughput of the system for message insertion by distributing the write operations over a large number of physical servers. The net result is much higher insertion rate compared to the current EHA system.

Queries benefit from the physical distribution of the data as well by executing on multiple partitions scattered over many server machines. The process of running queries in parallel, sometimes referred to as fan out, and aggregating the responses pays greater rewards the more complex the query and the bigger and more distributed the overall data set. Customers, especially in heavily regulated industries, frequently perform full-text searches using date ranges and other qualifiers. The more structured the query the more relevant the results. Our measurements with real customer messages show that queries with a high degree of fan-out often execute an order of magnitude faster compared to a single instance of a server.

The new generation archive will be available later this summer. It is very exciting to build a system which uses physical distribution of data to meet the scale and performance requirements of a large enterprise application. The self-managing system simplifies a host of administrative functions and makes those more reliable.