SQL Azure Team Blog : T-SQL

Updated CTP for SQL Azure Database includes complete feature set for PDC 2009!

davidrob — Wed, 14 Oct 2009 16:58:47 GMT

A few short weeks ago we announced the SQL Azure Database August CTP. Since the announcement, tens of thousands in the community have signed up for the service and have provided us with a tremendous amount of positive feedback. Today, the SQL Azure Database October CTP (CTP 2) update is another major milestone as we get ready for the Professional Developers Conference (PDC) on November 17th - 20th, 2009. We are also excited to announce that this CTP represents the complete feature set that will be available in the SQL Azure Database at PDC.

“We see SQL Azure as the perfect fit for information that needs to be kept in one safe, well-structured, accessible spot. Partitioning the data helps ensure scalability as more and more users interact with the site, and it also isolates tracking resources, helping avoid single points of failure,” said Luigi Rosso, Chief Technology Officer at leading interactive firm, Archetype. “To establish the kind of default redundant installation we get with SQL Azure, we’d have to purchase at least two dedicated servers and hire IT staff to administer them. A reliable system in place that is managed, efficient, and fast is critical for us. With SQL Azure, we don’t have to buy servers or manage and monitor them for capacity to ensure availability for our customers.”

The October CTP has been deployed to one of our go-live production clusters. This production cluster is significantly larger and more powerful than the machine cluster that is supporting the August CTP but is a completely separate machine cluster serviced by a dedicated developer portal (https://sql.azure.com). Accounts for all existing users of the current CTP (August CTP) have been automatically provisioned for access to the new October CTP and environment. Simply go to the developer portal (https://sql.azure.com) to activate your account and create servers on the new environment. Servers you create on this new environment will be reachable through a new address(<servername>.database.windows.net – we’ve dropped the ‘ctp’ moniker from the base address name).

When SQL Azure Database becomes generally available, this environment will automatically roll over into a fully supported production environment and all your databases and data in this environment will be converted into an active subscription to the SQL Azure Database service based on the subscription offer you choose.

Note: Existing servers and databases that you created in the August CTP environment will still be fully accessible and available through the current service URL (<servername>.ctp.database.windows.net), in parallel with the October CTP environment. The developer portal for the August CTP environment will also continue to be available, although through a new URL (https://ctpportal.database.windows.net ). As noted above, the previous URL will be re-used to support the new production environment.

As the August CTP is running on hardware below our production standard, that environment will not roll over into a production environment as part of our go-live plans. That environment will be decommissioned by the end of the year (Dec ’09).

We invite you to begin to use the new cluster to take advantage of all the cool new relational features available as part of the updated service!

The key new features included in this October CTP are listed below.

Firewall Support – The new firewall features allows a customer to specify an allow list of IP address that can access their SQL Azure Server. Security is a concern for companies looking at storing data in the cloud and with this new feature you can rest assured that only hosts you specify will be allowed to connect. Please be aware that your firewall will deny all connections by default, so please go to the SQL Azure Portal and configure your allow list so that existing clients can continue to connect.
Support for Bulk Insert – One of the pain points we heard from customers was around the speed at which they were able to load data into the system. We have taken this feedback and have enabled support for Bulk Insert. This will improve the rate at which you are able to load data into the system by a few orders of magnitude. This change also enables you to use the SqlBulkCopy class from within ADO.Net
Database Edition Selection – This allows you to select which SQL Azure Database edition (Web Edition (up to 1GB relational database) or Business Edition (up to 10GB relational database)) is created during the database provisioning process. This is surfaced both in the SQL Azure Portal and in the T-SQL Create Database statement. For example, to create a Business Edition database the T-SQL command would be as follows:

CREATE DATABASE foo (MAXSIZE = 10GB)

Updates to SQL Azure Portal – We have made numerous functionality changes and bug fixes in the portal. These include database edition selection when creating a database in the portal, viewing the size of your database and also the ability to configure your firewall settings.
Additional T-SQL Support – Due to customer demand, we have enabled support for additional T-SQL statements. The complete list can be found on MSDN and it includes support for items like Synonyms, Types, Table Value Parameters, additional systems views and more.
Updated SQL Azure Server Properties – We have finalized the internal SQL Server engine properties so you can reliably tell if you are connecting to an instance of SQL Server on premises or connecting to SQL Azure in the cloud. The major changes include a new value for Edition of “SQL Azure” and a new Engine Edition value of “5”. For further details please refer to the SQL Azure documentation on MSDN.
Updated SQL Azure Error Messages – We have enhanced the error messages returned should an error occur. Our goal to ensure that should you experience an error, the message received is meaningful and actionable by you.

Consuming data stored in SQL Azure Database is also critical, so as a reminder, the updated SQL Server Driver for PHP 1.1 with support for SQL Azure Database was recently released and can be found here. In addition to SQL Azure Database support, this release includes significant enhancements for PHP development including support for PHP version 5.3, performance improvements, and new features such as scrollable results sets, row count, and support for UTF-8 encoding. If you would like to find out more, you can check out the documentation on MSDN. Also, you should download the refreshed Windows Azure Platform Training Kit that reflects the changes in this October CTP.

A good portion of the new features and enhancements for this CTP came to us as requests from the user community. We listen to and appreciate your feedback as it helps us to build a better platform that will provide you critical capabilities for your database workloads. Please keep the feedback coming.

When we started this journey our goal was to extend the SQL Server Platform to the cloud and to provide our customers with relational database features and availability with cost efficiencies. We are on track to deliver that goal at PDC. There is much more to come as we continue to light up new scenarios and experiences. There will be more on this at PDC ‘09 and the SQL Azure team hopes to see everyone there.

Try SQL Azure Database CTP Today

davidrob — Tue, 18 Aug 2009 18:01:00 GMT

Check out the free trial of Microsoft® SQL Azure Database Community Technology Preview (CTP). SQL Azure Database, a part of the Windows Azure platform, is a cloud-based database service built on Microsoft SQL Server technology. With SQL Azure organizations will benefit from a pay-as-you grow model with enterprise-class availability, data protection, scalability, and security.

Also available as a CTP is the SQL Server Driver for PHP 1.1, which provides new capabilities for building PHP applications and support for SQL Azure, enabling developers to build PHP apps with relational database capabilities using SQL Server or SQL Azure Database. The CTP for the SQL Server Driver for PHP is available for download today.

With the explosion of data-driven Web 2.0 applications, along with the need to extend existing applications and build innovative data solutions in the cloud – these are exciting milestones. With SQL Azure, developers building Web 2.0, ASP.NET and PHP applications can use familiar tools and data models to develop on a pay-as-you-grow, secure, scalable and highly available database service at minimal infrastructure cost. In fact, there are really no comparable solutions available today --- we are leading the industry by offering a relational database service. When hosting your application in Windows Azure and with SQL Azure as your cloud database, building credible, reliable cloud-based applications becomes very easy.

The SQL Azure Database CTP includes:

· Relational data model supporting Transact-SQL (T-SQL), including T-SQL stored procedures. Access SQL Azure with familiar data access APIs such as ODBC, ADO.Net, PHP, etc. The high degree of compatibility with SQL Server enables easy migration of existing Line of Business (LOB) or Web applications to the cloud.

· Self-provisioning, auto-healing and disaster recovery, with high availability and no physical database administration. Self service provisioning means you can provision any number of databases and not have to worry about machines, disks, or server configuration.

Pay-as-you-grow multi-tenant scalable service model.
Efficiencies from an enterprise class data center without the administrative overhead.
Support for familiar tools so developers can leverage existing skills to speed time to solution

SQL Server Driver for PHP CTP will include:

Support for SQL Azure, PHP version 5.3, and UTF-8
Support for Scrollable results and row count
Migration to the SQL Server 2008 Native Client framework with enhanced performance

SQL Azure Database will be available as a free trial until the service goes live in November. SQL Azure Database will be available in two editions: Web Edition, which will include up to 1 GB of relational data, priced at $9.99 per month, and Business Edition, which will include up to 10 GB of data, priced at $99.99 per month.

Other Resources:

· Register for SQL Azure Database CTP (Note: due to high demand there may be delays in issuing invitation codes as we onboard new participants.)

· Additional information on SQL Azure

· Windows Azure Platform Training Kit

· Download SQL Server Driver for PHP 1.1

· Additional information on SQL Server Driver for PHP.

First round of Questions and Answers

davidrob — Thu, 12 Mar 2009 17:38:00 GMT

Tuesday was an exciting day for the team, and more importantly, for the industry. I think one of the best responses I got from someone was that they “stood up from their chair and did a dance of joy”.

Here is the first round of questions I have received and the answers you are looking for.

How will existing accounts be migrated over to the new version?

We haven’t worked out all the details yet, but our current plans are to send invites to all registered users when we launch the CTP by mid-year 2009.

Are we losing some BASE capabilities to grant ACID capabilities?

The whole theory behind BASE (basically available, soft state, eventually consistent) is to gain scalability at the cost of consistency. We have always supported full ACID capabilities in the service and will continue to do so.

Are we losing scalability or partition support to guarantee consistency?

The short answer is no. The foundation of SDS has not changed. All of the guarantees around scalability and consistency still apply. In the ACE model, each authority had one or more containers with each container being the unit of consistency for query and update. The new SDS model hasn’t changed this if you think of each authority now being a server and each container a database. You now get the richness of the T-SQL model over your databases!

When? or to quote JamieT “When do I get to party on this new service”?

We’re on track to deliver a public CTP mid-calendar year 2009 and ship in the second half of calendar year 2009.

If Microsoft can deliver on the features for SQL Data Services (SDS) announced today then they have set the bar for SQL and Cloud database vendors

Yes, we believe we have.

Will SDS support Database Encryption (certificate and key management). Any support for row level versioning?

Database encryption? Not initially, but it’s on the list and as we have demonstrated – if there is sufficient customer demand, it will be one of the first things we add after v1. As far as row level versioning, this can mean a few different things. Do we support statement level snapshot transactions? Yes. Do we support Change Data Capture? It is still being evaluated.

The blog entry states “If it works with SQL Server, it will largely work with SQL Data Services.”. That word “largely” bothers me a little – it suggests the functionality is going to be reduced slightly. Details please?

We will be providing documentation soon on what is and is not supported in SDS. I’ll post an entry to the blog once the guidance is available and you can also keep an eye out for it on our MSDN Dev Center. But, to answer the question – We say *largely* due to the fact that there are things that just don’t apply in a cloud based world like setting the location of a data or log file or making server wide configuration changes. In v1 we expect to deliver a surface area that will support the vast majority of SQL Server database applications.

What artificial limits will be in place?

The database size will be capped. We are still evaluating what the cap will be, but the plan is to ensure that the allowed database size supports most, if not all, departmental and web application workloads.

Do we have to pay per instance, per MB of storage or per bandwidth usage?

The pricing options and levels are still being finalized.

Will it be SQL Server 2005 or 2008? Enterprise or Standard?

The core engine is based on the SQL Server 2008 technology foundation. The feature set does not map to a specific SQL Server SKU – in fact you should think of the service as its own SKU.

Will we be able to connect to our cloud instances from SQL Server Management Studio (SSMS)? And will we be able to buy just the client tools without an on-premises server license?

Our plans are to support SSMS, but it might be a requirement to install a new version or a hotfix or two. As far as buying just the client tools, using SSMS Express is a great option. You can find out more info here.

Will replication between instances be offered to aid with Business Intelligence?

Not initially. After you start working with the service let us know what functionality to include or scenarios to support. We want to hear from you!

Will you offer hosted SSIS/SSAS/SSRS?

It’s on the product roadmap, but I can’t comment on specifics or timing.

The blog entry states that only SQL Server authentication (username/password) will be offered initially. Can we assume that eventually the Microsoft Services Connector will be used to offer Windows Authentication?

Post v1 you can expect a more robust authentication story. I can’t discuss implementation details yet.

Will we have the option to expose a REST head using ADO.NET Data Services as part of SDS’s offerings (i.e. at the flip of a switch) or will we have to implement ADO.NET Data Services separately ourselves on Windows Azure?

During the v1 timeframe you will need to implement this yourself on the Azure Services Platform. This approach is very similar to using ADO.NET Data Services against an on-premises SQL Server. There are many key capabilities of the framework that you specify (security, etc.) and so this does require some coding.

You say, you will no longer support the ACE model since Windows Azure has the same data model. If I’m going to put my application in Windows Azure (as I did) my best option for data is SDS, right? As I understood, Microsoft’s approach for cloud computing was Windows Azure as the platform for developers and SDS to store the data. And now you say I can use Windows Azure storage… what’s the difference? What’s the path to follow?

The best storage option for an Azure Services Platform application depends on your application. At a very high level, if you require the features of a relational database, use SDS. If you require basic blob or “schemaless” storage, then Windows Azure Storage is for you. Both will be key capabilities available to developers in the overall Azure Services platform.

Will the new SDS features support accessing SQL Server in the Cloud using LINQ to Entities and the ADO.NET Entity Framework?

It will work based on our client library story. Interop through ADO.NET is one of our top scenarios.

I just finished reading the latest post on Data Platform Insider Blog. It says that customers who wish to use REST based interfaces to SDS can do it through customized ADO.NET Data Services. Does this mean that customers can consume the REST based ADO.NET Data Services through any platform?

Yes that is correct. I would also like to point out that we have drivers for SQL Server to work with most popular development stacks like PHP, Java and Ruby. Those drivers work with SDS as well.

Your latest blog posting says that the ACE model will no longer be supported – I guess you are referring to the SDS storage, does this also include the Windows Azure sStorage?

These announcements do not affect Windows Azure Storage in any way. That is still a key foundational piece of the Azure Services Platform.

How can an existing applications based on the ACE model be migrated to the newer version?

There is no direct migration path from the ACE based SDS to the relational based SDS. What you can do, however, is use the existing SSIS adaptor to move the data from your containers down to tables within a SQL Server or SQL Server Express database and then start using all of the relational features in the product. When the CTP of SDS goes live mid-calendar year, just deploy your schema and database to SDS via the available tools (e.g. Visual Studio)

Does Windows Azure Storage and SDS both use SQL Server under the hood? Is there really a difference between the two storage options?

I don’t have all the details on how Windows Azure Storage works under the hood, but these are two distinct storage options of the Azure Services Platform – similar to the difference between saving data to your local file system and saving it to a database.

With ADO.NET Data Services compatibility, do you mean I can work against the System.Data.SqlClient and in the theory only change my connection string to switch between a local database and SDS?

That’s exactly what I mean. Pretty cool isn’t it?

The no spin details on the new SDS features

davidrob — Tue, 10 Mar 2009 18:50:00 GMT

Today we announced the details of our plans to accelerate the delivery of core relational database features as part of SDS. There has been quite a bit of buzz about SDS over the past couple weeks and it is great to be able to share the details more broadly.

If we flash back about a year ago to Mix 08, Nigel Ellis got up on stage to introduce the community to SDS which, at the time, was a flexible entity based cloud database that you accessed using standard internet protocols. We made this announcement with the promise that more relational capabilities would be coming - and they did. But the universal feedback we received from our TAP partners and other early adopters was the need for a relational database delivered as a service. This was extremely valuable feedback and drove us to more aggressively investigate ways in which we could deliver these features. As a result of that work and based on the progress we’ve since made in the product team, we are announcing that SDS will deliver full relational database capabilities as a service.

While we knew we needed to accelerate our plans we also knew we needed to hold true to some on the founding principles we had when we started our journey. Things like High Availability, Fault Tolerance, Friction Free Provisioning, Pay As You Grow Scaling, Immediate Consistency. We are still delivering on these promises and have added to the mix true relational capabilities, T-SQL and compatibility with the existing developer and management tools ecosystem.

What does this mean for developers? Developers will be able to very easily provision themselves a logical server and database and begin developing against it immediately using the existing tools and technologies that they are accustomed to. We are providing an experience where a developer can take an existing application and just change the connection string to point it to the cloud and have it just work.

How will we do it? Three letters TDS. TDS stands for Tabular Data Stream and it's the published protocol that clients use to communicate with SQL Server. From its inception, SDS has always been built on the SQL Server technology foundation and it just made sense to allow our users to access their data via TDS. Most importantly for developers, this means symmetric SQL Server functionality and behavior combined with compatibility with the existing tools you are familiar with.

Tables?...Check

Stored Procedures?...Check

Triggers?...Check

Views?...Check

Indexes?...Check

Visual Studio Compatibility?...Check

ADO.Net Compatibility?...Check

ODBC Compatibility?...Check

To be clear, the above is not a complete list of supported features. However, given the feature set we are planning to support in SDS v1, a majority of database applications will “just work”, allowing developers to target on and off-premises deployments with essentially the same code base. The initial scenarios we are targeting are things like web and departmental applications. We will be posting some content to our MSDN Dev Center in the coming weeks with specifics and getting started guidance but I encourage everyone to download SQL Express and the Windows Azure SDK to get started.

The core foundational components of SDS have not changed. This is still the same architecture that we have been telling you about for the past year and that underlies the current CTP bits. It is the same architecture that is powering some of Microsoft's key service properties and in the next few months will be used to store 100’s of terabytes of data in production deployments. Our early adopters (both internal and external) have shaken it down pretty well and we feel very confident about these bits. The only difference is we are now providing a rich SQL model while maintaining the high availability, fault tolerant and scale aspects of the system.

What about the ACE (Authority, Container, Entity) data model and developer experience? Since Windows Azure storage has a similar data model (property bag) and developer experience, we will stop supporting the current ACE Model sometime in the future. Does this mean you can't access your relational data via internet friendly protocols like REST? Not at all. You can still access your relational data (located on premises or in the cloud) via HTTP/REST using the ADO.Net Data Services framework. The compatibility with existing tools and technologies is a really important point to drive home and a super important value add that Microsoft provides.

Breadth and OSS developer support will continue to be a high priority for us and we will continue to support and provide breadth development libraries for all mainstream development technologies including PHP, Ruby and Java. If it works with SQL Server, it will largely work with SQL Data Services.

What about Security? All communications with our service is SSL encrypted and our initial authentication will be using SQL Authentication.

Of course, SQL Data Services remains one of the key developer services of the Azure Services Platform - that hasn't changed. Consuming SDS from within an Azure application has never been easier and we will continue to ensure this is a feature rich, friction-free experience.

I have said a lot, so go ahead and digest all of this. What else do you want to know? As I am sure you all have more questions, feel free to email me at david.robinson@microsoft.com and I'll post the questions and answers for all to see.