SQL Azure Team Blog : TDS

First round of Questions and Answers

davidrob — Thu, 12 Mar 2009 17:38:00 GMT

Tuesday was an exciting day for the team, and more importantly, for the industry. I think one of the best responses I got from someone was that they “stood up from their chair and did a dance of joy”.

Here is the first round of questions I have received and the answers you are looking for.

How will existing accounts be migrated over to the new version?

We haven’t worked out all the details yet, but our current plans are to send invites to all registered users when we launch the CTP by mid-year 2009.

Are we losing some BASE capabilities to grant ACID capabilities?

The whole theory behind BASE (basically available, soft state, eventually consistent) is to gain scalability at the cost of consistency. We have always supported full ACID capabilities in the service and will continue to do so.

Are we losing scalability or partition support to guarantee consistency?

The short answer is no. The foundation of SDS has not changed. All of the guarantees around scalability and consistency still apply. In the ACE model, each authority had one or more containers with each container being the unit of consistency for query and update. The new SDS model hasn’t changed this if you think of each authority now being a server and each container a database. You now get the richness of the T-SQL model over your databases!

When? or to quote JamieT “When do I get to party on this new service”?

We’re on track to deliver a public CTP mid-calendar year 2009 and ship in the second half of calendar year 2009.

If Microsoft can deliver on the features for SQL Data Services (SDS) announced today then they have set the bar for SQL and Cloud database vendors

Yes, we believe we have.

Will SDS support Database Encryption (certificate and key management). Any support for row level versioning?

Database encryption? Not initially, but it’s on the list and as we have demonstrated – if there is sufficient customer demand, it will be one of the first things we add after v1. As far as row level versioning, this can mean a few different things. Do we support statement level snapshot transactions? Yes. Do we support Change Data Capture? It is still being evaluated.

The blog entry states “If it works with SQL Server, it will largely work with SQL Data Services.”. That word “largely” bothers me a little – it suggests the functionality is going to be reduced slightly. Details please?

We will be providing documentation soon on what is and is not supported in SDS. I’ll post an entry to the blog once the guidance is available and you can also keep an eye out for it on our MSDN Dev Center. But, to answer the question – We say *largely* due to the fact that there are things that just don’t apply in a cloud based world like setting the location of a data or log file or making server wide configuration changes. In v1 we expect to deliver a surface area that will support the vast majority of SQL Server database applications.

What artificial limits will be in place?

The database size will be capped. We are still evaluating what the cap will be, but the plan is to ensure that the allowed database size supports most, if not all, departmental and web application workloads.

Do we have to pay per instance, per MB of storage or per bandwidth usage?

The pricing options and levels are still being finalized.

Will it be SQL Server 2005 or 2008? Enterprise or Standard?

The core engine is based on the SQL Server 2008 technology foundation. The feature set does not map to a specific SQL Server SKU – in fact you should think of the service as its own SKU.

Will we be able to connect to our cloud instances from SQL Server Management Studio (SSMS)? And will we be able to buy just the client tools without an on-premises server license?

Our plans are to support SSMS, but it might be a requirement to install a new version or a hotfix or two. As far as buying just the client tools, using SSMS Express is a great option. You can find out more info here.

Will replication between instances be offered to aid with Business Intelligence?

Not initially. After you start working with the service let us know what functionality to include or scenarios to support. We want to hear from you!

Will you offer hosted SSIS/SSAS/SSRS?

It’s on the product roadmap, but I can’t comment on specifics or timing.

The blog entry states that only SQL Server authentication (username/password) will be offered initially. Can we assume that eventually the Microsoft Services Connector will be used to offer Windows Authentication?

Post v1 you can expect a more robust authentication story. I can’t discuss implementation details yet.

Will we have the option to expose a REST head using ADO.NET Data Services as part of SDS’s offerings (i.e. at the flip of a switch) or will we have to implement ADO.NET Data Services separately ourselves on Windows Azure?

During the v1 timeframe you will need to implement this yourself on the Azure Services Platform. This approach is very similar to using ADO.NET Data Services against an on-premises SQL Server. There are many key capabilities of the framework that you specify (security, etc.) and so this does require some coding.

You say, you will no longer support the ACE model since Windows Azure has the same data model. If I’m going to put my application in Windows Azure (as I did) my best option for data is SDS, right? As I understood, Microsoft’s approach for cloud computing was Windows Azure as the platform for developers and SDS to store the data. And now you say I can use Windows Azure storage… what’s the difference? What’s the path to follow?

The best storage option for an Azure Services Platform application depends on your application. At a very high level, if you require the features of a relational database, use SDS. If you require basic blob or “schemaless” storage, then Windows Azure Storage is for you. Both will be key capabilities available to developers in the overall Azure Services platform.

Will the new SDS features support accessing SQL Server in the Cloud using LINQ to Entities and the ADO.NET Entity Framework?

It will work based on our client library story. Interop through ADO.NET is one of our top scenarios.

I just finished reading the latest post on Data Platform Insider Blog. It says that customers who wish to use REST based interfaces to SDS can do it through customized ADO.NET Data Services. Does this mean that customers can consume the REST based ADO.NET Data Services through any platform?

Yes that is correct. I would also like to point out that we have drivers for SQL Server to work with most popular development stacks like PHP, Java and Ruby. Those drivers work with SDS as well.

Your latest blog posting says that the ACE model will no longer be supported – I guess you are referring to the SDS storage, does this also include the Windows Azure sStorage?

These announcements do not affect Windows Azure Storage in any way. That is still a key foundational piece of the Azure Services Platform.

How can an existing applications based on the ACE model be migrated to the newer version?

There is no direct migration path from the ACE based SDS to the relational based SDS. What you can do, however, is use the existing SSIS adaptor to move the data from your containers down to tables within a SQL Server or SQL Server Express database and then start using all of the relational features in the product. When the CTP of SDS goes live mid-calendar year, just deploy your schema and database to SDS via the available tools (e.g. Visual Studio)

Does Windows Azure Storage and SDS both use SQL Server under the hood? Is there really a difference between the two storage options?

I don’t have all the details on how Windows Azure Storage works under the hood, but these are two distinct storage options of the Azure Services Platform – similar to the difference between saving data to your local file system and saving it to a database.

With ADO.NET Data Services compatibility, do you mean I can work against the System.Data.SqlClient and in the theory only change my connection string to switch between a local database and SDS?

That’s exactly what I mean. Pretty cool isn’t it?

The no spin details on the new SDS features

davidrob — Tue, 10 Mar 2009 18:50:00 GMT

Today we announced the details of our plans to accelerate the delivery of core relational database features as part of SDS. There has been quite a bit of buzz about SDS over the past couple weeks and it is great to be able to share the details more broadly.

If we flash back about a year ago to Mix 08, Nigel Ellis got up on stage to introduce the community to SDS which, at the time, was a flexible entity based cloud database that you accessed using standard internet protocols. We made this announcement with the promise that more relational capabilities would be coming - and they did. But the universal feedback we received from our TAP partners and other early adopters was the need for a relational database delivered as a service. This was extremely valuable feedback and drove us to more aggressively investigate ways in which we could deliver these features. As a result of that work and based on the progress we’ve since made in the product team, we are announcing that SDS will deliver full relational database capabilities as a service.

While we knew we needed to accelerate our plans we also knew we needed to hold true to some on the founding principles we had when we started our journey. Things like High Availability, Fault Tolerance, Friction Free Provisioning, Pay As You Grow Scaling, Immediate Consistency. We are still delivering on these promises and have added to the mix true relational capabilities, T-SQL and compatibility with the existing developer and management tools ecosystem.

What does this mean for developers? Developers will be able to very easily provision themselves a logical server and database and begin developing against it immediately using the existing tools and technologies that they are accustomed to. We are providing an experience where a developer can take an existing application and just change the connection string to point it to the cloud and have it just work.

How will we do it? Three letters TDS. TDS stands for Tabular Data Stream and it's the published protocol that clients use to communicate with SQL Server. From its inception, SDS has always been built on the SQL Server technology foundation and it just made sense to allow our users to access their data via TDS. Most importantly for developers, this means symmetric SQL Server functionality and behavior combined with compatibility with the existing tools you are familiar with.

Tables?...Check

Stored Procedures?...Check

Triggers?...Check

Views?...Check

Indexes?...Check

Visual Studio Compatibility?...Check

ADO.Net Compatibility?...Check

ODBC Compatibility?...Check

To be clear, the above is not a complete list of supported features. However, given the feature set we are planning to support in SDS v1, a majority of database applications will “just work”, allowing developers to target on and off-premises deployments with essentially the same code base. The initial scenarios we are targeting are things like web and departmental applications. We will be posting some content to our MSDN Dev Center in the coming weeks with specifics and getting started guidance but I encourage everyone to download SQL Express and the Windows Azure SDK to get started.

The core foundational components of SDS have not changed. This is still the same architecture that we have been telling you about for the past year and that underlies the current CTP bits. It is the same architecture that is powering some of Microsoft's key service properties and in the next few months will be used to store 100’s of terabytes of data in production deployments. Our early adopters (both internal and external) have shaken it down pretty well and we feel very confident about these bits. The only difference is we are now providing a rich SQL model while maintaining the high availability, fault tolerant and scale aspects of the system.

What about the ACE (Authority, Container, Entity) data model and developer experience? Since Windows Azure storage has a similar data model (property bag) and developer experience, we will stop supporting the current ACE Model sometime in the future. Does this mean you can't access your relational data via internet friendly protocols like REST? Not at all. You can still access your relational data (located on premises or in the cloud) via HTTP/REST using the ADO.Net Data Services framework. The compatibility with existing tools and technologies is a really important point to drive home and a super important value add that Microsoft provides.

Breadth and OSS developer support will continue to be a high priority for us and we will continue to support and provide breadth development libraries for all mainstream development technologies including PHP, Ruby and Java. If it works with SQL Server, it will largely work with SQL Data Services.

What about Security? All communications with our service is SSL encrypted and our initial authentication will be using SQL Authentication.

Of course, SQL Data Services remains one of the key developer services of the Azure Services Platform - that hasn't changed. Consuming SDS from within an Azure application has never been easier and we will continue to ensure this is a feature rich, friction-free experience.

I have said a lot, so go ahead and digest all of this. What else do you want to know? As I am sure you all have more questions, feel free to email me at david.robinson@microsoft.com and I'll post the questions and answers for all to see.