Pablo Castro's blog : Services

ADO.NET Data Services in Windows Azure: pushing scalability to the next level

pabloc — Sun, 02 Nov 2008 05:40:40 GMT

The announcement of Windows Azure is a big milestone for us in the Astoria team. We got a chance to add our little contribution to the platform by providing data service interfaces for a couple of the Azure services.

Currently there are two services that use the ADO.NET Data Services runtime: the Windows Azure Tables Service, which was announced this week as part of the whole Windows Azure story, and SQL Data Services, which has been around for a while but got a new experimental Data Services interface this week to coincide with the PDC.

These services -and others that will come in the future also based on Data Services- share a common aspect: they have extreme scalability requirements.

In order to enable them to use our Data Services server runtime we had to extend the data service framework to make it scale in various new dimensions. In the rest of this post I'll summarize some of the walls we hit and the changes we made to the system to handle these scenarios.

Things that already scaled

The Data Services runtime already incorporates many design principles that help with scalability.

For example, the system does not keep any required state between requests (we do cache stuff, but we can throw it away at any time), so scale out of the front-end servers of the storage systems is relatively straightforward. This allows the existing runtime to handle an arbitrarily large number of requests by throwing more front-ends to the problem (as long as the back-end systems can take it, of course).

Also, we don't make any assumptions around the size of the data and provide mechanisms to push-down filters in requests to the data source, so effectively in principle there are no limits to the amount of data that a data service may be fronting.

Hitting the scalability wall

While some things scaled, there are certain aspects in which we ran into a scalability wall that required a number of changes in the system.

Using .NET types to represent the shape of the services is great in a single application, but not-so-great if you have millions of users with hundreds or thousands. We needed another way of describing the "shape of the data in the service", that is the metadata or schema of the service.

Since you can't practically create a distinct type for every user/application/table in the system, that means that the instances of objects that represent data flowing through the data services runtime cannot be of a specific type for each entity type. Instead, we needed independent of the flow format with respect of the declared types.

Metadata and service schema

The data services runtime needs to know the "schema" of each service it exposes. That is, the list of entity-sets, the entity-types of the instances living in those entity sets and the relationships between the various entities.

In a typical data service, the service exposes data for a given application or domain-specific service, so the schema of the service is known and static (within a given version at least) and all the front-end servers simple share the same schema.

The way a service author specifies the schema of a service in the shipping version of the Data Services runtime is by using .NET classes or an Entity Framework model (which in turn generates .NET classes). That works great for application developers, because .NET classes are a simple and natural way of defining the shape of your objects.

Now, if the requirement is to be able to handle millions of applications, each of which can have hundreds or thousands of tables, does that mean that we have to create a .NET type for each service and for each table, and the corresponding number of properties and such? And if so, since the front-end systems are stateless and potentially don't have any affinity to parts of the data, does that mean that any given system may end up having to load up millions of types in memory? To complicate things further, once you load an assembly (the only container in which .NET types can exist), you can't unload it unless you unload the AppDomain.

.NET types are a great solution for the scenarios where the schema is known and more or less bounded, and will continue to be the primary way of creating services in that context. However, we needed something else to handle the high-end side of the spectrum.

To address this need we introduced a new interface that data services can optionally implement. We already had the internals of the system organized more or less like this, but didn't expose it in the first release. The idea is that there is main split between the "upper half" of the runtime that deals with URL translation, LINQ expression tree generation, interceptors, policies and all aspects that make a Data Service look like a Data Service. The "bottom half" is the "data service provider", and is responsible for describing the shape of the service among other things. There are two built-in "data service providers", the Entity Framework provider which is what you use when you create a data service over an Entity Framework model, and the reflection-based provider which is what you use when creating a service on top of an arbitrary object graph. With the new change you can now create new implementations of these data service provider thingies that can obtain and manage metadata any way they want.

The way we interact with the provider is carefully designed to avoid requiring long term state state in the provider or the consumer of the provider in any way, while at the same time allowing the provider to do caching of metadata and control information if desired.

First, we never hold on to information returned by the provider beyond the scope of a single request. So for all we know the provider could be reloading all the metadata in every request. In practice, providers will probably cache this metadata in some way or another.

Second, we load metadata on demand and piecemeal. For example, during URI translation we do a small scale version of the usual binding and semantic analysis that any compiler does, and for that we need metadata. In those cases we don't load all the metadata, but only the pieces we need to do type checks, symbol lookups, etc.

Making metadata dynamic

Another aspects around metadata to consider is the fact that the shape of one of these data services can be altered at any time. For example, the Azure Table service has the concept of tables, and you can add and remove tables whenever you want.

The new scheme with custom data service providers make this possible because we don't remember anything at all across requests. So all the provider needs to do when the underlying shape of the data changes is report a different schema on the next request, and the data services runtime will happily take it.

With .NET types this would have meant creating and re-distributing new types (or creating them on demand on each node), and dealing with not being able to unload the old types from memory. Clearly not an option at this scale.

Flow format independence

With the addition of the "data service provider" interfaces we no longer have .NET types to use for the instances of each entity-type that flows through the system (e.g. from the data source to the runtime via the IEnumerables returned in LINQ queries, and from there to the serialization stack).

Another important change we made in the system is that we no longer assume anything about the shape of each CLR object returned by the query. We treat instances just as "object" all over the code base. When we need to access a member, we use methods in the data service provider interface to do that, imagine something like GetPropertyValue(object o, string name).

That means it's now possible to use some form of generic record type across the system. Not only this avoids the need for specific types, but also allows providers to piggyback control information in the instances themselves, avoid copies from the original format into CLR objects just to flow them through our runtime and a few more benefits.

Impact on LINQ expressions

While having flow format independence is great, it did complicate things for query formulation.

We typically translate URLs to expression trees, and since we have all the CLR types in the server that correspond to the entity types, all the expression trees are nice and clear.

When we're operating against unknown types we can't generated "typed" expression trees anymore. In those cases we still produce expression trees, but the member-access operations (and certain operators) are represented using custom calls to a well-known set of static members. The providers that enable this feature need to know about this and do proper translation of these expression trees.

Extension to the data model

We did one more major change that while it's not directly related to scalability it has a lot to do with the database/storage services in Windows Azure.

In the current version of Data Services types are "closed" in the sense that they have a structure that's final. You list a set of properties for each type and instances of that type cannot have properties added dynamically.

It turns out that the data services we have online have a more flexible model, where each entity has a fixed portion but also a dynamic portion. Typically the fixed portion includes a key or some sort and a version property. The dynamic portion is a property bag where you can add any name/typed-value pair.

We call these types that can be extended on a per-instance basis at runtime "open types". We introduced support for open types in the Data Services runtime such that you can mark a given entity type as "open" in metadata and that would cause the system to allow unknown properties to be set, as well as the use of unknown properties in queries (e.g. in filter predicates).

There is a lot of details around open types that I won't go into here, maybe the topic for another post, but I wanted to point out the change because it was a significant addition.

What do these changes mean for developers?

What does all this mean to current users of data services. Well...not much other than some background on how the system is evolving. Other than open types, services created with custom metadata/custom flow formats are indistinguishable from the ones created the "classic" way.

Furthermore, we will preserve the existing model where creating a service based on some .NET objects or an Entity Framework schema is really straightforward, and we consider that our primary scenario for developers.

At the same time, addressing the needs for the highest-end services out there is important, so many (if not all) of these changes will eventually make it into the shipping product so that other folks out there can use them if they chose to. Beware that these interfaces are not designed to be "nice", but rather optimized for control and efficiency, so it may not be exactly a fun experience, but you'll get all the scalability you'll need out of them.

-pablo

Now you know...it's Windows Azure

pabloc — Wed, 29 Oct 2008 04:02:29 GMT

Since we shipped ADO.NET Data Services v1 in .NET 3.5 SP1 (and actually before that as well) I've been working on a few things that I could share (such as offline/sync support for data services) and some that I couldn't discuss publicly until all the big plans where announced.

This week at PDC Microsoft announced Windows Azure. A lot has been written about it, so I won't go into the details.

On our side, in the data services team, we made our small contribution to the big picture.

The Windows Azure table service is a structured storage facility that's part of the core part of Azure. Access to the table service is done through a data-services compatible RESTful interface that uses the Astoria conventions over an HTTP binding. That means that you use either any client with an HTTP stack to talk to it, or you can use the ADO.NET Data Services client, which does a nice job exposing data as .NET objects, letting you write simple queries using LINQ instead of URLs, etc.

Another cool thing about the table service (and the blobs and queuing service for that matter) is that they are accessible both from the virtual compute environment and from anywhere in the Internet. In both cases, if you're using .NET, you can use the data services client to interact with it. In the case of code running in the Windows Azure hosting environment, the client is already present (the environment includes .NET 3.5 SP1) so you can use it without worrying about taking new dependencies.

You find out more about the table service you can watch Brad's PDC session for a discussion of the service itself, and this other session than Niranjan and I did together (or "will do" if you're reading this before Wed in the PDC week) for a drill down on how to program the Windows Azure table service. If you're not at PDC no worries, these talks are accessible to online.

On the next layer up from the core, the Windows Azure service layer, SQL Data Services also is making big announcements in this PDC. We're introducing more relational capabilities into the system, and also experimenting with a data services-compatible interface. This PDC talk from Patrick will discuss and demo the new interface, and you can follow how this effort goes here.

-pablo

Coming from under the rocks just to celebrate for a bit

pabloc — Tue, 12 Aug 2008 05:46:00 GMT

I've been sort of under a rock for a while, but I thought I'd come out for a minute to celebrate. Today we made available .NET 3.5 SP1 and Visual Studio 2008 SP1. There are two components in the release I spent a bunch of time on, which interestingly enough have very different origins and get to RTM through very different processes. One is the ADO.NET Entity Framework, which has been cooking for several years and survived controversies, comparisons with non-shipped previous attempts and other natural disasters; the other one is the ADO.NET Data Services Framework or Project Astoria, which was built, well...fast.

I won't go into details of the release, folks have discussed the Astoria, Entity Framework and general data-related features in the release already.

Why have I been under a rock? In the last few months I've been spending time working on various things related to Astoria, online services and data interfaces. Some I can discuss, some will need to wait a bit until the stakeholders are comfortable to talk about it publicly.

Moving Astoria as a framework forward: we were ready (modulo bug fixing and last minute tweaks) some time ago, and we've been thinking about the next steps for the Astoria framework. In Mix 08 we mentioned that we were working on "Astoria Offline" and showed a prototype. We've been working hard in that topic. There is also a bunch of features we want to take on for the next release. I'm sure we'll post something in the Astoria blog at some point about our thinking and give a change for folks to give feedback.

Online services: as you can imagine there is a number of things going on around online services these days, and a number of them involve Astoria one way or the other. I've been working with several of them, varying from providing guidance all the way to writing custom "v.next" versions of Astoria to experiment with their needs. An example of these efforts is the work we're doing to align SQL Server Data Services and the ADO.NET Data Services framework. We would like to see them as the "service" and the "framework" pieces, both using the same HTTP interface, same client interfaces, etc., so we've been spending a bunch of time exploring how to bring them together.

Anyway, there, a bit of a celebration.

-pablo

REST and Concurrency Control

pabloc — Wed, 23 Apr 2008 04:13:00 GMT

As part of the Astoria design process we scanned through many topics, some of them are straightforward, some are hard but mostly mechanical, but there are some that become interesting, fundamental aspects to address.

I found the problem of concurrency control over REST interfaces very interesting to explore. The problem is actually well addressed in HTTP so it wasn't that we had to invent a bunch of stuff. It was more about learning what was out there, reading about good/bad stories, and then mapping it to our model and see how it fitted.

In the end we landed on what I think is a quite nice place. If you're interested, check out this write up in the Astoria team blog on the topic.

-pablo

Unifying service interfaces

pabloc — Sat, 01 Mar 2008 01:37:00 GMT

As David Treadwell announced yesterday, we are starting to align the Windows Live services interfaces to use the AtomPub protocol, and to have a uniform set of conventions that are shared across internet services and the Project Astoria bits.

What does that mean? It means that starting now (and more in the future) you can interact with many consumer and infrastructure services in the same way. It also means that those Microsoft services out there and your own services -online or on-premises- created with the ADO.NET Data Services Framework look and behave in highly compatible ways.

You won't need to learn yet another API every time a new service comes out. Furthermore, we (or you, or services providers out there) won't need to create new APIs for every service. Highly uniform interfaces bring the opportunity for extensive client library and tools re-use across services of broad nature.

We've been talking about this for a while...and it's exiting to see the first pieces start to come in place.

If you want to know more, see live demos of services and tools working together, and chat about the details, we'll be going through all of this at Mix 2008. There is a talk specifically about this called "Accessing Windows Live Services using AtomPub" where I'll be telling the whole story and showing the thing working live.

More details about our Mix sessions here.

-pablo

"Data Friction", spot-on

pabloc — Thu, 21 Feb 2008 03:22:00 GMT

Jon Udell wrote a brief piece on how data is locked on servers behind UIs that were not designed for data sharing. He views this as "data friction"...it's just the perfect way to describe the problem.

I couldn't agree more with Jon's take. I would even take it further: an operation-centric approach to interfaces is good for closed systems or systems where semantics are completely centered around behavior; however, the data behind those operations is still somewhat locked-in. Data-centric APIs that expose a uniform interface for clients to consume is how the class of scenarios that Jon talks about will come to life. We can't tell upfront how all those applications out there will use our data, it's just too hard to predict, so a "function centric" interface just won't do. Instead, those systems need put out the data plus a way to interact with it that enforces proper semantics while not getting in the way of how a consumer wants to explore that data.

We're hoping to help at least a bit in that space with Project Astoria...we'll see how it goes :)

-pablo

A data service is not a database

pabloc — Fri, 10 Aug 2007 04:13:00 GMT

Since we put out the new version of the experimental Astoria online service there has been a broad set of questions coming, which is great because it was the goal of the thing...we're learning a bunch from it.

A class of questions that we are getting are "database questions". Things such as "how can I enforce a constraint" or "can I create an index?" or "what if I want to use hints in my queries".

I think that a key thing to understand is that a data service is not a database. There are many aspects that are fundamentally different when you design, secure, deploy, do capacity planning and such between these two.

Schema

Database schemas are typically heavily optimized for performance. At the logical level tables are often arranged to balance grouping important data together with row-density to make common scans over a subset of the columns as fast as possible; normalization and de-normalization decisions are made so that the database remains reasonably queryable and maintainable while avoiding too many joins, etc. Beyond that, in a database you have access to the physical level, and there it's common to tweak indexes, create materialized views, create covering indexes, organize data in file-groups, and so on.

In a data service, your schema becomes part of your public interface. When you publish an Astoria data service (or something similar), the URI space is dictated by the set of top-level resource containers, the members of the types it can contain and related constructs. The shape of the payloads will also be based on the various types contained in the various resource containers exposed by the service.

So in a service you'll want the schema to be optimized for its target use, so semantics tend to drive it. In a database schema will be about data organization and performance.

The gap between the two, in the case of Astoria, is filled by the ADO.NET Entity Framework mapping engine, which you can use if you have a service schema that's different than your database schema, which I think will be the common case for anything that's not a quick sample.

Physical aspects

As I mentioned when discussing "schema", when you create and manage a database you worry a lot about the physical level. This ranges from indexes to file-groups to spreading the data across disks in a smart way.

A service is a different beast in this sense as well. In a service you just put the data "up there". The service will choose the appropriate physical organization for the data, regardless of the visible service schema. (for example, in the Astoria experimental online service you describe your data as entities and associations and the system figures out a logical/physical schema to support it, along with a mapping to translate between them). The system supporting the service may or may not use a relational database, or even a database at all (there are many large-scale storage systems that use other models instead of traditional relational to avoid the impact of global metadata and the complexity of partitioning highly structured schemas for scale-out).

Interaction model

Even the way clients interact is different. Databases have a concept of a "session" which has a bunch of state in it; you can even do transactions, which carry even more state. In a service that has an HTTP interface it would be quite unnatural to have sessions, and certainly you don't want to have something like a transaction that spans more than a single HTTP request if you want to scale to large concurrent workloads.

Also, particularly in larger databases, access to the database is often restricted to middle-tier systems that are "trusted" by the database. In some cases those systems may even be allowed to do certain level of dynamic SQL (usually the larger the system or the scalability requirements, the less likely to allow dynamic SQL). In the service side of things, while you want to allow certain level of flexibility on how the data is obtained and shaped, you may not want to take arbitrary programs in a language as expressive as SQL and execute them in your systems.

Semantics and business logic

While sometimes some business logic is pushed to the database for practical reasons such as performance, typically most of it happens in the middle-tier, and then you connect all your client components to the middle-tier.

When you have a data service exposed to the web you cannot assume anything about the client. You cannot trust that the client is legitimate (there are ways to do this...it depends on your target scenario if you want to be restrictive or not), you may need to handle varying versions and you may need to handle clients written by other people that may or may not follow your requirements. All that means is that the data service does need to do some validation/business logic execution before it allows you to see data or to manipulate it. In some form, it acts as a middle tier that sits across the web from the client.

There are several other aspects I did not elaborate on here that differentiate these things. The bottom line is that while there are some similarities, in general it is important to avoid the tendency to assume that these things are the same.

-pablo

We'll host an experimental Astoria data service for you

pabloc — Sat, 04 Aug 2007 01:49:00 GMT

When we announced Project Astoria at Mix 2007 last May we made the toolkit available along with a few samples as an online service. The missing piece there was the ability to create your own service online in the experimental setting we had setup. Now Mike has just announced the availability of the new version of the experimental online service that allows you to describe the schema of the data you'd like for your data service, and we'll create and host it for you.

So go ahead, create your data service and let us know how it works for you. To access the online service you can use HTTP directly or the client library that was included in the May 2007 CTP.

-pablo