ADO.NET Data Services Team Blog : Design Video

Design Notes: Row Count

dpblogs — Tue, 16 Dec 2008 00:18:34 GMT

One of the scenarios we have heard feedback around is the ability for the a client of a data service to determine the total number of entities in a set without having to retrieve them all. The following video discusses this scenario in more detail and some of our thoughts regarding how to make the experience better.

Astoria Design Walkthrough: Introducing $count as an URI query option

Additional design details:

- The count value would be calculated after $filter expressions are applied. For example /Customers?$filter=City eq 'London' would return a count value made up of only the customers living in London and not all the customers.

- If a query included $top, $skip or $orderby, the count value would be unaffected. For example /Customers?$top=1&$skip=2&$orderby=Name would return the total number of customers stored by the service.

- If a query includes a $expand, then the count applies to the outer entity set. For example /Customers?$expand=Orders would return the number of customers stored by the service

What do you think? Would this be useful in your data services applications?

Mike Flasko

ADO.NET Data Services, Program Manager

This post is part of the transparent design exercise in the Astoria Team. To understand how it works and how your feedback will be used please look at this post.

Alpha preview of Project Codename "Astoria Offline" coming soon

dpblogs — Tue, 04 Nov 2008 02:01:58 GMT

As we already discussed in a previous blog post, one of the problem spaces related to data services we’d like to take on is synchronization-capable services, enabling offline scenarios. We’re still in the early stages of the project, where direction can be adjusted and good feedback can be very influential.

Giving good feedback without actually having something to look at it’s tricky, so we want to ship bits. Earlier at the PDC talk that showed this technology end-to-end for the first time we announced that we’ll be shipping a very early release of the bits that extend our entity platform, including the Entity Framework and ADO.NET Data Services, with synchronization capabilities through integration with the Microsoft Sync Framework and a nice toolset that makes development of offline-capable applications reasonably straightforward.

Here is a brief video with Waseem and myself discussing the release. Waseem was probably the most involved developer so far in this effort and has been dealing with a lot of the challenges that we hit as we put all the pieces together and build new ones on top.

Astoria Design Walkthrough: Alpha preview of Project Codename "Astoria Offline" coming soon

Just like the first release of “Astoria” almost two years ago, the goal if this alpha release is to paint the picture of what we want to build, what scenarios we consider interesting and how we want to approach the tooling around it. It’s definitely not a goal to ship something that’s usable in production or even something that can be used to start writing real applications now. Things will change substantially as we hear feedback from all of you.

You can expect the release to ship before the end of this year, and to include both runtime and tool components for the whole thing. If you care about synchronization and offline applications, giving this release a try and sending us your thoughts, comments and suggestions would help build the right technology in this space.

Pablo Castro
Software Architect
Microsoft Corporation

This post is part of the transparent design exercise in the Astoria Team. To understand how it works and how your feedback will be used please look at this post.

Astoria futures: offline-enabled data services

dpblogs — Thu, 23 Oct 2008 04:32:00 GMT

We mentioned that we were doing some early thinking of “Astoria Offline” back in Mix 2008, where we even demo’ed an early proof of concept. Now we’ve been working on various design aspects of Data Services for its future versions, and synchronization/offline support is one of them. It’s still an experimental thing with no official home or release vehicle, so this is the best time to follow the design process if you find the scenario interesting, as this is when it’s easiest to influence the direction we’ll go for.

A short way of describing this can be: “imagine you can point Visual Studio to a data service and say ‘take it offline’, and things just happen”.

Of course, the real world is more complicated than that :-)

Astoria Design Walkthrough: Thinking of a future with sync & offline

In this first note I’ll just touch on the scenarios we want to hit and go over a few guiding principles. In future posts I’ll elaborate more on the details.

We have many scenarios in mind for this infrastructure. The ones we’re thinking of tackling first:

· Outlook type of apps: I’m sure there is a fancier way of saying this, but anyone that has used Microsoft Outlook knows what I mean. The application is basically a 1-tier app that interacts with a local (embedded) database. In the background -and independent from UI activity- 2-way synchronization with a data service (e.g. a Microsoft Exchange server) happens. Often sync’ing against a database is not quite what you want…”Astoria Offline” will let you sync against your data-service layer, where the usual business logic/validation/etc will run just like in the online path.

· The description above sort of implies that server and client are built in collaboration, perhaps as part of the same development team. That’s certainly an scenario and we can do some things easier when that’s the case. But the other scenario we want to tackle is when client and server in a synchronization relationship are independent from each other (e.g. sync a service that’s just available for sync on the web).

· Local replicas of cloud-stored data: as more online services offer structured storage capabilities, and more of them use the Data Services REST interface, it becomes more interesting to be able to synchronize that data locally either for latency reduction, offline operation or other reasons.

· Data consolidation: if you have multiple data services that expose data from a variety of sources (some databases, some online/”cloud” stores, some custom repositories), you may want to synchronize a slice of data of each store to a local database, and then work with the data locally.

A couple of guiding principles:

· We will stick to a simple and open interface. What that means is that while we will definitely build a nice end-to-end integrated story for Visual Studio, it will be on top of a well-documented underlying data exchange using just HTTP and known formats. Anybody with an HTTP client and enough knowledge of our sync strategy should be able to synchronize with a data service.

· Data independence will remain there for sync as it is already for online access. Today when you access a data service the interface is the same regardless of whether the service is backed by a database, a cloud store, some custom application or whatever. With sync, the same applies. If the data service is sync-enabled you can sync with it, no matter what backs it.

· We are targeting data services for structured stores and business applications. That implies certain level of sophistication in the shape of data, such as assuming cross-item dependencies, store-level and application-level constraints that dictate consistent states of data, the need for making partial progress during synchronization, etc. Such support does come with some extra complexity, but we think it’s the right target.

We’re just taking on this space, so any feedback you may have is good. Are our initial scenarios interesting? Do you need this thing at all? Does the initial direction we’re looking at sound reasonable?

btw – if you are going to be at PDC, we have a full talk on this at the event.

I hope this “short video” format that Andy wants to do for our design notes adds a good little twist and makes them more interesting.

Pablo Castro
Software Architect
Microsoft Corporation
http://blogs.msdn.com/pablo

This post is part of the transparent design exercise in the Astoria Team. To understand how it works and how your feedback will be used please look at this post.

Making Feeds Friendly

dpblogs — Mon, 29 Sep 2008 01:17:49 GMT

The goal of Astoria in V1 is to easily expose a data source using the REST approach to the web (with support for concurrency, versioning, auth, etc). In general, the way we think about serialization formats (ie. how we represent entites on the wire) is that they are a tool and an application will select the best tool (ie. serialization format) for the job. For example, AJAX developers will likely opt to interact with Astoria services using our JSON serialization format while .NET developers may choose AtomPub since our .NET client library uses that format under the covers. In V1, Astoria supports using the AtomPub and JSON formats as a general purpose data exchange representation and has fixed requirements regarding how an entity is to be represented using the formats. While this works well for a number of app scenarios, we’ve received feedback regarding use cases (one example being mashups) where an app/tool/etc is written to consume general Atom feeds and can then make assumptions about the data given (ie. what the title is, who was the author, etc). This area of thinking/exploration (uses of feeds in the wild and how to make Astoria’s feeds just work in certain scenarios) has been dubbed by our team as “Friendly Feeds”.

One of the specific scenarios we’ve discussed is a mapping site (ie. virtual earth, etc) which accepts feeds as an input and then displays the feed information on the appropriate spots on the map. In this scenario the mapping site would pull metadata from the well known Atom elements (title, author, content, etc), but also needs to locate geo information to be able to determine where on the map to place the data. For this we’ve seen some feeds start to embed micro formats in feeds such as georss. To make this more concrete, imagine being able to drive the example shown here via an Astoria feed.

We are still thinking about this space, but right now the high level goals we see as necessary for such a feature are listed below. What do you think?

Goals:

· Enable feeds to be easily customized such that a generic feed viewer can render an Astoria feed

· The feed customization approach could be applied to feed-based formats other than Atom. JSON serialization will not support this type of “Friendliness”

· Enable entity properties to be serialized as microformats within a feed

· Serialization customizations “just work” with V1 data service client libraries wherever possible. Changes that require a protocol version number increase should be explicit and off by default

· The ADO.NET Data Services client library (for .NET Fx and Silverlight) would be able to roundtrip a “Friendly Feed” in the same way the client roundtrips data in Astoria V1.

· We would also teach our code generation tools to understand such “Friendly Feeds” and generate the appropriate clients.

· It is NOT a goal to make ADO.NET Data Services into a blog server -- there are better domain-specific tools for that

Oh, I almost forgot……a nice benefit is that with this feature Astoria Atom feeds could likely be made directly viewable in IE. Folks who have inspected AtomPub-based feeds in web browsers today know what I’m referring to.

Below is a short video we recorded that discussed this area as well. Andy (Astoria Dev Lead) got a new camera and came up with the idea of adding these videos. Let us know what you think of having short video snippets such as this to accompany our design notes as we blog them....

Astoria Design Walkthrough: Friendly Feeds Part 1

In a future post, I'll include some specific examples to show how we will enable this in our server runtime and client libraries.

Cheers,
Mike Flasko

ADO.NET Data Services, Program Manager

This post is part of the transparent design exercise in the Astoria Team. To understand how it works and how your feedback will be used please look at this post.

ADO.NET Data Services Team Blog : Design Video

Design Notes: Row Count

Alpha preview of Project Codename &quot;Astoria Offline&quot; coming soon

Astoria futures: offline-enabled data services

Making Feeds Friendly

Alpha preview of Project Codename "Astoria Offline" coming soon