Pablo Castro's blog : REST

Every SharePoint 2010 server is a Data Services server

pabloc — Tue, 20 Oct 2009 06:26:00 GMT

I haven't been writing much here, mostly because I've been way too busy but also because I couldn't discuss publicly many of the things I'm doing. Now that SharePoint 2010 has been announced and its feature set published everywhere, I can finally discuss one of the coolest things we've been up to lately.

SharePoint is a repository of resources (list items and documents in document libraries) that are collected and manipulated collaboratively. Resources have a bunch of security and business logic attached to them, such as who can see each item, who gets to change it, or whether a particular column in a list needs to conform to a particular validation formula.

When SharePoint folks said they wanted a RESTful interface this was great news...the system is just a perfect fit. Not only it's a perfect fit for RESTful services in general, but also for Astoria in particular. In the end SharePoint is very data-centric in nature, it already supports queries and business logic as part of the uniform interface.

So we're really excited to announce that as of SharePoint 2010, every SharePoint server is an Astoria server out of the box. No configuration required or anything, just make sure the proper version of ADO.NET Data Services is in the box. For SharePoint 2010 beta, the "right" version is ADO.NET Data Services v1.5 CTP2. We'll put details out there for future iterations as they come.

Official write up in the Astoria team blog.

The SharePoint Data Service head is not just a side integration deal, it's a full-on REST-over-HTTP head for SharePoint. It supports browsing as well as modifying data using regular HTTP verbs (GET, PUT, DELETE, etc.), it does ETags for concurrency control, enforces business logic as part of side-effecting methods, and it handles the full range of Data Services conventions for URLs, Atom and JSON payload formats, etc. It also exposes full metadata like any other Astoria service, allowing Visual Studio and any other metadata-driven client to give you a great experience on the client side. Now if you need to get or manipulate data in SharePoint from any platform in any language, all you need is an HTTP stack.

The other cool aspect is that the SharePoint folks where able to build this entirely on top of public bits, using our new fancy provider model for advanced data sources I discussed here some time ago.

SharePoint joins a growing family of Microsoft products that enable users to share data through the use of a simple RESTful interface that follows the Astoria conventions when needed (e.g. URLs, structured data in Atom). Another example on the server side include SQL Server Reporting Services, which in SQL Server 2008 R2 can now render any report as Atom feed that follows the Astoria conventions (examples here). On the client, in addition to developer-oriented options such as .NET, Silverlight, AJAX, PHP, Java and more, we now also have PowerPivot (f.k.a. Gemini) which can pull data from any data service and do fancy analysis and publishing over it, making it trivial to bring data from Data Services into Microsoft Excel 2010 among other things.

I find this to be extremely important. A simple, uniform way of sharing data at the protocol level, and in a way that truly enables the lowest possible bar of entry, is key to enable broad integration across products and break data silos that form around applications.

I'll be talking about this at the SharePoint conference 2009 in Las Vegas this week, and at the Professional Developer Conference (PDC 2009) in late November. If you're attending any of these and this sounds interesting, these sessions will drill into plenty of details.

-pablo

PHP toolkit for ADO.NET Data Services

pabloc — Fri, 21 Aug 2009 19:42:00 GMT

Folks in the interoperability team at Microsoft just announced something they've been cooking for a while, developed Persistent Systems: a client PHP toolkit for Astoria services. It follows more or less the model of the .NET client where you can run a tool during development to get code-gen based on a data service metadata, and then it has a runtime library that can be used to send queries, obtain responses as PHP objects, track and submit changes, etc. They implemented both the Atom and JSON formats, and even threw in batching support.

The best part is that it's all written in PHP (both code-gen tool and runtime), so it runs in any environment where PHP and the toolkit dependencies (XML, XSLT, CURL) are available.

For more details check out the post in the interoperability team's blog, the kick off Channel 9 video, and the PHP toolkit project in codeplex.

-pablo

Adding support for JSONP and URL-controlled format to ADO.NET Data Services

pabloc — Thu, 26 Feb 2009 06:29:33 GMT

JSONP is a common way of making data accessible in client-side mashups even when the requests need to be cross-domain.

While the current version of the ADO.NET Data Services framework does not support this, it’s possible to build it on top. There are a couple of ways of doing this. Here is what’s probably the simplest way. There is some downsides to this approach, but overall is the most straightforward path to get there.

The default transport layer for Data Services is WCF, which has a many extensibility points across the stack. For the case of JSONP support, IDispatchMessageInspector comes in handy.

There are two things needed to support JSONP properly:

The ability to control the response format. Data Services uses standard HTTP content type negotiation to select what representation of a given resource should be sent to the client (e.g. JSON, Atom). That requires that the caller can set the Accept request header, which is not possible when doing the JSONP trick (which basically just uses <script> tags). We need to add the ability to use the query string in the URL to select format. (e.g. /People(1)/Friends?$orderby=Name&$format=json).
A new option to wrap the response in a callback if such callback was provided in the request (also in the query string). For example /People(1)/Friends?$orderby=Name&$format=json&$callback=loaded.

What we’ll do is register a message inspector and adjust the request/response when we see these new options coming in.

In order to support the $format=json option we can intercept the message before it gets dispatched to the Astoria runtime, at the IDispatchMessageInspector.AfterReceivedRequest method. If we see the query string option then we’ll a) strip it out from the URL so Data Services does not generate an error and b) change the “Accept” header to “application/json”, so the rest of the system just thinks that the client asked for a JSON response in the first place.

For the second part, where we need to wrap the response into a Javascript call if the $callback option was used, we have the IDispatchMessageInspector.BeforeSendReply method which gives us the perfect spot to rewrite the response. One unfortunate side-effect of this is that the response will get buffered and re-encoded; that said, in many cases this won’t make any noticeable difference.

Finally, we need to register the interceptor with WCF’s dispatchers. For that we create an attribute that implements IServiceBehavior, so we get called during service initialization. When we get called we can register our message interceptor.

The net effect is that if you include this code in your project, you just need to add a single attribute to your Data Service to make it support JSONP:

[JSONPSupportBehavior]
public class SampleService : DataService<ContactsData>
{
// your service code here...
}

Once that's in place you can use JSONP by adding $format and $callback to URLs, for example:

http://<host>/SampleService.svc/People?$format=json&$callback=cb

Of course, you can still use all the other Data Services URL options in addition to these.

The implementation and a small sample service are available at MSDN code gallery, here:

http://code.msdn.microsoft.com/DataServicesJSONP

-pablo

REST and Concurrency Control

pabloc — Wed, 23 Apr 2008 04:13:00 GMT

As part of the Astoria design process we scanned through many topics, some of them are straightforward, some are hard but mostly mechanical, but there are some that become interesting, fundamental aspects to address.

I found the problem of concurrency control over REST interfaces very interesting to explore. The problem is actually well addressed in HTTP so it wasn't that we had to invent a bunch of stuff. It was more about learning what was out there, reading about good/bad stories, and then mapping it to our model and see how it fitted.

In the end we landed on what I think is a quite nice place. If you're interested, check out this write up in the Astoria team blog on the topic.

-pablo