Marcelo's WebLog : Astoria

Download Data Services Update for .NET 3.5 SP1

marcelolr — Thu, 17 Dec 2009 22:00:00 GMT

Yesterday the availability announcement went up on the team blog, so why wait? Get yourself some updated bits and start data binding, blob'ing, paging, projecting, and counting entities, all from within the comfort of your programming chair!

If you haven't already, now is a great time to subscribe to the feed on the team blog. New content to help you make the most of the update will be posted regularly now that the release is out.

Enjoy!

Vistual Studio visualizer for OData

marcelolr — Thu, 19 Nov 2009 22:00:00 GMT

Fabian Winternitz is one of our awesome tools developers, and you can see some of his work in this post. That's a Visual Studio extension that will display searchable diagrams for the model exposed by any service endpoint that exposes OData metadata. Which these days includes a lot of software.

Personally, I think it's great to see on one hand the level of support for the ecosystem, and on the other hand the balance of power, expressiveness and simplicity that OData provides. Back when everything started in the early days of Project "Astoria", I don't think anyone had any idea as to how broad a spectrum of software components would play in this space, but it looks like sticking to simplicity and proven standards paid off.

Enjoy!

WCF Data Services

marcelolr — Wed, 18 Nov 2009 22:00:00 GMT

Over at the team blog, you can find this post about ADO.NET Data Services changing its name to WCF Data Services in the .NET Framework 4 time frame. This is goodness by every account - over time, this just means a more integrated experience when developing for these stacks, with less seams and caveats.

Enjoy! - in anticipation if nothing else :)

Project Codename "Astoria Offline", Alpha Preview

marcelolr — Wed, 11 Mar 2009 00:00:00 GMT

As recently announced, early preview bits are available for Astoria that provide synchronization capabilities, which allow you to create applications that work offline (among other things).

Note that this announcement was made close to the "1.5" made not too long ago, but they are separate, distinct things that will not play well if installed on the same machine. You really, really don't want to mix the two.

The idea here is really to reach out, give people a feeling of what the problem space looks like and what a solution might look like, and help us get feedback and have a conversation around this. As usual, the team blog and the forums are your best friends.

Enjoy!

A plea for ADO.NET Data Services black belt

marcelolr — Tue, 17 Feb 2009 01:00:00 GMT

Phani, master and commander of all things Astoria, has in his posession pictures of the latest ski trip... (yes, the team definitely knows how to have fun)

This is just my way of publicly calling unto him to post them. Peer pressure in action!

ADO.NET Data Services @ PDC

marcelolr — Mon, 27 Oct 2008 20:00:00 GMT

I am certain you ardently desire to keep up to date on everything Astoria (I mean, ADO.NET Data Services), so if you're going to the PDC, the sessions you'll want to go are listed here, along with a video showing Pablo and Mike plugging them.

I especially enjoy how the still frame always managed to catch Pablo in the most awkward pose possible.

Enjoy!

ADO.NET Data Services History

marcelolr — Fri, 29 Aug 2008 02:00:00 GMT

Pablo puts on his historian hat over at http://blogs.msdn.com/pablo/archive/2008/08/20/timeline-of-project-astoria.aspx and tells the tale of how Astoria (now ADO.NET Data Services) was born and raised.

Some of my own memories:

The very first mail on my 'Astoria' mail folder, sent to Pablo, titled Pinging for OOF message (I guess he was out and I wanted to get his out of office message, which typically includes information as to when someone is coming back). He eventually replied nonetheless with instructions on the version control client configuration and a couple of links to documents that had already been prepared.
Lovingly organizing our team's SharePoint Wiki. Today it has grown to 74 pages with all sorts of information, from links to the Microsoft download site for releases to tips on useful breakpoint locations.
Everyone ordering copies of RESTful Web Services, by Leonard Richarson and Sam Ruby. I somehow ended up with at least two copies at some point.
The design marathon, which actually lasted longer than a month (in my mind, anyway). I remember we used to get together every other day for two to three hours to nail down the specifics of what we wanted.
The people on the team. A great combination of being very smart, very easygoing and a lot of fun to work with. The RESTing on the slopes pictures have achieved everlasting fame by now.

Good times! I can only hope that developers that use our platform have as much fun as we did creating it.

System.Data.Services

marcelolr — Tue, 12 Aug 2008 05:29:00 GMT

The documentation is online... isn't this URI so very clean?

http://msdn.microsoft.com/en-us/library/system.data.services.aspx

Here's the one-sentence-or-less explanation of what's in this namespace, to give you a sense of what's used where.

Setting Up Your Service
DataService<T> is the class that is typically subclassed, using the data source type as the 'T' type parameter, and brings everything together.

QueryInterceptorAttribute and ChangeInterceptorAttribute are used to plug into the read and write operations (UpdateOperations is used in the latter). You can also plug in by overriding the OnStartProcessingRequest method, which takes a ProcessRequestArgs argument.

IDataServiceConfiguration is used to set up configuration limits and uses EntitySetRights and ServiceOperationRights to figure out what operations are allowed where.

Data Support
ETagAttribute, IgnorePropertiesAttribute and MimeTypeAttribute are used to annotate models when you're not using the ADO.NET Entity Framework.

SingleResultAttribute is used on service operations to indicate that they return a single item, and mostly affects how URLs over it are interpreted and composed.

IUpdatable is the interface to implement to support insert/update/delete operations when you're not using the ADO.NET Entity Framework.

Expand Support
This will be very, very rarely used - you would only have to know about any of this if you wanted to use your own implementation of the $expand query option. IExpandProvider is the interface to implement, ExpandSegmentCollection and ExpandSegment are used to talk to it, and IExpandedResult is used to return wrapped results when expansion cannot occur in-line with the object model.

Hosting the Service
DataServiceHost, DataServiceHostFactory, and IRequestHandler are used to get the service working with Windows Communication Foundation (WCF), and usually there's nothing to be done to get it up and running - the item template for an ADO.NET Data Service has the right directive to wire everything up. IDataServiceHost is the interface to implement if you want to host the data service without using WCF.

Error Handling
DataServiceException is an exception that knows about HTTP status codes and "safe for network" messages.

HandleExceptionArgs is used as an argument in the DataService<T> HandleException method, and allows you to tweak an exception that was caught for processing.

Well, that's it for the namespace - as short an explanation I can give while still giving context as to the use for each type.

Enjoy!

ADO.NET Data Services RTMs!

marcelolr — Tue, 12 Aug 2008 00:52:00 GMT

The team blog has the announcement over at http://blogs.msdn.com/astoriateam/archive/2008/08/11/rtm-is-here.aspx.

All in all, we're very happy with this release - we hope you like it too, and it allows you to do a better and more productive job of writing software that moves data across the network.

Enjoy!

Service Operations in ADO.NET Data Services

marcelolr — Tue, 22 Jan 2008 10:06:00 GMT

One way in which you can expose additional resources from your ADO.NET Data Service is to implement "service operations" on your WebDataService subclass.

For example, let's say we want to return all customers in a given city in a pre-baked entry point. We can write this code on the server:

public class WebDataService1 : WebDataService< Model.Entities >
{
    public static void InitializeService(IWebDataServiceConfiguration config)
    {
        config.SetResourceContainerAccessRule("Customers", ResourceContainerRights.AllRead);
        config.SetServiceOperationAccessRule("CustomersInLondon", ServiceOperationRights.All);
    }

    [WebGet]
    public IQueryable<Model.Customers> CustomersInLondon()
    {
        return from c in this.CurrentDataSource.Customers
               where c.City == "London"
               select c;
    }
}

Here's what's going on:

In InitializeService, we make sure that the resource set we have is visible.
Next, we make sure that the service operation we have is also visible.
We declare a method, CustomersInLondon, which returns an IQueryable of customers.
We add an attribute to the method, WebGet, which indicates we want to allow GET operations on this method.
Finally, we return the query we're interested in. Now that there is a property CurrentDataSource, of type Model.Entities (the 'T' in WebDataService<T>), which we can use as the querying context.

Now we can hit F5 and run our project, and navigate to the following URL to get customers from London.

http://localhost/WebDataService1.svc/CustomersInLondon

Also, if we look at metadata on http://localhost/WebDataService1.svc/$metadata, we'll see the following bit of CSDL:

<FunctionImport Name="CustomersInLondon" EntitySet="Customers" ReturnType="Collection(Model.Customers)" />

Note that because we're returning an IQueryable, we can keep composing over the returned results, for example by filtering them:

http://localhost/WebDataService1.svc/CustomersInLondon?$filter=ContactTitle%20eq%20'Sales%20Manager'

Or navigating through them:

http://localhost/WebDataService1.svc/CustomersInLondon('AROUT')/Orders

Let's say that we didn't want to allow clients to do this - instead, we want to return a very "locked down" set of results. In that case, we simply need to change our method to return an IEnumerable result rather than an IQueryable. Note that even if the actual type is an IQueryable, as long as the result type is declared as an IEnumerable, the server won't allow composition over it.

[WebGet]
public IEnumerable<Model.Customers> CustomersInLondon()
{
    return from c in this.CurrentDataSource.Customers
           where c.City == "London"
           select c;
}

So this URL still returns results as expected: http://localhost/WebDataService1.svc/CustomersInLondon

But this fails with a message 'Query options $expand, $filter, $orderby, $skip and $top cannot be applied to the requested resource.': http://localhost/WebDataService1.svc/CustomersInLondon?$filter=ContactTitle%20eq%20'Sales%20Manager'

Let's say that now want to allow the customers to tell us which specific city they're interested in, rather than hard-coding the city name.

We can add the following method to our class to enable this.

...
public static void InitializeService(IWebDataServiceConfiguration config)
{
...
config.SetServiceOperationAccessRule("CustomersInCity", ServiceOperationRights.All);
}

[WebGet]
public IQueryable<Model.Customers> CustomersInCity(string cityName)
{
    return from c in this.CurrentDataSource.Customers
           where c.City == cityName
           select c;
}

Now we can get results for this operation with the following URL: http://localhost/WebDataService1.svc/CustomersInCity?cityName='London'

As you can see, the syntax is simply the parameter name, an equals, and the value literal as you would have used in a filter or a key. The only parameter types currently supported are primitive types (numbers, string, DateTime, byte array, Guid). This is in following the the form encoding used by web browser for FORM tags.

If you want to pass parameters in the request body rather than the query portion of the URL, you can use WebInvoke on the service operation method instead of WebGet. The clients then use POST rather than GET, and pass the queries in the body of the request, using application/x-www-form-urlencoded encoding.

There are two more attributes that are interesting for service operations. You can use the Microsoft.Data.Web.SingleResultAttribute attribute to indicate a single result will be returned from a specific operation.

...
public static void InitializeService(IWebDataServiceConfiguration config)
{
...
config.SetServiceOperationAccessRule("LastOrderDate", ServiceOperationRights.All);
}

[SingleResult]
[WebGet]
public IQueryable<DateTime> LastOrderDate()
{
    var result =
        from o in this.CurrentDataSource.Orders
        orderby o.OrderDate
        select o;
    return result.Take(1).Select((o) => o.OrderDate.Value);
}

You can now access this value from the following URL: http://localhost/WebDataService1.svc/LastOrderDate

And finally, the MimeTypeAttribute can be returned to indicate that a given MIME type applies to the returned value of an operation.

...
public static void InitializeService(IWebDataServiceConfiguration config)
{
...
config.SetServiceOperationAccessRule("WelcomePage", ServiceOperationRights.All);
}

[MimeType("text/html")]
[SingleResult]
[WebGet]
public IQueryable<string> WelcomePage()
{
    return new string[]
    {
        "<html><head><title>Welcome</title></head>" +
        "<body><h1>Welcome!</h1><p>Currently we have " +
        this.CurrentDataSource.Customers.Count().ToString() +
        " customers.</p></body></html>"
    }.AsQueryable();
}

You can now access this information through the following URL: http://localhost/WebDataService1.svc/WelcomePage/$value

Now, there are far better methods for generating web pages, but I just wanted to give a little taste of how having a (simple!) uniform interface for accessing resources allows you to integrate disparate systems, like a web browser and an ADO.NET Data Service.

This post is part of the transparent design exercise in the Astoria Team. To understand how it works and how your feedback will be used please look at this post.

Arithmetic and built-in functions for $filter

marcelolr — Wed, 16 Jan 2008 04:03:00 GMT

While the syntax described for filter in the previous post allows you to do some nifty things, there are still a few more things that an ADO.NET Data Service supports.

The first are operators for arithmetic: +, -, *, /, %. These also have mnemonics:

+	add
-	sub
/	div
*	mul
%	mod

So the following is a very elaborate but silly filter:

/Region?$filter=(1 add 1 eq 2) and (1 sub 1 eq 0) and (1 mul 1 eq 1) and (1 div 1 eq 1) and (1 mod 1 eq 0)

There are also a number of built-in functions that you can use. These are fairly simple and are likely to be implemented in some way by the data provider for your data service. These functions must be recognized by the provider for the data source to be usable; the provider for Linq to Objects and Linq to Entities already recognize them and in the latter case, the Entity Framework is able to have them execute directly on the database server, for best efficiency.

The functions are the following. I'll give an obvious example of each that always evaluates to true, just to show how they can be used. Remember you can always use properties or more complex expressions in place of constants.

String Functions

contains - checks whether a string is contained in another one
/Region?$filter=contains('abc', 'ab')

endswith - checks whether a string ends with another one
/Region?$filter=contains('abc', 'bc')

startswith - checks whether a string starts with another one
/Region?$filter=contains('abc', 'b')

indexof - gets the index of a substring (zero-based)
/Region?$filter=2 eq indexof('abc', 'c')

insert - inserts a string inside another one
/Region?$filter='abcd' eq insert('ad', 1, 'bc')

tolower - converts a string to lowercase
/Region?$filter='abc' eq tolower('ABC')

toupper - converts a string to uppercase
/Region?$filter='ABC' eq toupper('abc')

trim - trims leading and trailing whitespace from a string
/Region?$filter='abc' eq trim(' abc ')

remove - removes characters from a string from a given offset (length optional)
/Region?$filter='abc' eq remove('a00bc', 1, 2)
/Region?$filter='a' eq remove('a00bc', 1)

substring - gets characters from a string from a given offset (length optional)
/Region?$filter='00' eq substring('a00bc', 1, 2)
/Region?$filter='00bc' eq substring('a00bc', 1)

concat - concatenates two strings
/Region?$filter='abc' eq concat('ab', 'c')

length - gets the length of a string
/Region?$filter=3 eq length('abc')

DateTime Functions

year - gets the year of a DateTime value
/Region?$filter=1990 eq year('1990-12-20')

month - gets the month of a DateTime value
/Region?$filter=12 eq month('1990-12-20')

day - gets the day of the month of a DateTime value
/Region?$filter=20 eq day('1990-12-20')

hour - gets the hour of the day of a DateTime value
/Region?$filter=14 eq hour('1990-12-20T14:10:20')

minute - gets the minute of the hour of a DateTime value
/Region?$filter=10 eq minute('1990-12-20T14:10:20')

second - gets the second of the minute of a DateTime value
/Region?$filter=20 eq second('1990-12-20T14:10:20')

Math Functions

round - rounds a value to its nearest integral value
Region?$filter=2 eq round(1.8)

floor - gets the largest integral value (in magnitude) that is less than or equal a given value
/Region?$filter=1 eq floor(1.8)
/Region?$filter=-2 eq floor(-1.8)

ceiling - gets the smallest integral value (in magnitude) that is greater than or equal a given value
/Region?$filter=2 eq ceiling(1.8)
/Region?$filter=-1 eq ceiling(-1.8)

Type Functions

isof - checks whether something is of the given type - useful in inheritance scenarios
/Region?$filter=isof('Model.Region') -> in the ATOM payload, I see 'Model.Region as the adsm:type - this depends on the namespace for your types)

cast - casts something to a given type - useful in inheritance scenarios
/Region?$filter=(cast('Model.Region'))/RegionID ge 0

Whew! Well, that's it for functions. Put them to good use!

This post is part of the transparent design exercise in the Astoria Team. To understand how it works and how your feedback will be used please look at this post.

Shaping the ADO.NET Data Services Silverlight Add-On

marcelolr — Sat, 12 Jan 2008 12:01:00 GMT

So, as per the announcement on this team blog post, there is now a library you can use from a Silverlight 1.1 Alpha project to access your data.

This is a great time to let us know what you think, in the post comments or in the forums. Is is too hard to use? Does it over-simply something that's important to you? Were you expecting something completely different?

There are more things that we'd like to do for this library, and you get to help us define the shape it will take. Speak up!

This post is part of the transparent design exercise in the Astoria Team. To understand how it works and how your feedback will be used please look at this post.

$filter Query Option in ADO.NET Data Services

marcelolr — Fri, 11 Jan 2008 08:24:00 GMT

Let's say that we have a data service exposing all of Northwind. We can get all customers in the database by accessing this URL.

http://localhost/WebDataService1.svc/Customers

That could be more data than we wanted. Let's say we only care about customers in London. We can use the $filter query option to get all customers that match this condition by accessing this URL.

/Customers?$filter=City eq 'London'

(I’m shortening and unescaping such URLs for readability; we actually follow standard URI conventions and the URI you’ll see will be something like http://localhost/WebDataService1.svc/Customers?$filter=City%20eq%20'London')

What can go in a $filter? The most basic thing to do is to test properties of the resources we're returning, which you can access simply by name, like we have done above. Literals for things like strings and numbers use the same syntax as in the key portion (see my last post and the original URL post for more details).

To actually test for a condition, you can use comparison operators, which look a lot like the symbolic ones you'd expect in C# but use short mnemonics to make them easier to write and not have to deal with escaping/unescaping in the URL.

eq	equals
ne	not equals
lt	less than
le	less than or equal
gt	greater than
ge	greater than or equal

So all of these are valid URLs:

/Orders?$filter=OrderDate lt '1996-08-01'
/Orders?$filter=Customers/CustomerID eq 'ALFKI' -> Note that you can traverse navigation properties that reference a single resource. The "Customers" plural is an artifact of the default mapping generated, the property is really a to-one navigation property.

To include more than one condition, you can use 'and', 'or' and of course 'not'.

/Orders?$filter=Customers/CustomerID eq 'ALFKI' or Customers/CustomerID eq 'WOLZA'
/Orders?$filter=ShipCountry eq 'France' and ShipCity eq 'Toulouse'

To group conditions logically, you can use parenthesis.

/Orders?$filter=ShipCountry eq 'France' and (ShipCity eq 'Toulouse' or ShipCity eq 'Marseille')

Today we don’t support operations on collections, which has two practical points.

First, we don’t lift values from collections, so you can’t have a filter such as /Customers?$filter=Orders/Name eq 'Foo' (this would imply a quantifier such as ‘any’ or ‘all’).
Second, we don’t have aggregates, so you can’t write a query such as /Customers?$filter=count(Orders) gt 5. This restriction helps to set bounds to any single operation and simplifies the queries that the server and clients need to deal with.

Next time, more on what you can put in a $filter.

This post is part of the transparent design exercise in the Astoria Team. To understand how it works and how your feedback will be used please look at this post.

Updates to URL syntax for December CTP of ADO.NET Data Services

marcelolr — Wed, 09 Jan 2008 06:43:00 GMT

This post talks a bit about updates from the last post on the URI format.

In that post, we discussed how we were considering having a "full form" and a "compressed form" for syntax. With the convergence of using ATOM and JSON as the serialization formats, we’ve been able to converge on the URI format as well and thus we no longer have both forms - we simply have the resource set name and a comma-separated list of key values in parents: /Customers('ALFKI')

The use of parens for values is a classic CSV-style design: a character to separate (,); a character to delimit the values so you can include the separator in them ('Lopez Ruiz, Marcelo','another string'), and a convention to double the delimiter to use it ('Marcelo''s World','Party time! Excellent!').

There are other types that don't really require delimiters, like dates, but because they come in handy when used in expressions on the query side of the string (after the '?'), we've kept them in the key portion as well. This way client libraries only need to deal with a single format. If this doesn't make much sense, wait a bit and I'll go into it in a post in the near future.

Everything else works pretty much as described in the post - feel free to post here or in the forums if you have any questions or want to provide feedback on this.

This post is part of the transparent design exercise in the Astoria Team. To understand how it works and how your feedback will be used please look at this post.

Fast, simple test app for ADO.NET Data Services

marcelolr — Thu, 03 Jan 2008 02:02:00 GMT

If you're currently looking at the ADO.NET Data Services functionality that is available through the ASP.NET 3.5 Extensions Preview download, you'll notice that using the project item to add a new service is fabulously simple.

If you're not doing anything serious and just want to play around with server models and look at the wire formats, however, it is very simple to create a quick app. There is very little to add over my last post in terms of setting things up - the only extra bit I've added here is a simple class that returns some sample data (using some very nice object and collection initializers). Getting the service going is also very simple - Christian Weyer goes into more detail as to how this all works together, and Mike Taulty already found this as well.

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using Microsoft.Data.Web;

namespace ConsoleApplication2
{
    public class Customer
    {
        public int ID { get; set; }
        public string Name { get; set; }
    }

    public class SampleContext
    {
        public IQueryable<Customer> Customers
        {
            get
            {
                return new List<Customer>()
                {
                    new Customer() { ID = 1, Name = "Marcelo" },
                    new Customer() { ID = 2, Name = "Andy" },
                }.AsQueryable();
            }
        }
    }

    public class SampleService : WebDataService<SampleContext>
    {
        public static void InitializeService(IWebDataServiceConfiguration c)
        {
            c.SetResourceContainerAccessRule("*", ResourceContainerRights.All);
        }
    }

    class Program
    {
        static void Main(string[] args)
        {
            string location = "http://localhost:20000/";
            DataServiceHost host = new DataServiceHost(
                typeof(SampleService),
                new Uri[] { new Uri(location) });
            host.Open();
            Process p = Process.Start(location);
            if (p != null) p.Dispose();
            Console.WriteLine("Press any key to shut down the service at " + location);
            Console.ReadKey();
            host.Close();
        }
    }
}

This posting is provided "AS IS" with no warranties, and confers no rights. Use of included script samples are subject to the terms specified at http://www.microsoft.com/info/cpyright.htm.