If you didn't get to TechEd - no problem - videos of all the sessions are available here.

Including my Best Practices: Creating an OData Service using WCF Data Services session.

If you watch that video and want to follow try it out - you can download all the source code I uploaded previously.

Enjoy.

At TechEd I got a lot of questions about how to expose Data as OData

By now you probably know you can use Data Services and the Entity Framework to expose data from a database as an OData Service. You might even know you can use Data Services with a custom Data Service Provider to expose arbitrary data from anywhere.

But did you know about the Data Services Reflection provider?.

Turns out reflection provider  is VERY simple to use.

To show you just how simple I am going to create an odata service to expose some in memory data.

First you need some data, so how about this topical data:

[EntityPropertyMapping("Name", 
                        SyndicationItemProperty.Title, 
                        SyndicationTextContentKind.Plaintext, 
                        false)]
[EntityPropertyMapping("Odds", 
                        SyndicationItemProperty.Summary, 
                        SyndicationTextContentKind.Plaintext, 
                        false)]
public class Team {     
   public static List<Team> Teams = new List<Team>{
         new Team {ID = 1, Name = "New Zealand", Odds = "1:1000"},
         new Team {ID = 2, Name = "Paraquay", Odds = "1:50"},
         …     }; 

    public int ID{get;set;}
    public string Name {get;set;}
    public string Odds { get; set; }
}
[DataServiceKey("Name")]
[EntityPropertyMapping("Name", 
                        SyndicationItemProperty.Title,
                        SyndicationTextContentKind.Plaintext, 
                        false)]
public class Group
{
    public static List<Group> Groups = new List<Group>{ 
      new Group { 
         Name = "A", 
         Teams = new List<Team>{ 
             Team.Teams[0],
             Team.Teams[1],
             Team.Teams[2],
             Team.Teams[3]
         }
       },      
      new Group {
         Name = "B",
         Teams = new List<Team>{
             Team.Teams[4],
             Team.Teams[5],
             Team.Teams[6],
             Team.Teams[7]
          }
       },
      …
    };
    public string Name {get;set;}
    public List<Team> Teams {get;set;}
}

Notice the use of the [DataServiceKey] and [EntityPropertyMapping] attributes:

• [DataServiceKey] tells Data Services which property(s) is the key. This is only required if the key isn’t ID or {ClassName}ID.
• [EntityPropertyMapping] tells Data Services to map the property to a standard atom element. This is not required – but it is a best practice – it makes the resulting feed easier to read in a browser.

Next you need a class to act as your Data Source. Data Services will expose all the IQueryable properties as Feeds and infer types for all the types exposed by those Feeds.

So if I want two feeds, to expose Groups and Teams from the world cup, I’d do it like this:

public class WorldCupData
{
     public IQueryable<Team> Teams{
        get{
            return Team.Teams.AsQueryable();
        }
     }
     public IQueryable<Group> Groups{
        get{ 
           return Group.Groups.AsQueryable();

       }
     }
}

Now all you need to do is create the Data Service, and expose our sets. Simply add a WCF Data Service to you web application and modify the generated code to look like this:

public class WorldCup : DataService<WorldCupData>
{
     // This method is called only once to initialize service-wide policies.
     public static void InitializeService(DataServiceConfiguration config)     {
         config.SetEntitySetAccessRule("*", EntitySetRights.AllRead);
         config.SetEntitySetPageSize("*", 100);
         config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V2;
     }
}

That’s it you are done:

A couple of points worth noting:

• This service is read-only. If you want to make it read-write you have to implement IDataServiceUpdateProvider.
• In this example the data comes from in-memory arrays, however if you have an IQueryable that supports it your data can come from anywhere.
• Yes I know my data is all wrong, I’m writing this on a SouthWest flight from New Orleans to Denver with no internet. But thankfully there is a real OData Service for the World Cup here.

Enjoy.

Yesterday I promised to share all the code from my Best Practices – Creating an OData Service using WCF Data Services session at TechED.

Note: you can find a recording of that session here.

So here goes, essentially this is what I did:

1. Downloaded, unzipped, opened and ran the MVC Music Store Sample. 
2. Added an album to my cart, registered and ordered the album.
3. Added a data service to that project to expose the Entity Framework model already in the Music Store sample:
   
   public class MusicService : DataService<MusicStoreEntities>
       
4. Added specific EntitySetAccessRules:
   
   // We don’t want the Data service to expose carts at all config.SetEntitySetAccessRule(
      "Carts",
       EntitySetRights.None
   );
   // You can only get 1 OrderDetail at a time
   config.SetEntitySetAccessRule(
      "OrderDetails",
       EntitySetRights.ReadSingle
   );
   // Everything else you can read & query.
   config.SetEntitySetAccessRule(
      "*", 
      EntitySetRights.AllRead
   );
   
5. Added Server Driven Paging limits:
   
   config.SetEntitySetPageSize("*", 5);

6. Added a query interceptor to only allow users to see just the own orders:

   [QueryInterceptor("Orders")]
   public Expression<Func<Order, bool>> OrdersFilter()
   {
       var user = HttpContext.Current.User.Identity.Name;
       if (string.IsNullOrEmpty(user))
           return (Order o) => false;
       else if (user== "Administrator")
           return (Order o) => true;
       else
           return (Order o) => o.Username == user;
   }

7. Made my service web browser friendly by Configuring the EDMX to map Genre.Name to the Entry/Title and Genre.Description to the Entry/Summary.
   
   See the EDMX in the final copy of the source to see how.
8. Added a ClientAccessPolicy.xml so that Silverlight apps hosted on different sites can interact with our Data Service.
   
   NOTE: The two sites in question need to be in the same internet zone!
    
9. Demoed using some JQuery JSON code that accesses the MusicService from the same site.
   
   NOTE: this doesn’t work x-site, for that you need JSONP.
   
10. Added support for JSONP by applying the [DataServicesJSONP.JSONPSupportBehavior] attribute to our MusicService.
    
    You can download the source from CodeGallery.
11. Demoed some JQuery JSONP code that accesses the MusicService from a remote site.
12. Configured ASP.NET to expose an authentication service, so non-browser agents can easily connect and logon to the Forms Authentication service, by adding this to the web.config:

<system.web.extensions>
  <scripting>
    <webServices>
      <authenticationService enabled="true" requireSSL="false"/>
    </webServices>
  </scripting>
</system.web.extensions>

• Tested the authentication service using Fiddler by sending this RAW request to authenticate:
  
  POST http://localhost:1397/Authentication_JSON_AppService.axd/Login HTTP/1.1
  Content-Type: application/json
  
  { "userName": "Alex", "password": "password", "createPersistentCookie":false}
  
• Opened this Data Services client application to show how to authenticate against the forms authentication service.

You can download the finally copy of the Music Service code if you want.

Enjoy and good luck.

Background info: This post builds on Parts 1 thru 8 which teach you how to create a typed Data Service Provider.

But now the part we’ve all been waiting for- an un-typed DSP – which is what I expect most real world DSP implementations will be based on.

What do we mean by Un-typed?

An un-typed DSP is needed whenever there isn’t a matching CLR class for each ResourceType. Typically you there is some general purpose class used for every ResourceType, something like a Dictionary for instance.

Over the course of the rest of this Post we will look at converting our DSP to expose product and category data stored in two Lists of Dictionaries, i.e. a List for each ResourceSet/Feed and a Dictionary for each Resource.

Why Un-typed?

Un-typed DSPs have a lot of interesting advantages:

1. You don’t need CLR classes for all the different ResourceTypes we want to expose
2. You don’t need to recompile / restart the service if the ResourceTypes changes in some way, i.e. you add a new Property.
3. You can store a pointer to the metadata for the ResourceType directly in the ResourceType itself.
4. You can store extra data along with a Resource very easily, which potentially allows us to take advantage of Data Services support for OpenTypes and OpenProperties.
5. You can evolve you model at runtime to expose more ResourceTypes/Sets

So now we know why, lets get started.

Metadata Changes:

The first thing we need to do create all our ResourceTypes:

var productType = new ResourceType(
    // CLR type backing this Resource  
    typeof(Dictionary<string,object>), 
    // Entity, ComplexType etc        
    ResourceTypeKind.EntityType,  
    null, // BaseType
    "Namespace", // Namespace
    "Product", // Name
    false // Abstract?
);
productType.CanReflectOnInstanceType = false;

Note there are two big differences between this and how we did things for Strongly Typed Resource Types (see part 3):

1. We tell Data Service that Products are stored in a Dictionary<string,object>
2. We tell Data Service that Dictionary<string,object> doesn’t have the same CLR inheritance hierarchy as the ResourceType itself – via CanReflectOnInstanceType. 
   This means whenever Casts are required rather than injecting standard CLR casts into the generated Query Expression, Data Services will inject methods calls the underlying injecting query provider is expected to translate.

Next we create our properties just like before:

var prodKey = new ResourceProperty(
   "ProdKey",
   ResourcePropertyKind.Key |
   ResourcePropertyKind.Primitive,
   ResourceType.GetPrimitiveResourceType(typeof(int))
);

But we add one extra call:

prodKey.CanReflectOnInstanceTypeProperty = false;

This tells DataServices that the declaring type – in this case Dictionary<string,object> - doesn’t actually have a “ProdKey” property, so a PropertyAccessor expression can’t be used.

With this setup this:

GET ~/Products(2)

Will no longer produce an (pseudo) expression like this:

from p in _queryProvider.GetQueryRootForResourceSet(productsSet)
where p.ProdKey == 2
select p;

Which would be invalid because ProdKey is not a property of Dictionary<string,object>, instead it will produce something like this:

Which gives your query provider the opportunity to re-write the expression as:

from p in _queryProvider.GetQueryRootForResourceSet(productsSet)
where ((int) p["ProdKey"]) == 2               
select p;

Which for example LINQ to Objects can handle.

NOTE: If your data isn’t in memory – maybe it is in a database – you still have to translate this expression into something like SQL, so it might just be easier to do that in one visit.

Once you’ve updated all your types accordingly you should have no problem getting ~/root.svc or ~/root.svc/$metadata working.

Data Source Changes:

Next we need to make some changes to our Data Sources so that the data is stored in Dictionaries.

If you remember we had this abstract base class from which our Data Source should inherit:

public abstract class DSPContext
{
    public abstract IQueryable GetQueryable(
       ResourceSet resourceSet);

    public abstract object CreateResource(
       ResourceType resourceType);

    public abstract void AddResource(
       ResourceSet resourceSet, object resource);

    public abstract void DeleteResource(
       object resource);

    public abstract void SaveChanges();
}

Here is an implementation that works over dictionaries:

public class SampleUntypedContext: DSPContext
{
    public const string RESOURCE_TYPE_KEY = "_ResourceType";

    static Dictionary<string, List<Dictionary<string, object>>>
       dataSources =
       new Dictionary<string, List<Dictionary<string, object>>>{
        {"Products", new List<Dictionary<string,object>>()},
        {"Categories", new List<Dictionary<string,object>>()} 
    };

    public override IQueryable GetQueryable(
        ResourceSet resourceSet)
    {
        return GetDataSource(resourceSet.Name).AsQueryable();
    }

    public override void AddResource(
        ResourceSet resourceSet, object resource)
    {
        var resourceInstance = GetResourceInstance(resource);
        var instanceTypeName = GetResourceType(resourceInstance);
        if (instanceTypeName != resourceSet.ResourceType.FullName)
            throw new InvalidOperationException("Unexpected Resource Type");

        var dataSource = GetDataSource(resourceSet.Name);
        dataSource.Add(resourceInstance);
    }

    public override void DeleteResource(object resource)
    {
        var resourceInstance = GetResourceInstance(resource);
        var resourceInstanceType = GetResourceType(resource);

        List<Dictionary<string,object>> datasource = null;
        if (resourceInstanceType == "Product")
            datasource = GetDataSource("Products");
        else if (resourceInstanceType == "Category")
            datasource = GetDataSource("Categories");
        else
            throw new Exception("ResourceSet not found");
        datasource.Add(resourceInstance);
    }
    public override void SaveChanges()
    {
        var products = GetDataSource("Products");
        var categories = GetDataSource("Categories");

        var prodKey = products.Max(p => (int)p["ProdKey"]);
        foreach (var prod in products
                    .Where(p => ((int) p["ProdKey"])== 0))
        {
            prod["ProdKey"] = ++prodKey;
        }

        var catKey = categories.Max(c => (int)c["ID"]);
        foreach (var cat in categories
                     .Where(c => ((int)c["ID"]) == 0))
        {
            cat["ID"] = ++catKey;
        }
    }
    private Dictionary<string, object> GetResourceInstance(
        object obj)
    {
        var x = obj as Dictionary<string, object>;
        if (x == null || !x.ContainsKey(RESOURCE_TYPE_KEY))
            throw new InvalidCastException(
                "Object is not a resource"
            );
        return x;
    }
    private string GetResourceType(object obj)
    {
        var x = GetResourceInstance(obj);
        return x[RESOURCE_TYPE_KEY] as string;
    }

    public override object CreateResource(
        ResourceType resourceType)
    {
        var instance = new Dictionary<string, object> {
           { RESOURCE_TYPE_KEY, resourceType.FullName }
        };
        foreach (var property in resourceType.Properties)
        {
            instance[property.Name] = GetDefaultValue(property);
        }
        return instance;
    }

    public static List<Dictionary<string, object>> GetDataSource(
        string resourceSetName)
    {
        if (!dataSources.ContainsKey(resourceSetName))
            throw new Exception("ResourceSet not found");
        return dataSources[resourceSetName];
    }

    private object GetDefaultValue(ResourceProperty prop)
    {
        if ((prop.Kind & ResourcePropertyKind.Primitive)
            == ResourcePropertyKind.Primitive)
        {
            if (!prop.ResourceType.InstanceType.IsValueType)
                return null;
            else
                return Activator.CreateInstance(
                    prop.ResourceType.InstanceType
                );
        }
        else if (
            prop.Kind==ResourcePropertyKind.ResourceSetReference
        ){
            return new List<Dictionary<string, object>>();
        }
        else if (
            prop.Kind== ResourcePropertyKind.ResourceReference
        ){
            return null;
        }
        throw new NotSupportedException();
    }
}

Again you can see that this has some repetitive code that we could be generalize.

We hold our Lists of Dictionary<string,object> in a Dictionary keyed on the ResourceSet name.

And as before our SaveChanges() function mimic server generated values for primary keys.

Perhaps the most interesting function is the implementation of CreateResource which adds all the properties to the Dictionary and assigns default values for each property, mimicking a strongly typed class constructor.

Now that we have a Data Source we need to fill it with some data. In the typed example we did this by overriding CreateDataSource, so no surprises, we need to do that again:

protected override SampleUntypedContext CreateDataSource()
{
    var products = SampleUntypedContext
                        .GetDataSource("Products");
    var categories = SampleUntypedContext
                        .GetDataSource("Categories");
    if (products.Count == 0)
    {
        var bovril = new Dictionary<string,object>
        {
            {"_ResourceType", "Namespace.Product"},
            {"ProdKey",  1},
            {"Name", "Bovril"},
            {"Cost", 4.35M},
            {"Price", 6.49M}
        };
        var marmite = new Dictionary<string, object>
        {
            {"_ResourceType", "Namespace.Product"},
            {"ProdKey",  2},
            {"Name", "Marmite"},
            {"Cost", 4.97M},
            {"Price", 7.21M}
        };
        var food = new Dictionary<string, object>
        {
            {"_ResourceType", "Namespace.Category"},
            {"ID",  1},
            {"Name", "Food"}
        };
        // Build bi-directional relationships
        food["Products"] = new List<Dictionary<string, object>> {
            marmite,
            bovril
        };
        marmite["Category"] = food;
        bovril["Category"] = food;

        // Store resources
        products.Add(marmite);
        products.Add(bovril);
        categories.Add(food);
    }
    return base.CreateDataSource();
}

IDataServiceMetadataProvider changes:

There are no real changes required here, because essentially this is just returning metadata, and the same code we wrote for the Typed provider will work still.

IDataServiceQueryProvider changes:

Here we need to change three things.

The way we work out the ResourceType for a resource:

public ResourceType GetResourceType(object target)
{
    return GetResourceType(GetResource(target));
}

private Dictionary<string, object> GetResource(object target)
{
    var dict = target as Dictionary<string, object>;
    if (dict == null)
    {
        throw new InvalidCastException(
            "Resource is not a dictionary"
        );
    }
    return dict;
}

private ResourceType GetResourceType(
    Dictionary<string, object> resource)
{
    ResourceType type = null;
    if (!resource.ContainsKey("_ResourceType"))
        throw new InvalidOperationException(
            "ResourceType not known"
        );

    if (!_metadata.TryResolveResourceType(
        resource["_ResourceType"].ToString(),
        out type)
    )
        throw new InvalidOperationException(
            "ResourceType not found"
        );
    return type;
}

The way we get property values:

public object GetPropertyValue(
    object target,
    ResourceProperty resourceProperty)
{
    var resource = GetResource(target);
    if (resource.ContainsKey(resourceProperty.Name))
        return resource[resourceProperty.Name];
    throw new InvalidOperationException("Property not found");
}

And finally the way we return query roots.

If we continue to do what we did in part 5:

public IQueryable GetQueryRootForResourceSet(
    ResourceSet resourceSet)
{
    return _currentDataSource.GetQueryable(resourceSet);
}

It will work for things URLs like
~/Products
~/Categories

But as soon as we do anything interesting like

~/Products(1)
~/Products(1)/Category
~/Categories(1)/Products
~/Products/?$filter=Name eq 'Marmite'

It will all fall over in a crashing heap.

Because we are Un-typed, DataServices will inject calls to DataServiceProviderMethods.XXX into the query, and then LINQ to Objects will just blindly call those methods – which unfortunately can’t be called directly.

We need to write a wrapped IQueryable that wraps the LINQ to Objects queryable and rewrites the expression so LINQ to Objects can deal with it.

Turns out I wrote a blog post just yesterday on this :)

In that blog post I created an InterceptedQuery<> and InterceptingProvider that wrap an IQueryable and allow you to visit and alter the expression – which is exactly what we need to do – before it passing it on to the underlying query provider.

So our method now looks like this:

public IQueryable GetQueryRootForResourceSet(
    ResourceSet resourceSet)
{
    var underlying = _currentDataSource.GetQueryable(resourceSet);
    return InterceptingProvider.Intercept(underlying, dspVisitor);
}

Visitor Implementation

The interesting piece of course is dspVisitor. This is an instance of a class derived from System.Linq.Expressions.ExpressionVisitor that looks like this:

public class DSPExpressionVisitor: ExpressionVisitor
{
    static readonly MethodInfo GetValueMethodInfo =
        typeof(DataServiceProviderMethods)
        .GetMethod(
            "GetValue",
            BindingFlags.Static | BindingFlags.Public,
            null,
            new Type[] {
                typeof(object),
                typeof(ResourceProperty) 
            },
            null
        );

    static readonly MethodInfo GetSequenceValueMethodInfo =
        typeof(DataServiceProviderMethods)
        .GetMethod(
            "GetSequenceValue",
            BindingFlags.Static | BindingFlags.Public,
            null,
            new Type[] {
                typeof(object),
                typeof(ResourceProperty)
            },
            null
        );

    static readonly MethodInfo ConvertMethodInfo =
        typeof(DataServiceProviderMethods)
        .GetMethod(
           "Convert",
           BindingFlags.Static | BindingFlags.Public
        );

    static readonly MethodInfo TypeIsMethodInfo = 
        typeof(DataServiceProviderMethods)
        .GetMethod(
            "TypeIs",
            BindingFlags.Static | BindingFlags.Public
        );

// Replace calls to DataServiceProviderMethods.GetValue
// with this expression.
// we cast to a dictionary, and then use the dictionary
// item accessor to get the property.
static readonly Expression<Func<object, ResourceProperty, object>> GetValueReplacement = 
    (o, rp) => (o as Dictionary<string,object>)[rp.Name];

// To check that something is a particular type we cast to
// Dictionary and compare the _ResourceType field with the
// fullname of the ResourceType.
// Obviously this approach won’t handle inheritance.
static readonly Expression<Func<object, ResourceType, bool>> TypeIsReplacement =
    (o, rt) => ((string)(o as Dictionary<string, object>)["_ResourceType"]) == rt.FullName;

   protected override Expression VisitMethodCall(
       MethodCallExpression node
   )
   {
        if (node.Method == GetValueMethodInfo)
        {
            // Arguments[0] - the resource to get property from
            // Arguments[1] - the ResourceProperty to get
            // Invoke the replacement expression, passing the
            // appropriate parameters.
            return Expression.Invoke(
               Expression.Quote(GetValueReplacement), 
               node.Arguments[0],
               node.Arguments[1]
            );
        }
        else if (node.Method.IsGenericMethod && 
            node.Method.GetGenericMethodDefinition() == 
           GetSequenceValueMethodInfo)
        {
            // Arguments[0] - the resource 
            // Arguments[1] - the Property that is a sequence 
            // Just call the GetValueReplacement(0,1) and
            // cast it to IEnumerable<T> which is the 
            // correct return type
            return Expression.Convert(
                Expression.Invoke(
                    Expression.Quote(GetValueReplacement),
                    node.Arguments[0],
                    node.Arguments[1]
                ),
                node.Method.ReturnType);
        }
        else if (node.Method == TypeIsMethodInfo)
        {
            // Arguments[0] – the resource
            // Arguments[1] – the ResourceType
            // Invoke the TypeIsReplacement expression
            // binding to the resource & resourceType
            return Expression.Invoke(
               Expression.Quote(TypeIsReplacement),
               node.Arguments[0],
               node.Arguments[1]
            );
        }
        else if (node.Method == ConvertMethodInfo)
        {
            // Arguments[0] – the resource
            // Arguments[0] – the ResourceType
            // no need to do anything, so just
            // return the argument
            return this.Visit(node.Arguments[0]);
        }
        return base.VisitMethodCall(node);
    }
}

This visitor class grabs hold of MethodInfo’s for each of the DataServiceProviderMethods methods, and stores them once for performance reasons.

Then it overrides VisitMethodCall looking for each of those MethodInfo’s in turn, replacing them with an expression that LINQ to Objects can handle.

To do this we use a little trick to feed in our replacementExpressions:

Expression.Invoke(
   Expression.Quote(replacementLambda),
   argument0,
   argument1
)
  
Using this approach we don’t have to manually construct the lambda expression, which I think makes our visitor easier to read.

Remember while you are doing this translation / replacement that you need to create expression that your underlying IQueryable provider can actually handle.

For example LINQ to Objects can handle these expressions without any problem, but the Entity Framework would choke because it can’t translate Invoke Expressions in to SQL.

NOTE: if you using .NET 3.5 – where System.Linq.Expressions.ExpressionVisitor isn’t available – you can still use a visitor derived IQToolkit’s base visitor or from this one.

IDataServiceUpdateProvider Changes:

This one is not too tricky: you just need to change each methods so they work against dictionaries rather than strongly typed classes.

Basically we need to take the IDataServiceUpdateProvider implementation in part 7 and update each method in turn to work against a dictionary.

Generally this actually makes things easier (and faster) because we no longer need to use reflection.

For example SetValue is a lot easier now:

public void SetValue(
   object targetResource,
   string propertyName,
   object propertyValue)
{
     try
     {
         _actions.Add(() => ReallySetValue(
             targetResource,
             propertyName, 
             propertyValue)
         );
     }
     catch { }
}

public void ReallySetValue(object targetResource, string propertyName, object propertyValue)
{
    var resource = targetResource as Dictionary<string, object>;
    resource[propertyName] = propertyValue;
}

Putting it altogether

If you make all these changes, you should now have a fully functional, Read/Write Un-typed Data Service Provider!

Congratulations, the world is now officially your oyster…

Over the last couple of years I’ve found myself in lots of situations where I’ve wanted to get ‘under the hood’ and see what is happening inside an IQueryable, but I haven’t had an easy solution, at least until now.

Getting down and dirty like this is interesting because it means you can:

• Log Queries before they are executed
• Rewrite expressions, for example replacing expressions that are not supported by a provider - EF, LINQ to SQL, LINQ to Objects etc - with expressions that are.

Anyway, I was very pleased a while back when reviewing some sample code by Vitek, who has a new blog by the way, and I realized I could generalize it to create an InterceptedQuery<> and an InterceptingProvider.

The basic idea is that you can use it like this:

public IQueryable<Customer> Customers
{
   get{
      return InterceptingProvider.CreateQuery(_ctx.Customers, visitor);    
   }
}

Here visitor, is an ExpressionVisitor that will visit, and potentially rewrite, any queries composed with Customers before it is handed off to the underlying queryable (in this case the Entity Framework).

Implementation

Thanks to Vitek, the implementation is actually pretty simple.

Lets start with the implementation of the InterceptedQuery<> which is trivial:

public class InterceptedQuery<T> : IOrderedQueryable<T>
{
    private Expression _expression;
    private InterceptingProvider _provider;

    public InterceptedQuery( 
       InterceptingProvider provider, 
       Expression expression) 
    { 
        this._provider = provider; 
        this._expression = expression; 
    }  
    public IEnumerator<T> GetEnumerator() 
    { 
        return this._provider.ExecuteQuery<T>(this._expression); 
    }  
    IEnumerator IEnumerable.GetEnumerator() 
    { 
        return this._provider.ExecuteQuery<T>(this._expression); 
    }  
    public Type ElementType 
    { 
        get { return typeof(T); } 
    }  
    public Expression Expression 
    { 
        get { return this._expression; } 
    }  
    public IQueryProvider Provider 
    { 
        get { return this._provider; } 
    } 
}

Next lets look at InterceptingProvider which is much more interesting:

public class InterceptingProvider : IQueryProvider
{
    private IQueryProvider _underlyingProvider;
    private Func<Expression,Expression>[] _visitors;

    private InterceptingProvider(
        IQueryProvider underlyingQueryProvider,
        params Func<Expression,Expression>[] visitors)
    {
        this._underlyingProvider = underlyingQueryProvider;
        this._visitors = visitors;
    }

    public static IQueryable<T> Intercept<T>(
        IQueryable<T> underlyingQuery,
        params ExpressionVisitor[] visitors)
    {
        Func<Expression, Expression>[] visitFuncs =
            visitors
            .Select(v => (Func<Expression, Expression>) v.Visit)
            .ToArray();
        return Intercept<T>(underlyingQuery, visitFuncs);
    }

    public static IQueryable<T> Intercept<T>(
        IQueryable<T> underlyingQuery,
        params Func<Expression,Expression>[] visitors)
    {
        InterceptingProvider provider = new InterceptingProvider(
            underlyingQuery.Provider,
            visitors
        );
        return provider.CreateQuery<T>(
            underlyingQuery.Expression);
    }
    public IEnumerator<TElement> ExecuteQuery<TElement>(
        Expression expression)
    {
        return _underlyingProvider.CreateQuery<TElement>(
            InterceptExpr(expression)
        ).GetEnumerator();
    } 
    public IQueryable<TElement> CreateQuery<TElement>(
        Expression expression)
    {
        return new InterceptedQuery<TElement>(this, expression);
    } 
    public IQueryable CreateQuery(Expression expression)
    {
        Type et = TypeHelper.FindIEnumerable(expression.Type);
        Type qt = typeof(InterceptedQuery<>).MakeGenericType(et);
        object[] args = new object[] { this, expression };

        ConstructorInfo ci = qt.GetConstructor(
            BindingFlags.NonPublic | BindingFlags.Instance,
            null,
            new Type[] { 
                typeof(InterceptingProvider),
                typeof(Expression)
            },
            null);

        return (IQueryable)ci.Invoke(args);
    } 
    public TResult Execute<TResult>(Expression expression)
    {
        return this._underlyingProvider.Execute<TResult>(
            InterceptExpr(expression)
        );
    } 
    public object Execute(Expression expression)
    {
        return this._underlyingProvider.Execute(
            InterceptExpr(expression)
        );
    } 
    private Expression InterceptExpr(Expression expression)
    {
        Expression exp = expression;
        foreach (var visitor in _visitors)
            exp = visitor(exp);
        return exp;
    }
}

Notice that whenever the Query is executed, we Intercept the current Expression, which involves calling all registered ‘visitors’ in turn, and then executing the resulting expression against the underlying provider.

Implementation Notes:

At its foundation our implementation uses Func<Expression,Expression> rather than .NET 4.0’s System.Linq.Expressions.ExpressionVisitor, mainly because .NET was a little late to the party, so there are lots of visitors that don’t derive from System.Linq.Expressions.ExpressionVisitor today.

You need look no further than Matt Warren’s excellent IQToolkit, for many such examples.

Nevertheless we want to encourage the use of System.Linq.Expressions.ExpressionVisitor so there is a convenience overload for that too.

Remember also that if you are wrapping the Entity Framework and doing any non-trivial expression rewrites you have to avoid any Invoke Expressions - see Colin’s blog post.

One of IQToolkit useful visitors is called ExpressionWriter and with a few minor mods – making both its constructor and the base Visit method public – you can use it to write the expression to the console before any Entity Framework query is executed:

CustomersContext _ctx = new CustomersContext();
ExpressionWriter _writer = new ExpressionWriter(Console.Out);

public IQueryable<Customer> Customers{
   get{
        return InterceptingProvider.Intercept(_ctx.Customers, _writer.Visit);
   }
} 

You’ll also notice that we are using the IQToolkit’s useful TypeHelper class in our untyped CreateQuery method, it really helps us create the correct generic InterceptedQuery<> instance.

Thanks again Matt!

Putting it all together:

Lets show an end to end example.

Here I mimic what a WCF/ADO.NET Data Service has to do to handle this request:

GET ~/People/?$filter=Surname eq ‘James’

if it is backed by an un-typed DSP.

// Create some data
List<Dictionary<string, object>> data = new List<Dictionary<string, object>>();
data.Add(
    new Dictionary<string, object>{{"Surname", "James"}, {"Firstname", "Alex"}}
);
data.Add(
    new Dictionary<string, object>{{"Surname", "Guard"}, {"Firstname", "Damien"}}
);
data.Add(
    new Dictionary<string, object>{{"Surname", "Meek"}, {"Firstname", "Colin"}}
);
data.Add(
    new Dictionary<string, object>{{"Surname", "Karas"}, {"Firstname", "Vitek"}}
);
data.Add(
    new Dictionary<string, object>{{"Surname", "Warren"}, {"Firstname", "Matt"}}
);

// Create a couple of visitors
var writer = new ExpressionWriter(Console.Out);
var dspVisitor = new DSPExpressionVisitor();

// Intercept queries to the L2O IQueryable
var queryRoot = InterceptingProvider.Intercept(
   data.AsQueryable(), // L2O’s iqueryable
   writer.Visit, // What does the expression look like first?
   dspVisitor.Visit, // Replace GetValue().
   writer.Visit // What does the expression look like now?
);

// Create a Data Services handle for the Surname property
ResourceProperty surname = new ResourceProperty(
   "Surname",
   ResourcePropertyKind.Primitive,
   ResourceType.GetPrimitiveResourceType(typeof(string))
);

// Create a query without knowing how to access the Surname
// from x.
var query =
  from x in queryRoot
  where ((string) DataServiceProviderMethods.GetValue(x, surname))
         == "James"
  select x;

// Execute the query and print some results
foreach (var x in query)
    Console.WriteLine("Found Match:{0}",
       x["Firstname"].ToString()
    );

As can see we have some People data in a list of dictionaries, and we are trying to find just the People with a Surname of ‘James’.

The problem is that Data Services doesn’t know how to get the Surname from a Dictionary. So it injects a call to DataServiceProviderMethods.GetValue(..).

Fine.

Unfortunately at this point the LINQ to Objects query provider doesn’t have enough context to process this query - blindly calling GetValue like we are doing in this query will fail.

So we intercept the query, and the DSPExpressionVisitor (which I won’t go into here) simply replaces things like this:

DataServiceProviderMethods.GetValue(x, surname)

with this

x[surname.Name]

which if you look at surname you can see is the same as:

x["Surname"]

So when the whole expression is visited you end up with something like this:

Which Linq to Objects can handle just fine!

Summary

This is a general purpose solution that allows you to layer one IQueryable over another, and translate / rewrite / log the query expression before it is passed to the underlying provider.

Enjoy.