Stuart Leeks : Extension methods

Improving ObjectQuery.Include – Updated

stuartle — Fri, 24 Apr 2009 14:25:00 GMT

Having spent some time using the sample from my previous post on ObjectQuery.Include, I’ve encountered a bug! It turns out that the code generates the wrong include string for

context.Customers.Include(c => c.Order.SubInclude(o=>o.OrderDetail))

The fix for this is a small change to the BuildString method to recurse up the MemberExpression if necessary. The updated code is below - usual disclaimers apply!

    public static class ObjectQueryExtensions
    {
        public static ObjectQuery<TSource> Include<TSource, TPropType>(this ObjectQuery<TSource> source, Expression<Func<TSource, TPropType>> propertySelector)
        {
            string includeString = BuildString(propertySelector);
            return source.Include(includeString);
        }
        private static string BuildString(Expression propertySelector)
        {
            switch (propertySelector.NodeType)
            {
                case ExpressionType.Lambda:
                    LambdaExpression lambdaExpression = (LambdaExpression)propertySelector;
                    return BuildString(lambdaExpression.Body);

                case ExpressionType.Quote:
                    UnaryExpression unaryExpression = (UnaryExpression)propertySelector;
                    return BuildString(unaryExpression.Operand);

                case ExpressionType.MemberAccess:

                    MemberExpression memberExpression = (MemberExpression)propertySelector;
                    MemberInfo propertyInfo = memberExpression.Member;

                    if (memberExpression.Expression is ParameterExpression)
                    {
                        return propertyInfo.Name;
                    }
                    else
                    {
                        // we've got a nested property (e.g. MyType.SomeProperty.SomeNestedProperty)
                        return BuildString(memberExpression.Expression) + "." + propertyInfo.Name;
                    }

                case ExpressionType.Call:
                    MethodCallExpression methodCallExpression = (MethodCallExpression)propertySelector;
                    if (IsSubInclude(methodCallExpression.Method)) // check that it's a SubInclude call
                    {
                        // argument 0 is the expression to which the SubInclude is applied (this could be member access or another SubInclude)
                        // argument 1 is the expression to apply to get the included property
                        // Pass both to BuildString to get the full expression
                        return BuildString(methodCallExpression.Arguments[0]) + "." +
                               BuildString(methodCallExpression.Arguments[1]);
                    }
                    // else drop out and throw
                    break;
            }
            throw new InvalidOperationException("Expression must be a member expression or an SubInclude call: " + propertySelector.ToString());

        }

        private static readonly MethodInfo[] SubIncludeMethods;
        static ObjectQueryExtensions()
        {
            Type type = typeof(ObjectQueryExtensions);
            SubIncludeMethods = type.GetMethods().Where(mi => mi.Name == "SubInclude").ToArray();
        }
        private static bool IsSubInclude(MethodInfo methodInfo)
        {
            if (methodInfo.IsGenericMethod)
            {
                if (!methodInfo.IsGenericMethodDefinition)
                {
                    methodInfo = methodInfo.GetGenericMethodDefinition();
                }
            }
            return SubIncludeMethods.Contains(methodInfo);
        }

        public static TPropType SubInclude<TSource, TPropType>(this EntityCollection<TSource> source, Expression<Func<TSource, TPropType>> propertySelector)
            where TSource : class, IEntityWithRelationships
            where TPropType : class
        {
            throw new InvalidOperationException("This method is only intended for use with ObjectQueryExtensions.Include to generate expressions trees"); // no actually using this - just want the expression!
        }
        public static TPropType SubInclude<TSource, TPropType>(this TSource source, Expression<Func<TSource, TPropType>> propertySelector)
            where TSource : class, IEntityWithRelationships
            where TPropType : class
        {
            throw new InvalidOperationException("This method is only intended for use with ObjectQueryExtensions.Include to generate expressions trees"); // no actually using this - just want the expression!
        }
    }

UPDATE: Alex James has a great post on Eager Loading Strategies that I'd recommend reading.

DataServiceQuery.Expand

stuartle — Mon, 15 Sep 2008 11:25:00 GMT

ADO.Net Data Services allows you to expose your LINQ To Entities model (or LINQ To SQL model, or even your custom IQueryable model) via a RESTful API with minimal coding. For example, if you’re working with the Northwind database you can use the URL http://server/Service.svc/Customers(‘ALFKI’)/Orders to retrieve the orders for customer ALFKI. This simplicity makes it easy to retrieve data in a variety of scenarios – you simply need to be able to issue HTTP requests to access the data. To make it really easy to consume the data from client-side javascript, you can even specify that you want the data to be in JSON format. In rich client scenarios, you can either work with HTTP requests or if you like to make things really easy you can create a LINQ data model to access your ADO.Net Data Service. Simply point DataSvcUtil at your service and it will generate the LINQ classes you need. There’s a very good article by Shawn Wildermuth in the September 2008 issue of MSDN Magazine called ‘Creating Data-Centric Web Applications With Silverlight 2’ [1] that takes you through the process of creating a service and the LINQ classes to access the service from Silverlight. The article also discusses the mechanisms that ADO.Net Data Services uses to discover your entities if you want to expose your custom model (or if you’re just interested in how it works). I’ve also included a few extra links in the references at the end of the post if you want to find out more. Now, back to the point of the post...

The client-side code can use the DataServiceQuery<T>.Expand method to specify what properties should also be retrieved (much like the Include method in my last post Improving ObjectQuery<T>.Include). So you can write the following query to retrieve the customer ALFKI from the Northwind database and also pull back the orders:

var q = from c in context.Customers.Expand("Orders/Order_Details")
        where c.CustomerID == "ALFKI"
        select c;

Notice the call to the Expand Method. Much like the Include method for LINQ To Entities (see my last post), this lets yoe specify that you want certain related data to be pre-fetched. If you wanted to get the order details included as well then you could change the query to:

var q = from c in context.Customers.Expand("Orders/Order_Details")
        where c.CustomerID == "ALFKI"
        select c;

One point to notice is that it uses a forward slash (‘/’) rather than a decimal (‘.’) as the path separator, other than that it’s pretty similar to Include. Applying the same logic as in my last post, we can quickly create extension methods that allow us to write the above queries as

var q = from c in context.Customers.Expand(c => c.Orders)
        where c.CustomerID == "ALFKI"
        select c;

and

var q = from c in context.Customers.Expand(c => c.Orders.SubExpand(o=> o.Order_Details))
        where c.CustomerID == "ALFKI"
        select c;

Note that you can also expand multiple properties on the same entity. So if you were querying for orders, you can also bring back the customer and order details:

var q = from o in context.Orders.Expand(o=>o.Customer).Expand(o=>o.Order_Details)
        where o.Customer.CustomerID == "ALFKI"
        select o;

The code for the extension method is shown below – note that the code isn’t production ready (and as always, is subject to the standard disclaimer: “These postings are provided "AS IS" with no warranties, and confer no rights. Use of included script samples are subject to the terms specified at http://www.microsoft.com/info/cpyright.htm”). Since this code is very similar to my previous post, and gives you the same benefits, I’ll refer you to that post if you want to know how the code works and what the benefits are!

public static class DataServiceQueryExtensions
{
    public static DataServiceQuery<TSource> Expand<TSource, TPropType>(this DataServiceQuery<TSource> source, Expression<Func<TSource, TPropType>> propertySelector)
    {
        string expandString = BuildString(propertySelector);
        return source.Expand(expandString);
    }
    private static string BuildString(Expression propertySelector)
    {
        switch (propertySelector.NodeType)
        {
            case ExpressionType.Lambda:
                LambdaExpression lambdaExpression = (LambdaExpression)propertySelector;
                return BuildString(lambdaExpression.Body);

            case ExpressionType.Quote:
                UnaryExpression unaryExpression = (UnaryExpression)propertySelector;
                return BuildString(unaryExpression.Operand);

            case ExpressionType.MemberAccess:
                MemberInfo propertyInfo = ((MemberExpression)propertySelector).Member;
                return propertyInfo.Name;

            case ExpressionType.Call:
                MethodCallExpression methodCallExpression = (MethodCallExpression)propertySelector;
                if (IsSubExpand(methodCallExpression.Method)) // check that it's a SubExpand call
                {
                    // argument 0 is the expression to which the SubExpand is applied (this could be member access or another SubExpand)
                    // argument 1 is the expression to apply to get the expanded property
                    // Pass both to BuildString to get the full expression
                    return BuildString(methodCallExpression.Arguments[0]) + "/" +
                           BuildString(methodCallExpression.Arguments[1]);
                }
                // else drop out and throw
                break;
        }
        throw new InvalidOperationException("Expression must be a member expression or an SubExpand call: " + propertySelector.ToString());

    }

    private static readonly MethodInfo[] SubExpandMethods;
    static DataServiceQueryExtensions()
    {
        Type type = typeof(DataServiceQueryExtensions);
        SubExpandMethods = type.GetMethods().Where(mi => mi.Name == "SubExpand").ToArray();
    }
    private static bool IsSubExpand(MethodInfo methodInfo)
    {
        if (methodInfo.IsGenericMethod)
        {
            if (!methodInfo.IsGenericMethodDefinition)
            {
                methodInfo = methodInfo.GetGenericMethodDefinition();
            }
        }
        return SubExpandMethods.Contains(methodInfo);
    }

    public static TPropType SubExpand<TSource, TPropType>(this Collection<TSource> source, Expression<Func<TSource, TPropType>> propertySelector)
        where TSource : class
        where TPropType : class
    {
        throw new InvalidOperationException("This method is only intended for use with DataServiceQueryExtensions.Expand to generate expressions trees"); // no actually using this - just want the expression!
    }
    public static TPropType SubExpand<TSource, TPropType>(this TSource source, Expression<Func<TSource, TPropType>> propertySelector)
        where TSource : class
        where TPropType : class
    {
        throw new InvalidOperationException("This method is only intended for use with DataServiceQueryExtensions.Expand to generate expressions trees"); // no actually using this - just want the expression!
    }
}

References:

1: Creating Data-Centric Web Applications With Silverlight 2, Shawn Wildermuth. MSDN Magazine, September 2008. http://msdn.microsoft.com/en-us/magazine/cc794279.aspx

2 ADO.Net Data Services team blog. http://blogs.msdn.com/astoriateam/

3 ADO.Net Data Services homepage on MSDN. http://msdn.microsoft.com/en-us/data/bb931106.aspx

4 Mike Taulty has a number of blog posts on ADO.Net Data Services. http://mtaulty.com/CommunityServer/blogs/mike_taultys_blog/archive/category/1027.aspx

Improving ObjectQuery.Include

stuartle — Thu, 28 Aug 2008 01:35:00 GMT

** UPDATE: There’s a bug in the code below – see this post for the update!

One of the great features of LINQ To SQL and LINQ To Entities is that the queries you write are checked by the compiler, which eliminates typing errors in your query. Unfortunately, the ObjectQuery<T>.Include function (which is used to eager-load data that isn’t directly included in the query) takes a string parameter, opening up opportunities for typos to creep back in. In this post I’ll present some sample code that illustrates one way that you can work round this. To start with, let’s take a quick look at a query against an entity model on the Northwind database.

var query = from customer in context.Customers
            where customer.Orders.Count > 0
            select customer;

This query simply retrieves customers with an order. If the code that uses the query results then needs to make use of the order data, it won’t have been loaded. We can use the Include function to ensure that this data is loaded up front:

var query = from customer in context.Customers.Include("Orders")
            where customer.Orders.Count > 0
            select customer;

Notice the Include(“Orders”) call that we’ve inserted which instructs Entity Framework to retrieve the Orders for each Customer. It would be much nicer if we could use a lambda expression to specify what property to load:

var query = from customer in context.Customers.Include(c => c.Orders)
            where customer.Orders.Count > 0
            select customer;

It turns out that this is very easy to achieve by using an extension method:

public static class ObjectQueryExtensions
{
    public static ObjectQuery<TSource> Include<TSource, TPropType>(this ObjectQuery<TSource> source, Expression<Func<TSource, TPropType>> propertySelector)
    {
        MemberExpression memberExpression = propertySelector.Body as MemberExpression;
        if (memberExpression== null)
        {
            throw new InvalidOperationException("Expression must be a member expression" + propertySelector);
        }
        MemberInfo propertyInfo = memberExpression.Member;
        return source.Include(propertyInfo.Name);
    }
}

This Include extension method allows the query syntax above with the lambda expression. When the Include method is called, it inspects the Expression Tree. If the method is used as intended, the tree will describe a accessing a member of the TSource class. We can then us the name of the member to call the original Include function.

Whilst this solves the problem as I described it above, what if the code consuming the query also needed the order details? With the original Include function we can write

var query = from customer in context.Customers.Include("Orders.Order_Details") 
            where customer.Orders.Count > 0
            select customer;

Notice that we can pass a path to the properties to include. The extension method we wrote doesn’t give us a way to handle this case. I imagine using something like the syntax below to describe this situation

var query = from customer in context.Customers.Include(c => c.Orders.SubInclude(o => o.Order_Details))
            where customer.Orders.Count > 0
            select customer;

Here, we’ve specified that we want to include Orders, and then also that we want to include Order_Details. Adding this support is a bit more code, but not too bad:

public static class ObjectQueryExtensions
{
    public static ObjectQuery<TSource> Include<TSource, TPropType>(this ObjectQuery<TSource> source, Expression<Func<TSource, TPropType>> propertySelector)
    {
        string includeString = BuildString(propertySelector);
        return source.Include(includeString);
    }
    private static string BuildString(Expression propertySelector)
    {
        switch(propertySelector.NodeType)
        {
            case ExpressionType.Lambda:
                LambdaExpression lambdaExpression = (LambdaExpression)propertySelector;
                return BuildString(lambdaExpression.Body);

            case ExpressionType.Quote:
                UnaryExpression unaryExpression= (UnaryExpression)propertySelector;
                return BuildString(unaryExpression.Operand);

            case ExpressionType.MemberAccess:
                MemberInfo propertyInfo = ((MemberExpression) propertySelector).Member;
                return propertyInfo.Name;

            case ExpressionType.Call:
                MethodCallExpression methodCallExpression = (MethodCallExpression) propertySelector;
                if (IsSubInclude(methodCallExpression.Method)) // check that it's a SubInclude call
                {
                    // argument 0 is the expression to which the SubInclude is applied (this could be member access or another SubInclude)
                    // argument 1 is the expression to apply to get the included property
                    // Pass both to BuildString to get the full expression
                    return BuildString(methodCallExpression.Arguments[0]) + "." +
                           BuildString(methodCallExpression.Arguments[1]);
                }
                // else drop out and throw
                break;
        }
        throw new InvalidOperationException("Expression must be a member expression or an SubInclude call: " + propertySelector.ToString());

    }

    private static readonly MethodInfo[] SubIncludeMethods;
    static ObjectQueryExtensions()
    {
        Type type = typeof (ObjectQueryExtensions);
        SubIncludeMethods = type.GetMethods().Where(mi => mi.Name == "SubInclude").ToArray();
    }
    private static bool IsSubInclude(MethodInfo methodInfo)
    {
        if (methodInfo.IsGenericMethod)
        {
            if (!methodInfo.IsGenericMethodDefinition)
            {
                methodInfo = methodInfo.GetGenericMethodDefinition();
            }
        }
        return SubIncludeMethods.Contains(methodInfo);
    }

    public static TPropType SubInclude<TSource, TPropType>(this EntityCollection<TSource> source, Expression<Func<TSource, TPropType>> propertySelector)
        where TSource : class, IEntityWithRelationships
        where TPropType : class
    {
        throw new InvalidOperationException("This method is only intended for use with ObjectQueryExtensions.Include to generate expressions trees"); // no actually using this - just want the expression!
    }
    public static TPropType SubInclude<TSource, TPropType>(this TSource source, Expression<Func<TSource, TPropType>> propertySelector)
        where TSource : class, IEntityWithRelationships
        where TPropType : class
    {
        throw new InvalidOperationException("This method is only intended for use with ObjectQueryExtensions.Include to generate expressions trees"); // no actually using this - just want the expression!
    }
}

This code still has the Include method with the original signature, and adds a couple of SubInclude extension methods. You can see that the code to extract the property name has been pulled out into a separate method (BuildString). This now also handles some additional NodeTypes so that we can handle the SubInclude calls inside the Include call. There are some checks in to ensure that we are dealing with the SubInclude calls at this point (using the IsSubInclude method). With this code, we can write the previous query as well as:

var query = from customer in context.Customers.Include(c => c.Orders.SubInclude(o => o.Order_Details).SubInclude(od=>od.Products))
            where customer.Orders.Count > 0
            select customer;

This query includes the Orders, Order Details, and Products as if we’d called Include(“Orders.Order_Details.Product”) and in fact this is what the code will do! Additionally, it doesn’t matter whether you chain the SubInclude calls (as above) or nest them:

var query = from customer in context.Customers.Include(c => c.Orders.SubInclude(o => o.Order_Details.SubInclude(od => od.Products)))
            where customer.Orders.Count > 0
            select customer;

Both of these queries have the same effect, so it’s up to you which style you prefer.

The code isn’t production ready (and as always, is subject to the standard disclaimer: “These postings are provided "AS IS" with no warranties, and confer no rights. Use of included script samples are subject to the terms specified at http://www.microsoft.com/info/cpyright.htm”). However, there are a couple of other features that I think are worth briefly mentioning:

The IsSubInclude function works against a cached MethodInfos for the SubInclude methods. Because these methods are generic, we have to get the generic method definition to test for comparison
The SubInclude functions are not intended to be called at runtime - they are purely there to get the compiler to generate the necessary expression tree!
Generic type inference is at work when we write the queries. Notice that we could write Include(c => c.Orders) rather than Include<Customer>(c => c.Orders). Imagine how much less readable the code would become, especially when including multiple levels.

I found it quite interesting putting this code together as it pulls in Extension Methods, LINQ To Entities and Expression Trees. The inspiration for this came from the LinkExtensions.ActionLink methods in ASP.Net MVC framework which do the same sort of thing for ActionLinks. I'’m not sure if I’m entirely satisfied with the syntax for including multiple levels, but it is the best I’ve come up with so far. If you’ve any suggestions for how to improve it then let me know!