meek : LINQ to Entities

InvocationExpression and LINQ to Entities

cmeek — Sun, 07 Dec 2008 17:02:00 GMT

I talked a little bit about patterns using InvocationExpression in a previous post (you might want to review the post for some context). It turns out my advice wasn’t that easy to implement… Fortunately, the EFExtensions helper library now includes an ExpandInvocations method that implements the same functionality (and a little bit more). I’ll return to the same example I used in my previous post, “all cars that are red or cheap”:

Expression<Func<Car, bool>> theCarIsRed = c1 => c1.Color == "Red";

Expression<Func<Car, bool>> theCarIsCheap = c2 => c2.Price < 10.0;

ParameterExpression p = theCarIsRed.Parameters.Single();

Expression<Func<Car, bool>> theCarIsRedOrCheap = Expression.Lambda<Func<Car, bool>>(

Expression.Or(theCarIsRed.Body, Expression.Invoke(theCarIsCheap, p)), p);

var query = carQuery.Where(theCarIsRedOrCheap);

Unfortunately, the Entity Framework does not support InvocationExpressions, so the query fails with a NotSupportedException. Our new helper method does the work of rewriting the query to avoid the unsupported invocations:

// Rewrite query to remove InvocationExpression

query = query.ExpandInvocations();

A lot easier! If you’re interested in the implementation, read on…

Remember math class? Let’s define a couple of functions:

f(x) = x + 1, g(y) = f(y + 2) * f(y + 3)

To evaluate g(4), we just expand the values inline:

g(4) = f(4 + 2) * f(4 + 3) = ((4 + 2) + 1) * ((4 + 3) + 1)

We just need to teach EFExtensions how to do the same trick. Whenever an invocation is encountered (e.g. “f(y + 2)” in our example), we want to replace the parameter in the body of the function with the corresponding argument. If we’re expanding an expression tree, we just need to remember which argument values correspond to which parameters:

private sealed class InvocationExpander : ExpressionVisitor {

private readonly ParameterExpression _parameter;

private readonly Expression _expansion;

private readonly InvocationExpander _previous;

…

protected override Expression VisitInvocation(InvocationExpression iv) {

if (iv.Expression.NodeType == ExpressionType.Lambda) {

LambdaExpression lambda = (LambdaExpression)iv.Expression;

return lambda

.Parameters

// zip together parameters and the corresponding argument values

.Zip(iv.Arguments, (p, e) => new { Parameter = p, Expansion = e })

// add to the stack of available parameters bindings (this class doubles as an immutable stack)

.Aggregate(this, (previous, pair) => new InvocationExpander(pair.Parameter, pair.Expansion, previous))

// visit the body of the lambda using an expander including the new parameter bindings

.Visit(lambda.Body);

}

return base.VisitInvocation(iv);

}

…

A couple of details of the VisitInvocation method might need explanation. First of all, I’m using the (very handy) Zip method, which allows me to consider the elements of two sequences pair-wise, like the teeth of a zipper coming together. (Good news: Zip is in .NET 4.0!) In this case, the lambda expression’s Parameters are aligned pair-wise with the invocation expression’s Arguments. Second of all, I’m using the call stack to keep track of which parameters are currently in scope. If you squint, InvocationExpander looks a lot like an immutable stack implementation… The _previous field is the ‘tail’ and the _parameter and _expansion fields are the ‘head’. The use of the Aggregate method gives me implicit the Push behavior I’m looking for, adding parameter bindings like layers in an onion. When control returns to the caller of VisitInvocation, the parameters are Popped out of scope, or peeled away, as a side effect of ceding control to the _previous visitor. At every position in the call stack, the InvocationVisitor instance that is in scope also captures the relevant parameter stack. When we run into a parameter expression that potentially requires expansion, we just iterate over the stack until we find a match:

protected override Expression VisitParameter(ParameterExpression p) {

InvocationExpander expander = this;

while (null != expander) {

if (expander._parameter == p) {

return base.Visit(expander._expansion);

}

expander = expander._previous;

}

return base.VisitParameter(p);

}

OK… There might be easier ways to write this solution, but I like immutable types in general. Why? I’m not smart enough to figure out state changes. I hate reviewing code and having to ask questions like “have I initialized field x at this point?”, “what happens if someone modifies collection y?” or “do I need to use a lock here?” I’m sure you can find plenty of other good reasons to like them.

EFExtensions update

cmeek — Sun, 07 Dec 2008 16:46:00 GMT

The EFExtensions helper library has a few updates:

- CreateCommand now respects the CommandTimeout property on ObjectContext. A user of the library reported this bug to me a while ago, and I’m finally getting around to fixing it.

- One tricky scenario involves multiple stored procedure result sets that need to be combined to produce a single object graph. No new code in the library is required, but I have added a new sample illustrating how you can return a set of categories and their related products through a single stored procedure call. See the Materializer_MultipleResultSetsMerge sample for details. See this post for background information.

- We get frequent requests for LINQ patterns involving InvocationExpressions. I’ve added a couple of helper ExpandInvocations overload to simplify use of these patterns in the Entity Framework (EF). See the Misc_ExpandInvocationExpressions sample for details. See this post for details of the implementation soon.

LINQ to Entities: Combining Predicates

cmeek — Sat, 03 May 2008 03:28:00 GMT

Someone asked a great question on the ADO.NET Entity Framework forums yesterday: how do I compose predicates in LINQ to Entities? I’ll give three answers to the question.

Answer 1: Chaining query operators

Basically, you have some query and you have some predicates you want to apply to that query (“the car is red”, “the car costs less than $10”). If both conditions need to be satisfied, you can just chain together some calls to Where (“the car is red and costs less than $10”):

Expression<Func<Car, bool>> theCarIsRed = c => c.Color == "Red";

Expression<Func<Car, bool>> theCarIsCheap = c => c.Price < 10.0;

IQueryable<Car> carQuery = …;

var query = carQuery.Where(theCarIsRed).Where(theCarIsCheap);

If you’re willing to exceed the $10 budget for cars that are red, you can chain Unions instead (“the car is red or the car costs less than $10”):

var query2 = carQuery.Where(theCarIsRed).Union(carQuery.Where(theCarIsCheap));

This last query has a couple of problems: it’s inefficient (because of the unions) and it eliminates duplicates in the results, something that would not happen if I applied a single predicate.

Answer 2: Build expressions manually

The LINQ Expressions API includes factory methods that allow you to build up the predicate by hand. I can define the conditions (with respect to a “car” parameter) as follows:

ParameterExpression c = Expression.Parameter(typeof(Car), "car");

Expression theCarIsRed = Expression.Equal(Expression.Property(c, "Color"), Expression.Constant("Red"));

Expression theCarIsCheap = Expression.LessThan(Expression.Property(c, "Price"), Expression.Constant(10.0));

Expression<Func<Car, bool>> theCarIsRedOrCheap = Expression.Lambda<Func<Car, bool>>(

Expression.Or(theCarIsRed, theCarIsCheap), c);

var query = carQuery.Where(theCarIsRedOrCheap);

Building queries by hand isn’t very convenient. If you’re already building expressions from scratch, this is a good approach but otherwise I’d suggest something different…

Answer 3: Composing Lambda Expresions

The Albaharis suggest combining bodies of lambda expressions in their C# 3.0 book (a great resource for all things C# and LINQ). This allows you to describe the parts of the expression using the lambda syntax and build an aggregate expression:

Expression<Func<Car, bool>> theCarIsRed = c1 => c1.Color == "Red";

Expression<Func<Car, bool>> theCarIsCheap = c2 => c2.Price < 10.0;

Expression<Func<Car, bool>> theCarIsRedOrCheap = Expression.Lambda<Func<Car, bool>>(

Expression.Or(theCarIsRed.Body, theCarIsCheap.Body), theCarIsRed.Parameters.Single());

var query = carQuery.Where(theCarIsRedOrCheap);

I’m taking the bodies of the two conditions and Oring them in a new lambda expression. There is a subtle problem however: the parameter for the merged expression (c1) is taken from “theCarIsRed”, which leaves us with a dangling parameter (c2) from “theCarIsCheap”. The resulting query is invalid. How can I force “theCarIsCheap” to use the same parameter? The answer is to invoke the expression using the common parameter:

ParameterExpression p = theCarIsRed.Parameters.Single();

Expression<Func<Car, bool>> theCarIsRedOrCheap = Expression.Lambda<Func<Car, bool>>(

Expression.Or(theCarIsRed.Body, Expression.Invoke(theCarIsCheap, p)), p);

Here’s the problem: LINQ to Entities does not support InvocationExpressions. Rather than invoking the expression with c1, I can manually rebind the parameter. Matt Warren’s series of articles on IQueryable providers includes an ExpressionVisitor implementation that makes it easy to rewrite expression trees. If you do any LINQ expression manipulation, this class is a crucial tool. Here’s an implementation of the visitor that rebinds parameters:

public class ParameterRebinder : ExpressionVisitor {

private readonly Dictionary<ParameterExpression, ParameterExpression> map;

public ParameterRebinder(Dictionary<ParameterExpression, ParameterExpression> map) {

this.map = map ?? new Dictionary<ParameterExpression, ParameterExpression>();

}

public static Expression ReplaceParameters(Dictionary<ParameterExpression, ParameterExpression> map, Expression exp) {

return new ParameterRebinder(map).Visit(exp);

}

protected override Expression VisitParameter(ParameterExpression p) {

ParameterExpression replacement;

if (map.TryGetValue(p, out replacement)) {

p = replacement;

}

return base.VisitParameter(p);

}

Now I can write a general utility method to compose lambda expressions without using invoke (I’ll call it Compose), and leverage it to implement EF-friendly And and Or builder methods:

public static class Utility {

public static Expression<T> Compose<T>(this Expression<T> first, Expression<T> second, Func<Expression, Expression, Expression> merge) {

// build parameter map (from parameters of second to parameters of first)

var map = first.Parameters.Select((f, i) => new { f, s = second.Parameters[i] }).ToDictionary(p => p.s, p => p.f);

// replace parameters in the second lambda expression with parameters from the first

var secondBody = ParameterRebinder.ReplaceParameters(map, second.Body);

// apply composition of lambda expression bodies to parameters from the first expression

return Expression.Lambda<T>(merge(first.Body, secondBody), first.Parameters);

}

public static Expression<Func<T, bool>> And<T>(this Expression<Func<T, bool>> first, Expression<Func<T, bool>> second) {

return first.Compose(second, Expression.And);

}

public static Expression<Func<T, bool>> Or<T>(this Expression<Func<T, bool>> first, Expression<Func<T, bool>> second) {

return first.Compose(second, Expression.Or);

}

To combine lambda expressions, I can write:

Expression<Func<Car, bool>> theCarIsRed = c => c.Color == "Red";

Expression<Func<Car, bool>> theCarIsCheap = c => c.Price < 10.0;

Expression<Func<Car, bool>> theCarIsRedOrCheap = theCarIsRed.Or(theCarIsCheap);

var query = carQuery.Where(theCarIsRedOrCheap);

I’ll use this last answer as an excuse to discuss variations on the visitor pattern in a future post...