Jomo Fisher -- Sharp Things : Design Patterns

LINQ to SQL Beta2 Performance Numbers and the Dynamic Compilation Pattern

Jomo Fisher — Fri, 06 Jul 2007 18:42:00 GMT

Jomo Fisher--Rico Mariani has been posting about LINQ to SQL perfomance and has finally posted the performance numbers for Beta2: http://blogs.msdn.com/ricom/archive/2007/07/05/dlinq-linq-to-sql-performance-part-4.aspx

One of the tricks Rico and Matt Warren used to get this stellar performance was to generate IL at run time. I learned this design pattern from Matt and used it on a completely different problem (see http://blogs.msdn.com/jomo_fisher/archive/2007/03/28/fast-switching-with-linq.aspx) to get a 900% performance improvement. I don't know whether this pattern already has a name (Dynamic Compilation seems to fit) but it's hard to ignore its potential for significant performance improvement in a particular class of problem. A further benefit is that you can hide a lot of complex, deep thinking code behind a simple interface that you present to your user.

Correction: Now that I think about it, I may have originally learned this pattern from my high school friend Brandon D'Souza who used it to compile equations into Apple II machine language. This was for a spread sheet application our teacher wanted to call 'Rickards High Scool 1-2-3-4.'

This posting is provided "AS IS" with no warranties, and confers no rights.

Dealing with Linq’s Immutable Expression Trees

Jomo Fisher — Wed, 23 May 2007 23:37:00 GMT

Jomo Fisher --I recently got a question via my blog that dovetailed nicely with something I’ve been working on:

I know that expression trees are (or at least appear to be) immutable - which requires that you rewrite the entire tree if you want a tree that is similar, but not the same as, an existing tree.

My questions is really whether there is anything in the near term pipeline from Microsoft that might address this gap.

Linq’s expression trees are indeed immutable. I’ve worked with mutable trees—LINQ to SQL has an internal tree structure that’s mutable—and immutable trees like expression trees from the System.Linq.Expression namespace. I’ve really come to believe that, maybe counter-intuitively, immutable trees are far easier to work with. See this blog article for a discussion of the sorts of problems that can arise with mutable classes.

The problems are actually magnified when a tree structure is mutable because they’re intrinsically more complex and because it’s easy to unintentionally turn the tree into a graph. This latter problem is relieved in immutable trees because a node must exist before it can be assigned to be the child of another node.

Another advantage of immutable trees is that you can effectively duplicate them just by reference assignment. Here’s what I mean:

Expression<Func<int, int>> f = (n => 1 - n);

Expression<Func<int, int>> g = f;

f = MyRewriteFunction(f);

Here, the variable ‘g’ keeps the original expression tree value while ‘f’ is rewritten with a new value. Notice that this is a very fast reference copy. Without the immutability guarantee you would have to physically copy the tree to get the same guarantee.

Changing a Mutable Tree without Complete Rewrite

This original concern, though, is about the need to completely rewrite the tree in order to change it. It turns out, though, that you only need to rewrite the slice of the tree from the point of the change along with parents all the way to the root expression. While you do need to recreate the parents you can reuse the unchanged siblings. This is actually a very powerful concept:

You can create a forest of similar expression trees in which only the differences between the trees occupy extra memory.

For example, if you have a system that rewrites an expression tree in many phases then you can keep all of the intermediate rewrite steps with little effort and without consuming unnecessary memory.

Another example might be a Genetic Programming example in which the mutations are kept in a forest of expression trees.

Yet another example might be a theorem prover in which you need to search for a particular tree but you need to be able to backtrack if you reach a dead end.

I should back this up with an example. My example is based on the chain-of-responsibility code that I posted here. You’ll need that code to make this work. What we need is a visitor over Linq Expression trees. This visitor needs to recognize when a child node has changed and rewrite parent nodes all the way to the root.

(Note: This is is post-Beta1 code and won't compile with Beta1 and earlier of Orcas. At the bottom, I've attached the equivalent Beta1 ExprOp class and sample code).

public static class ExprOp {

static public Func<Expression, Expression> Visit = FuncOp.Create<Expression, Expression>(

(self, expr) => {

switch (expr.NodeType) {

case ExpressionType.Coalesce:

var c = (BinaryExpression)expr;

var left = self(c.Left);

var right = self(c.Right);

var conv = self(c.Conversion);

return (left == c.Left && right == c.Right && conv == c.Conversion) ? expr : Expression.Coalesce(left, right, (LambdaExpression)conv);

case ExpressionType.TypeIs:

var tbe = (TypeBinaryExpression)expr;

var tbex = self(tbe.Expression);

return (tbe.Expression == tbex) ? expr : Expression.TypeIs(tbex, tbe.TypeOperand);

case ExpressionType.Conditional:

var ce = (ConditionalExpression)expr;

var t = self(ce.Test);

var it = self(ce.IfTrue);

var @if = self(ce.IfFalse);

return (t == ce.Test && it == ce.IfTrue && @if == ce.IfFalse) ? expr : Expression.Condition(t, it, @if);

case ExpressionType.MemberAccess:

var ma = (MemberExpression)expr;

var maex = self(ma.Expression);

return (maex == ma.Expression) ? expr : Expression.MakeMemberAccess(maex, ma.Member);

case ExpressionType.Call:

var mce = (MethodCallExpression)expr;

var o = self(mce.Object);

var ca = self.VisitExpressionList(mce.Arguments);

return (o == mce.Object && ca == mce.Arguments) ? expr : Expression.Call(o, mce.Method, ca);

case ExpressionType.Lambda:

var le = (LambdaExpression)expr;

var b = self(le.Body);

return (b == le.Body) ? expr : Expression.Lambda(le.Type, b, le.Parameters);

case ExpressionType.New:

var ne = (NewExpression)expr;

var nar = self.VisitExpressionList(ne.Arguments);

if (nar != ne.Arguments)

return (ne.Members != null) ? Expression.New(ne.Constructor, nar, ne.Members)

: Expression.New(ne.Constructor, nar);

return expr;

case ExpressionType.NewArrayInit:

case ExpressionType.NewArrayBounds:

var na = (NewArrayExpression)expr;

var inits = self.VisitExpressionList(na.Expressions);

if (inits != na.Expressions)

return (na.NodeType == ExpressionType.NewArrayInit) ?

Expression.NewArrayInit(na.Type.GetElementType(), inits) :

Expression.NewArrayBounds(na.Type.GetElementType(), inits);

return expr;

case ExpressionType.Invoke:

var inv = (InvocationExpression)expr;

var args = self.VisitExpressionList(inv.Arguments);

var ie = self(inv.Expression);

return (args == inv.Arguments && ie == inv.Expression) ? expr : Expression.Invoke(ie, args);

case ExpressionType.MemberInit:

var mi = (MemberInitExpression)expr;

var n = (NewExpression)self(mi.NewExpression);

var bindings = self.VisitBindingList(mi.Bindings);

return (n == mi.NewExpression && bindings == mi.Bindings) ? expr : Expression.MemberInit(n, bindings);

case ExpressionType.ListInit:

var li = (ListInitExpression)expr;

var lin = (NewExpression)self(li.NewExpression);

var lii = VisitElementInitializerList(self, li.Initializers);

return (lin == li.NewExpression || lii == li.Initializers) ? expr : Expression.ListInit(lin, lii);

}

if (expr.IsBinary()) {

var b = (BinaryExpression)expr;

var left = self(b.Left);

var right = self(b.Right);

return (left == b.Left && right == b.Right) ? expr : Expression.MakeBinary(expr.NodeType, left, right);

}

else if (expr.IsUnary()) {

var u = (UnaryExpression)expr;

var op = self(u.Operand);

return (u.Operand == op) ? expr : Expression.MakeUnary(u.NodeType, op, expr.Type);

}

return expr;

}

);

public static bool IsBinary(this Expression expr) {

return expr is BinaryExpression;

}

public static bool IsUnary(this Expression expr) {

return expr is UnaryExpression;

}

public static MemberBinding VisitBinding(this Func<Expression, Expression> self, MemberBinding b) {

switch (b.BindingType) {

case MemberBindingType.Assignment:

return self.VisitMemberAssignment((MemberAssignment)b);

case MemberBindingType.MemberBinding: return self.VisitMemberMemberBinding((MemberMemberBinding)b);

}

return self.VisitMemberListBinding((MemberListBinding)b);

}

public static MemberAssignment VisitMemberAssignment(this Func<Expression, Expression> self, MemberAssignment assignment) {

var e = self(assignment.Expression);

return (e == assignment.Expression) ? assignment : Expression.Bind(assignment.Member, e);

}

public static MemberMemberBinding VisitMemberMemberBinding(this Func<Expression, Expression> self, MemberMemberBinding binding) {

var bindings = self.VisitBindingList(binding.Bindings);

return (bindings == binding.Bindings) ? binding : Expression.MemberBind(binding.Member, bindings);

}

public static MemberListBinding VisitMemberListBinding(this Func<Expression, Expression> self, MemberListBinding binding) {

var initializers = self.VisitElementInitializerList(binding.Initializers);

return (initializers == binding.Initializers) ? binding : Expression.ListBind(binding.Member, initializers);

}

public static ElementInit VisitElementInitializer(this Func<Expression, Expression> self, ElementInit initializer) {

var arguments = self.VisitExpressionList(initializer.Arguments);

return (arguments == initializer.Arguments) ? initializer : Expression.ElementInit(initializer.AddMethod, arguments);

}

public static ReadOnlyCollection<Expression> VisitExpressionList(this Func<Expression, Expression> self, ReadOnlyCollection<Expression> original) {

return VisitList(original, e => self(e));

}

public static ReadOnlyCollection<MemberBinding> VisitBindingList(this Func<Expression, Expression> self, ReadOnlyCollection<MemberBinding> original) {

return VisitList(original, e => self.VisitBinding(e));

}

public static ReadOnlyCollection<ElementInit> VisitElementInitializerList(this Func<Expression, Expression> self, ReadOnlyCollection<ElementInit> original) {

return VisitList(original, e => VisitElementInitializer(self, e));

}

private static ReadOnlyCollection<T> VisitList<T>(ReadOnlyCollection<T> original, Func<T, T> op) {

List<T> @new = null;

for (int i = 0, n = original.Count; i < n; i++) {

T init = op(original[i]);

if (@new != null)

@new.Add(init);

else if (!ReferenceEquals(init, original[i])) {

@new = new List<T>(n);

for (int j = 0; j < i; j++)

@new.Add(original[j]);

@new.Add(init);

}

return (@new == null) ? original : @new.AsReadOnly();

}

Once I pasted this in, it looks like a lot of code (~150 lines). But keep in mind that this is complete, reusable mutating visitor for Linq expression trees.

Ok, now to see it in action. This code chains to the visitor above and creates an expression tree rewriter that replaces all adds with subtracts:

var RewriteSubtractToAdd = ExprOp.Visit.Chain(

(self, last, expr) => {

switch (expr.NodeType) {

case ExpressionType.Add:

var b = (BinaryExpression)expr;

return Expression.Subtract(b.Left, b.Right);

default:

return last(expr);

}

);

Here's the how to use the rewriter:

Expression<Func<int, int>> SubtractOneExpr = (n => n - 1);

var AddOneExpr = (LambdaExpression)RewriteSubtractToAdd(SubtractOneExpr);

var AddOne = (Func<int,int>)AddOneExpr.Compile();

Console.WriteLine(AddOne(5));

In this example, I take an expression that subtracts one and rewrite it as an expression that adds one. Then I compile the resulting expression into a delegate and invoke it to prove it work. The chain-of-responsibility expression visitor does the heavy lifting.

(Follow-up 5-25-2007: Aaron's got a different utility class for dealing with Expression trees. Check out ExpressionBuilder: http://blog.magenic.com/blogs/aarone/archive/2007/05/24/Announcing-MetaLinq-_2D00_-Linq-to-Expressions.aspx)

(Follow-up 7-31-2007: Matt Warren (they guy in the office next to mine has posted a different expression visitor. His is pretty close to the one we use internally: http://blogs.msdn.com/mattwar/archive/2007/07/31/linq-building-an-iqueryable-provider-part-ii.aspx)

This posting is provided "AS IS" with no warranties, and confers no rights.

Visitor Revisitted: LINQ, Function Composablity and Chain of Responsibility

Jomo Fisher — Mon, 07 May 2007 22:53:00 GMT

Jomo Fisher—Last time, I wrote about constructing an inline visitor using new C# language features. It worked fine for what it did, but it completely falls down when you want to extend existing visitors that you’ve created. What if I wanted to modify the PrintTree delegate to print negative numbers inside parentheses? There’s not a way to do it. This is a serious limitation, especially when you have hundreds of node types in your tree structure.

In order to fix this problem I went back to first principles. Without worrying about how to implement it, how do I really want functions to compose and recurse in C#? Here’s what I sketched out using the venerable Fibonacci sequence:

var fib = n => n > 1 ? self(n - 1) + self(n - 2) : n

var fib2x = fib chainto n => last(n) * 2;

var fib6x = fib2x chainto n => last(n) * 3;

Here, I’m pretending to add two new keywords to C#—self and last—which are analogous to this and base for classes. Also, I’m pretending to add a third keyword—chainto—which is used for composing functions. The last keyword is only valid on the right side of a chainto and it causes a call to the left side. The call to self always calls the outermost function into which it is composed.

The thing I want is a language encoding of the Chain of Responsibility pattern with a twist. The twist is that the nested responsibility handlers can call the outermost level through self.

Back to Reality

I’m not free to add keywords to C# so I need a way to work with what I have. Here’s a literal transcoding of my idealized functionality into C# 3.0.

var fib = FuncOp.Create<int, int>(

(self, n) => n > 1 ? self(n - 1) + self(n - 2) : n

);

var fib2x = fib.Chain(

(self, last, n) => last(n) * 2

);

var fib6x = fib2x.Chain(

(self, last, n) => last(n) * 3

);

The pretend keywords self and last become lambda expression parameters who’s type is a delegate. The chainto becomes an extension method named Chain.

To my eyes, it’s nicely concise.

Now that I know what I want it to look like, I have to figure out a way to make it work. This turned out to require all kinds of non-obvious plumbing. I think it needs some explaining.

Smuggling Baggage in Static Delegates

First, I want the fib delegate to be directly callable, so the return type from Create(...) needs to be a delegate that takes an int and returns an int. Something like Func<int,int>.

So that means that the Chain method needs to operate on Func<int,int>. At this point, I’m already in trouble. The reason is that I need to repoint fib’s calls to self(...) to call the result of the call to Chain. The problem is that I have a Func<int, int> and what I want is the original Func<Func<int,int>, int, int> that was originally passed in to Chain. I need to smuggle the ‘unbound’ version of fib around with the regular fib but hide it from the user.

The trick is to realize that fib is a static delegate. So this means that its Delegate.Target member is always null. Since its not being used for its intended purpose (to hold the delegate’s calling target for non-static calls) I’ll use it for my own purposes. To see what I mean, here’s an example of smuggling a random string around with a static delegate:

var addOne = new Func<int, int>(

new Smuggler("My totally random string.", n => n + 1).Invoke);

Console.WriteLine(addOne(5));

Console.WriteLine(((Smuggler)addOne.Target).MyString);

class Smuggler {

private Func<int, int> f;

public string MyString { get; private set; }

public Smuggler(String myString, Func<int, int> f) {

this.f = f;

this.MyString = myString;

}

public int Invoke(int n) { return f(n); }

}

Using this trick, I smuggle around the unbound version of the delegate along with a pointer to the prior function (thus forming a linked list of delegates that have been composed).

Simulated ref Lambda Expression

Each call to Chain needs to repoint all calls to ‘self’ for all prior functions in the chain. In order to do this, I needed to curry a lambda expression against a ref to delegate:

void Function(ref Func<int,int> self) {

Func<int, int> f = (n) => self(n);

}

Unfortunately for me, the C# language doesn’t allow ref parameters inside delegates. I’m not sure why, there’s no obvious technical reason.

This problem can be solved by making a RefOf class to hold the delegate.

private class RefOf<T> {

public T Ref { get; set; }

}

void Function(RefOf<Func<int, int>> self) {

Func<int, int> f = (n) => self.Ref(n);

}

Putting it all Together

Now that we have the tools, let’s put it all together:

static class FuncOp {

private class Binding<T, TResult> {

public Func<T, TResult> Bound { get; private set; }

public Func<Func<T, TResult>,

Func<T, TResult>, T, TResult> Unbound { get; private set; }

public Binding<T, TResult> LastBinding { get; private set; }

public Binding(Func<T, TResult> bound,

Func<Func<T, TResult>,

Func<T, TResult>, T, TResult> unbound,

Binding<T, TResult> lastBinding) {

Bound = bound;

Unbound = unbound;

LastBinding = lastBinding;

}

public TResult Invoke(T t) { return Bound(t); }

}

private class RefOf<T> {

public T Ref { get; set; }

}

public static Func<T, TResult> Create<T, TResult>(

Func<Func<T, TResult>, T, TResult> function) {

var selfRef = new RefOf<Func<T,TResult>>();

var unbound = new Func<Func<T, TResult>,

Func<T, TResult>, T, TResult>(

(s, l, t) => function(s, t)

);

selfRef.Ref = Chain(selfRef, unbound, null);

return selfRef.Ref;

}

private static Func<T, TResult> Chain<T, TResult>(

RefOf<Func<T, TResult>> selfRef,

Func<Func<T, TResult>,

Func<T, TResult>, T, TResult> unbound,

Binding<T, TResult> lastBinding) {

Func<T, TResult> last = null;

Func<T, TResult> bound = (t) => unbound(selfRef.Ref, last, t);

if (lastBinding != null)

last = Chain(selfRef, lastBinding.Unbound,

lastBinding.LastBinding);

var binding = new Binding<T, TResult>(bound, unbound,

lastBinding);

return new Func<T, TResult>(binding.Invoke);

}

public static Func<T, TResult> Chain<T, TResult>(

this Func<T, TResult> lastFunction,

Func<Func<T, TResult>,

Func<T, TResult>, T, TResult> selfFunction) {

var selfRef = new RefOf<Func<T, TResult>>();

var lastBinding = (Binding<T, TResult>)lastFunction.Target;

selfRef.Ref = Chain<T, TResult>(selfRef, selfFunction,

lastBinding);

return selfRef.Ref;

}

Composing Functions in Action

Ok, now I want to see how well this solves my original problem: reusing the PrintTree delegate while putting negative numbers in parenthesis. Here’s the original PrintTree transcoded into the new form:

Node addNumbers = Node.Add(

Node.Add(Node.Constant(-1), Node.Constant(-2)),

Node.Add(Node.Constant(3), Node.Constant(5)));

var UnhandledNode = FuncOp.Create<Node, Node>(

(self, n) => { throw new Exception("Unhandled node type "

+ n.NodeType); }

var PrintTree = UnhandledNode.Chain(

(self, last, n) => {

switch (n.NodeType) {

case NodeType.Add:

BinaryNode bn = (BinaryNode)n;

self(bn.Left);

Console.Write("+");

self(bn.Right);

return n;

case NodeType.Constant:

ConstantNode cn = (ConstantNode)n;

Console.Write(cn.Value);

return n;

default:

return last(n);

}

);

PrintTree(addNumbers);

I like this a lot better than the prior solution. Aside from being composable, I also get to use the plain old C# switch\case instead of inventing my own parallel mechanism. Also, the code pretty much looks like what it does.

Here’s the code that will extend PrintTree to parenthesize negative numbers:

var PrintParens = PrintTree.Chain(

(self, last, n) => {

switch (n.NodeType) {

case NodeType.Constant:

ConstantNode cn = (ConstantNode)n;

if (cn.Value is int) {

int v = (int)cn.Value;

if (v < 0) Console.Write("(" + v + ")");

else Console.Write(v);

} else {

Console.Write(cn.Value);

}

return n;

default:

return last(n);

}

);

PrintParens(addNumbers);

As expected, there’s no need to handle NodeType.Add at this level.

It's interesting to me that turning a Visitor inside-out gives you a Chain of Responsibility. I wonder if there are other OO<-->Functional design pattern dualities like this.

This posting is provided "AS IS" with no warranties, and confers no rights.

(Note 5-24-2007: I put this pattern to good use here: http://blogs.msdn.com/jomo_fisher/archive/2007/05/23/dealing-with-linq-s-immutable-expression-trees.aspx)