Thus Quoth the Humble Programmer

wesdyer — Sat, 23 Dec 2006 02:20:01 GMT

I love Scheme! It is such a beautiful language. I was first introduced to it during college. At that time, I thought it was an interesting language but I didn't see the power of the language until later. That moment came when I experienced reading The Structure and Interpretation of Computer Programs. It was a great joy to read the text and work through the exercises. But the most fulfilling part was thinking about what it all meant. I definitely encourage anyone who wants to expand their mind to read this wonderful text.

Quoting in Scheme

Scheme (or any other LISP dialect) allows programmers to define variables, define lambdas, and evaluate expressions (I know that I am greatly simplifying here). But one feature that is particularly interesting is the ability to quote. Quoting an expression causes the expression not be evaluated but instead to return the structure of the expression. For example (line numbers added for discussion purposes):

1:    (define f1 (lambda (x y) (+ x y)))
2:    f1
3:    (define e (quote (lambda (x y) (+ x y))))
4:    e
5:    (define f2 (eval e))
6:    (f1 3 4)
7:    (f2 3 4)

This program produces the following results.

#<procedure:f1>
(lambda (x y) (+ x y))
7
7

Line number 1 defines a variable f1 to be the result of evaluating a lambda expression, thus f1 is a function. Line 2 evaluates f1 which simply returns the function. Line 3 defines e to be the result of evaluating the quote of the same lambda expression that appeared in line 1. But now e is not a function but rather a structure describing the lambda. Line 4 evaluates e, which displays the structure it references. Line 5 defines a variable f2 to be the result of apply eval to e. Eval runs the Scheme interpreter on its arguments. Recall, that e is a structure describing a lambda. So applying eval to e we get a lambda and so f2 is now a function described by this lambda. Finally, on lines 6 and 7 we apply f1 and f2 respectively to arguments 3 and 4. Since they are both functions that correspond to the same lambda structure we expect to get the same results and indeed we do.

Quoting in C#

With C# 3.0 we can do essentially the same thing (again line numbers added for discussion purposes).

1:    Func<int, int, int> f1 = (x, y) => x + y;
2:    Console.WriteLine(f1);
3:    Expression<Func<int, int, int>> e = (x, y) => x + y;
4:    Console.WriteLine(e);
5:    Func<int, int, int> f2 = e.Compile();
6:    Console.WriteLine(f1(3, 4));
7:    Console.WriteLine(f2(3, 4));

This program displays the following:

System.Linq.Func`3[System.Int32,System.Int32,System.Int32]
(x, y) => (x + y)
7
7

Notice how remarkably similar both the programs and that outputs are. Again, on line 1 we define a delegate to be the result of evaluating a lambda expression. On line 2, we display this delegate. On line 3, we assign e the result of evaluating a lambda expression. Notice that in C# both assignments (lines 1 and 3) look the same except for the type of the variable to which the lambda is assigned. Lambdas can be converted either to delegates or to expression trees depending on the usage. Delegates can be invoked while expression trees preserve the structure of the lambda. On line 4, we display the expression tree for e which corresponds to the structure of the lambda. On line 5, we do the equivalent of the eval in Scheme by compiling the expression tree structure to a delegate. We can now invoke f2 like we can invoke f1. On lines 6 and 7, we invoke these two delegates and display their results.

So What?

While this is all very remarkable, it may seem a bit esoteric. Why take the trouble to have a variable with a value that corresponds to the structure of a lambda and then convert that to an invocable form? Why not just always create the delegate immediately? The reason is with the expression tree form of the lambda, we can reason about the structure and then take action on it. Consider queries. When a query is translated it creates a number of lambdas, but one of the key questions is: Are those lambdas then converted to delegates or expression trees? For example:

var q = from x in foo
where x > 1
select x + 1;

This is translated to:

var q = foo.Where(x => x > 1)
.Select(x => x + 1);

We have already discussed that which "Where" and "Select" methods are called depends upon what the type of foo is and which extension methods are in scope. If the "Where" and "Select" methods take a delegate then the lambdas will be converted to delegates; however, if they take an expression tree then the lambdas are converted to expression trees instead. So a LINQ provider (or someone who has implemented the query pattern methods like "Where" and "Select") can choose whether they would like the lambdas in delegate or expression tree form. Now lets look at a few providers and see what they do. LINQ to objects (or the in-memory query operators that apply to IEnumerable<T>) take delegates. This is because they will invoke the delegates to do things like evaluate predicates or projections. However, LINQ to SQL on the other hand takes expression trees. It does not "invoke" the lambdas but instead reasons over the structure of the lambdas and then translates the expression trees to equivalent SQL code given a schema for a database. This is the beauty of the LINQ pattern.

Adding the ability to quote lambdas enables all sorts of possiblities. You might easily imagine other uses for expression trees: remoting evaluation of expressions, query planning, executing them on a GPU, ....

What can you think of doing?

Yet Another Language Geek : Scheme

Thus Quoth the Humble Programmer