Don's McBlog

Expression Editor Quirks (or CodeDom Quirks II)

dmccrady — Fri, 05 Jan 2007 22:45:00 GMT

It's been a while since I've updated this. My apologies, but I do tend to disappear from work for the month of December, which is what I did this year too. Happy 2007!

This article can be considered somewhat of a sequel to my last one (CodeDom Quirks), but now I'll focus on some quirkiness of the WF Rules expression editors. The same editor is used to author and edit both Declarative Rule Conditions (the Condition editor) as well as Rule conditions and actions (the RuleSet editor). Both support a natural algebraic expression language that parses a large subset of what C# can express.

As I mentioned in the previous article, the parser behind these designers consumes an expression and translates into an expression object model used by WF Rules, namely CodeDom. Thus if you enter the following into the condition editor:

2 + 3 == 5

the editor parses this and saves it as a CodeDom tree, just as if you had created the following CodeDom tree yourself:

new CodeBinaryOperatorExpression(

new CodePrimitiveExpression(2),

CodeBinaryOperatorType.Add,

new CodePrimitiveExpression(3)),

CodeBinaryOperatorType.ValueEquality,

new CodePrimitiveExpression(5))

This CodeDom is then serialized out to the .rules file, where you can see that it shows up just as XML serialized CodeDom. Note that the original text (2+3==5) was never saved. When you re-edit the expression (in fact, as soon as you press <TAB> in the dialog), the CodeDom is decompiled back into a canonical string form. It turns out that this is the source of many of the "quirks" described in this article, because the decompile may not preserve the exact syntax you wrote.

SPACING

At its most basic level, this decompilation process will strip out any excess spaces, or add spaces. For example:

2+ 3 ==5

will simply end up being decompiled as:

2 + 3 == 5

That's fairly innocuous, but there are a few instances where the canonical form may surprise you (and I admit, there are times when it may annoy you).

PARENTHESES

The decompilation uses the minimum amount of parenthesization necessary to preserve operator precedence. Thus if you enter the following into the condition editor:

((2 - 3) + 4) == 3

and hit <TAB>, your expression will be replaced by:

2 - 3 + 4 == 3

All the parentheses disappear because they're all redundant.

ZERO MINUS

Remember in the last article when I said the unary negation operator was not supported by CodeDom. No problem, the expression editor will still parse "-x" and turn it into "0 - x". The decompilation is even smart enough to show "0 - x" as "-x". Pretty smart, eh? However, if you enter the following into the condition editor:

3 + (0 - 5) == (0 - 2)

and hit <TAB>, your expression will be rewritten as:

3 + -5 == -2

The "pretty smart" decompilation really can't tell whether you typed "0 - 5" or whether you typed "-5" because they're both represented by the same CodeDom tree.

I do like to think that the canonical form is simpler in this case.

EQUAL FALSE

Similarly, recall that CodeDom does not support a unary NOT operation. The expression editor still parses "!x" and turns it into "x == false". Once again the decompilation is smart enough to show "x == false" as "!x". But once again, if you entered:

(2 > 5) == false

and hit <TAB>, your expression will be rewritten as:

!(2 > 5)

This may or may not be more readable depending on your taste, but it is the canonical representation of the "== false" pattern. Note that if you'd entered:

(2 == 5) == false

the decompilation would be even smarter and rewrite this as:

2 != 5

which is, in my opinion, a much simpler expression.

ALTERNATE OPERATOR SPELLINGS

To make the condition editor's expression language more approachable to a wider audience of developers, we chose to support some alternate operator spellings. Most of these alternate spellings come from VB. So instead of "!=" you can use "<>". Instead of unary "!" you can use "NOT". Here's a complete list:

Inequality: != <>
Unary NOT: ! NOT
Boolean AND: && AND
Boolean OR: || OR
Modulus: % MOD
This reference: this me
Null reference: null nothing

However, only the operator spellings in bold will be preserved by the decompilation. Thus, if you type:

me.x <> 5 AND me.y == nothing

and hit <TAB>, the expression will get rewritten to:

this.x != 5 && this.y == null

This may definitely fall into the category of "annoying", but simply reflects our teams C# bias. My apologies to all esteemed VB'ers out there.

QUALIFIED TYPE NAMES

The expression editor will allow you to refer to unqualified type names as long as they have unambiguous names. For example, if you have types "Foo.Glorp" and "Bar.Glorp", you cannot unambiguously refer to the type as "Glorp"; you have to use the fully qualified type name. On the other hand the type "Console" refers unambiguously to "System.Console", so you can refer to it just as "Console".

The decompilation does not know what type names are unique, so it always fully-qualifies them. Thus if you type:

DateTime.Now.Year == 2007

and press <TAB>, this will get rewritten as:

System.DateTime.Now.Year == 2007

EXPLICIT "THIS" QUALIFICATION

Like C#, the expression editor will allow you to refer to an instance member of the "ambient" class without qualifying it with "this.". In a workflow application, the ambient class is always the root workflow class. If your workflow type has an instance field "int foo", you can enter the following expression in the editor:

foo > 25

However, when you press <TAB>, the canonicalized representation will be decompiled as:

this.foo > 25

We chose to be explicit about "this." qualification because unlike an instance method within a C# class, it is not obvious what the ambient class is. Making "this." qualification explicit helps make developers aware of the context in which they are accessing data.

SUMMARY

If you use the WF Rules expression editor enough, you may run into other CodeDom decompilation "quirks" that I have not mentioned. By all means let me know if you find anything else surprising and/or annoying.

CodeDom Quirks

dmccrady — Wed, 11 Oct 2006 20:53:00 GMT

I'm going to talk a little bit here about the expression model used by WF rules, and some of the idiosyncrasies, caveats, and downright quirks that arise from our chosen expression model.

First, a little history. Rather than (re)inventing our own private Rule expression object model, the WF Rules team opted to leverage an existing expression model, namely CodeDom. CodeDom was originally intended as an object model for code emitters: i.e., VS-based designers like WinForms, and the like. The object model forms an abstraction that enables these designers to target C# and VB developers without having to bake language-specific notions into their tools.

WF Rules takes CodeDom to the next level by enabling semantic validation and execution of CodeDom trees. Using CodeDom, WF Rules supports a very large expression language subset that is very close to what C# and VB support. If you can model it in CodeDom (and sometimes even if you can't, as we'll see), you can execute it using WF Rules. It's actually pretty heady stuff, and includes complex name resolution, type checking, and method overloading. In future releases we will consider operator overloading as well.

CodeDom is a fairly rich object model for expressions, supporting a large subset of arithmetic and programmatic expressions. However, it has some annoying gaps.

Value Inequality: CodeDom's CodeBinaryOperatorType enumeration supports "ValueEquality", but there's no "ValueInequality" operator. That's like being able to write "x == y", but not "x != y". This does seem quite limiting, especially in a Boolean-heavy environment like Rules. Fortunately, there's a simple way to express inequality in terms of the supported "ValueEquality":

x != y → (x == y) == false

In CodeDom, this would be constructed using something like:

new CodeBinaryOperatorExpression(

CodeBinaryOperatorType.ValueEquality,

y),

CodeBinaryOperatorType.ValueEquality,

new CodePrimitiveExpression(false))

Left shift, right shift, and bitwise XOR: While CodeDom supports bitwise AND and bitwise OR operations, there is no support for left or right shifting, and no support for the XOR operator. I don't lose much sleep over this, since these operators aren't often used, especially in high-level Rule Sets.

Boolean NOT: The unary "not" operator cannot be modeled directly in CodeDom. (Actually, CodeDom does not support any unary operators.) Once again, this seems kind of limiting for a Boolean-heavy feature such as Rules, but again there's a fairly simple workaround that is very similar to our "Value Inequality" transformation above:

not x → x == false

In CodeDom, this would be constructed using something like:

new CodeBinaryOperatorExpression(

CodeBinaryOperatorType.ValueEquality,

new CodePrimitiveExpression(false))

As we can see, the Boolean "not" operation is just a variant of the "Value Inequality" case.

Unary Negation: The unary negation operator cannot be directly modeled in CodeDom. Fortunately, negation can be expressed in terms of binary subtraction:

-x → 0 - x

In CodeDom, this would be constructed using something like:

new CodeBinaryOperatorExpression(

new CodePrimitiveExpression(0),

CodeBinaryOperatorType.Subtract,

Bitwise NOT: This is another unary operator that cannot be modeled directly in CodeDom. In theory, this operation can be implemented using the substitution:

~x → x xor 0xFFFFFFFF

(assuming "x" is an unsigned 32-bit integer). However, recall that CodeDom cannot model the XOR operator either. So we're stuck in this case; CodeDom cannot represent bitwise NOT. However, like the shift operators, I believe bitwise NOT is a rarely used operator, which we can live without.

WORKAROUNDS

In the end, if an operator is not explicitly supported by CodeDom, its result can almost always be achieved by calling a helper method.

WF Rules also supports custom extensions to CodeDom, so you can add whatever missing operations you like and use them in your rule expressions. I'll try to cover this in more detail in a later article.

DESIGN-TIME IMPLICATIONS

The Condition editor and the RuleSet editors have a text-based, parser-driven interface that allows WF Rules users to enter their expressions in a natural syntax, and not be too particularly concerned with the fact that it's doing nothing more than building CodeDom. In fact, the parser behind it will happily consume the following expressions:

-x

x != y

and do the appropriate transformations into CodeDom for you automatically.

In a future article, I'll talk about more design-time implications that arise from our use of an expression object model to represent Rules expressions.

Rete?

dmccrady — Sun, 10 Sep 2006 07:14:00 GMT

It seems that every thing that is to be called a Rules Engine must claim to implement the Rete Algorithm. Microsoft's BRE does, Oracle's new Business Rules product does, iLog does, and the list goes on. Going against the tide, the Rules Engine in Windows Workflow Foundation (WF Rules) does not implement the Rete algorithm. That is not to say that it won't in the future, but in the near term, it won't.

Why not?

Let me first give a simple practical reason. WF Rules was developed on a short development cycle, so we just didn't have time to implement it, let alone test it.

There are plenty of resources out there that describe what the Rete algorithm does and why it is so often used in Rules engines. At the end of the day, Rete is an optimization. You don't need a Rete engine to implement a forward-chaining rules engine; however, the Rete optimization may improve the runtime performance of a Rule Set. That's what optimizations do. In particular, Rete reduces the number of predicate evaluations by remembering the most recent value and re-using it on subsequent re-evaluations.

Ironically, I love optimization. I have a compiler-development background, and local & global compiler optimization techniques are fascinating subjects, and fun projects. Implementing the Rete optimization for WF Rules would have been a cool way to spend a coding milestone.

However, I also know that before you try applying a lot of cool optimizations to a project, you should try to make sure that they solve real bottlenecks. Turning on the C++ compiler's global optimizer will do a lot of cool optimizing transformations to your program; but if your program's performance is dominated by network I/O, it won't make a bit of difference.

The same logic applies to Rules solutions. Sometimes (quite often, I'd actually argue), a Rule Set is structured in such a way that there are no opportunities for the Rete optimization to make any difference. For example, if each rule evaluates totally independent expressions against different facts, the Rete algorithm will yield no improvement. In fact, its intrinsic overhead may result in a net loss of performance.

I certainly concede that there are many examples of Rule Sets that contain a lot of common predicates. In those cases, a Rete-based Rules engine like BRE will crush a non-Rete engine like WF Rules. However, in cases where there are few common predicates, the reverse has also been demonstrated: WF Rules and its non-Rete execution mechanism will crush Rete-based BRE.

My point is simply that Rete is an optimization strategy that is not necessarily appropriate for all Rules-based solutions. For WF Rules, we placed much more importance on how much you can express in your Rules, how easy it is to express it, and how well it integrates into Windows Workflow Foundation and the .NET framework.

Performance optimizations for WF Rules are certainly coming in the future. But even without Rete, WF Rules is already a highly performant Rules engine.

Intro

dmccrady — Sun, 10 Sep 2006 06:53:00 GMT

I suppose the first entry in blog should be an introduction of sorts. Since this is a Microsoft blog, I'll mention immediately that I do work at Microsoft, and have since 1997. Prior to that I worked at IBM for 8 years. I am currently the development lead for the Windows Workflow Foundation Rules team. The content of this blog will therefore focus mostly on the WF Rules technology that I helped develop, and that I currently lead.

I should point out that I'm not a big fan of blogs. Heck, I don't even like the word "blog", which frankly sounds more like a bodily function than anything else ("... he got so drunk, he blogged all over the place"). However, they're undeniably popular and occasionally useful sources of information, and I believe I can add my tiny bit of wisdom to those who happen to land on this site via a link or a search.

Finally, for a glimpse at my other life, you can check out my stargazing & astrophotography web site at http://starryvistas.net. Thanks.