Fabulous Adventures In Coding : precedence

Precedence vs order, redux

Eric Lippert — Mon, 10 Aug 2009 17:06:00 GMT

Once more I'm revisting the myth that order of evaluation has any relationship to operator precedence in C#. Here's a version of this myth that I hear every now and then. Suppose you've got a field arr that is an array of ints, and some local variables index and value:

int index = 0;
int value = this.arr[index++];

When all is said and done, value will contain the value that was in this.arr[0], and index will be 1, right?

Right. Now, the myth. I often hear this explained as "because the ++ comes after the variable, the increment happens after the array is dereferenced."

Wrong! "After" implies a relationship based on a sequence of events in time, and the logical sequence of events is extremely well-defined. The sequence of events for this program fragment is:

1) store zero in "index"
2) fetch a reference to this.arr and remember the result
3) fetch the value in "index" and remember the result
4) add one to the result of step 3 and remember the result
5) store the result of step 4 in "index"
6) look up the value in the reference from step 2 at the index from step 3 and remember the result
7) store the result of step 6 in "value"

I emphasize that the logical sequence is well-defined because the compiler, jitter and processor are all allowed to change the actual order of events for optimization purposes, subject to the restriction that the optimized result must be indistinguishable from the required result in single-threaded scenarios. Reorderings are sometimes observable in multithreaded scenarios. Analyzing the consequences of that fact is insanely complicated. I might blog about those issues at some point if I feel brave. But for normal scenarios, you should assume that the ordering of events in time precisely follows the rules laid out in the specification.

In fact, you can demonstrate that the increment happens before the indexing with this little self-referential gem:

int[] arr = {0};
int value = arr[arr[0]++];

What happens? First we fetch arr[0], which is 0, and remember that. Then we increment arr[0], so arr[0] becomes 1. Then we fetch the value of arr[0] because we remember the 0, and we get 1. If we had done the increment after the (outer) indexing then the result would be zero because the increment would not happen until after the value had been fetched.

-- Eric is on vacation; this posting was prerecorded --

Precedence vs Associativity vs Order

Eric Lippert — Fri, 23 May 2008 17:19:00 GMT

Raymond has written about this, I have written about Raymond writing about it, but I still frequently get questions from people who are unclear on the difference between precedence, associativity and evaluation order.

I suspect that this confusion arises from the difference between how most people are trained to evaluate arithmetical expressions versus how compilers generate code to evaluate arithmetical expressions. As a child it was drilled into me that the way to evaluate an arithmetical expression was to recursively apply the PEDMAS rule. That is, evaluate anything in parentheses first. Then evaluate any exponentiations. Then divisions, multiplications, additions and subtractions, in that order. So if you had 4 x 5 x (20 - 12 / 3) you start by evaluating what's in parentheses: 20 - 12 / 3. In there, there are no parens or exponents, so start with the division. Replace 12 / 3 with 4 to get 20 - 4. Then evaluate the subtraction to get 16. Now we have the value for the parens, and we are down to 4 x 5 x 16. Evaluate one of the multiplications -- but wait, we do not know what order to evaluate the multiplications in. But we can do it in any old order, so lets say 5 x 16 is 80, so we have 4 x 80, which is 320, done.

You'll notice that the algorithm that I was taught emphasizes that you do the work on whatever is in the deepest set of parentheses first, no matter what. Many people believe that arithmetical expressions in programming languages work the same way. They do not.

Rather, in programming languages, parentheses indicate how the results of evaluations are combined, but not necessarily the order in which the calculations are carried out. In languages with no side effects, the order of evaluation is irrelevant. But in languages where side effects might occur, order becomes relevant.

The evaluation of an arithmetical expression is controlled by three sets of rules: precedence rules, associativity rules, and order rules.

Precedence rules describe how an underparenthesized expression should be parenthesized when the expression mixes different kinds of operators. For example, multiplication is of higher precedence than addition, so 2 + 3 x 4 is equivalent to 2 + (3 x 4), not (2 + 3) x 4.

Associativity rules describe how an underparenthesized expression should be parenthesized when the expression has a bunch of the same kind of operator. For example, addition is associative from left to right, so a + b + c is equivalent to (a + b) + c, not a + (b + c). In ordinary arithmetic, these two expressions always give the same result; in computer arithmetic, they do not necessarily. (As an exercise can you find values for a, b, c such that (a + b) + c is unequal to a + (b + c) in C#?)

Now the confusing one.

Order of evaluation rules describe the order in which each operand in an expression is evaluated. The parentheses just describe how the results are grouped together; "do the parentheses first" is not a rule of C#. Rather, the rule in C# is "evaluate each subexpression strictly left to right".

The expression F() + G() * H() is equivalent to F() + (G() * H()), but C# does NOT evaluate G() * H() before F(). Rather, this is equivalent to:

temp1 = F();
temp2 = G();
temp3 = H();
temp4 = temp2 * temp3;
result = temp1 + temp4;

Another way to look at it is that the rule in C# is not "do the parentheses first", but rather to parenthesize everything then recursively apply the rule "evaluate the left side, then evaluate the right side, then perform the operation".

This is not a rule of C++. In C++, F(), G() and H() can be evaluated in whatever order the compiler chooses, so long as it combines the results in the right way. A legal C++ compiler might do left to right, right to left, parentheses first, whatever the compiler writer felt like.

The way this topic usually comes up is when someone has an expression chock full of side effects -- assignments, increments, decrements, pointer stores and so on, which they are attempting to convert from C++ to C#, and report the "bug" in the C# compiler to me that C# does not follow the "rules" of C++. Which is ironic, because since there are not actually any rules in C++ about order of evaluation between two sequence points. Thus the bug actually is that their code was never portable C++, and only worked because the code author happened to know (or guess) what ordering the compiler writer chose.

C++ and the Pit Of Despair

Eric Lippert — Tue, 14 Aug 2007 18:25:00 GMT

Raymond has an interesting post today about two subtle aspects of C#: how order of evaluation in an expression is specified as strictly left-to-right, and how the rules regarding local shadowing ensure that an identifier has exactly one meaning in a local scope. He makes an educated guess that the reason for these sorts of rules is to "reduce the frequency of a category of subtle bugs".

I'd like to take this opportunity to both confirm that guess and to expand upon it a bit.

Eliminating Subtle Bugs

You remember in The Princess Bride when Westley wakes up and finds himself locked in The Pit Of Despair with a hoarse albino and the sinister six-fingered man, Count Rugen? The principle idea of a pit of despair is twofold. First, that it is a pit, and therefore easy to fall into but difficult work to climb out of. And second, that it induces despair. Hence the name.

I often think of C++ as my own personal Pit of Despair Programming Language. Unmanaged C++ makes it so easy to fall into traps. Think buffer overruns, memory leaks, double frees, mismatch between allocator and deallocator, using freed memory, umpteen dozen ways to trash the stack or heap -- and those are just some of the memory issues. There are lots more "gotchas" in C++. C++ often throws you into the Pit of Despair and you have to climb your way up the Hill of Quality. (Not to be confused with scaling the Cliffs of Insanity. That's different.)

Now, as I've said before, the design of C# is not a subtractive process. It is not "C++ with the stupid parts taken out". But that said, it would be rather foolish of us to not look at what problems people have typically had with other languages and work to ensure that those exact same problems do not crop up for C# users. I would like C# to be a "Pit of Quality" language, a language where its rules encourage you to write correct code in the first place. You have to work quite hard to write a buffer overrun bug into a C# program, and that's on purpose.

I have never written a buffer overrun in C#. I have never written a bug where I accidentally shadowed a variable in another scope in C#. I have never used stack memory after the function returned in C#. I've done all those things in C++ multiple times, and it's not because I'm an idiot, it's because C++ makes it easy to do all those things accidentally and C# makes it very hard. Making it easy to do good stuff is obviously goodness; thinking about how to make it hard to do bad is actually more important.

Now, given that the design of C# is not subtractive, we have to consider the pros and cons of each decision. Is there any compelling user benefit in a deliberate failure to specify what order functions in an expression are evaluated? The only benefit I can think of is "not breaking some existing users of two existing incompatible implementations by declaring one of the implementations to be wrong", which is the situation that the C standardization committee frequently found itself in, I'm sure. When C# was a new language that wasn't an issue, so we were free to pin that down early. Doing so has compelling benefits; it prevents subtle bugs, and as I'll discuss in a moment, there are other benefits as well.

So, long story short, yes, designing the language so as to prevent certain categories of subtle bugs is a huge priority for us. However, there are other reasons too. For instance:

Uncertainty sometimes has high value but only in special contexts

Like Vroomfondel said, "we demand rigidly defined areas of doubt and uncertainty!" Ideally, those areas should be small and should afford some way for users to eliminate the uncertainty. Nondeterministic finalization is a good example. We deliberately do not specify when and in what order finalizers run because:

the vast majority of the time it makes no difference,
relying on a particular timing or ordering of finalization some small percentage of the time is probably a subtle bug waiting to happen,
specifying it would require us to simplify the implementation to the point where it actually could be specified, thereby destroying much of its value; there is value in that complexity
specifying it ties our hands to make algorithm improvements in the future

But we do provide a mechanism (the "using" statement) whereby if you do need to ensure that a finalizer runs at a particular point, there is an easy syntax for it.

With the possible exception of point 2, the order of evaluation of sub-expressions does not have these problems. It often does make a difference, the implementation is already simple enough that the specification is trivial, and its unlikely that we are going to come up with some incredible improvement in the algorithm which determines what order subexpressions are evaluated in. And if by some miracle we do, the specification does take care to call out that if we can prove that out-of-order evaluation cannot introduce subtle bugs, then we reserve the right to do it.

Uncertainty is hard on the language implementation team

When I am implementing part of the language specification, of course I want great freedom to decide how to implement a particular chunk of semantics. The language specification is not in the business of telling me whether we should be using a hand-written recursive descent parser or a parser-generator that consumes the grammar, etc. But I hate, hate, hate when I'm reading the specification and I'm left with a choice of what semantics to implement. I will choose wrong. I know this, because when I ask you guys what the "intuitively obvious" thing to do is, significant fractions of the population disagree!

Uncertainty is also hard on our testers -- they would like to know whether the language is correct, and being told "the specification is unclear, therefore whatever we do is correct" makes their jobs harder because that's a lie. Crashing horribly is probably not correct. Deadlocking is probably not correct. Somewhere on that spectrum between clearly correct behaviour and clearly incorrect behaviour is a line, and one of testing's jobs is to ensure that the developers produced an implementation that falls on the "correct" side of that line. Not knowing precisely where the line is creates unnecessary work for them, and they are overworked already.

And uncertainty is hard on our user education people. They want to be able to write documentation which clearly describes what semantics the implementation actually implements, and they want to be able to write sample programs to illustrate it.

Uncertainty erodes confidence

And finally, uncertainty erodes confidence in our users that we have a quality product that was designed with deep thought, implemented with care, and behaves predictably and sensibly. People entrust the operation of multi-million dollar businesses to their C# code; they're not going to do that if we can't even reliably tell them what "(++i) + i + (i++)" does!

*****

Next time on FAIC I think I'll talk a bit about a related principle of language design: how we think about issues involving "breaking changes". I'll be expanding upon an excellent article written a couple of years ago by my colleague Neal, so you might want to start there.

Bad Recursion Revisited

Eric Lippert — Thu, 28 Apr 2005 21:12:00 GMT

We have internal email lists for questions about programming languages. Here's one that came across recently that I thought illustrated a good point about language design.

An interview candidate gave the following awful implementation of the factorial function. (Recall that factorial is notated "n!", and is defined as the product of all the integers from 1 to n. 0! is defined as 1. So 4! = 4 x 3 x 2 x 1 = 24.)

If you note that n! = n x ((n-1)!) then a recursive solution comes to mind. When asked to implement the factorial function in C, an interview candidate came up with this:

int F(int x){
return (x > 1) ? (x * F(--x)) : x;
}

Now, leaving aside for the moment the fact that this badly-named function does no bounds checking on the inputs, potentially consumes the entire stack, returns a wrong or nonsensical answer for inputs less than one, and is a recursive solution to a problem that can easily be solved with a simple lookup table, it has another big problem -- it doesn't even return the correct answer for any input greater than one either! F(4) will return 6, not 24, in Microsoft C.

And yet the seemingly equivalent C#, JScript and VBScript programs return 24.

The question was "What the heck is up with that?"

This looks perfectly straightforward. If x is 4, then we evaluate the consequence of the trinary operator. 4 * F(3) = 4 * 3 * F(2) = 4 * 3 * 2 * F(1) = 4 * 3 * 2 * 1 = 24, right? What is broken with C?

Page 52 of K&R has the answer.

"C, like most languages, does not specify the order in which the operands of an operator are evaluated. (The exceptions are &&, ||, ?: and ','.) For example, in a statement like x = f()+g(); f may be evaluated before g or vice versa; thus if either f or g alters a variable on which the other depends, x can depend on the order of evaluation."

The Microsoft C compiler actually evaluates the function call first, which causes x to be decremented before the multiplication. So this is actually the same as 3 * F(3) = 3 * 2 * F(1) = 3 * 2 * 1 = 6.

Why does the specification call out that the compiler can choose any order for subexpression evaluation? Because then the compiler can choose to optimize the order so that it consumes a small number of stack or register slots for the results.

Of course, C was written back in the dark ages -- the 1970's -- when indeed most languages were pretty ill-specified and full of terrible "gotchas" like this. JScript, VBScript, C#, and most modern languages do guarantee that functions will be evaluated in left-to-right order. In all these languages,

x = f() + g() * h();

will evaluate f, then g, then h.

As K&R notes "The moral is that writing code that depends on order of evaluations is a bad programming practice in any language." Amen to that!