Fabulous Adventures In Coding : scope

Simple names are not so simple, Part Two, plus, volcanoes and fried foods

Eric Lippert — Thu, 05 Nov 2009 14:52:00 GMT

I've returned from a brief vacation, visiting friends on the island of Maui. I'd never been to that part of the world before. Turns out, it's a small island in the middle of the Pacific Ocean, entirely made out of volcanoes. Weird! But delightful.

The most impressive thing about the Hawaiian Islands for me was just how obvious were -- even to my completely untrained eyes -- the geomechanical and fluvial processes which shaped the landscape. The mountains and craters and river valleys and red sand beaches and easily-fractured rocks were very different from the (also somewhat volcanic) much older mountainous landscape I've lived in for the past decade.

Also quite amusing to me was learning to read and pronounce Hawaiian place names. It is all very logical once you know the system; before long I could easily pronounce signs like WAINAPANAPA STATE PARK -- wa-ee-napa-napa -- or PUUNENE AVENUE -- pu-oo-nay-nay -- or MAILIBEHANAMONOTANA STREET -- "Miley-Stewart-is-really-Hannah-Montana".

Many thanks to K and R and D for putting me and Leah up for a week; if you're going to Hawai'i and can stay with locals, I highly recommend it, particularly if they are awesome people. Everyone in Maui was awesome, with the exception of the rangers at (stunningly beautiful, even for Maui) Wa'inapanapa State Park, who are apparently consistently grumpy. As one Hawaiian, himself in the camping sector of the economy put it to me, "They do not have the big aloha".

The most amusing encounter was on the Hana Highway. There are numerous little stops along the way, where someone has erected a hut or parked a trailer and is selling coconuts, smoothies, banana bread, and so on. Hand-lettered signs, stunning natural beauty, middle of nowhere, you get the picture I'm sure. At one of the larger such stops there was a young fellow, probably in his late twenties, serving a variety of fried foods. It was mostly traditional American-style Chinese food, but also he had french fries, fish'n'chips, and so on. He was clearly not a native speaker of English, but spoke understandably with a strong accent. We were waiting behind a middled-aged woman with a typically midwestern American accent. Their conversation went something like this:

Her: I'm not very hungry, can I just get the fish without the chips?

Him, not quite following her: Half order?

Her, louder: How much without the fries?

This went back and forth for some time, both sides becoming increasingly frustrated by the communication breakdown, until:

Her, even louder: Can I speak to your manager?

Leah and K and I silently boggled -- there is no other word for it -- at each other for a moment. Where on earth did she imagine that a manager was going to emerge from? There was a counter, behind that, a trailer with a wok in it, behind that, jungle, and behind that, huge jagged lava rocks followed immediately by the Pacific Ocean. And what sort of management structure does she think one really needs to manage a single guy selling pineapple fried rice at the side of a highway? My conclusion: people have strange beliefs. Sometimes their beliefs cause them to leave in a huff with neither fish nor chips, even when fish and chips are both plentiful and reasonably priced. Hopefully she had better luck in Hana.

Anyway, enough travelogue. Regarding the puzzle from last time: the code is correct, and compiles without issue. I was quite surprised when I first learned that; it certainly looks like it violates our rule about not using the same simple name to mean two different things in one block.

The key is to understanding why this is legal is that the query comprehensions and foreach loops are specified as syntactic sugars for another program, and it is that program which is actually analyzed for correctness. Our original program:

static void Main()
{
int[] data = { 1, 2, 3, 1, 2, 1 };
foreach (var m in from m in data orderby m select m)
System.Console.Write(m);
}

is transformed into

static void Main()
{
int[] data = { 1, 2, 3, 1, 2, 1 };
{
    IEnumerator<int> e = ((IEnumerable<int>)(data.OrderBy(m=>m)).GetEnumerator();
    try
    {
      int m;
      while(e.MoveNext())
      {
        m = (int)(int)e.Current;
        Console.Write(m);
      }
    }
    finally
    {
      if (e != null) ((IDisposable)e).Dispose();
    }
}
}

There are five usages of m in this transformed program; it is:

1) declared as the formal parameter of a lambda.
2) used in the body of the lambda; here it refers to the formal parameter.
3) declared as a local variable
4) written to in the loop; here it refers to the local variable
5) read from in the loop; here it refers to the local variable

Is there any usage of a local variable before its declaration? No.

Are there any two declarations that have the same name in the same declaration space? It would appear so. The body of Main defines a local variable declaration space, and clearly the body of Main contains, indirectly, two declarations for m, one as a formal lambda parameter and one as a local. But I said last time that local variable declaration spaces have special rules for determining overlaps. It is illegal for a local variable declaration space to directly contain a declaration such that another nested local variable declaration space contains a declaration of the same name. But an outer declaration space which indirectly contains two such declarations is not an error. So in this case, no, there are no local variable declarations spaces which directly contain a declaration for m, such that a nested local variable declaration space also directly contains a declaration for m. Our two local variable declarations spaces which directly contain a declaration for m do not overlap anywhere.

Is there any declaration space which contains two inconsistent usages of the simple name m? Yes, again, the outer block of Main contains two inconsistent usages of m. But again, this is not relevant. The question is whether any declaration space directly containing m has an inconsistent usage. Again, we have two declaration spaces but they do not overlap each other, so there's no problem here either.

The thing which makes this legal, interestingly enough, is the generation of the loop variable declaration logically within the try block. Were it to be generated outside the try block then this would be a violation of the rule about inconsistent usage of a simple name throughout a declaration space.

Simple names are not so simple

Eric Lippert — Mon, 02 Nov 2009 14:23:00 GMT

C# has many rules that are designed to prevent some common sources of bugs and encourage good programming practices. So many, in fact, that it is often quite confusing to sort out exactly which rule has been violated. I thought I might spend some time talking about what the different rules are. We'll finish up with a puzzle.

To begin with, it will be vital to understand the difference between scope and declaration space. To refresh your memory of my earlier article: the scope of an entity is the region of text in which that entity may be referred to by its unqualified name. A declaration space is a region of text in which no two things may have the same name (with an exception for methods which differ by signature.) A "local variable declaration space" is a particular kind of declaration space used for declaring local variables; local variable declaration spaces have special rules for determining when they overlap.

The next thing that you have to understand to make any sense o this is what a "simple name" is. A simple name is always either just a plain identifier, like "x", or, in some cases, a plain identifier followed by a type argument list, like "Frob<int, string>".

Lots of things are treated as "simple names" by the compiler: local variable declarations, lambda parameters, and so on, always have the first form of simple name in their declarations. When you say "Console.WriteLine(x);" the "Console" and "x" are simple names but the "WriteLine" is not. Confusingly, there are some textual entities which have the form of simple names, but are not treated as simple names by the compiler. We might talk about some of those situations in later fabulous adventures.

So, without further ado, here are some relevant rules which are frequently confused. It's rules 3 and 4 that people find particularly confusing.

1) It is illegal to refer to a local variable before its declaration. (This seems reasonable I hope.)
2) It is illegal to have two local variables of the same name in the same local variable declaration space or nested local variable declaration spaces.
3) Local variables are in scope throughout the entire block in which the declaration occurs. This is in contrast with C++, in which local variables are in scope in their block only at points after the declaration.
4) For every occurrence of a simple name, whether in a declaration or as part of an expression, all uses of that simple name within the immediately enclosing local variable declaration space must refer to the same entity.

The purpose of all of these rules is to prevent the class of bugs in which the reader/maintainer of the code is tricked into believing they are referring to one entity with a simple name, but are in fact accidentally referring to another entity entirely. These rules are in particular designed to prevent nasty surprises when performing what ought to be safe refactorings.

Consider a world in which we did not have rules 3 and 4. In that world, this code would be legal:

class C
{
int x;
void M()
{
    // 100 lines of code
    x = 20; // means "this.x";
    Console.WriteLine(x); // means "this.x"
    // 100 lines of code
    int x = 10;
    Console.WriteLine(x); // means "local x"
}
}

This is hard on the person reading the code, who has a reasonable expectation that the two "Console.WriteLine(x)" lines do in fact both print out the contents of the same variable. But it is particularly nasty for the maintenance programmer who wishes to impose a reasonable coding standard upon this body of code. "Local variables are declared at the top of the block where they're used" is a reasonable coding standard in a lot of shops. But changing the code to:

class C
{
int x;
void M()
{
    int x;
    // 100 lines of code
    x = 20; // no longer means "this.x";
    Console.WriteLine(x); // no longer means "this.x"
    // 100 lines of code
    x = 10;
    Console.WriteLine(x); // means "local x"
}
}

changes the meaning of the code! We wish to discourage authoring of multi-hundred-line methods, but making it harder and more error-prone to refactor them into something cleaner is not a good way to achieve that goal.

Notice that the original version of this program, rule 3 means that the program violates rule 1 -- the first usage of "x" is treated as a reference to the local before it is declared. The fact that it violates rule 1 because of rule 3 is precisely what prevents it from being a violation of rule 4! The meaning of "x" is consistent throughout the block; it always means the local, and therefore is sometimes used before it is declared. If we scrapped rule 3 then this would be a violation of rule 4, because we would then have two inconsistent meanings for the simple name "x" within one block.

Now, these rules do not mean that you can refactor willy-nilly. We can still construct situations in which similar refactorings fail. For example:

class C
{
int x;
void M()
{
    {
      // 100 lines of code
      x = 20; // means "this.x";
      Console.WriteLine(x); // means "this.x"
    }
    {
      // 100 lines of code
      int x = 10;
      Console.WriteLine(x); // means "local x"
    }
}
}

This is perfectly legal. We have the same simple name being used two different ways in two different blocks, but the immediately enclosing block of each usage does not overlap that of any other usage. The local variable is in scope throughout its immediately enclosing block, but that block does not overlap the block above. In this case, it is safe to refactor the declaration of the local to the top of its block, but not safe to refactor the declaration to the top of the outermost block; that would change the meaning of "x" in the first block. Moving a declaration up is almost always a safe thing to do; moving it out is not necessarily safe.

Now that you know all that, here's a puzzle for you, a puzzle that I got completely wrong the first time I saw it:

using System.Linq;
class Program
{
static void Main()
{
    int[] data = { 1, 2, 3, 1, 2, 1 };
    foreach (var m in from m in data orderby m select m)
      System.Console.Write(m);
}
}

It certainly looks like name "m" is being used multiple times to mean different things. Is this program legal? If yes, why do the rules for not re-using simple names not apply? If no, precisely what rule has been violated?

[Eric is on vacation; this posting was pre-recorded.]

What's The Difference, Part Two: Scope vs Declaration Space vs Lifetime

Eric Lippert — Mon, 03 Aug 2009 17:08:00 GMT

"Scope" has got to be one of the most confusing words in all of programming language design. People seem to use it casually to mean whatever is convenient at the time; I most often see it confused with lifetime and declaration space. As in "the memory will be released when the variable goes out of scope".

In an informal setting, of course it is perfectly acceptable to use "scope" to mean whatever you want, so long as the meaning is clearly communicated to the audience. In a more formal setting, like a book or a language specification, it's probably better to be precise.

The difference between scope and declaration space in C# is subtle.

The scope of a named entity is the region of program text in which it is legal to refer to that entity by its unqualified name.

There are some subtleties here. The implication does not "go the other way" -- it is not the case that if you can legally use the unqualified name of an entity, that the name refers to that entity. Scopes are allowed to overlap. For example, if you have:

class C
{
int x;
void M()
{
int x;
}
}

then the field is in scope throughout the entire body text of C, including the entirity of M. Local variable x is in scope throughout the body of M, so the scopes overlap. When you say "x", whether the field or the local is chosen depends on where you say it.

A declaration space, by contrast, is a region of program text in which no two entities are allowed to have the same name. For example, in the region of text which is body of C excluding the body of M, you're not allowed to have anything else named x. Once you've got a field called x, you cannot have another field, property, nested type, or event called x.

Thanks to overloading, methods are a bit of an oddity here. One way to characterize declaration spaces in the context of methods would be to say that "the set of all overloaded methods in a class that have the same name" constitutes an "entity". Another way to characterize methods would be to tweak the definition of declaration space to say that no two things are allowed to have the same name except for a set of methods that all differ in signature.

In short, scope answers the question "where can I use this name?" and declaration space answers the question "where is this name unique?"

Lifetime and scope are often confused because of the strong connection between the lifetime and scope of a local variable. The most succinct way to put it is that the contents of a local variable are guaranteed to be alive at least as long as the current "point of execution" is inside the scope of the local variable. "At least as long" of course implies "or longer"; capturing a local variable, for example, extends its lifetime.