|
|
Eric Lippert's Blog
-
Joel had a great series of articles many years ago about the benefits of writing functional specifications, that is, specifications of how the product looks to its users. I want to talk a bit about technical specifications, that is, a specification of how something actually works behind the scenes. A while back, I described how I wrote a seemingly-trivial piece of code by first writing the technical spec, and then writing the code to exactly implement the spec, and the test cases to test each statement of the spec. I use this technique frequently; it almost always saves me time and effort. If you don't write a spec first, it's very easy to get bogged down in a morass of bugs, and hard to know when you're actually done writing the code.
Here's an example of another seemingly-trivial function that smart people gave a good half-dozen completely wrong answers to because they didn't write a spec. Notice how once I state what the spec is, the code follows very naturally.
I thought I might go through a recent example of a slightly less trivial problem. I needed a function in the compiler which could take a valid list of arguments A1 to an applicable method M, and give me back two lists. The first result list is a list of local variable declarations, L, the second, another argument list, A2. Here's the kicker: evaluating each member of A2 must have no side effects. And, the combination of executing L followed by M(A2) must give exactly the same results as executing M(A1). (I also knew that the members of A1 had already had all the conversions generated to make them match the types of M's parameter list, and so on. Overload resolution was completely done at this point.)
Why I needed such a function is not particularly important; perhaps you can speculate as to why I needed it.
So I started writing a spec, starting with the description above: the function T takes a valid list... blah blah blah.
That's not enough of a spec to actually write code.
I realized that some expressions in A1 might have side effects, and some might not. If an expression had side effects, then I could declare a temp local, execute the side effects, assign the result to the local, and then use the local as the expression in the same place in A2.
Super. Now I can write more sentences in my spec.
For each argument x in A1, we potentially generate a temporary value as follows: First, if x has no side effects, then simply keep x in the appropriate place in A2. Otherwise, if x does have side effects, then generate a temporary variable declaration in L, var temp = x. Then put an expression that evaluates the temporary variable in the appropriate place in A2.
Then I thought "can I correctly implement this spec?"
I started thinking about the possible cases for the input, A1. The members of A1 could be passed by value, out, or ref; I already knew they were correct, since the particular method M was an applicable method of argument list A1. In this particular point in the compilation process, there was no need to worry about whether there were "params" arguments, whether there were missing arguments with default values, and so on; that had already been taken care of. I also noted that whether they were "ref" or "out" was irrelevant; ref and out are the same thing behind the scenes. As far as the compiler is concerned, the only difference between them is that the definite assignment rules are different for each. So we had two cases for each argument: it's being passed by value, or by ref/out.
If it's being passed by ref/out, then what we actually do is we pass the managed address of a variable to the method. In M(ref int x), the type of x is actually a special type int&, "managed reference to variable of type int", which is not a legal type in C#. That's why we hide it from you by requiring a goofy "ref" syntax any time you want to talk about a managed reference to a variable.
Unfortunately, our IL generation pass was written with the assumption that all local variables are never of these magic managed-reference-to-variable types. I was then faced with a choice. Either come up with a technique for working with this limitation, or rewrite the most complicated code in the IL gen to support this scenario. (The code which figures out how to deal with managed references is quite complex.) I decided to opt for the former strategy. Which meant more spec work, since our spec now doesn't handle these cases!
For each argument x in A1, we potentially generate a temporary value as follows:
First, if x has no side effects, then simply keep x in the appropriate place in A2.
Otherwise, if x does have side effects and corresponds to a value parameter, then generate ... etc.
Otherwise, if x does have side effects and corresponds to a ref/out parameter, then a miracle happens.
Clearly that last bit needed some work.
So I wrote down all the possible cases I could think of. Clearly x has to be a variable if its type is "reference to variable", so that makes for a small number of cases:
Otherwise, if x does have side effects and corresponds to a ref/out parameter, then the possible cases are:
* x is a local variable
* x is a value parameter
* x is an out parameter
* x is a ref parameter
* x is a field of an instance
* x is a static field
* x is an array element
* x is a pointer dereferenced with *
* x is a pointer dereferenced with [ ]
and for each, a miracle happens.
Again, clearly this is not good enough. I re-examined my list to see if I could eliminate any of these cases. Locals, parameters and static fields never have side effects, so we can eliminate them.
Also, at this point in our analysis, I knew that pointer dereferences of the form "pointer[index]" had already been rewritten into the form "*(pointer + index)", which meant that in practice, we would never actually hit that last case in this algorithm; the second-last case would take care of both.
Otherwise, if x does have side effects and corresponds to a ref/out parameter, then the possible cases are:
* x is a field of an instance
* x is an array element
* x is a pointer dereferenced with *
and for each, a miracle happens.
I then started to think of what side effects could happen for each. We could have "ref instance.field", "ref array[index]", or "ref *pointer", and "instance", "array", "index" and "pointer" can all be expressions that have a side effect. ("field" cannot, it merely names a field.) So now we can use the same specification as before:
Otherwise, if x does have side effects and corresponds to a ref/out parameter, then the possible cases are:
- x is ref/out instance.field: in this case, add var temp=instance to L and ref/out temp.field to A2.
- x is ref/out array[index]: in this case, add var t1 = array and var t2 = index to L and ref/out t1[t2] to A2.
- x is ref/out *pointer: in this case, add var temp = pointer to L and ref/out *temp to A2.
And now we have something I can implement. So I sent this proposed spec around for review, while I started plugging away at the implementation.
This spec is wrong. Can you spot the bugs in my implementation that my coworkers found by reading my spec?
Next time, the thrilling conclusion.
|
-
Thanks to everyone who left thoughtful and insightful comments on last week's post.
More countries really ought to implement Instant Runoff Voting; it would certainly appeal to the geek crowd. Many people left complex opinions of the form "I'd prefer to make the change, but if you can't do that then make it a warning". Or "don't make the change, do make it a warning", and so on. But what I can deduce from reading the comments is that there is a general lack of consensus on what the right thing to do here is. In fact, I just did a quick tally:
Commenters who expressed support for a warning: 26
Commenters who expressed the sentiment "it's better to not make the change": 24
Commenters who expressed the sentiment "it's better to make the change": 25
Wow. I guess we'll flip a coin. :-) (*)
Four people suggested to actually make it an error to do this. That's a pretty big breaking change, particularly since we would be breaking not just "already broken" code, but plenty of code that works perfectly well today -- see below. That's not likely to happen.
People also left a number of interesting suggestions. I thought I'd discuss some of those a little bit.
First off, I want to emphasize that what we're attempting to address here is the problem that the language encourages people to write code that has different semantics than they think it has. The problem is NOT that the language has no way to express the desired semantics; clearly it does. Just introduce a new variable explicitly into the loop.
A number of suggestions were for ways that the language could more elegantly express that notion. Some of the suggestions:
foreach(var x in c) inner
foreachnew(var x in c)
foreach(new var x in c)
foreach(var x from c)
foreach(var x inside c)
Though we could do any of those, none of them by themselves solve the problem at hand. Today, you have to know to use a particular pattern with foreach to get the semantics you want: declare a variable inside the loop. With one of these changes, you still have to know to use a particular keyword to get the semantics you want, and it is still easy to accidentally do the wrong thing.
Furthermore, a change so small and so targetted at such a narrow scenario probably does not provide enough benefit to justify the large cost of creating a new syntax, particularly one which is still easily confused with an existing syntax.
C++ luminary Herb Sutter happened to be in town and was kind enough to stop by my office to describe to me how they are solving a related problem in C++. Apparently the next version of the C++ standard will include lambdas, and they're doing this:
[q, &r] (int x) -> int { return M(x, q, r); }
This means that the lambda captures outer variable q by value, captures r by reference, takes an int and returns an int. Whether the lambda captures values or references is controllable! An interesting approach but one that doesn't immediately solve our problem here; we cannot make lambdas capture by value by default without a huge breaking change. Capturing by value would have to require new syntax, and then we're in the same boat again: the user has to know to use the new syntax when in a foreach loop.
A number of people also asked what the down sides of adding a warning are. The down side is that a warning which warns about correct behaviour is a very bad warning; it makes people change working code, and frequently they break working code in order to eliminate a warning that shouldn't have been present in the first place. Consider:
foreach(var insect in insects)
{
var query = frogs.Where(frog=>frog.Eats(insect));
Console.WriteLine("{0} is eaten by {1} frogs.", insect, query.Count());
}
This makes a lambda closed over insect; the lambda never escapes the loop, so there's no problem here. But the compiler doesn't know that. The compiler sees that the lambda is being passed to a method called Where, and Where is allowed to do anything with that delegate, including storing it away to be called later. Which is exactly what Where does! Where stores away the lambda into a monad that represents the execution of the query. The fact that the query object doesn't survive the loop is what keeps this safe. But how is the compiler supposed to suss out that tortuous chain of reasoning? We'd have to give a warning for this case, even though it is perfectly safe.
It gets worse. A lot of people are required by their organizations to compile with "warnings are errors" turned on. Therefore, any time we introduce a new warning for a pattern that is often actually safe and frequently used, we are effectively causing an enormous breaking change. A vaccine which kills more healthy people than the disease would have is probably not a good bet. (**)
This is not to say that a warning is a bad idea, but that it is not the obvious slam dunk good idea that it initially appears to be.
A number of people suggested that the problem was in the training of the developers, not in the design of the language. I disagree. Obviously modern languages are complex tools that require training to use, but we are working hard to make a language where people's natural intuitions about how things work lead them to write correct code. I have myself made this error a number of times, usually in the form of writing code like the code above, and then refactoring it in such a manner that suddenly some part of it escapes the loop and the bug is introduced. It is very easy to make this mistake, even for experienced developers who thoroughly understand closure semantics. That's a flaw in the design of the language.
And finally, a number of people made suggestions of the form "make it a warning in C# 4, and an error in C# 5", or some such thing. FYI, C# 4 is DONE. We are only making a few last-minute "user is electrocuted"-grade bug fixes, mostly based on your excellent feedback from the betas. (If you have bug reports from the beta, please keep sending them, but odds are good they won't get fixed for the initial release.) We are certainly not capable of introducing any sort of major design change or new feature at this point. And we try to not introduce semantic changes or new features in service packs. We're going to have to live with this problem for at least another cycle, unfortunately.
********
(*) Mr. Smiley Face indicates that Eric is indulging in humourous japery.
(**) I wish to emphasize that I am 100% in favour of vaccinations for deadly infectious diseases, even vaccines that are potentially dangerous. The number of people made ill or killed by the smallpox vaccine was tiny compared to the number of people who did not contract this deadly, contagious (and now effectively extinct) disease as a result of mass vaccination. I am a strong supporter of vaccine research. I'm just making an analogy here people.
|
-
I don't know why I haven't blogged about this one before; this is the single most common incorrect bug report we get. That is, someone thinks they have found a bug in the compiler, but in fact the compiler is correct and their code is wrong. That's a terrible situation for everyone; we very much wish to design a language which does not have "gotcha" features like this.
But I'm getting ahead of myself. What's the output of this fragment?
var values = new List<int>() { 100, 110, 120 };
var funcs = new List<Func<int>>();
foreach(var v in values)
funcs.Add( ()=>v );
foreach(var f in funcs)
Console.WriteLine(f());
Most people expect it to be 100 / 110 / 120. It is in fact 120 / 120 / 120. Why?
Because ()=>v means "return the current value of variable v", not "return the value v was back when the delegate was created". Closures close over variables, not over values. And when the methods run, clearly the last value that was assigned to v was 120, so it still has that value.
This is very confusing. The correct way to write the code is:
foreach(var v in values)
{
var v2 = v;
funcs.Add( ()=>v2 );
}
Now what happens? Every time we re-start the loop body, we logically create a fresh new variable v2. Each closure is closed over a different v2, which is only assigned to once, so it always keeps the correct value.
Basically, the problem arises because we specify that the foreach loop is a syntactic sugar for
{
IEnumerator<int> e = ((IEnumerable<int>)values).GetEnumerator();
try
{
int m; // OUTSIDE THE ACTUAL LOOP
while(e.MoveNext())
{
m = (int)(int)e.Current;
funcs.Add(()=>m);
}
}
finally
{
if (e != null) ((IDisposable)e).Dispose();
}
}
If we specified that the expansion was
try
{
while(e.MoveNext())
{
int m; // INSIDE
m = (int)(int)e.Current;
funcs.Add(()=>m);
}
then the code would behave as expected.
It's compelling to consider fixing this for a hypothetical future version of C#, and I'd like to hear your feedback on whether we should do so or not. The reasons FOR making the change are clear; this is a big confusing "gotcha" that real people constantly run into, and LINQ, unfortunately, only makes it worse, because it is likely to increase the number of times a customer is going to use a closure in a loop. Also, it seems reasonable that the user of the foreach loop might think of there being a "fresh" loop variable every time, not just a fresh value in the same old variable. Since the foreach loop variable is not mutable by user code, this reinforces the idea that it is a succession of values, one per loop iteration, and not "really" the same variable over and over again. And finally, the change has no effect whatsoever on non-closure semantics. (In fact, in C# 1 the spec was not clear about whether the loop variable went inside or outside, since in a world without closures, it makes no difference.)
But that said, there are some very good reasons for not making this change.
The first reason is that obviously this would be a breaking change, and we hates them, my precious. Any developers who depend on this feature, who require the closed-over variable to contain the last value of the loop variable, would be broken. I can only hope that the number of such people is vanishingly small; this is a strange thing to depend on. Most of the time, people do not expect or depend on this behaviour.
Second, it makes the foreach syntax lexically inconsistent. Consider foreach(int x in M()) The header of the loop has two parts, a declaration int x and a collection expression, M(). The int x is to the left of the M(). Clearly the M() is not inside the body of the loop; that thing only executes once, before the loop starts. So why should something to the collection expression's left be inside the loop? This seems inconsistent with our general rule that stuff to the left logically "happens before" stuff to the right. The declaration is lexically NOT in the body of the loop, so why should we treat it as though it were?
Third, it would make the "foreach" semantics inconsistent with "for" semantics. We have this same problem in "for" blocks, but "for" blocks are much looser about what "the loop variable" is; there can be more than one variable declared in the for loop header, it can be incremented in odd ways, and it seems implausible that people would consider each iteration of the "for" loop to contain a fresh crop of variables. When you say for(int i; i < 10; i += 1) it seems dead obvious that the "i += 1" means "increment the loop variable" and that there is one loop variable for the whole loop, not a new fresh variable "i" every time through! We certainly would not make this proposed change apply to "for" loops.
And fourth, though this is a nasty gotcha, there is an easy workaround, and tools like ReSharper detect this pattern and suggest how to fix it. We could take a page from that playbook and simply issue a compiler warning on this pattern. (Though adding new warnings brings up a whole raft of issues of its own, which I might get into in another post.) Though this is vexing, it really doesn't bite that many people that hard, and it's not a big deal to fix, so why go to the trouble and expense of taking a breaking change for something with an easy fix?
Design is, of course, the art of compromise in the face of many competing principles. "Eliminate gotchas" in this case directly opposes other principles like "no breaking changes", and "be consistent with other language features". Any thoughts you have on pros or cons of us taking this breaking change in a hypothetical future version of C# would be greatly appreciated.
|
-
Three baseball umpires are having lunch together. The first umpire says "Well, a lot of them are balls, and a lot of them are strikes, but I always calls 'em as I sees 'em."
The second umpire says "Hmph. I calls 'em as they are."
The third umpire slowly looks at his two colleagues and declares "They ain't nothin' until I calls 'em."
Those of you unfamiliar with the bizarre rules of baseball might need a brief primer. Suppose the pitcher throws a pitch and the batter swings and misses. Such a failure to strike the ball is, bizarrely enough, called a "strike", and counts against the batter; three strikes and the batter is out. But what if the batter fails to swing at all? If the umpire decides that the pitch was "inside the strike zone" then the pitch counts as a strike against the batter. If the pitch was outside the strike zone then the pitch counts as a "ball" (another bizarre and confusing name; obviously the object pitched is also called a ball). If the pitcher pitches four "balls" before the batter accumulates three "strikes" then the batter gets to "walk" to first base for free. (At least the walk is sensibly named.)
Formally, the strike zone is reasonably well-defined by the rules (though as the Wikipedia article linked to above indicates, there are some subtle points left out of the definition.) But the formal definition is actually irrelevant; the rules of baseball also state that a strike is any pitch that the umpire says is a strike. Umpires are given wide lattitude to declare what is a strike, and there are no appeals allowed.
And hence the fundamental disagreement between the three umpires. The first umpire believes that whether a pitch was in the strike zone or not is a fact about an objective reality, and that the call is a sometimes-imperfect subjective judgment about that reality. The second umpire seems to be basically agreeing with the objective, materialist stance of the first umpire, and simply bragging about having 100% accuracy in judgment. The third umpire's position is radically different from the first two: that the rules of baseball say that regardless of the objective reality of the path of the baseball, what makes a pitch into a ball or a strike is the umpire's call, no more, no less.
I think about the three umpires a lot. The C# language has a clear and mostly unambiguous definition of "the strike zone"; the specification should in theory allow us to classify any finite set of finite strings of text into either "a legal C# program" or "not a legal C# program". As a spec author, I take the third umpire's position: what the spec says is the definition of what is legal, end of story. But as an all-too fallible compiler writer, I take the first umpire's position: the compiler calls 'em as it sees 'em. Sometimes an illegal program accidentally (or deliberately; we implement a small number of extensions to the formal C# language) makes it through the compiler. And sometimes a legal program is incorrectly flagged as an error, or cannot be successfully compiled because it causes the compiler to run out of stack space or some other resource. (Also, though I believe that the compiler always in theory terminates, there are ways to build short C# programs that take exponentially long to analyze, making the compiler a sometimes impractical tool for deciding correctness.) But we calls 'em as we sees 'em, and if we get it wrong, then that's a bug.
But as a practical matter for our customers, the compiler is more like the third umpire: the arbiter of correctness, with no appeal. And of course I haven't even begun to consider the runtime aspects of correctness! Not only should the compiler decide what programs are legal, it should also generate correct code for every legal program. And again, the code generator plus the CLR's verifier and jitter act like our third umpire; the de facto arbiter of what "the right behaviour" actually is.
|
-
I've returned from a brief vacation, visiting friends on the island of Maui. I'd never been to that part of the world before. Turns out, it's a small island in the middle of the Pacific Ocean, entirely made out of volcanoes. Weird! But delightful.
The most impressive thing about the Hawaiian Islands for me was just how obvious were -- even to my completely untrained eyes -- the geomechanical and fluvial processes which shaped the landscape. The mountains and craters and river valleys and red sand beaches and easily-fractured rocks were very different from the (also somewhat volcanic) much older mountainous landscape I've lived in for the past decade.
Also quite amusing to me was learning to read and pronounce Hawaiian place names. It is all very logical once you know the system; before long I could easily pronounce signs like WAINAPANAPA STATE PARK -- wa-ee-napa-napa -- or PUUNENE AVENUE -- pu-oo-nay-nay -- or MAILIBEHANAMONOTANA STREET -- "Miley-Stewart-is-really-Hannah-Montana".
Many thanks to K and R and D for putting me and Leah up for a week; if you're going to Hawai'i and can stay with locals, I highly recommend it, particularly if they are awesome people. Everyone in Maui was awesome, with the exception of the rangers at (stunningly beautiful, even for Maui) Wa'inapanapa State Park, who are apparently consistently grumpy. As one Hawaiian, himself in the camping sector of the economy put it to me, "They do not have the big aloha".
The most amusing encounter was on the Hana Highway. There are numerous little stops along the way, where someone has erected a hut or parked a trailer and is selling coconuts, smoothies, banana bread, and so on. Hand-lettered signs, stunning natural beauty, middle of nowhere, you get the picture I'm sure. At one of the larger such stops there was a young fellow, probably in his late twenties, serving a variety of fried foods. It was mostly traditional American-style Chinese food, but also he had french fries, fish'n'chips, and so on. He was clearly not a native speaker of English, but spoke understandably with a strong accent. We were waiting behind a middled-aged woman with a typically midwestern American accent. Their conversation went something like this:
Her: I'm not very hungry, can I just get the fish without the chips?
Him, not quite following her: Half order?
Her, louder: How much without the fries?
This went back and forth for some time, both sides becoming increasingly frustrated by the communication breakdown, until:
Her, even louder: Can I speak to your manager?
Leah and K and I silently boggled -- there is no other word for it -- at each other for a moment. Where on earth did she imagine that a manager was going to emerge from? There was a counter, behind that, a trailer with a wok in it, behind that, jungle, and behind that, huge jagged lava rocks followed immediately by the Pacific Ocean. And what sort of management structure does she think one really needs to manage a single guy selling pineapple fried rice at the side of a highway? My conclusion: people have strange beliefs. Sometimes their beliefs cause them to leave in a huff with neither fish nor chips, even when fish and chips are both plentiful and reasonably priced. Hopefully she had better luck in Hana.
Anyway, enough travelogue. Regarding the puzzle from last time: the code is correct, and compiles without issue. I was quite surprised when I first learned that; it certainly looks like it violates our rule about not using the same simple name to mean two different things in one block.
The key is to understanding why this is legal is that the query comprehensions and foreach loops are specified as syntactic sugars for another program, and it is that program which is actually analyzed for correctness. Our original program:
static void Main()
{
int[] data = { 1, 2, 3, 1, 2, 1 };
foreach (var m in from m in data orderby m select m)
System.Console.Write(m);
}
is transformed into
static void Main()
{
int[] data = { 1, 2, 3, 1, 2, 1 };
{
IEnumerator<int> e = ((IEnumerable<int>)(data.OrderBy(m=>m)).GetEnumerator();
try
{
int m;
while(e.MoveNext())
{
m = (int)(int)e.Current;
Console.Write(m);
}
}
finally
{
if (e != null) ((IDisposable)e).Dispose();
}
}
}
There are five usages of m in this transformed program; it is:
1) declared as the formal parameter of a lambda.
2) used in the body of the lambda; here it refers to the formal parameter.
3) declared as a local variable
4) written to in the loop; here it refers to the local variable
5) read from in the loop; here it refers to the local variable
Is there any usage of a local variable before its declaration? No.
Are there any two declarations that have the same name in the same declaration space? It would appear so. The body of Main defines a local variable declaration space, and clearly the body of Main contains, indirectly, two declarations for m, one as a formal lambda parameter and one as a local. But I said last time that local variable declaration spaces have special rules for determining overlaps. It is illegal for a local variable declaration space to directly contain a declaration such that another nested local variable declaration space contains a declaration of the same name. But an outer declaration space which indirectly contains two such declarations is not an error. So in this case, no, there are no local variable declarations spaces which directly contain a declaration for m, such that a nested local variable declaration space also directly contains a declaration for m. Our two local variable declarations spaces which directly contain a declaration for m do not overlap anywhere.
Is there any declaration space which contains two inconsistent usages of the simple name m? Yes, again, the outer block of Main contains two inconsistent usages of m. But again, this is not relevant. The question is whether any declaration space directly containing m has an inconsistent usage. Again, we have two declaration spaces but they do not overlap each other, so there's no problem here either.
The thing which makes this legal, interestingly enough, is the generation of the loop variable declaration logically within the try block. Were it to be generated outside the try block then this would be a violation of the rule about inconsistent usage of a simple name throughout a declaration space.
|
-
C# has many rules that are designed to prevent some common sources of bugs and encourage good programming practices. So many, in fact, that it is often quite confusing to sort out exactly which rule has been violated. I thought I might spend some time talking about what the different rules are. We'll finish up with a puzzle.
To begin with, it will be vital to understand the difference between scope and declaration space. To refresh your memory of my earlier article: the scope of an entity is the region of text in which that entity may be referred to by its unqualified name. A declaration space is a region of text in which no two things may have the same name (with an exception for methods which differ by signature.) A "local variable declaration space" is a particular kind of declaration space used for declaring local variables; local variable declaration spaces have special rules for determining when they overlap.
The next thing that you have to understand to make any sense o this is what a "simple name" is. A simple name is always either just a plain identifier, like "x", or, in some cases, a plain identifier followed by a type argument list, like "Frob<int, string>".
Lots of things are treated as "simple names" by the compiler: local variable declarations, lambda parameters, and so on, always have the first form of simple name in their declarations. When you say "Console.WriteLine(x);" the "Console" and "x" are simple names but the "WriteLine" is not. Confusingly, there are some textual entities which have the form of simple names, but are not treated as simple names by the compiler. We might talk about some of those situations in later fabulous adventures.
So, without further ado, here are some relevant rules which are frequently confused. It's rules 3 and 4 that people find particularly confusing.
1) It is illegal to refer to a local variable before its declaration. (This seems reasonable I hope.)
2) It is illegal to have two local variables of the same name in the same local variable declaration space or nested local variable declaration spaces.
3) Local variables are in scope throughout the entire block in which the declaration occurs. This is in contrast with C++, in which local variables are in scope in their block only at points after the declaration.
4) For every occurrence of a simple name, whether in a declaration or as part of an expression, all uses of that simple name within the immediately enclosing local variable declaration space must refer to the same entity.
The purpose of all of these rules is to prevent the class of bugs in which the reader/maintainer of the code is tricked into believing they are referring to one entity with a simple name, but are in fact accidentally referring to another entity entirely. These rules are in particular designed to prevent nasty surprises when performing what ought to be safe refactorings.
Consider a world in which we did not have rules 3 and 4. In that world, this code would be legal:
class C
{
int x;
void M()
{
// 100 lines of code
x = 20; // means "this.x";
Console.WriteLine(x); // means "this.x"
// 100 lines of code
int x = 10;
Console.WriteLine(x); // means "local x"
}
}
This is hard on the person reading the code, who has a reasonable expectation that the two "Console.WriteLine(x)" lines do in fact both print out the contents of the same variable. But it is particularly nasty for the maintenance programmer who wishes to impose a reasonable coding standard upon this body of code. "Local variables are declared at the top of the block where they're used" is a reasonable coding standard in a lot of shops. But changing the code to:
class C
{
int x;
void M()
{
int x;
// 100 lines of code
x = 20; // no longer means "this.x";
Console.WriteLine(x); // no longer means "this.x"
// 100 lines of code
x = 10;
Console.WriteLine(x); // means "local x"
}
}
changes the meaning of the code! We wish to discourage authoring of multi-hundred-line methods, but making it harder and more error-prone to refactor them into something cleaner is not a good way to achieve that goal.
Notice that the original version of this program, rule 3 means that the program violates rule 1 -- the first usage of "x" is treated as a reference to the local before it is declared. The fact that it violates rule 1 because of rule 3 is precisely what prevents it from being a violation of rule 4! The meaning of "x" is consistent throughout the block; it always means the local, and therefore is sometimes used before it is declared. If we scrapped rule 3 then this would be a violation of rule 4, because we would then have two inconsistent meanings for the simple name "x" within one block.
Now, these rules do not mean that you can refactor willy-nilly. We can still construct situations in which similar refactorings fail. For example:
class C
{
int x;
void M()
{
{
// 100 lines of code
x = 20; // means "this.x";
Console.WriteLine(x); // means "this.x"
}
{
// 100 lines of code
int x = 10;
Console.WriteLine(x); // means "local x"
}
}
}
This is perfectly legal. We have the same simple name being used two different ways in two different blocks, but the immediately enclosing block of each usage does not overlap that of any other usage. The local variable is in scope throughout its immediately enclosing block, but that block does not overlap the block above. In this case, it is safe to refactor the declaration of the local to the top of its block, but not safe to refactor the declaration to the top of the outermost block; that would change the meaning of "x" in the first block. Moving a declaration up is almost always a safe thing to do; moving it out is not necessarily safe.
Now that you know all that, here's a puzzle for you, a puzzle that I got completely wrong the first time I saw it:
using System.Linq;
class Program
{
static void Main()
{
int[] data = { 1, 2, 3, 1, 2, 1 };
foreach (var m in from m in data orderby m select m)
System.Console.Write(m);
}
}
It certainly looks like name "m" is being used multiple times to mean different things. Is this program legal? If yes, why do the rules for not re-using simple names not apply? If no, precisely what rule has been violated?
[Eric is on vacation; this posting was pre-recorded.]
|
-
Here's a statement I read the other day about making comparisons between objects of reference type in C#:
Object.ReferenceEquals(x,y) returns true if and only if x and y refer to the same object.
True or false?
My wife Leah recently acquired a Honda Fit, thanks to the imminant failure of the automatic transmission solenoids in her aged Honda Civic. The back seats in the Fit fold down flat. You can fit a llama or a whole pile of hula hoops or whatever into that thing. It's quite handy. Not what I would call a powerful engine by any means, but for quick trips around town, it certainly gets the job done.
Since we were married when she bought the car, and we continue to be married, what's mine is hers and what's hers is mine. So if x = Eric's Honda Fit, and y = Leah's Honda Fit, then x and y are "reference equals". Those two things refer to the same object, viz, the shiny black object full of llamas and hula hoops in my driveway.
Now, we could have bought a different car. Say, a Ford Focus. But we did not. We own a total of zero Ford Foci. Suppose I said that x = Eric's Ford Focus, and y = Leah's Ford Focus. What's the sensible way to characterize the nature of x and y? Do we say that x and y refer to the same Ford Focus, namely that they refer to the Ford Focus that does not exist? The mind boggles at the repugnant and paradoxical implication that there exists a Ford Focus that is the Ford Focus that does not exist! (*) Rather, the right way to characterize this is to say that neither x nor y refer to any object. They're "null references" -- references that do not have any referent, but rather, capture the notion of "a lack of referent".
And that's why it's incorrect to say that Object.ReferenceEquals(x,y) returns true if and only if x and y refer to the same object.If x and y both do not refer to any object, then clearly they do not refer to the same object, because neither refers to an object in the first place. The correct way to characterize the behaviour of reference equality is
Object.ReferenceEquals(x,y) returns true if and only if either x and y refer to the same object, or x and y are both null references.
***********
(*) And yet I am a fan of the "null object pattern". Life is just full of these little contradictions.
[Eric is on vacation this week; this posting is pre-recorded.]
|
-
Somehow it has happened again; people just keep on recording videos of me and putting them on the internet.
In these videos you find out what I look like when lit from above and behind. Kinda spooky. We should have made the room entirely dark and held a flashlight underneath my face. That would be, like, ten times scarier. Anyway, if you're interested in me blathering on about my favourite feature in C# 4, covariance and contravariance of interface and delegate types, here are two little demo videos: Part One, Part Two. (There seems to be some minor sound sync issues here and there, but it's not really a problem; most of the audio is voice-over.)
Charlie has been crazy busy getting these little videos together; here are some more of his recent efforts, including some good ones from my colleagues Chris and Sam talking about all the other far more awesome features of C# 4.0: dynamic interop, improved interop with Office, named and optional parameters, and so on. Links to all of our recent videos are here: http://blogs.msdn.com/charlie/archive/2009/10/19/community-convergence-lvi.aspx.
Have an amusing and safe Hallowe'en -- I'll be going to Hallowe'en parties on a small island this year, just for a change of pace.
[Eric is on vacation this week; this posting was pre-recorded]
|
-
In my last post I described the syllogism "Photogenic people look good in photograps; Michelle Pfeiffer is photogenic; therefore, Michelle Pfeiffer looks good in photographs" as "begging the question". A few people commented on that, so I thought I'd address this point of English usage.
In modern usage, "begging the question" has come to mean nothing more than "the situation suggests that an obvious question to raise at this time is blah blah blah." For example, "The global financial meltdown begs the question: was there insufficient federal oversight of the American mortgage industry?" Though this usage is certainly common in civic discourse and the media, it is entirely a modern departure from the historic usage of the phrase. I try to eschew this modern usage when I say "begs the question".
"Begs the question" is also sometimes used to mean "this argument raises additional questions which require additional investigation before we can accept the argument". Though this is considerably closer to the traditional definition of the phrase, this is also not exactly what I mean.
When I say "begs the question", I mean it in the traditional sense of "this argument is fallacious because it takes as a premise an assumption which is at least as strong as the thing being proven, and is therefore an unwarranted assumption."
Let me give you another example of question begging, in the traditional sense, which might be more clear.
Suppose I asked "why are diamonds very hard but butter is very soft?" and you answered "diamond and butter are both made out of atoms; the atoms of diamonds are hard and the atoms of butter are soft." You would have begged the question; your answer to my question "why are some things hard and some things soft" is "because some things are made out of stuff that is hard and some things are made out of stuff that is soft" -- that is, you've avoided answering the question by providing an "explanation" that itself cannot be understood without answering the original question -- namely, why is some stuff hard and some stuff soft? This pseudo-explanation has no predictive power; it doesn't tell us anything new, it just circles back on itself. The explanatory assumption -- that some atoms are hard and some atoms are soft -- is stronger than the thing we are trying to investigate -- the hardness and softness of two substances.
A non-question-begging answer would be "diamond and butter are both made of atoms; the atoms of a diamond are all identical and arranged in a stable, rigid lattice where every point in the lattice is reinforced by a strong bond to four other points. The atoms of butter are a disorganized collection of many different atoms grouped into different kinds of relatively complex molecules; though the molecules themselves are quite strong, each molecule of butter holds weakly to each other molecule. It takes only a small force to disrupt the loose arrangement of butter molecules but a very large force to disrupt the strong arrangement of diamond atoms. We perceive this difference in required force as 'hardness' on the human scale, but in fact it is a property that arises from the sub-microscopic-scale properties of each substance."
Now, this explanation does *raise* more questions. It raises questions like "why are some lattices strong and some weak?" and "why are some objects composed of many different kinds of atoms organized into molecules, and some composed of just one atom?" Question-begging is not the act of raising more questions. Every good explanation raises more questions. What makes this explanation a good one is that it is testable and has predictive power; we can investigate the hardness or softness of other substances, and make predictions about what sorts of atomic structures they will have -- or, vice versa, we can look at an atomic structure and try to figure out from it how hard the substance will be. We can invent other techniques for determining atomic structure, like x-ray diffraction crystallography or spectroscopic analysis, and use those to cross-check our "atomic theory of hardness".
But the "because she's photogenic" pseudo-explanation is clearly question-begging. Why does she look so good? Because she's photogenic. Why is she photogenic? Because she looks so good. We have learned nothing about photogenicity (or the lovely Ms. Pfeiffer).
Similarly, if you ask "why is this code thread-safe?" and the answer is "because it can be correctly called on multiple threads", we've begged the question. Why is it thread-safe? Because it's correct. Why is it correct? Because it's thread-safe. Again, we have learned nothing about the nature of thread safety.
|
-
Caveat: I am not an expert on multi-threading programming. In fact, I wouldn't even say that I am competent at it. My whole career, I've needed to write code to spin up a secondary worker thread probably less than half a dozen times. So take everything I say on the subject with some skepticism.
A question I'm frequently asked: "is this code thread safe?" To answer the question, clearly we need to know what "thread safe" means.
But before we get into that, there's something I want to clear up first. A question I am less frequently asked is "Eric, why does Michelle Pfeiffer always look so good in photographs?" To help answer this pressing question, I consulted Wikipedia:
"A photogenic subject is a subject that usually appears physically attractive or striking in photographs."
Why does Michelle Pfeiffer always look so good in photographs? Because she's photogenic. Obviously.
Well, I'm glad we've cleared up that mystery, but I seem to have wandered somehwat from the subject at hand. Wikipedia is just as helpful in defining thread safety:
"A piece of code is thread-safe if it functions correctly during simultaneous execution by multiple threads."
As with photogenicity, this is obvious question-begging. When we ask "is this code thread safe?" all we are really asking is "is this code correct when called in a particular manner?" So how do we determine if the code is correct? We haven't actually explained anything here.
Wikipedia goes on:
"In particular, it must satisfy the need for multiple threads to access the same shared data, ..."
This seems fair; this scenario is almost always what people mean when they talk about thread safety. But then:
"...and the need for a shared piece of data to be accessed by only one thread at any given time."
Now we're talking about techniques for creating thread safety, not defining what thread safety means. Locking data so that it can only be accessed by one thread at a time is just one possible technique for creating thread safety; it is not itself the definition of thread safety.
My point is not that the definition is wrong; as informal definitions of thread safety go, this one is not terrible. Rather, my point is that the definition indicates that the concept itself is completely vague and essentially means nothing more than "behaves correctly in some situations". Therefore, when I'm asked "is this code thread safe?" I always have to push back and ask "what are the exact threading scenarios you are concerned about?" and "exactly what is correct behaviour of the object in every one of those scenarios?"
Communication problems arise when people with different answers to those questions try to communicate about thread safety. For example, suppose I told you that I have a "threadsafe mutable queue" that you can use in your program. You then cheerfully write the following code that runs on one thread while another thread is busy adding and removing items from the mutable queue:
if (!queue.IsEmpty) Console.WriteLine(queue.Peek());
Your code then crashes when the Peek throws a QueueEmptyException. What is going on here? I said this thing was thread safe, and yet your code is crashing in a multi-threaded scenario.
When I said "the queue is threadsafe" I meant that the queue maintains its internal state consistently no matter what sequence of individual operations are happening on other threads. But I did not mean that you can use my queue in any scenario that requires logical consistency maintained across multiple operations in a sequence. In short, my opinion of "correct behaviour" and your opinion of the same differed because what we thought of as the relevant scenario was completely different. I care only about not crashing, but you care about being able to reason logically about the information returned from each method call.
In this example, you and I are probably talking about different kinds of thread safety. Thread safety of mutable data structures is usually all about ensuring that the operations on the shared data always operate on the most up-to-date state of the shared data as it mutates, even if that means that a particular combination of operations appears to be logically inconsistent, as in our example above. Thread safety of immutable data structures is all about ensuring that use of the data across all operations is logically consistent, at the expense of the fact that you're looking at an immutable snapshot that might be out-of-date.
The problem here is that the choice about whether to access the first element or not is based on "stale" data. Designing a truly thread-safe mutable data structure in a world where nothing is allowed to be stale can be very difficult. Consider what you'd have to do in order to make the "Peek" operation above actually threadsafe. You'd need a new method:
if (!queue.Peek(out first)) Console.WriteLine(first);
Is this "thread safe"? It certainly seems better. But what if after the Peek, a different thread dequeues the queue? Now you're not crashing, but you've changed the behaviour of the previous program considerably. In the previous program, if, after the test there was a dequeue on another thread that changed what the first element was, then you'd either crash or print out the up-to-date first element in the queue. Now you're printing out a stale first element. Is that correct? Not if we always want to operate on up-to-date data!
But wait a moment -- actually, the previous version of the code had this problem as well. What if the dequeue on the other thread happened after the call to Peek succeeded but before the Console.WriteLine call executed? Again, you could be printing out stale data.
What if you want to ensure that you are always printing out up-to-date data? What you really need to make this threadsafe is:
queue.DoSomethingToHead(first=>{Console.WriteLine(first);});
Now the queue author and the queue user agree on what the relevant scenarios are, so this is truly threadsafe. Right?
Except... there could be something super-complicated in that delegate. What if whatever is in the delegate happens to cause an event that triggers code to run on another thread, which in turn causes some queue operation to run, which in turn blocks in such a manner that we've produced a deadlock? Is a deadlock "correct behaviour"? And if not, is this method truly "safe"?
Yuck.
By now you take my point I'm sure. As I pointed out earlier, it is unhelpful to say that a building or a hunk of code is "secure" without somehow communicating which threats the utilized security mechanism are and are not proof against. Similarly, it is unhelpful to say that code is "thread safe" without somehow communicating what undesirable behaviors the utilized thread safety mechanisms do and do not prevent. "Thread safety" is nothing more nor less than a code contract, like any other code contract. You agree to talk to an object in a particular manner, and it agrees to give you correct results if you do so; working out exactly what that manner is, and what the correct responses are, is a potentially tough problem.
************
(*) Yes, I'm aware that if I think something on Wikipedia is wrong, I can change it. There are two reasons why I should not do so. First, as I've already stated I'm not an expert in this area; I leave it to the experts to sort out amongst themselves what the right thing to say here is. And second, my point is not that the Wikipedia page is wrong, but rather that it illustrates that the term itself is vague by nature.
|
-
User: Recently I found out about a peculiar behaviour concerning division by zero in floating point numbers in C#. It does not throw an exception, as with integer division, but rather returns an "infinity". Why is that?
Eric: As I've often said, "why" questions are difficult for me to answer. My first attempt at an answer to a "why" question is usually "because that's what the specification says to do"; this time is no different. The C# specification says to do that in section 4.1.6. But we're only doing that because that's what the IEEE standard for floating point arithmetic says to do. We wish to be compliant with the established industry standard. See IEEE standard 754-1985 for details. Most floating point arithmetic is done in hardware these days, and most hardware is compliant with this specification.
User: It seems to me that division by zero is a bug no matter how you look at it!
Eric: Well, since clearly that is not how the members of the IEEE standardization committee looked at it in 1985, your statement that it must be a bug "no matter how you look at it" must be incorrect. Some industry experts do not look at it that way.
User: Good point. What motivated this design decision?
Eric: I wasn't there; I was busy playing Jumpman on my Commodore 64 at the time. But my educated guess is that it is desirable for all possible operations on all floats to produce a well-defined float result. Mathematicians would call this a "closure" property; that is, the set of floating point numbers is "closed" over all operations.
Positive infinity seems like a reasonable choice for dividing a positive number by zero. It seems plausible because of course the limit of 1 / x as x goes to zero (from above) is "positive infinity", so why shouldn't 1/0 be the number "positive infinity"?
Now, speaking as a mathematician, I find that argument specious. A thing and its limit need not have any particular property in common; it is fallacious to reason that just because, say, a sequence has a particular limit that a fact about the limit is also a fact about the sequence. Mathematically, "positive infinity" (in the sense of a limit of a real-valued function; let's leave transfinite ordinals, hyperbolic geometry, and all of that other stuff out of this discussion) is not a number at all and should not be treated as one; rather, it's a terse way of saying "the limit does not exist because the sequence diverges upwards".
When we divide by zero, essentially what we are saying is "solve the equation x * 0 = 1"; the solution to that equation is not "positive infinity", it is "I cannot because there is no solution to that equation". It's just the same as asking to solve the equation "x + 1 = x" -- saying "x is positive infinity" is not a solution; there is no solution.
But speaking as a practical engineer who uses floating point numbers to do an imprecise approximation of ideal arithmetic, this seems like a perfectly reasonable choice.
User: But surely it is impossible for the hardware to represent "infinity".
Eric: It certainly is possible. You've got 32 bits in a single-precision float; that's over four billion possible floats. All bit patterns of the form
?11111111???????????????????????
are reserved for "not-a-number" values. That's over sixteen million possible NaN combinations. Two of those sixteen million NaN bit patterns are reserved to mean positive and negative infinity. Positive infinity is the bit pattern 01111111100000000000000000000000 and negative infinity is 11111111100000000000000000000000.
User: Do all languages and applications use this convention of division-by-zero-becomes-infinity?
Eric: No. For example, C# and JScript do but VBScript does not. VBScript gives an error if you do that.
User: Then how do language implementors get the desired behaviour for each language if these semantics are implemented by the hardware?
Eric: There are two basic techniques. First, many chips which implement this standard allow the programmer to make float division by zero an exception rather than an infinity. On the 80x87 chip, for example, you can use bit two of the precision control register to determine whether division by zero returns an infinity or throws a hardware exception.
Second, if you don't want it to be a hardware exception but do want it to be a software exception, then you can check bit two of the status register after each division; it records whether there was a recent divide-by-zero event.
The latter strategy is used by VBScript; after we perform a division operation we check to see whether the status register recorded a divide-by-zero operation; if it did, then the VBScript runtime creates a divide-by-zero error and the usual VBScript error management process takes over, same as any other error.
Similar bits exist for other operations that seem like they might be better treated as exceptions, like numeric overflow.
The existence of the "hardware exception" bits creates problems for the modern language implementor, because we are now often in a world where code written in multiple languages from multiple vendors is running in the same process. Control bits on hardware are the ultimate "global state", and we all know how irksome it is to have global, public state that random code can stomp on.
For example: I might be misremembering some details, but I seem to recall that Delphi-authored controls set the "overflows cause exceptions" bit. That is, the Delphi implementors did not use the VBScript strategy of "try it, allow it to succeed, and check to see whether the overflow bit was set in the status register". Rather, they used the "make the hardware throw an exception and then catch the exception" strategy. This is deeply unfortunate. When a VBScript script calls a Delphi-authored control, the control flips the bit to force exceptions but it never "unflips" it. If, later on in the script, the VBScript program does an overflow, then we get an unhandled hardware exception because the bit is still set, even though the Delphi control might be long gone! I fixed that by saving away the state of the control register before calling into a component and restoring it when control returns. That's not ideal, but there's not much else we can do.
User: Very enlightening! I will be sure to pass this information along to my coworkers. I would be delighted to see a blog post on this.
Eric: And here you go!
|
-
Today, two more subtly incorrect myths about C#.
As you probably know, C# requires all local variables to be explicitly assigned before they are read, but assumes that all class instance field variables are initially assigned to default values. An explanation of why that is that I sometimes hear is "the compiler can easily prove that a local variable is not assigned, but it is much harder to prove that an instance field is not assigned. And since the class's default constructor automatically assigns all instance fields to default values, you don't need to do the analysis for fields."
Both statements are subtly incorrect.
The first statement is incorrect because the compiler in fact cannot and does not prove that a local variable is not assigned. Proving that is (1) impossible, and (2) does not give us any useful information we can act upon. It's impossible because proving that a given variable is assigned a value is equivalent to solving the Halting Problem:
int x;
if (/*condition requiring solution of the halting problem here*/) x = 10;
print(x);
If what we wanted to do was prove that x was unassigned then we would have to at compile time prove that the condition was false. Our compiler is not that sophisticated!
But the deeper point here is that we're not interested in proving for certain that x is unassigned. We're interested in proving for certain that x is assigned! If we can prove that for certain, then x is "definitely assigned". If we cannot prove that for certain then x is "not definitely assigned". We're only interested in "definitely unassigned" insofar as "definitely unassigned" is a stronger version of "not definitely assigned". If x is read from when it is "not definitely assigned", that's a bug.
That is, we're attempting to prove that x is assigned, and our failure to prove that at every point where it is read is what motivates the error. That failure could be because of a bona fide bug in your program, or it could be because our flow analyzer is extremely conservative. For example:
int x, y = 0;
if (0 * y == 0) x = 10;
print(x);
You and I know that x is definitely assigned, but in C# 3 the compiler is deliberately not smart enough to prove that. (Interestingly enough, it was smart enough in C# 2. I broke that to bring the compiler into line with the spec; being smarter but in violation of the spec is not necessarily a good thing.)
This example again shows that we do not prove that x is unassigned; if we did prove that, then clearly our prover would contain an error, since you and I both know that x is definitely assigned. Rather, we fail to prove that x is assigned.
This is an interesting twist on the believers vs skeptics argument that goes like this: the skeptic says "there's no reliable evidence that bigfoot exists, therefore, bigfoot does not exist". The believer says "absence of reliable evidence is not itself evidence of absence; and yes, bigfoot does exist". In both cases, reasoning from a position of lacking reliable evidence is seldom good reasoning! But in our case, it is precisely because we lack reliable evidence that we are coming to the conclusion that we do not know enough to allow you to read from x.
(The relevant principle for tentatively concluding that bigfoot is mythical based on a lack of reliable evidence is "extraordinary claims require extraordinary evidence". It is reasonable to assume that an extraordinary claim is false until reliable evidence is produced. When overwhelmingly reliable evidence is produced of an extraordinary claim -- say, the extraordinary claim that time itself slows down when you move faster -- then it makes sense to believe the extraordinary claim. Overwhelming evidence has been provided for the theory of relativity, but not for the theory of bigfoot.)
The second myth is that the default constructor of a class initializes the fields to their default values. This can be shown to be false by several arguments.
First, a class need not have a default constructor, and yet its fields are always observed to be initially assigned. If there is no default constructor, then something else must be initializing the fields.
Second, even if a class does have a default constructor, there's no guarantee that it will be called. Some other constructor could be called.
Third, the field initializers of a class run before any constructor body runs, therefore it cannot be the constructor body that does the initialization; that would be wiping out the results of the field initializers.
Fourth, constructors can call other constructors; if each of those constructors was initializing the fields to zero, then that would be wasteful; we'd be unneccessarily re-initializing already-wiped-out fields.
What actually happens is that the CLI memory allocator guarantees that the memory allocated for a given class instance will be initialized to all zeros before the constructor is called. By the time the constructors run the object is already freshly zeroed out and ready to go.
|
-
Most people will tell you that the difference between "(Alpha) bravo" and "bravo as Alpha" is that the former throws an exception if the conversion fails, whereas the latter returns null. Though this is correct, and this is the most obvious difference, it's not the only difference. There are pitfalls to watch out for here.
First off, since the result of the "as" operator can be null, the resulting type has to be one that takes a null value: either a reference type or a nullable value type. You cannot do "as int", that doesn't make any sense. If the argument isn't an int, then what should the return value be? The type of the "as" expression is always the named type so it needs to be a type that can take a null.
Second, the cast operator, as I've discussed before, is a strange beast. It means two contradictory things: "check to see if this object really is of this type, throw if it is not" and "this object is not of the given type; find me an equivalent value that belongs to the given type". The latter meaning of the cast operator is not shared by the "as" operator. If you say
short s = (short)123;
int? i = s as int?;
then you're out of luck. The "as" operator will not make the representation-changing conversions from short to nullable int like the cast operator would. Similarly, if you have class Alpha and unrelated class Bravo, with a user-defined conversion from Bravo to Alpha, then "(Alpha) bravo" will run the user-defined conversion, but "bravo as Alpha" will not. The "as" operator only considers reference, boxing and unboxing conversions.
And finally, of course the use cases of the two operators are superficially similar, but semantically quite different. A cast communicates to the reader "I am certain that this conversion is legal and I am willing to take a runtime exception if I'm wrong". The "as" operator communicates "I don't know if this conversion is legal or not; we're going to give it a try and see how it goes".
|
-
I'm frequently asked "you guys added extension methods to C# 3, so why not add extension properties as well?"
Good question.
First, let me talk a bit about C# 3. Clearly the big feature in C# 3 was LINQ. In a sense we had only three features in C# 3:
- everything necessary for LINQ -- implicitly typed locals, anonymous types, lambda expressions, extension methods, object and collection initializers, query comprehensions, expression trees, improved method type inference
- partial methods
- automatically implemented properties
The latter two were tiny compared to the items in that first bucket. On the design side, the syntax and semantics of both features are straightforward. On the implementation side, we already had the mechanisms in place to remove a call site; partial methods and conditional methods are pretty much the same thing behind the scenes. Auto props were also straightforward to analyze and generate code for. The testing burden was not particularly large for these features either.
The C# team was "the long pole" for the 2008 release of Visual Studio and the .NET Framework. By that I mean that if you took the amount of time required to do the work each team signed up for, given our level of staffing, blah blah blah, and made a pole proportionally long for each team in Developer Division, the C# pole would have been the longest pole. Which means that every other team in devdiv had slack in their schedule, but if we slipped our schedule a day, the new VS/CLR would also ship a day late. If any other team slipped a day, well, as long as that didn't make their pole longer than ours, they were still OK. Within the team, dividing up the work amongs the various development team members, the "long pole" work was the lambda binding and method type inference work, which was mine. So in a sense, every day that I was late, we'd slip the whole product that many days. (No pressure!)
Fortunately we have an excellent team here, we all supported each other very well through that release, picked up each other's slack when necessary, and delivered a quality release in plenty of time. My point is simply that unless a feature was either necessary for LINQ, or small and orthogonal and easy to cut if necessary (like the other two), it got cut immediately. There was no way we were going to risk slipping the entire product for any feature that was both complex and unnecessary.
It was of course immediately obvious that the natural companion to extension methods is extension properties. It's less obvious, for some reason, that extension events, extension operators, extension constructors (also known as "the factory pattern"), and so on, are also natural companions. But we didn't even consider designing extension properties for C# 3; we knew that they were not necessary and would add risk to an already-risky schedule for no compelling gain.
So now we come to C# 4.
As I'm fond of pointing out, the answer to every question of the form "why doesn't product X have feature Y?" is the same. It's because in order for a product to have a feature, that feature must be:
- thought of in the first place
- desired
- designed
- specified
- implemented
- tested
- documented
- shipped to customers
You've got to hit every single one of those things, otherwise, no feature.
When we started working on C# 4, we made a list of every feature request we'd heard of. It had hundreds of features on it. As I described last year, we categorized that list into "gotta have / nice to have / bad idea" buckets. Extension properties were in the "gotta have" bucket. We then looked at our available budget -- which was not so much measured in dollars as in available designers, developers, testers, writers and management multiplied by available time -- and determined that we did not have the resources to do more than about half the things in the "gotta have" bucket. So we cut half that stuff. Extension properties made it past that cut.
We then designed the feature. We had many hours of debate about the proposed syntax on the declaration side, how to call an extension property getter or setter "directly" as a static method, and so on. We came up with a syntax we could agree was acceptable, designed the semantics, wrote up a draft specification, and started writing code and test plans.
By the time the code was in reasonable shape -- not yet seriously tested, but usable and compliant with the spec -- we'd been having meetings with the WPF team. WPF developers were the assumptive primary consumers of extension properties. WPF already has a mechanism that resembles extension properties; it would be nice to unify that mechanism with our mechanism. Unfortunately, after taking a deep look at their real-world scenarios, we came to the disappointing conclusion that we had designed the wrong thing; this was not actually a feature that would solve their problems.
This is, admittedly, in a sense a failure of the design process. In retrospect, we should have gotten input and feedback from the primary customer much earlier. Had we done that then they could have either influenced the design to make something that would work for them, or we could have cut the feature without taking on the expense of writing spec, code and test plans.
But when you're in an unfortunate situation, you've got to decide to stop throwing good money after bad. Rather than take on the additional costs -- testing, documentation, shipping, and then maintenance of the feature for the rest of the life of the language -- in exchange for a feature that did not serve the needs of our customers, we cut the feature. At that point we did not feel confident that we had enough cycles remaining to redesign the feature the right way.
Therefore, sadly, no extension properties in C# 4. Perhaps in a hypothetical future version of C#.
|
-
Another good question from StackOverflow. Why is there an implicit conversion from char to ushort, but only an explicit conversion from ushort to char? Why did the designers of the language believe that these asymmetrical rules were sensible rules to add to the language?
Well, first off, the obvious things which would prevent either conversion from being implicit do not apply. A char is implemented as an unsigned 16 bit integer that represents a character in a UTF-16 encoding, so it can be converted to or from a ushort without loss of precision, or, for that matter, without change of representation. The runtime simply goes from treating this bit pattern as a char to treating the same bit pattern as a ushort, or vice versa.
It is therefore possible to allow either implicit conversion. Now, just because something is possible does not mean it is a good idea. Clearly the designers of the language thought that implicitly converting char to ushort was a good idea, but implicitly converting ushort to char is not. (And since char to ushort is a good idea, it seems reasonable that char-to-anything-that-ushort-goes-to is also reasonable, hence, char to int is also good.)
Unlike you guys, I have the original notes from the language design team at my disposal. Digging through those, we discover some interesting facts.
The conversion from ushort to char is covered in the notes from April 14th, 1999, where the question of whether it should be legal to convert from byte to char arises. In the original pre-release version of C#, this was legal for a brief time. I've lightly edited the notes to make them clear without an understanding of 1999-era pre-release Microsoft code names. I've also added emphasis on important points:
[The language design committee] has chosen to provide an implicit conversion from bytes to chars, since the domain of one is completely contained by the other. Right now, however, [the runtime library authors] only provide Write methods which take chars and ints, which means that bytes print out as characters since that ends up being the best method. We can solve this either by providing more methods on the writer class or by removing the implicit conversion.
There is an argument for why the latter is the correct thing to do. After all, bytes really aren't characters. True, there may be a useful mapping from bytes to chars, but ultimately, 23 does not denote the same thing as the character with ASCII value 23, in the same way that the byte 23 denotes the same thing as the long 23. Asking [the library authors] to provide this additional method simply because of how a quirk in our type system works out seems rather weak.
The notes then conclude with the decision that byte-to-char should be an explicit conversion, and integer-literal-in-range-of-char should also be an explicit conversion.
Note that the language design notes do not call out why ushort-to-char was also made explicit at the same time, but you can see that the same logic applies. When passing a ushort to a method overloaded as M(int) and M(char), odds are good that you want to treat the ushort as a number, not as a character. And a ushort is not a character representation in the same way that a ushort is a numeric representation, so it seems reasonable to make that conversion explicit as well.
The decision to make char go to ushort implicitly was made on the 17th of September, 1999; the design notes from that day on this topic simply state "char to ushort is also a legal implicit conversion", and that's it. No further exposition of what was going on in the language designers' heads that day is evident in the notes.
However, we can make educated guesses as to why implicit char-to-ushort was considered a good idea. The key idea here is that the conversion from number to character is a "possibly dodgy" conversion. It's taking something that you do not know is intended to be a character, and choosing to treat it as one. That seems like the sort of thing you want to call out that you are doing explicitly, rather than accidentally allowing it. But the reverse is much less dodgy. There is a long tradition in C programming of treating characters as integers -- to obtain their underlying values, or to do mathematics on them.
In short: it seems reasonable that using a number as a character could be an accident and a bug, but it also seems reasonable that using a character as a number is deliberate and desirable. This asymmetry is therefore reflected in the rules of the language.
|
|
|
|
