Fabulous Adventures In Coding : Code Quality

Always write a spec, Part Two

Eric Lippert — Mon, 23 Nov 2009 15:01:00 GMT

Upon submitting that specification for review, even before seeing my code, Chris found a bug and an omission.

The omission is that I neglected to say what happens when the ref variable is an index into a fixed-size array buffer. As it turns out, that case is also rewritten as a pointer dereference by the time we get to this point in the code, so missing that case turned out to not be a big deal.

The bug is that this line is wrong:

if x is ref/out instance.field then add var temp=instance to L and ref/out temp.field to A2.

because it does not meet the stated goal of the method, namely, to produce the same results after the transformation:

struct S { public int q; }
static void M(ref int r) { r = r + 1; }
static int Zero() { Console.WriteLine("hello!"); return 0; }
...
S[] arr = new[] { new S() };
M(ref arr[Zero()].q);
Console.WriteLine(arr[0].q); // 1

Here we have a variable expression of the form instance.field which has a side effect; it prints "Hello". So according to the spec, we rewrite that as

S temp = arr[Zero()];
M(ref temp.q);
Console.WriteLine(arr[0].q); // 0 !

Once more, mutable value types are revealed to be pure concentrated evil. We just mutated temp.q, which is a copy of arr[0].

The interesting thing here is that Chris found the bug by reading the spec and thinking about whether I'd missed any interesting cases. Bugs are always cheaper to fix the earlier you find them; a bug you fix as a result of reading the spec is a bug that you don't have to pay QA to find, it's a bug that your customers never see, it's a bug that never causes a backwards-compatibility breaking change, and so on.

I thought hard about this and realized that the painful case only happens when we have ref/out instance.field with a side effect and instance is a variable of value type. That's the missing case. Furthermore, we can solve this by recursing!

if x is ref/out instance.field and instance is a variable of value type, then recurse. Compute T("ref instance"), as if we were invoking a method that takes a single parameter of this ref type. That will give us a resulting declaration list, which we merge onto the end of L, and a resulting argument list with one element on it, r. Add "ref r.field" to A2.
if x is a ref/out instance.field but instance is not a variable of value type, then generate var temp=instance on L, ref temp.field on A2.

So if we have, say, M(ref arr[index].q), that ultimately is generated as var t1 = arr, var t2 = index, M(ref t1[t2].q), which is correct.

I fixed up my implementation, wrote some test cases to test the spec, and sent it off to testing for further analysis.

There's still a bug in this spec, which our tester David found during his analysis of my spec plus proposed implementation. Hint: the problem has nothing to do with ref/out semantics; it's a more fundamental error in reasoning.

Feel free to pause and reflect for a moment if you want a chance to figure it out for yourself.

...

The error is in the default case:

First, if x has no side effects, then simply keep x in the appropriate place in A2.

Whether x has side effects or not is completely irrelevant; what is relevant is whether the computation of x depends on the order of every other side effect!

For example, our spec would have that

int k = 0;
M( k, arr[k=1] );

is the same as

int k = 0;
var t1 = arr;
var t2 = (k = 1);
M(k, t1[t2]);

which is clearly not correct; the first argument is supposed to be zero, not one. The fact that the first argument has no side effects is irrelevant; it depends on the second argument's side effect taking effect after the first argument's value is computed.

We have to fix up the spec (and implementation) to say that if the argument corresponds to a value parameter, you always save away the value in a temporary, regardless of whether it has side effects or not. If it corresponds to, say, a ref parameter, and the argument is a ref to a local then we can still simply generate the ref to the local normally; regardless of whether the contents of the local change, the managed address of the local is always the same, and computing the address of a local causes no side effects.

I note that we could also make an exception in the case that the value is, say, a compile-time constant. Clearly the computation of its value does not have any dependency on the ordering of other side effects. But for the sake of not further complexifying my spec and implementation, I glossed over this detail. The optimizer will probably take care of it.

Commenter Pavel Minaev found several more corner cases that none of us found. The big one is that we should be clear about what happens with multidimensional arrays, and in fact, I had a bug in my implementation for multidimensional arrays, so thanks Pavel! He also found some obscure ones. One is that accessing a static field does have a side effect; if this is the first access to the class then that causes the static class constructor to run, which could have observable side effects. Another is that the exact semantics of when bad arguments cause exceptions is slightly changed by this transformation.

At this point I now believe that I have a correct specification of my function T (modulo Pavel's weird issues, which I'm still mulling over. But those are bizarre corner cases that might require some language spec work to sort out.) Since the implementation is a straightforward transformation of the specification into C++, I also believe I have a correct implementation. I've written test cases that verify every case in the spec, and handed it off to testing to look for practical cases I missed.

Anyone see any other problems with this spec? Anyone?

And anyone have any guesses as to why I needed this function implemented?

Always write a spec, part one

Eric Lippert — Thu, 19 Nov 2009 14:45:00 GMT

Joel had a great series of articles many years ago about the benefits of writing functional specifications, that is, specifications of how the product looks to its users. I want to talk a bit about technical specifications, that is, a specification of how something actually works behind the scenes. A while back, I described how I wrote a seemingly-trivial piece of code by first writing the technical spec, and then writing the code to exactly implement the spec, and the test cases to test each statement of the spec. I use this technique frequently; it almost always saves me time and effort. If you don't write a spec first, it's very easy to get bogged down in a morass of bugs, and hard to know when you're actually done writing the code.

Here's an example of another seemingly-trivial function that smart people gave a good half-dozen completely wrong answers to because they didn't write a spec. Notice how once I state what the spec is, the code follows very naturally.

I thought I might go through a recent example of a slightly less trivial problem. I needed a function in the compiler which could take a valid list of arguments A1 to an applicable method M, and give me back two lists. The first result list is a list of local variable declarations, L, the second, another argument list, A2. Here's the kicker: evaluating each member of A2 must have no side effects. And, the combination of executing L followed by M(A2) must give exactly the same results as executing M(A1). (I also knew that the members of A1 had already had all the conversions generated to make them match the types of M's parameter list, and so on. Overload resolution was completely done at this point.)

Why I needed such a function is not particularly important; perhaps you can speculate as to why I needed it.

So I started writing a spec, starting with the description above: the function T takes a valid list... blah blah blah.

That's not enough of a spec to actually write code.

I realized that some expressions in A1 might have side effects, and some might not. If an expression had side effects, then I could declare a temp local, execute the side effects, assign the result to the local, and then use the local as the expression in the same place in A2.

Super. Now I can write more sentences in my spec.

For each argument x in A1, we potentially generate a temporary value as follows: First, if x has no side effects, then simply keep x in the appropriate place in A2. Otherwise, if x does have side effects, then generate a temporary variable declaration in L, var temp = x. Then put an expression that evaluates the temporary variable in the appropriate place in A2.

Then I thought "can I correctly implement this spec?"

I started thinking about the possible cases for the input, A1. The members of A1 could be passed by value, out, or ref; I already knew they were correct, since the particular method M was an applicable method of argument list A1. In this particular point in the compilation process, there was no need to worry about whether there were "params" arguments, whether there were missing arguments with default values, and so on; that had already been taken care of. I also noted that whether they were "ref" or "out" was irrelevant; ref and out are the same thing behind the scenes. As far as the compiler is concerned, the only difference between them is that the definite assignment rules are different for each. So we had two cases for each argument: it's being passed by value, or by ref/out.

If it's being passed by ref/out, then what we actually do is we pass the managed address of a variable to the method. In M(ref int x), the type of x is actually a special type int&, "managed reference to variable of type int", which is not a legal type in C#. That's why we hide it from you by requiring a goofy "ref" syntax any time you want to talk about a managed reference to a variable.

Unfortunately, our IL generation pass was written with the assumption that all local variables are never of these magic managed-reference-to-variable types. I was then faced with a choice. Either come up with a technique for working with this limitation, or rewrite the most complicated code in the IL gen to support this scenario. (The code which figures out how to deal with managed references is quite complex.) I decided to opt for the former strategy. Which meant more spec work, since our spec now doesn't handle these cases!

For each argument x in A1, we potentially generate a temporary value as follows:

First, if x has no side effects, then simply keep x in the appropriate place in A2.

Otherwise, if x does have side effects and corresponds to a value parameter, then generate ... etc.

Otherwise, if x does have side effects and corresponds to a ref/out parameter, then a miracle happens.

Clearly that last bit needed some work.

So I wrote down all the possible cases I could think of. Clearly x has to be a variable if its type is "reference to variable", so that makes for a small number of cases:

Otherwise, if x does have side effects and corresponds to a ref/out parameter, then the possible cases are:

* x is a local variable
* x is a value parameter
* x is an out parameter
* x is a ref parameter
* x is a field of an instance
* x is a static field
* x is an array element
* x is a pointer dereferenced with *
* x is a pointer dereferenced with [ ]

and for each, a miracle happens.

Again, clearly this is not good enough. I re-examined my list to see if I could eliminate any of these cases. Locals, parameters and static fields never have side effects, so we can eliminate them.

Also, at this point in our analysis, I knew that pointer dereferences of the form "pointer[index]" had already been rewritten into the form "*(pointer + index)", which meant that in practice, we would never actually hit that last case in this algorithm; the second-last case would take care of both.

Otherwise, if x does have side effects and corresponds to a ref/out parameter, then the possible cases are:

* x is a field of an instance
* x is an array element
* x is a pointer dereferenced with *

and for each, a miracle happens.

I then started to think of what side effects could happen for each. We could have "ref instance.field", "ref array[index]", or "ref *pointer", and "instance", "array", "index" and "pointer" can all be expressions that have a side effect. ("field" cannot, it merely names a field.) So now we can use the same specification as before:

Otherwise, if x does have side effects and corresponds to a ref/out parameter, then the possible cases are:

x is ref/out instance.field: in this case, add var temp=instance to L and ref/out temp.field to A2.
x is ref/out array[index]: in this case, add var t1 = array and var t2 = index to L and ref/out t1[t2] to A2.
x is ref/out *pointer: in this case, add var temp = pointer to L and ref/out *temp to A2.

And now we have something I can implement. So I sent this proposed spec around for review, while I started plugging away at the implementation.

This spec is wrong. Can you spot the bugs in my implementation that my coworkers found by reading my spec?

Next time, the thrilling conclusion.

Events and Races

Eric Lippert — Wed, 29 Apr 2009 15:00:00 GMT

Here’s a question similar to one I saw on stackoverflow the other day. Suppose you have an event:

public event Action Foo;

The standard pattern for firing this event is:

Action temp = Foo;
if (temp != null)
temp();

What the heck is up with that? Why not just call “Foo()” ?

First off, this pattern ensures that the thing that is invoked is not the null delegate reference, which would cause a null reference exception to be thrown. But if that’s what we want, then surely we could have skipped the temporary variable – just “if (Foo != null) Foo();” would do. Why the temp?

The temporary variable ensures that this is “thread safe”. If the event’s handlers are being modified on another thread it is possible for Foo to be non-null at the point of the check, then set to null on a different thread, and then the invocation throws.

Using the temporary variable effectively makes a copy of the current set of event handlers. Remember, multi-cast delegates are immutable; when you add or remove a handler, you replace the existing multi-cast delegate object with a different delegate object that has different behaviour. You do not modify an existing object, you modify the variable that stores the event handler. Therefore, stashing away the current reference stored in that variable into a temporary variable effectively makes a copy of the current state.

Is that clear? Make sure it is, because now things start to get really confusing.

A common criticism of this pattern is that it trades one race condition for another. Let’s consider that scenario again more carefully:

The event handler contains a single handler, H. Thread Alpha makes a copy of the delegate to H in temp and determines that it is not null. Thread Beta decides that H must no longer be called when the event is fired, so it sets the handler to null. Thread Beta then assumes that H will never be called, and destroys a bunch of state that H needs to execute successfully. Thread Alpha then regains control and executes H, which behaves crazily since its necessary state has been destroyed. Thread Beta’s attempt to ensure that H is not called has been defeated by the race condition.

A frequently-stated principle of good software design is that code which calls attention to bugs by throwing exceptions is better than code which hides bugs by muddling on through and doing the wrong thing. This code has a race condition that results in incorrect behaviour. Perhaps an application of that principle is to stop using a temporary and instead crash if it races. That is to say, this code has a failure mode; surely it is better to highlight that failure with a crisp exception than to turn it into crazy wrong behaviour.

That seems plausibly argued, I agree, but the conclusion is wrong.

Suppose we remove the temporary variable but keep the null check. Does that solve the problem? No! We still have the same race conditions. Suppose the event handler delegate contains a reference to H. Thread Alpha checks to see whether it is null; it is not. Thread Alpha pushes the object to invoke on the runtime stack. Between the push of the delegate value and the call to invoke it, thread Beta sets the event handler to null. And once again, H will be invoked after thread Beta has removed the handler.

Suppose we remove the temporary and the null check and just invoke the delegate directly. Does that help? No, we still have the same race condition. The contents of the event handling variable can still be changed between the push of the delegate object onto the stack and the invocation of the delegate.

Removing the code around the call site does not decrease the number of race conditions in the code, and it does increase the number of crashes. Worse, doing so makes the race condition harder to detect by shrinking the window in which the race can occur without eliminating it.

Removing the null check and/or the temporary is a bad idea; the already-bad situation is only made worse.

Essentially there are two problems here that are being conflated. The two problems are:

1) The event handler delegate can be null at any time.
2) A “stale” handler can be invoked even after it has been removed.

These two problems are actually orthogonal and have different solutions. The onus for solving the first problem is laid upon the code which does the invocation; it is required to ensure that it does not dereference null. The store-in-temporary-and-test pattern ensures that null is never dereferenced. (There are other ways to solve this problem; for example, initializing the handler to have an empty action that is never removed. But doing a null check is the standard pattern.)

The onus for solving the second problem is laid upon the code being invoked; event handlers are required to be robust in the face of being called even after the event has been unsubscribed. In the scenario I described, the bug is actually in H. It needs to be robust enough to check to see whether the state it needs is still there, and bail out cleanly if it is not. (Or, alternatively, some additional locking mechanism needs to be implemented which ensures that the code which fires the event cooperates with the code that changes the event handlers to ensure the desired behaviour.)

The point though is that the "null ref problem" and the "stale handler problem" are two separate problems that are easily confused because the symptoms of both arise at the exact same call site code. You've got to solve both problems if you want to do threadsafe events. How you do so is up to you; just don't confuse the solution of one problem for a solution of the other.

(Thanks to Microsofties Levi Broderick, Chris Burrows, Curt Hagenlocher and Wolf Logan; this article is based on a conversation about their analysis of this pattern.)

Good Names

Eric Lippert — Mon, 06 Apr 2009 19:30:00 GMT

Imagine a door with an unusual handle. The handle is five feet off the ground and rotates upwards to open the door. The door has no lock. Is this a good door design?

Sorry. That’s not an answerable question. The purpose of almost every door is to prevent something from going through it without preventing everything from going through it. Some doors exist only to mitigate heat loss while allowing anything else through. Some doors exist to keep everyone out except authorized personnel. The goodness or badness of a door design depends entirely on how well or poorly it performs the task of preventing undesired access and allowing desired access.

If I were describing a jail cell door or bank vault door, that would be an absurdly bad design. But for the teacher’s supply room in a kindergarden classroom, it’s pretty good. It doesn’t prevent access to any adults, but the kids are unlikely to be able to get through without adult supervision. Purpose matters.

Last time I talked a bit about what makes a book title better or worse. These ideas can be extended to naming just about anything, but it is important when considering the goodness or badness of a name to keep in mind the purpose of the name. Frequently when we name types and methods, it’s like my book example. The purpose of the code, like the purpose of the book, is to provide a benefit to the consumer. The purpose of the name is usually (*) to make it easy for the potential consumer to find the thing, and then quickly and accurately evaluate whether it is likely to provide the desired benefit.

With those purposes in mind, here are some guidelines that I use when trying to come up with a public name for something:

Everything in my Bad Names post.
The name should describe what the thing is (classes) or does (methods) or is used for (interfaces), as opposed to how it achieves any of those things.
Names should describe the unchanging aspects of the nature of the thing.

A small example of how I got this wrong recently: the style that I use for adding responses to comments in this blog is called “yellowbox”. If I ever decide to change it the look of the blog and want to make it blue, I’m going to look pretty silly having a style called “yellowbox” that draws a blue box. I should have called it “response”; it’s always going to be for responses.
The name should use the vocabulary of the consumer, not the jargon of the mechanisms used in the implementation.
The name should be as unique as possible, so that searching for the words in it rapidly narrows the field down.
The name should be as precise as possible.

How many “HandleSomething” methods have you seen? Usually these do something a lot more specific than “handling”. It might be important to the consumer to have a better idea of what you’re doing in there.
The name should not have any non-standard abbreviations; FindCustRec is unlikely to be found by searching for “customer” or “record”.

Those are just a few off the top of my head. What are some of the criteria you use to come up with good names for types and methods?

**************************

(*) There are unusual cases where names are deliberately chosen to work against those typical goals. Sometimes code is very special-purpose, designed to be used very rarely and then only by subject experts. In those cases, you do not want the code to be accidentally found by people who are unlikely to need it, and unlikely to use it correctly. In those cases, it’s desirable for the name to be laden with the jargon of the experts who will be using it. Doing so sends the message to potential users “if you do not know what these words mean, you probably should not be using this code”. Again, purpose matters.

Every Problem Looks Like A Nail

Eric Lippert — Mon, 30 Mar 2009 18:10:00 GMT

I wish all the questions I got were this straightforward:

“I need to compare two strings for non-culture-sensitive equality. I notice that there are methods String.Equals and String.Compare which can both do that. What is the guideline on which one I should use?”

I’ll answer your question, but first, a funny story (*).

My buddy Steve and I were fixing the flashing that is leaking around the seal between my roof and my chimney the other day. “Steve,” I said, “hand me that screwdriver.” Steve handed me his roofing hammer. “No, dude, the screwdriver. Thin circular shaft, handle at one end, Robertson bit on the other.”

“Dude, if you wanted the screw remover you should have said so.”

Ha ha ha ha ha, I crack myself up. That joke never gets old.

Anyway, I seem to have digressed slightly from the topic of today’s blog.

The documentation in MSDN clearly states that the purpose of String.Equals is for comparing strings for equality, and the purpose of String.Compare is for sorting a list of strings into a culture-sensitive alphabetical order.

You can use a hammer to drive screws, and in a world without screwdrivers, you might have to. But in a world with screwdrivers, why would you? You would use the tool that was designed for the task.

You can use String.Compare to test for equality if you want, but why would you? You have a method that is designed to do exactly and specifically what you want to do, so use it.

Unless there is a really compelling reason to do otherwise, always use the tool that was specifically designed for your problem, if there is one. Don’t use some other tool designed for a different task that just happens to work; that’s a recipe for bugs.

*******************

(*) Not a true story, except for the part about my roof leaking at the flashing. I need to fix that.

Santalic tailfans, part two

Eric Lippert — Fri, 06 Feb 2009 21:19:31 GMT

As I have said before many times, there is only one sensible way to make a performant application. (As an aside: perfectly good word, performant, deal with it!) That is:

Set meaningful, measurable, customer-focused goals.
Write the code to be as clear and correct as possible.
Carefully measure your performance against your goals.
Did you meet your goal? Great! Don't waste any time on performance analysis. Spend your valuable time on features, documentation, bug fixing, robustness, security, whatever.
If you did not meet your goal, use tools to discover what the worst-performing fixable thing is, and fix it.

Iterate on that process, updating your goals if it proves necessary, until you either have something that meets your goals or you give up.

My explicit goal for my little dictionary search program was that it give results fast enough that I would not be sitting there impatiently waiting for more than a few seconds. That's a very customer-focused goal, me being the only customer of this program. With only a very small amount of tweaking my program met that goal right away, so why would I spend any more time on it? The program takes just under two seconds to report the results for a typical query, which is faster than I can read.

But suppose that we did want to improve the performance of this program for some reason. How? Well, let's go down the list. We have goals, we have very clear code, we can measure the performance easily enough. Suppose we didn't meet the goal. The last thing on the list is "use tools to discover what the slowest fixable thing is".

A commenter conjectured that the performance bottleneck of this program was in the disk I/O. As you can see, every time we do a query we re-read the two megabyte dictionary file dozens of times. This has the benefit of using very little memory; we never have more than one line of the dictionary in memory at a time, instead of the whole four megs it would take to store the dictionary (remember, the dictionary is in ASCII on disk but two-byte Unicode if in strings in memory, so the in-memory size will double.)

That's a conjecture -- a reasonable conjecture -- but nevertheless, it's just a guess. If I've learned one thing about performance analysis it's that my guesses about where the bottleneck is are often wrong. I'll tell you right now that yes, the disk-hitting performance is bad, but it is not the worst thing in this program, not by far. Where's the real performance bottleneck? Any guesses? Could you know without using tools?

Here's the result of a timing profile run of a seven-letter queries with one blank, repeated 20 times:

43%: Contains
21%: ReadLine
14%: Sort
7%: ToUpperInvariant
15%: everything else

Holy goodness! The call to the Contains extension method in the query to test whether the dictionary word is in the rack set array is almost half the cost of this program!

Which makes sense, once you stop to think about it. The "Contains" method is by its highly general nature necessarily very naive. When given an array, 99.9% of the time it has to look through the entire 26-item array because 99.9% of the time, the word is not actually going to match any of the possible racks. It cannot take advantage of any "early outs" like you could do if you were doing a linear search on a sorted list. And each time through the array it has to do a full-on string comparison; there's no fancy-pants checks in there that take advantage of string immutability or hash codes or any such thing.

We have a data structure that is designed to rapidly tell you whether a member is contained in a set. And even better, it already does the "distinct" logic. When I replace

var racks = (from rack in ReplaceQuestionMarks(originalRack)
select Canonicalize(rack)).Distinct().ToArray();

with

    var racks = new HashSet<string>(
                from rack in ReplaceQuestionMarks(originalRack)
                select Canonicalize(rack));

suddenly performance improves massively. The "Contains" drops down to 3% of the total cost, and of course, the total cost is now half of what it was before.

Another subtle point here: notice how when I changed the type of the variable "racks" from "array of string" to "set of string", I didn't have to redundantly change the type thanks to implicit typing of local variables. I want to emphasize the semantics here, not the storage mechanism. If I felt that communicating the storage mechanism was an important part of this code -- because it has such a strong effect on performance -- perhaps I would choose to emphasize the storage by eschewing the "var".

With this change, the program performance improves to about one second per query and the profile now looks like this:

39%: ReadLine
23%: Sort
11%: ToUpperInvariant
7%: the iterator block goo in FileLines
5%: ToArray (called by Join)
15%: everything else

Now the bottleneck is clearly the combination of repeated file line reading (48%) and the string canonicalization of every dictionary line over and over again (37%).

With this information, we now have data with which to make sensible investments of time and effort. We could cache portions of the dictionary in memory to avoid the repeated disk cost. Is the potential increase in speed worth the potentially massive increase in memory usage? We could be smart about it and, say, only cache the seven- and eight-letter words in memory.

We could also attack the canonicalization performance problem. Should we perhaps precompute an index for the dictionary where every string is already in canonical form? This in effect trades increased disk space, increased program complexity and increased redundancy for decreased time. Should we use a different canonicalization algorithm entirely?

All of these decisions are driven by the fact that I have already exceeded my performance goal, so the answer is "no". Good enough is, by definition, good enough. If I were using this algorithm to actually build a game AI, it would not be good enough anymore and I'd go with some more clever solution. But I'm not.

Properties vs. Attributes

Eric Lippert — Tue, 03 Feb 2009 00:02:26 GMT

Here is yet another question I got from a C# user recently:

I have a class that represents a business rule. I want to add some rule metadata that could be used by consumers to retrieve a friendlier rule name, description, and anything else that makes sense. Should this information be exposed as an attribute or property on the class?

I would say to absolutely go for a property in this case, for four reasons.

First, properties are highly discoverable. Consumers of this class can use IntelliSense to see that there is a property on the class called "Description" much more easily than they can see that there is an attribute.

Second, properties are much easier to use than attributes. You don't want to muck around with the code to extract strings from metadata attributes unless you really have to.

Third, data such as names and descriptions is highly likely to be localized in the future. Making it a property means that the property getter code can read the string out of a resource, a resource which you can hand off to your localization experts when it comes time to ship the Japanese version.

And fourth, let's go back to basic object-oriented design principles. You are attempting to model something -- in this case, the class is modeling a "business rule".

Note that a business rule is not a class. Nor is a rule an interface. A rule is neither a property nor a method. A rule isn't any programming language construct. A rule is a rule; classes and structs and interfaces and whatnot are mechanisms that we use to implement model elements that represent the desired semantics in a manner that we as software developers find amenable to our tools. But let's be careful to not confuse the thing being modeled with the mechanisms we use to model it.

Properties and fields and interfaces and classes and whatnot are part of the model; each one of those things should represent something in the model world. If a property of a "rule" is its "description" then there should be something in the model that you're implementing which represents this. We have invented properties specifically to model the "an x has the property y" relationship, so use them.

That's not at all what attributes are for. Think about a typical usage of attributes:

[Obsolete]
[Serializable]
public class Giraffe : Animal
{ ...

Attributes typically do not have anything to do with the semantics of the thing being modeled. Attributes are facts about the mechanisms - the classes and fields and formal parameters and whatnot. Clearly this does not mean "a giraffe has an obsolete and a serializable." This also does not mean that giraffes are obsolete or serializable. That doesn't make any sense. This says that the class named Giraffe is obsolete and the class named Giraffe is serializable.

In short: use attributes to describe your mechanisms, use properties to model the domain.

Future-Proofing A Design

Eric Lippert — Fri, 16 Jan 2009 20:21:00 GMT

Last time on FAIC a user asked for guidance on the potential pitfalls of refactoring an automatic property into a regular explicit property. This is just an example of a far more general problem: how can we design programs so that they are easy to get right when things inevitably change in the future?

This is an incredibly difficult question to answer, one which whole books could be written on. Today, I'll just give three general points to think about.

First: Premature generality is expensive.

Designing code to be future-proof without a clear understanding of what the future holds often leads to excess generality; generality has very real costs, and they might be higher than the benefit accrued.

If you do a good job of solving the problem at hand with the tools at hand, if the code is clean and organized and has few moving parts, then it will be easier to generalize it in the future if you need to in order to solve a problem then.

Design your code to solve the well-understood problem. If you design it to solve an unknown future problem, odds are good that you're going to solve it poorly, making even more work for the future.

Second: Represent in your model only those things which are always in the problem domain and whose class relationships are unchanging.

The relationships between the classes TrafficLight, TrafficSign, Vehicle, Car, Truck, Pedestrian, Roadway, Intersection, SignalTiming and RightOfWayLaw are likely to be eternal and unchanging.

Things change. The semantics of a particular RightOfWayLaw might change subtly or grossly as the city council does its job. There might be new subclasses of Vehicle created in the future. A specific traffic light might have a faulty implementation that the rest of the system needs to handle gracefully. But each of these situations is about changing the implementation details of a class, never changing its relationships with other classes.

If the class relationships are future-proofed then it is a lot easier to edit the implementations of those classes without having to worry that the whole system will thereby get messed up.

The easiest way to keep the relationships straight is to base them as much as possible on concepts directly from the problem domain.

Third: Keep your policies away from your mechanisms.

The mechanism of a traffic light works the same way no matter what policies like light timings at rush hour are. If mechanisms (TrafficLight) are separated from policies (SignalTiming) then you can change the mechanism to a more efficient one without worrying that you’re going to inadvertently change policy, and can change policy without worrying that you’re going to break mechanism.

My earlier example of a bank balance was deliberately an example of what goes wrong when mechanism and policy become conflated. The “Balance” getter was originally a mechanism, but after the edit, it became a security policy enforcement tool.

You must then ask yourself, “where in this code do I care about enforcing the policy, and where do I care about executing the mechanism?” and make the appropriate edits. After the edit, everywhere that needs policy enforcement needs to use “Balance”, everywhere that needs to effect the action of the mechanism needs to use “balance”.

What are the odds that there is going to be a bug introduced, given that we now have a difference which is important and almost invisible? Pretty high! This difference should probably be made more visible by renaming the backing store to something that calls it out as semantically different from the property accessor.

Let's bring this back to the question at hand. What guidance do we propose for future-proofing turning automatic properties into regular properties? The guidance that I am proposing here is:

First, think about whether making the design general now to solve an unknown problem that you might not even have in the future is worthwhile.
Second, make sure that the design in general makes the relationships between classes unchanging, even if the specifics of each class change.
Third, consider whether that hypothetical future change to the property will be splitting apart its role as a mechanism from its role as an enforcer of policy; if this is going to be an important distinction, then consider getting that distinction into the class implementation early so that you don’t have to retrofit it in a tedious and error-prone manner later.

Automatic vs Explicit Properties

Eric Lippert — Wed, 14 Jan 2009 20:11:25 GMT

Here's a question I got from a C# user last year, a question I get fairly frequently:

User: With “regular” explicit properties, I tend to use the private backing field directly from within the class. Of course, with an automatic property, you can’t do this. My concern is that in the future, if I decide I need an explicit property for whatever reason, I’m left with the choice of changing the class implementation to use the new private field, or continue going through the property. I’m not sure what the right thing to do in this case is.

You say “for whatever reason”, and that's key. The answer to your question will depend entirely upon what the reason was that motivated the change.

If the reason that motivated the change from automatically implemented property to explicitly implemented property was to change the semantics of the property then you should evaluate whether the desired semantics when accessing the property from within the class are identical to or different from the desired semantics when accessing the property from outside the class.

If the result of that investigation is “from within the class, the desired semantics of accessing this property are different from the desired semantics of accessing the property from the outside”, then your edit has introduced a bug. You should fix the bug. If they are the same, then your edit has not introduced a bug; keep the implementation the same.

That is a bit abstract. Let’s get more concrete. Suppose you have

sealed class BankAccount
{
public decimal Balance { get; set; }

and somewhere in the class you have a calculation:

if (Balance > 0) …

Then one day you decide to change the semantics of Balance:

sealed class BankAccount
{
private decimal balance;
public decimal Balance
{
    get
    {
      if (!CurrentUser.HasReadAccess(this))
        throw new SecurityException();
      return balance;
…

OK, you’ve changed the semantics of Balance. Now, everywhere inside your class, do you intend all calculations that at present use Balance to have these security semantics, or do you intend them to not have these security semantics?

If the former, keep the calling code the way it is; it is correct; changing it will introduce a bug.

If the latter, change it; it is now wrong; changing it will fix a bug.

If the reason that you changed the property from automatic to explicit was NOT to modify the semantics then… then… then why on Earth did you change it? And why are you contemplating making further inessential changes?

I presume that in this case there must be some motivation for changing correct working code; if you have some motivation that is causing you to change correct, working code, I suppose it can be applied equally well to all the other correct, working code in the class that calls that property.

User: Thanks. This is on my mind because I’m starting my first C# 3.0 project, and am experimenting with the new features. The scenario that tripped me up is exactly as you state; someone making a future semantic change.

Then your question is actually an instance of a more general question: “how do I future-proof a design?” That is, how do I design and implement a class hierarchy now so that inevitable changes in the future are easier and less bug-prone?

That's a hard question, one which someone more knowledgeable than I am could write a whole book on. Next time on FAIC I'll give a few musings on this topic.

Arrays considered somewhat harmful

Eric Lippert — Mon, 22 Sep 2008 21:04:00 GMT

I got a moral question from an author of programming language textbooks the other day requesting my opinions on whether or not beginner programmers should be taught how to use arrays.

Rather than actually answer that question, I gave him a long list of my opinions about arrays, how I use arrays, how we expect arrays to be used in the future, and so on. This gets a bit long, but like Pascal, I didn't have time to make it shorter.

Let me start by saying when you definitely should not use arrays, and then wax more philosophical about the future of modern programming and the role of the array in the coming world.

You probably should not return an array as the value of a public method or property, particularly when the information content of the array is logically immutable. Let me give you an example of where we got that horridly wrong in a very visible way in the framework. If you take a look at the documentation for System.Type, you'll find that just looking at the method descriptions gives one a sense of existential dread. One sees a whole lot of sentences like "Returns an array of Type objects that represent the constraints on the current generic type parameter." Almost every method on System.Type returns an array it seems.

Now think about how that must be implemented. When you call, say, GetConstructors() on typeof(string), the implementation cannot possibly do this, as sensible as it seems.

public class Type {
   private ConstructorInfo[] ctorInfos;
   public ConstructorInfo[] GetConstructors()
   {
     if (ctorInfos == null) ctorInfos = GoGetConstructorInfosFromMetadata();
     return ctorInfos;
   }

Why? Because now the caller can take that array and replace the contents of it with whatever they please. Returning an array means that you have to make a fresh copy of the array every time you return it. You get called a hundred times, you’d better make a hundred array instances, no matter how large they are. It’s a performance nightmare – particularly if, like me, you are considering using reflection to build a compiler. Do you have any idea how many times a second I try to get type information out of reflection? Not nearly as many times as I could; every time I do it’s another freakin’ array allocation!

The frameworks designers were not foolish people; unfortunately, we did not have generic types in .NET 1.0. clearly the sensible thing now for GetConstructors() to return is IList<ConstructorInfo>. You can build yourself a nice read-only collection object once, and then just pass out references to it as much as you want.

What is the root cause of this malaise? It is simple to state: The caller is requesting values. The callee fulfills the request by handing back variables.

An array is a collection of variables. The caller doesn’t want variables, but it’ll take them if that’s the only way to get the values. But in this case, as in most cases, neither the callee nor the caller wants those variables to ever vary. Why on earth is the callee passing back variables then? Variables vary. Therefore, a fresh, different variable must be passed back every time, so that if it does vary, nothing bad happens to anyone else who has requested the same values.

If you are writing such an API, wrap the array in a ReadOnlyCollection<T> and return an IEnumerable<T> or an IList<T> or something, but not an array. (And of course, do not simply cast the array to IEnumerable<T> and think you’re done! That is still passing out variables; the caller can simply cast back to array! Only pass out an array if it is wrapped up by a read-only object.)

That’s the situation at present. What are the implications of array characteristics for the future of programming and programming languages?

Parallelism Problems

The physics aspects of Moore’s so-called “Law” are failing, as they eventually must. Clock speeds have stopped increasing, transistor density has stopped increasing. The laws of thermodynamics and the Uncertainty Principle are seeing to that. But manufacturing costs per chip are still falling, which means that our only hope of Moore’s "Law" continuing to hold over the coming decades is to cram more and more processors into each box.

We’re going to need programming languages that allow mere mortals to write code that is parallelizable to multiple cores.

Side-effecting change is the enemy of parallelization. Parallelizing in a world with observable side effects means locks, and locks means choosing between implementing lock ordering and dealing with random crashes or deadlocks. Lock ordering requires global knowledge of the program. Programs are becoming increasingly complex, to the point where one person cannot reasonably and confidently have global knowledge. Indeed, we prefer programming languages to have the property that programs in them can be understood by understanding one part at a time, not having to swallow the whole thing in one gulp.

Therefore we tools providers need to create ways for people to program effectively without causing observable side effects.

Of all the sort of “basic” types, arrays most strongly work against this goal. An array’s whole purpose is to be a mass of mutable state. Mutable state is hard for both humans and compilers to reason about. It will be hard for us to write compilers in the future that generate performant multi-core programs if developers use a lot of arrays.

Now, one might reasonably point out that List<T> is a mass of mutable state too. But at least one could create a threadsafe list class, or an immutable list class, or a list class that has transactional integrity, or uses some form of isolation or whatever. We have an extensibility model for lists because lists are classes. We have no ability to make an “immutable array”. Arrays are what they are and they’re never going to change.

Conceptual Problems

We want C# to be a language in which one can draw a line between code that implements a mechanism and code that implements a policy.

The “C” programming language is all about mechanisms. It lays bare almost exactly what the processor is actually doing, providing only the thinnest abstraction over the memory model. And though we want you to be able to write programs like that in C#, most of the time people should be writing code in the “policy” realm. That is, code that emphasizes what the code is supposed to do, not how it does it.

Coding which is more declarative than imperative, coding which avoids side effects, coding which emphasizes algorithms and purposes over mechanisms, that kind of coding is the future in a world of parallelism. (And you’ll note that LINQ is designed to be declarative, strongly abstract away from mechanisms, and be free of side effects.)

Arrays work against all of these factors. Arrays demand imperative code, arrays are all about side effects, arrays make you write code which emphasizes how the code works, not what the code is doing or why it is doing it. Arrays make optimizing for things like “swapping two values” easy, but destroy the larger ability to optimize for parallelism.

Practical Problems

And finally, given that arrays are mutable by design, the way an array restricts that mutability is deeply weird. All the contents of the collection are mutable, but the size is fixed. What is up with that? Does that solve a problem anyone actually has?

For this reason alone I do almost no programming with arrays anymore. Arrays simply do not model any problem that I have at all well – I rarely need a collection which has the rather contradictory properties of being completely mutable, and at the same time, fixed in size. If I want to mutate a collection it is almost always to add something to it or remove something from it, not to change what value an index maps to.

We have a class or interface for everything I need. If I need a sequence I’ll use IEnumerable<T>, if I need a mapping from contiguous numbers to data I’ll use a List<T>, if I need a mapping across arbitrary data I’ll use a Dictionary<K,V>, if I need a set I’ll use a HashSet<T>. I simply don’t need arrays for anything, so I almost never use them. They don’t solve a problem I have better than the other tools at my disposal.

Pedagogic Problems

It is important that beginning programmers understand arrays; it is an important and widely used concept. But it is also important to me that they understand the weaknesses and shortcomings of arrays. In almost every case, there is a better tool to use than an array.

The difficulty is, pedagogically, that it is hard to discuss the merits of those tools without already having down concepts like classes, interfaces, generics, asymptotic performance, query expressions, and so on. It’s a hard problem for the writer and for the teacher. Fortunately, for me, it's not a problem that I personally have to solve.

Vexing exceptions

Eric Lippert — Wed, 10 Sep 2008 22:28:00 GMT

Writing good error handling code is hard in any language, whether you have exception handling or not. When I'm thinking about what exception handling I need to implement in a given program, I first classify every exception I might catch into one of four buckets which I label fatal, boneheaded, vexing and exogenous.

Fatal exceptions are not your fault, you cannot prevent them, and you cannot sensibly clean up from them. They almost always happen because the process is deeply diseased and is about to be put out of its misery. Out of memory, thread aborted, and so on. There is absolutely no point in catching these because nothing your puny user code can do will fix the problem. Just let your "finally" blocks run and hope for the best.

Boneheaded exceptions are your own darn fault, you could have prevented them and therefore they are bugs in your code. You should not catch them; doing so is hiding a bug in your code. Rather, you should write your code so that the exception cannot possibly happen in the first place, and therefore does not need to be caught. That argument is null, that typecast is bad, that index is out of range, you're trying to divide by zero – these are all problems that you could have prevented very easily in the first place, so prevent the mess in the first place rather than trying to clean it up.

Vexing exceptions are the result of unfortunate design decisions. Vexing exceptions are thrown in a completely non-exceptional circumstance, and therefore must be caught and handled all the time.

The classic example of a vexing exception is Int32.Parse, which throws if you give it a string that cannot be parsed as an integer. But the 99% use case for this method is transforming strings input by the user, which could be any old thing, and therefore it is in no way exceptional for the parse to fail. Worse, there is no way for the caller to determine ahead of time whether their argument is bad without implementing the entire method themselves, in which case they wouldn't need to be calling it in the first place.

This unfortunate design decision was so vexing that of course the frameworks team implemented TryParse shortly thereafter which does the right thing.

You have to catch vexing exceptions, but doing so is vexing.

Try to never write a library yourself that throws a vexing exception.

And finally, exogenous exceptions appear to be somewhat like vexing exceptions except that they are not the result of unfortunate design choices. Rather, they are the result of untidy external realities impinging upon your beautiful, crisp program logic. Consider this pseudo-C# code, for example:

try
{
using ( File f = OpenFile(filename, ForReading) )
{
// Blah blah blah
}
}
catch (FileNotFoundException)
{
// Handle filename not found
}

Can you eliminate the try-catch?

if (!FileExists(filename))
// Handle filename not found
else
using ( File f = ...

This isn't the same program. There is now a "race condition". Some other process could have deleted, locked, moved or changed the permissions of the file between the FileExists and the OpenFile.

Can we be more sophisticated? What if we lock the file? That doesn't help. The media might have been removed from the drive, the network might have gone down…

You’ve got to catch an exogenous exception because it always could happen no matter how hard you try to avoid it; it’s an exogenous condition outside of your control.

So, to sum up:

• Don’t catch fatal exceptions; nothing you can do about them anyway, and trying to generally makes it worse.
• Fix your code so that it never triggers a boneheaded exception – an "index out of range" exception should never happen in production code.
• Avoid vexing exceptions whenever possible by calling the “Try” versions of those vexing methods that throw in non-exceptional circumstances. If you cannot avoid calling a vexing method, catch its vexing exceptions.
• Always handle exceptions that indicate unexpected exogenous conditions; generally it is not worthwhile or practical to anticipate every possible failure. Just try the operation and be prepared to handle the exception.

High maintenance

Eric Lippert — Mon, 08 Sep 2008 19:36:00 GMT

The other day I went to buy some snack from the snack machine in the kitchen. The snack I wanted was in slot B-10, so I put in my coins, press B - one - zero, hey wait a minute there's no zero button! And why is it serving me up the snack on the left end of the machine instead of the right? Aha, there is a button marked "10", which is the one I was supposed to press. Instead I got snack B1. How irksome!

And then I laughed at my plight, because of course Steve Maguire told the same story about Microsoft vending machines in Writing Solid Code fifteen years ago. Maguire went on to make an analogy between bad candy machine interfaces and bad software interfaces; a "candy machine interface" is one which leads the caller to make a plausible but wrong choice.

Coincidentally, I was asked to review a fragment of code recently that got me thinking again about candy machine interfaces. This isn't exactly the kind of bad interface that Maguire was talking about -- but I'm getting ahead of myself. Let's take a look at the code:

public static class StreamReaderExtensions
{
    public static IEnumerable<string> Lines(this StreamReader reader)
    {
        if (reader== null)
            throw new ArgumentNullException("reader");
        reader.BaseStream.Seek(0, SeekOrigin.Begin);
        string line;
        while ((line = reader.ReadLine()) != null)
            yield return line;
    }
}

The idea of this code is awesome. I use this technique all the time in my own programs when I need to manipulate a file. Being able to treat a text file as a sequence of lines is very handy. However, the execution of it has some problems.

The first flaw is the bug I discussed in my earlier post on psychic debugging. Namely, the null check is not performed when the method is called, but rather, when the returned enumerator is moved for the first time. That means that the exception isn't thrown until possibly far, far away from the actual site of the error, which is potentially confusing.

The second flaw is that the "while" loop condition is a bit hard to read because it tries to do so much in one line. Shorter code is not magically faster than longer code that does the same thing; write the code so that it is maximally readable.

But there's a deeper flaw here. To get at it, first let me state a crucial fact about the relationship between a stream reader and a stream:

A StreamReader “owns” its underlying stream. That is, once a stream has been handed to a stream reader by a caller, the caller should not muck with the stream ever again. The stream reader will be seeking around in the stream; mucking around with the stream could interfere with the stream reader, and the stream reader will interfere with anyone trying to use the stream. The stream reader emphasizes its ownership by taking responsibility for disposing the stream when the stream reader is itself disposed.

Now, “Lines” defines an object, and what does that object do right off the bat? It mucks around with the underlying stream. This is big red flag #1. This smells terrible. No one should be messing with that stream but the reader.

Furthermore, think about it from the caller’s point of view. Maybe the caller knows that there are a bunch of bytes that it wants to skip, so it deliberately hands a StreamReader to Lines() which has been positioned somewhere past the beginning of the file. But Lines() thinks that it knows best and ignores the information that the caller has given it. This is big red flag #2.

(The reason why the original code was seeking back was because the same reader was being "recycled" many times to read the same file over and over again, which is yet another red flag. Readers are cheap; you can have multiple readers on one file, there's no need to hoard them and reuse them.)

The third big red flag for me here is the ownership issue. When you hand a stream to a reader, the reader takes care of everything for you from then on – that’s the “contract” between the reader and the caller. “Lines()” does not have that contract. Lines()’s contract says “attention caller: I am taking ownership of this reader. I am going to muck with its underlying stream. You must never use this reader for anything ever again – except that I’m not going to dispose it for you. If you want the reader and its stream closed then you are going to have to keep a reference to it around until you can prove that I am done with it. And if you get it wrong, either I crash or you get a resource leak. So get it right.”

This is a terrible contract to impose upon a caller, and of course, the imposition here is in no way represented by the signature of the method. You just have to know that “Lines()” owns the reader for the purposes of all its functionality, but not for its cleanup, which is deeply weird.

In short, the caller needs to know all the details of what is happening in the method in order to use it correctly; this is a violation of the whole purpose of creating methods. Methods are supposed to abstract away an operation so that you do not have to know what they are doing.

A fourth red flag is that the task performed by Lines() does not have a clear meaning in terms of the logic of the program. In looking at every caller of this method it became clear that the desired semantics were “give me the lines of this text file one at a time”. But we haven’t written a method that does that. In order to get the lines of a text file from Lines() the caller is required to do all kinds of work to make Lines() happy – open the stream, create a reader, call Lines() but keep the reader around, close the reader after we know that the iterator is done.

This code makes the caller do a whole bunch of things -- things that have to be done in exactly the right order but potentially distributed out over long stretches of time and disparate code locations. This method is very "high-maintenance"! Everything has to be just right all the time in order for it to be happy and get along with others; anything out of place and things will start going wrong.

Finally, we have an example of premature generality here -- if the intention is to read a text file, then write a method that reads a text file. If you don't need the power of reading lines from an arbitrary stream, maybe don't implement that. Cut down on your testing burden.

Some relatively simple changes fix it up:

public static class FileUtilities
{
    public static IEnumerable<string> Lines(string filename)
    {
        if (filename == null)
            throw new ArgumentNullException("filename");
        return LinesCore(filename);
    }
    private static IEnumerable<string> LinesCore(string filename)
    {
        Debug.Assert(filename != null);
        using(var reader = new StreamReader(filename))
        {
            while (true)
           {
                string line = reader.ReadLine();
              if (line == null)
                  yield break;
               yield return line;
          }
        }
    }
}

And now everything is crystal clear. The purpose of this thing is to read the lines out of a text file. The caller gives it the name of a file, it gives you the lines, and we’re done. The caller does not have to worry about anything else – the iterator takes care of opening the file, cleaning up when its done, and so on. There’s no messing around with streams at all; we have now provided an abstraction over the file system to the caller.

Whenever you write a method think about the contract of that method. What burdens are you imposing upon the caller? Are they reasonable burdens? The purpose of a method should be to make the caller’s life easier; the original version of Lines() makes life harder on the caller. The new version makes life easier. Don't write high-maintenance methods.

Reading Code Over the Telephone

Eric Lippert — Fri, 16 May 2008 17:00:00 GMT

In my youth I once attended a lecture given by Brian Kernighan on the subject of code quality, which was very influential on my attitudes towards writing legible code. One of the things that Kernighan recommended was to endeavour write code that was so clear that it could be easily read over the phone and understood. Most people find code much easier to comprehend when read than when heard; if you can make it clear enough to be understood when heard, it's probably pretty clear code.

I was reminded of this when I got the following question from a colleague by email:

Subject: Stupid C# 3.0 lambda expression question

How does one read the => operator?

First off, I told my colleague that there are no stupid questions, only stupid people. No stupid people work here, so don't stress about it. This is a perfectly sensible question.

As far as I know, we do not have an "official" line on how to read this operator over the phone. In the absense of any other context, I personally would say c=>c+1 as "see goes to see plus one". Some variations that I've heard:

For a projection, (Customer c)=>c.Name: "customer see becomes see dot name"

For a predicate, (Customer c)=>c.Age > 21: "customer see such that see dot age is greater than twenty-one"

An unfortunate conflation is that the => operator looks a lot like ⇒, the "implies" operator in mathematics. Since => does not have the same semantics as ⇒, it is probably a bad idea to read => as "implies". (x⇒y would have the semantics of !x|y in C#.)

Incidentally, it is a little known fact that VB6 and VBScript implemented the ⇒ operator with the Imp keyword and the ⇔ operator with the Eqv keyword. They disappeared in VB.NET. Where did they go? It is a mystery!

Translating intentions and mechanisms

Eric Lippert — Tue, 25 Mar 2008 20:26:00 GMT

Before I get into today's blogging, a quick note about my recent post on How To Not Get A Question Answered. That was certainly not intended to be fishing for compliments or chiding people for never acknowledging help ten years ago; that said, I appreciate both. Thanks to everyone who made thoughtful comments.

Moving on; a theme that seems to be coming up over and over again in my recent technical conversations is that of intention vs. mechanism. We have tried hard in C# 3.0 to make a language where there is a good balance between the code reading as a declaration of what meaning you intend the code to represent, and reading as a list of imperative instructions specifying a mechanism which achieves those intentions. I've been accumulating anecdotes about this tension between representing intentions vs. mechanisms; expect this to be a recurring theme in the blog for the next while.

Here's a question I got recently which speaks to this tension with regards to the subject of porting code from one language to another:

I have some C++ code with a macro in it. A typical usage looks like this:
TRY_WAIT_OP(Execute());
This macro expands to code similar to this:
for(int i=0; i<10; i++) {
if (Execute()) break;
Sleep(i*1000);
}

We are translating this code into C#, but C# does not have a #define directive. How do I do textual replacement of code in C#?

Once more, we have someone looking for a thin metal ruler. There's a problem -- represent the meaning of a common operation in a programming language. C++ provides a solution mechanism -- using preprocessor-based metaprogramming to create a nonstandard control flow primitive. The natural tendency when doing a translation is to find the identical mechanism in the new language, and then translate the code to use that mechanism. But what if there is no such mechanism?

In that case, you've got to translate the intentions, which after all, is what you are trying to translate in the first place. Presumably the new code is intended to do the same thing as the old code. Translating the mechanisms is just a particularly easy road to translating the intentions. What is the meaning of this macro?

Clearly TRY_WAIT_OP means "execute some arbitrary code that returns a Boolean. If it returns true, you're done. If it returns false, wait some amount of time and try again, up to ten times".

Now think about how you would write code from scratch that implemented those intentions in C#. Don't think at all about how it was written in C++. Your goal here is to solve the same problem using a different tool, so don't use the same techniques that you used for the other tool if they're not appropriate. Use the techniques that are appropriate for this tool.

The way I would write that in C# is to write a method that takes as its argument some arbitrary code that returns a Boolean. "Arbitrary code" is represented in C# by a delegate. We can do a bit better than the macro while we're at it, and return a success code:

private static bool AttemptMultiple(Func<bool> action) {
    Debug.Assert(action != null);
    const int maxAttempts = 10;
    const int delay = 1000;
    for (int attempt = 0 ; attempt < maxAttempts ; ++attempt) {
        if (action()) return true;
        Sleep(attempt * delay);
    }
    return false;
}

Lambda expression syntax gives us a nice way to do the call:

AttemptMultiple(()=>Execute());

A code porting project is a good opportunity to review the design fundamentals:

Why 10 retrys? Should this be a parameter to AttemptMultiple?
Why wait 1000 milliseconds instead of some other delay? Should this also be a parameter?
Why is the increasing delay linear rather than constant, geometric, etc? Should the delay strategy be a parameter?

The above questions are good but they rather miss the point. The important question is: is this functionality even a good idea in the first place?

This last point is key. The "try it, fail, wait, try again" strategy is in general a dangerous one because it does not compose well with itself. Consider the following:

bool SendPackets() { ... if (!AttemptMultiple(()=>{ ... })) return false; ... }
bool TalkToSocket() { ... if (!AttemptMultiple(()=>SendPackets())) return false; ... }
bool SendData() { ... if (!AttemptMultiple(()=>TalkToSocket())) return false;... }
void HandleCommand() { ... if (command == SendData && !AttemptMultiple(()=>SendData())) ReportErrorToUser();... }

Now suppose that the user has a bad network card and SendPackets is always going to fail. If you look at any one of those lines, it looks like the attempt is being made ten times and will take a maximum of about one minute. In fact, the attempt to send the packet is made ten thousand times and will not report the error to the user for about a week.

Usually the right thing to do when something fails is to go into a failure state immediately. Tell the user that something failed and let them decide when and if to retry it.

What are some examples of a poor mismatch between intentions and mechanisms that you guys have seen? I'm interested in stories about:

mechanisms that subtly did not implement the intentions of the programmer
situations where the intention of the code was completely obscured by the mechanisms. How did you make the code better reflect the intentions?
situations where the mechanism of the code was important, but obscured by unnecessary emphasis on representing the intention. What were the negative consequences of obscuring the mechanism behind some abstraction?

Thanks!

Bad Names

Eric Lippert — Tue, 12 Jun 2007 21:59:00 GMT

Over the past year and a half of working on the C# compiler we’ve refactored a lot of crufty old code. But a surprising number of the “refactorings” have actually simply been renaming a structure, class, or method to something more sensible. The compiler is showing its age; there are a lot of structures, classes and methods inside the compiler which no longer have sensible names (if, in fact, they ever did). Going through the whole thing all at once and renaming everything without changing the semantics of any of the code is a fairly cheap and easy way to massively improve code readability. After all, what is easier to understand, FUNCBREC or StatementBindingContext? (I think FUNCBREC at one point stood for “function binding record”.)

One of the things we’ve been looking for is names that “smell bad”. Sometimes the thing simply needs to be renamed and that’s the end of the story. Sometimes, though, the existence of a bad name is a red flag in the code that points to some chunk of functionality which could use some serious thought and additional code refactoring applied to it.

Functions with names like DoTheOtherThingToIt are obviously badly named. There’s no way to look at the call site and have the faintest idea what’s happening on the callee side. But there are more subtle signs of badness. I’ve been compiling a list of these red flag particles, and here is what we’ve come up with so far:

Misc:

This is indicative that the Single Responsibility Principle has been violated. Functions or structures which do “miscellaneous” work almost certainly do two or more things which are unrelated.

Base/Core/Worker/Helper:

These are often a sign of too much or too little abstraction. Suppose InferTypeSignature calls InferTypeSignatureWorker. If InferTypeSignatureWorker is doing all the work of inferring a type signature then move it back into InferTypeSignature. If it is not doing all the work, then describe what work it is doing, don’t just say that it’s working on something.

Fake/Real:

In the compiler we had a structure called FakeMethodSymbol, deriving from MethodSymbol. OK, it’s a symbol, it represents a method, got it. But in what sense is it a “fake” method? Is it a compiler-generated method as opposed to a user-generated method? Is it some kind of stub for a missing method? When is it legal to use as a MethodSymbol? Who knows? I sure didn’t. “Fake” doesn’t tell you anything about its purpose.

(It turned out that this is used when representing the method signature of an expanded method call that uses a C++ style “varargs” argument list. We’ve renamed it ExpandedVarargsMethodSymbol, and now it is extremely clear what that thing is for.)

Special/Normal:

Describe the property that makes the case special, rather than just calling it out as special. And try not to call things “normal”, just call them things. Rather than NormalFoo and SpecialFoo, have Foo and FrobbingFoo. Or have a NonFrobbingFoo and a FrobbingFoo both derived from abstract base class Foo.

Ex/2:

Oh, the pain. These should be used to solve COM interface versioning problems, and even then only as a last resort.

Those are the ones we've found in the compiler so far. What sort of bad-smelling name particles do you guys encounter in crufty code?