Fabulous Adventures In Coding : Aargh!

Aargh! Part Eight, plus Boring Metablogging

Eric Lippert — Fri, 23 Jul 2004 21:51:00 GMT

Q: What's a pirate's second favourite mode of transportation?

A: A caaaaargh! Preferably a Jaguaaaaargh, but an early Oldsmobile Cutlass will do.

Q: Very amusing -- but what's a pirate's favourite mode of transportation?

A: A pirate ship, silly.

Gripe #10: Don't use _alloca

_alloca allocates memory off the stack instead of the heap. It's very convenient to use when you need a temporary buffer and don't want to worry about freeing it, but I try to avoid it whenever possible for two reasons:

(1) it wrecks the ability for the compiler to make optimizations based on how much stack a function uses, and

(2) when combined with the previous point, it leads to terrible, terrible crashes. If the user asks the machine for lots and lots of heap then the program churns away trying to allocate lots of memory before finally throwing an exception or returning null. The worst that can happen is not too bad. The worst that can happen if the stack is overallocated is that the stack guard page is hit for the second time and the process is terminated. (The exact behaviour of the win32 stack under low-stack conditions is interesting, and I will blog about it at some point; the script engines do some fancy footwork in this department to keep the stack from blowing.)

Here's the thing -- if you have an upper bound on the amount of stack you're going to use, you might as well just ask for it. If you don't have an upper bound -- if the upper bound is determined by something at runtime and it could be arbitrarily big -- then use the heap, because really big stack allocations are too risky.

For example, I'll often see code where _alloca is used to hold a temporary copy of a string when it is being converted from eight bit ASCII to UTF-16. Bad idea, particularly if the string can come from an untrusted internet script. The string could be a million bytes long, and blow the stack! Aaargh! It's drivin' me nuts!

The thing is, stack allocation is incredibly fast, and in many situations the common scenario is for the allocation to be small even if that's not guaranteed. Therefore I often use a "best of both worlds" strategy if I have data that is usually small but might be huge. First, I declare a stack buffer twice the size I expect to use. Then detect if more than that is required and allocate it off the heap if it is. (Of course, you have to remember to clean up the heap buffer if you do that.)

*********************************

You'll note how few new hit television series are called "Extreme Television Editing", or "New Issues In Satellite Uplink Futures Contract Law" -- though come to think of it, either of those would be more likely to catch my interest than the vast majority of the inaptly named "reality" television. Therefore I try to avoid talking about the blog in the blog -- I personally find blogs about blogging boring. I'll keep this short then.

A number of people have asked me questions like "can I quote your blog in my blog?", "can I post your article in my wiki?", "can I use this information in my MSDN Magazine article?" etc.

I'm putting this information on the web because I want it to reach the people who can use it, so by all means, disseminate the information widely. I ask only four things:

1) Please give me a heads-up before you do.
2) Please attribute the source, preferably with a link back to the original.
3) Please don't quote me out of context.
4) Please understand that by doing so, you remove my ability to correct mistakes in your copy.

UPDATE: I have posted a follow-up to these guidelines.

Aargh! Part Seven

Eric Lippert — Fri, 04 Jun 2004 17:17:00 GMT

Q: How do pirates keep their socks from falling down?

A: Thumbtacks.

I am insanely busy with bug fixing and performance testing today, so once more I'll dip into my endless archive of rants about irksome coding practices I've seen one time too many.

Gripe #8: Assert the truth, the whole truth, and nothing but the truth

Debug assertions should always describe invariants -- conditions that should always happen no matter what. They both help document your program, by making your invariants clear, and help catch bugs by alerting you when those invariants are violated. But assertions must describe what you believe to be always true about your algorithm, not what you hope is true:

pv = malloc(10);
Assert(NULL != pv);

Aargh! That's drivin' me nuts!

NULL is a legal return value of malloc and you therefore can't assert that malloc didn't return it. That condition, rare though it might be, has to be tested like any other. You can't simply declare that the world is going to turn out the way you'd like it.

Sometimes you want to pop up warnings in your debug build when incredibly rare things happen. That's a great idea -- but create a little function called CreateDebugWarning or some such thing, so that it does not get confused with Assert .

Gripe #9: Don't be so darn friendly

In C++ you can hide implementation details with the private keyword, but sometimes you want to have two classes which have the ability to party on each other's internals. For example, you might have a collection class and an enumerator class that need to have a private way to communicate with each other. Such classes are called "friendly classes". One day I grepped the headers of an unnamed Microsoft application for the word friend. It appeared over 600 times! Aargh!

This is a bad sign. One class had eleven different friend classes. When you have that many friends, there really is no difference between the private interface and the public interface. Visibility modifiers exist in the first place so that you can do information hiding and clean polymorphism. If you have to allow friend access to so many different classes then your public interfaces are not clean. You have a bunch of classes depending on each other's implementation details in order to work properly. The information hiding afforded by classes and interfaces is a feature of C++ specifically designed to reduce the complexity inherant in large software projects -- friend is a way to work around that limitation when necessary. And there certainly are times when it is necessary, but overuse makes code more and more complex, intertwined, buggy and unmaintainable.

Use friends very sparingly in C++ -- enumerators should be friends of collections, yes, but if documents are friends of views, you might have a problem on your hands.

In C#, JScript .NET and VB.NET there are no friendly classes. Rather, there is private, public and internal -- in C#, private really is private, but any class in the same assembly can party on internal data. This gives you a lot of the benefits of friendly classes while still being able to restrict access to stuff that is truly private.

Aaargh, Part Six: One More Thing About Comments

Eric Lippert — Wed, 05 May 2004 22:55:00 GMT

Gripe #7: Use The Right Struct For The Job

I meant to include this one in yesterday's gripe about comments, as this illustrates a time when I found a comment that should never have been there. The person who wrote the comment should have realized that the very fact that they needed to put a comment in indicated that the code needed to be rewritten. It slipped my mind at the time though, so here it is today.

Here's a fragment of code I found in an unnamed Microsoft product once. The code is part of a fairly complicated routine for clipping a rectangle -- the details are unimportant for the purposes of this discussion. The code is correct -- it performs the correct calculation and spits out the right answer -- but boy, is it hard to read:

if (client.x > rect.right)
default.x += ((client.x - rect.right) / 2 - rect.left); // right == width
if (client.y > rect.bottom)
default.y += ((client.y - rect.bottom) / 2 - rect.top); // bottom == height

Holy cow, was that ever a bug waiting to happen. The data member that holds the width was named right and the data member that holds the height was named bottom ! I needed to change this code and spent a good thirty solid minutes looking at the routine (which was much longer than shown here), running it in my head before I concluded that the comment and the code were in fact correct.

Someone got lazy -- they already had a struct defined for a {right, left, bottom, top} rectangle but they had the information {left, width, top, height}, so they just stuffed the information they were manipulating into the struct they had handy. Aaargh! It's drivin' me nuts!

Worse, the comment is cryptic in the extreme. It took me quite a while to figure out whether the equality operator in the comment was documenting a logical invariant (that is, the comment means “we know that the right coordinate is equal to the width because the left coordinate is always zero in this case“) or whether they were just using the equality operator to mean “has the semantics of“.

Please don't put cryptic comments in the code explaining why code is misleading -- fix the code so that it isn't misleading anymore! In this particular case, the rectangle in question was entirely local so there was no need to have it in a struct at all. I needed to modify this code anyway, so I got rid of the struct altogether and just declared four local variables for the left, width, top and height, appropriately named.

Aargh, Part Five: Comment Rot

Eric Lippert — Tue, 04 May 2004 19:04:00 GMT

Gripe #6: Comment Rot

If you've been reading my SimpleScript code you might have noticed that there are very few comments in my code. That's deliberate.

Why do we have comments in the first place? We have comments because sometimes the semantics of the code are not clear from the syntax. Remember, the syntax tells you what the code does, the semantics tell you why the developer wrote that code, what the purpose is.

I try to write code so that the semantics are clear just by reading the code, no comments required. Code with one comment for every line drives me nuts! Imagine if people wrote books that way -- that their English was so unclear, so hard to understand that they needed to have a footnote on every sentence and paragraph explaining in some other language what the intended meaning was! It would be awful.

Worse, excessive comments cause "comment rot". I don't know how many times I've been reading a piece of code, read a comment that said "there is such and such a problem with this code, I'll come back and fix it later", and sure enough, there is no problem anymore. Or there is a problem, but it’s a different problem. People change the code, compile it, test it, and never fix the comments. The more comments there are, the more likely it is that this will happen. Wrong comments are usually much worse than no comments at all.

Slightly less horrid than wrong comments are useless comments -- comments that do not explain the semantics, but restate the syntax. How many times have you seen this:

j = j + 1 ' Increment j

Thank you so much, developer, for letting me know what j = j + 1 does! In my advanced old age I was beginning to forget the first thing I ever learned about programming! Why do people do this? This would be better, as it explains the semantics:

j = j + 1 ' Go to the next line

but surely this is an opportunity to eliminate the comment entirely:

currentLine = currentLine + 1

When I write a piece of code with comments I want those comments to stand out, not get lost in a sea of triviality! If I write a comment, you'd better believe that it's usually because the implementation of the intentions simply cannot be clearly expressed by the code, or because something really bizarre and important is going on. (Like the weird garbage collection protection I mentioned earlier today.) Comments should be signposts, your eye should be drawn to them because they're important. Commenting style should train people to read the comments as they come across them in the code. Writing lots of irrelevant comments trains people to ignore the comments!

Earlier today I was talking about the default property semantics of IDispatch, and earlier I commented that I'd blog about a conversation Matt Curland and I once had about the differences between VBScript's and VB's default property resolution in their respective Intellisense engines. Now's a good chance to kill a few birds with one stone; one of my favourite comments I've ever written, and certainly one of the longest, is found in the VBScript Intellisense code that I wrote for Visual Interdev:

        // [Eric Lippert 1998-03-27] -- see Scripting bug 847
        //
        // We now have the following problem. We have "foo.bar(".
        // What if bar returns an object that has a default property?
        // Suppose bar is a collection, for instance, with a default
        // property "item". Then this should show statement completion
        // information for the "item" method, not for "bar" -- provided
        // that bar takes no arguments. If bar takes arguments, then
        // we should show statement completion for bar.
        //
        // This generalizes to chains of defaults -- if bar is the default
        // property of foo, and bar takes no arguments, and bar
        // returns an object that has default property baz that takes
        // an argument blah, then
        //
        // foo(
        // foo.bar(
        // foo.bar.baz(
        //
        // should all return statement completion information for baz(blah).
        //
        // So here's what we'll do:
        //
        // First we determine if "bar" returns an object. If it does,
        // we check and see if it has a default property chain. If so, we
        // get type info for the final available function on the default
        // property chain.
        //
        // Second, if bar is a function that takes arguments, we return
        // function information for bar.
        //
        // If bar does not take arguments and bar returns an object with
        // a default property, then we return info on the default property.
        //
        // Otherwise, we return information on bar.
        //
        // This is not actually what Visual Basic does. VB does the
        // following: (from email by Matt Curland, 1998-03-27)
        //
        // "If no parameters are supplied never call the default unless
        // you're in an assignment statement without a Set. If you're
        // in a non-ending statement of a call chain then only do
        // the default resolution if the specified function doesn't have
        // its own parameters and a parameter is actually specified."
        //
        // That is not exactly what we're doing here, but it's close enough
        // as far as I'm concerned.

Gross, gross, gross. And actually I've cut it short -- the comment then goes on to describe even more weirdnesses with the case where foo.bar(blah)( gets Intellisense on a method that returns a collection. This is very complicated code; I wanted anyone who was about to change it to understand fully what the meaning behind it was.

The other thing that amuses me greatly about this comment is that apparently there were only 847 bugs in the scripting database at the time!

Aargh, Part Four: There is no part four

Eric Lippert — Mon, 29 Mar 2004 17:36:00 GMT

I've got all these little rants queued up for days when I'm too busy to post. And Raymond has all these little articles queued up while he's on vacation. And in one of those weird Brouwer Fixed Point Theorem moments of synchronicity, Raymond just beat me to it. His post on how people write incorrect implementations of QueryInterface is different than the one I was going to post only insofar as his is more complete and has fewer pirate jokes.

In short: I've come across plenty of code where the QueryInterface responds to IPersistBlahBlahBlah but not IPersist. Drives me nuts! Argh! Read Raymond's post for more details.

http://weblogs.asp.net/oldnewthing/archive/2004/03/26/96777.aspx

Aaargh! Part Three

Eric Lippert — Thu, 25 Mar 2004 21:45:00 GMT

I'm still at VSLive. Both my talks are done, so its just booth duty from here on in. The talks went... OK. Running VSTO on top of Virtual PC on a laptop was too slow; we'll have to devirtualize that for the next time.

Unfortunately they put me in the keynote room, which seats over a thousand. Now, when 200 people show up for a talk, that's great -- but in such an enormous room, it's really, really hard to get the audience enthusiam up when they're spread out over what seemed like a couple of acres. But I got a lot of good questions afterwards, so hopefully a few people were awake and learned something.

No time to write, so another canned gripe about C++ idioms that are drivin' me nuts today -- but first, an excerpt from Pirate Riddles For Sophisticates.

Q: Whom did the pirate vote for in the first Haitian election?
A: ARRRistide.

Q: Wait. Why did they let a pirate vote in the Haitian election?
A: Remember, the nation was taking its first halting steps toward democracy, and balloting procedures were rather chaotic. The pirate just slipped in somehow. Arrr.

Gripe #5: Exception Handling and operator new

In C++ you can easily change the new operator to throw an exception when it runs out of memory instead of returning NULL. Cool idea, but it can be very dangerous because this causes a global change in the behaviour of a commonly used operator. What happens when you do that in a program that contains libraries that do not have that assumption?

try
{
m_pbar = m_pfoo->bar();
} catch // ...

where you grabbed the source code for that class and just kind of compiled it in:

Bar * CFoo::bar(void)
{
HANDLE h = NULL;
h = GoGetMeAFileHandle();
m_pBlah = new CBlah;
if (NULL == m_pBlah){
FreeFileHandle(h);
return NULL;
} // ...

You didn't write CFoo::bar, but you've just broken its error handling if you redefined new to throw an error instead of returning NULL. Now if the allocation of CBlah fails, you've leaked a kernel object. This is another example where idioms like smart pointers need to be consistently applied across an application in order to reap their benefits.

I have debugged leaks in IIS where these kinds of things happen and they are absolutely no fun for our customers.

I once debugged some code where the new operator had been redefined to throw an exception, and the authors did something very clever. (Aside: clever is bad -- clever is hard to figure out! If the code is clever, rewrite it until it is brain-dead obvious.) In the debug build, they wrote some magic into the operator so that it detected if it was being called without an exception handler and raised an assertion. Sounds cool, right? Any possible negative consequences of that?

Well, how about the fact that static objects which have constructors that call new cannot be linked in to the application without the program asserting on startup? The standard map, vector, etc, objects are such objects. Their constructors call new, and therefore you can't have a static map. Statics have no context to catch the possible throw, and therefore will assert when the debug application starts up. (Guess how I found that bug?)

That said, I can see how it is useful to have a new operator that throws an exception. You know what I'd do if I wanted that? I'd define an operator overload that took an argument, create a dummy identifier called "throws", and then say pBar = new(throws) CBar -- now it is perfectly clear what the semantics of that thing are.

Redefining the global new operator works great if you have 100% control over all contexts where the operator is used. Unfortunately, that's frequently not a realistic assumption!

Aaargh! Part Two

Eric Lippert — Tue, 23 Mar 2004 17:58:00 GMT

I'm insanely busy prepping for VSLive! today, so, once more, I complain about coding practice that are drivin' me nuts.

But first: why do pirates dislike Johnny Depp's Oscar-nominated performance in "Pirates of the Caribbean"?

Because it wasn't AAAAAARRRR-rated.

Gripe #3: More Complaining About Bad Macros

One day I was debugging a bunch of macro-ridden code that I didn't write. I wanted to know what the method signature of CForm::GetMouseIcon was. So, silly me, I grepped the source code for CForm::GetMouseIcon. Nothing. Must be defined in the class header itself! So I grepped for GetMouseIcon. Nothing. Aaargh! It was drivin' me nuts!

Inspiration hit -- I grepped for MouseIcon and found in some header somewhere

DEFINE_DERIVED_GETSET_METHODS(CForm, CServer, MouseIcon, IID_IPicture)

Aaargh! I still didn't know what the signature was until I managed to find that macro definition and decode it! Why do people write code like this? You can't easily debug into CForm::GetMouseIcon, you can't find it easily using grep, etc. Why define this stuff in macros? Does it really make anyone's life easier?

Gripe #4: Exception Handling, Return Values and Smart Pointers

You probably knew I couldn't actually not complain about smart pointers, the bane of my existence. I'm so predictable.

Before I start complaining about exception handling, please, I don't want to have a replay of that Joel On Software debate about the merits of structured exception handling. I think exception handling is super. I love writing code in C# that uses exception handling. catch allows you to isolate your error handling code into strictly scoped blocks, and finally allows you to isolate your cleanup code, and that's all good.

The problem I have with structured exception handling is that programs that mix libraries that use SEH and libraries that return error codes, (like, say, every single COM API) often turn into a godawful mess. You end up with the worst of both worlds: exceptions potentially going off all over the place acting as non-local gotos, and still having to pay careful attention to error return codes, thereby de-localizing the error handling code. C# and the .NET Framework were designed to use exception handling from day one, so it makes a lot of sense. COM was not!

As we saw last time, the tempting thing to do is to wrap every error-returning function with a macro that turns it into a throw, and wrap every must-be-released handle with a smart object that will clean up when it goes out of scope. But the resulting complexity means that often the cure is worse than the disease. We've already seen the havoc that macros wreak. What about smart pointers?

Case in point -- one of my coworkers and I were debugging a weird problem in some of our code just last week. We called into a particular API which was setting the GetLastError value. Something in our code was blowing it away, so we wrote some code to preserve it. It didn't work; when our function returned, GetLastError was still reporting that there had been no error. How could that be? What's wrong with this picture?

// Ensure that we've preserved the error state we captured earlier
SetLastError(blah);
return hr;
}

The caller of this method calls GetLastError , but the state we just set has been blown away. OK, smart people, where does that happen?

Turned out that a smart pointer in the current scope was holding on to a proxy for an out-of-process COM object, and when the smart pointer went away, the out-of-proc object was released, which resulted in a whole slew of calls behind the scenes, which resulted in the GetLastError value being overwritten. Of course, since this was a smart pointer destructor doing all this, the source code for the call site looked like this:

} <-- calls destructor

Aargh! It's drivin' me nuts! Call me crazy, but I want operations bearing complex semantics and causing side effects that break my error handling to look a little more like function calls and a little less like punctuation. Had the code been written using dumb pointers, no problem

// Ensure that we've preserved the error state we captured earlier
SetLastError(blah);
pProxy->Release();
return hr;
} <-- no op

Now it's obvious where the mistake is.

COM programming demands discipline and a keen understanding of how to write correct error-path code whether you also use exception handling or not. Therefore, I say don't use exception handling if you're going to be doing a whole lot of COM-style error handling too! I like to think that I'm a fairly bright guy, but I have great difficulty analyzing the semantics of code that has macros, template classes, smart pointers, exception raisers, error return values and cross-process COM calls all in the same routine. I'm not that smart!

I've developed a coding style that works very well with COM -- it is verbose, it is rigid, and it is very, very clear to even novice programmers exactly what the lifetime of all pointers in the code are -- and my code rarely has reference leaks because of it, even in error cases. And when it does, they are a snap to track down because every time a pointer is addrefed there is a call to AddRef and every time it is released there is a call to Release right there in the code, not tucked away in some template library. It's not hard to develop a good coding standard.

Next time: even more on bad interactions between exception handling and memory management

Aargh, Part One: A Pirate Walks Into A Bar…

Eric Lippert — Wed, 10 Mar 2004 18:20:00 GMT

One of my former housemates was fond of pirate jokes, as am I. My personal favourite of his was:

A pirate walks into a bar, and the barkeep says "Excuse me, cap'n, but did you know that you've got your ship's wheel stuck in your pantaloons?"

"Aye," says the pirate, "that thing be drivin' me nuts! Aaargh!"

Today: stuff that drives me nuts, part one of an ongoing series which I will continue whenever I'm too busy to write. (Complaining is easy.)

I've reviews a lot of code over the years, most of which was very high quality. But no code is perfect. There are some bad patterns I see over and over again that are easy to fall into but suboptimal in subtle ways. Most of this series will apply specifically to COM programming, but I may throw in a few scripting peeves as well.

I have already griped about smart pointers and bad Hungarian, so I'll give them a miss.

Gripe #1: Too Many Comments

Comments should explain places where the semantics is not obvious from the syntax. Here's some code I found deep in the persistence code of an unnamed Microsoft product:

// Read cItems
CheckHresult( pStm->Read( &cItems, sizeof(LONG), NULL), pStm, IID_IStream );
// Allocate a block which can hold cItems
pItems = new DWORD[cItems];
// Read the block of items
CheckHresult( pStm->Read(pItems, cItems * sizeof(LONG), NULL), pStm, IID_IStream);

Aargh! It's drivin' me nuts!

(Aside: Why no error checking on the new? More on that next time. Also, this reminds me that I want to do a blog on security aspects of serialization code some time.)

Having one comment per line which simply restates the following line in English instead of C++ doesn't help anyone understand the code, it just makes the code longer.

It's worse than that. It dilutes the value of comments -- you should be able to look through a piece of code and find all the really interesting bits by looking for the comments. If there are comments everywhere, 90% of which explain nothing that couldn't be gleaned from reading the code itself then the 10% of the useful comments -- the ones that explain the semantics -- will never be noticed.

Furthermore, comments "go bad" -- when you do this habitually, it is really easy to change the code and forget to change the comments. Then the comments no longer accurately describe how the code is, but rather how it was.

Comments should also be of appropriate size. Here is a fragment of a script I recently code reviewed.

''''''''''''''''''''''''''''''''''''''''''''''''''''''''
'' DISABLE SCREEN SAVER
''''''''''''''''''''''''''''''''''''''''''''''''''''''''

WScript.Echo "Disabling Screen Saver"
[ then code here that disabled the screen saver ]

OK, I'm reading the code, and I see the line

WScript.Echo "Disabling Screen Saver"

and I can probably guess what the next line of code does without having the five lines above it consumed by comments and whitespace! I want these huge block comments to be catching my eye for really incredibly important stuff. Otherwise, it's just consuming valuable screen real estate, preventing me from getting more semantics-bearing code onto one screen.

Gripe #2: Bad Macros

The sharp-eyed among you will have wondered about the CheckHresult call above. What the heck is that thing doing? It's turning HRESULTs into exceptions. (More on why that's a bad idea later.)

Unfortunately, it's a macro. Here's some code that I once wrote using this macro, from the aforementioned persistence code. I was adding the ability to save the state in a new format. Does there appear to be anything wrong with it? (LoadOldFileFormat doesn't return an error, it throws exceptions. LoadNewFileFormat returns an HRESULT.)

if (fOldFileFormat)
LoadOldFileFormat(pStorage);
else if (NULL != pNewStream)
CheckHresult(LoadNewFileFormat(pNewStream), (IPersistStream*)this, IID_IPersistStream);
else
// This file is not in any of our formats.
Throw(E_FAIL);

Imagine my surprise to see that the compiler produces a syntax error when attempting to compile this code.

I didn't write that macro, so I had no idea what it did -- I was cargo-cult programming! All the other code that returned error codes called this macro, so I figured I should do so as well. Bad developer! No biscuit!

I ran that through the C preprocessor, and take a look at what popped out: (I've added whitespace and removed some cruft for clarity.)

if (fOldFileFormat)
LoadOldFileFormat(pStorage);
else if (0 != pStream)
{
HRESULT _hr;
if ((_hr=LoadNewFileFormat(pXMLStream)) < 0)
Throw(_hr);
} ;
else
Throw(0x80004005L);

The trailing semi effectively terminates the if and the compiler then balks on the else.

Aaaaaargh! It's drivin' me nuts!

If you're going to have macros, don't write them so that they break C++'s lexical semantics in weird, hard-to-track-down ways. Obviously this wasn't intentional, which just makes my point even stronger -- it is very hard to write correct macros, so don't even try. Don't write macros that look like functions, write actual inline functions and let the compiler take care of optimizing them. Inline functions are guaranteed to not have weird lexical side effects that are not apparent from the text.

In this particular case, we have a macro which represents a statement block but can be used like a statement. The correct thing to do given the definition of this macro is to never terminate it with a semicolon, which looks weird -- no one will be able to remember to do the correct thing!

Next time, more on how mixing C++ exception handling and COM programming drives me nuts.