Progressive Development : assertions

Motley says: "Too many assertions make debugging very annoying!"

James Waletzky — Tue, 13 May 2008 18:01:00 GMT

Summary

Motley: Too many assertions make debugging very annoying!

Maven: Lots of assertions are great, as long as they validate the right assumptions. If an assertion fires it is likely manifested by a bug, so get to the root cause and fix it. Best practices for assertions include using them early and often, avoiding executable code within an assertion, and documenting the reason for an assertion.

______________________________

[Context: Motley is having some frustrations while debugging, particularly with components developed by other teams within the company]

Motley: I need to have a talk with the team. Our code has too many debug assertions. They are always firing and it is making debugging very annoying. I constantly have to dismiss the assertion dialog, and when I am executing the same path over and over to debug, it becomes really, really annoying.

Maven: Ah, assertions - one of my favorite topics. I am a huge advocate of defensive programming, and assertions are a great tool. As we have talked about in the past, I like to develop as if the debugger isn't even there. Assertions are mandatory to uphold that way of thinking.

Motley: I still think you're nuts about developing without a debugger. Anyway, assertions have their use, but you can have too many. We do. In this case the large number of assertions is preventing me from getting my work done.

Maven: I am not buying what you are selling. The number of assertions is not the problem. The fact that the assertions are firing on a regular basis IS the problem. In previous teams I have been on, an assertion that is firing that should NOT be firing is considered a priority 1 bug. Think of an assertion as an exception. It is considered a blocking bug from the debugging perspective as you are forcing other developers to continue execution every time they hit it. In addition, assertions validate assumptions, so obviously some assumption is off, which may affect a retail build as well.

Motley: That's true, I guess. I still think we have too many. We probably have one in every 20 lines of source code. And if someone is going to file a blocking bug on me every time they hit my assertion, then I am not sure I want any more than that in the code.

Maven: I would argue that one in 20 may not be enough! Debug assertions are put in place to ensure that you are maintaining a valid state and that your assumptions hold true. Having one in 10 lines of source code may still not be enough. You only see assertions in debug builds, and if the assertions are coded properly, you should never know they are there unless there is a legitimate bug.

Your argument around avoiding blocking bugs is bogus. Code your assertions properly. Give some thought to the conditions. Be thankful if an assertion fires as it is likely exposing a bug in the code. A co-worker gave some great advice around assertions: "If you are assuming anything without verifying it in either the code or an assert, you’re walking a tightrope without a safety net."

Motley: I did learn a little trick for temporarily getting around someone else's assertions when I am debugging without having to rebuild. If there are just a couple that are bothering me, I can disassemble the code around the assertion, look for Int3 which is a debug break, and replace it with no-op byte codes.

Maven: That is very inventive, but make sure you follow up on why the assertion fired in the first place and file a bug as appropriate.

Motley: Another thing to note is that all of this assumes you are running a debug build and not a retail build.

Maven: Absolutely. All of your verification as a developer should be done on debug builds so that you can see the assertions pop up. If you are going to run a retail build but compile only your components in debug, then you may miss some assumptions that were made around your code. You should compile in retail before checking in, but try to do developer validation on a debug build running under the debugger.

Motley: There are ways to misuse assertions too. Regardless of the volume of assertions in source code, here is a list of some stuff I have seen lately in code reviews:

A developer uses an assertion to check an error condition that is expected to happen. I saw an example yesterday where someone could pass in a bad parameter value to a public API and all there was to protect against the error was an assertion. This will work fine in a debug build, but you are in deep trouble in a retail build. The assertion code will be ignored at compile-time and there is no error check!

Maven: Good one! Assertions protect against conditions that should never happen. They are typically used to check internal state of your classes and invariants (refer to our discussion on design by contract). You still need a construct like exceptions to flag errors at run-time in retail builds. This is extremely important in public methods. In private methods you could get away with just assertions if the method is otherwise protected by public accessors.

Motley: Here's another one I saw last week:

A developer put executable code inside an assertion. That code will work in a debug build but not in retail. An example like this is bad news:

Debug.Assert(i++ < 100, "Bounds of 'I' exceeded 100");

Maven: Another good one! If that is taking place in a loop, at retail the loop index will not be incremented. Never put executable code inside a debug assertion. Actually, this is a bit of an argument to also do some testing in retail builds as well. To refine my statement above that developer validation should take place on debug builds only, I'd like to resubmit that you should at least run your unit tests on a retail build as well before check-in. You taught me something Mot!

Motley: You act like that isn't a frequent occurrence. Without me around you would probably be on the fast track to nowhere at this company.

Maven: Yeah… right. But I'll humor you. To finish up, here are a few best practices around assertions:

Use an assertion to protect a switch statement that would not have a valid default case. If you switch on an enumerated type, for example, and someone adds a value to the enumeration and forgot to update the switch, at least you'll find out about it in a debug build provided you assert on the default case.
Assert even on obvious things. An assertion that may seem totally obvious to you may not be obvious to another developer.
Assert early and often. The earlier you can validate an assumption in source code the more efficient you will be at finding the root cause.
Always include an explanatory message with your assertions. Don't just assert without providing justification and/or intention of the condition in the assert. Again, it may be obvious to you but not the next person. What will happen if the assertion is violated?

Motley: You always take a simple concept and blow it all up. I didn't realize there was quite that much thinking behind assertions. They seem like a fairly basic language construct at the high level.

Maven: Yeah, as with many computer science, there may be more than meets the eye. That's what keeps our jobs interesting!

Motley: And overly complicated to the point of making frequent errors producing buggy code and-

Maven: Don't be so negative!

Note: Special thanks to the development leads of the mobile phone and shell teams at Microsoft for their indirect contributions to this blog entry.

______________________________

Maven's Pointer: Think about assertions even at design-time. A design is typically full of assumptions. Document those assumptions at design-time and translate them to source code when implementation begins.

Maven's Resources:

Assertion-Based Design (Information Technology: Transmission, Processing and Storage) , by Harry D. Foster, Adam C. Krolnik, David J. Lacey, Kluwer Academic, ISBN: 1402080271, May 2004.
Code Complete, 2nd Edition, by Steve McConnell, Microsoft Press, ISBN: 0735619670, June 2004.

Motley says: "Rely on the debugger to write solid code"

James Waletzky — Sun, 25 Mar 2007 21:05:00 GMT

Summary

Motley: I rely on the debugger when I am writing code. Without it, I'd have a really hard time tracking down all the bugs in my code.

Maven: Write code as if the debugger does not even exist. Include unit tests, assertions, tracing, logging, static analysis, and internal consistency checks in your code to pinpoint problems as they occur. Time in the debugger is generally not efficient.

______________________________

[Context: Maven interrupts Motley as he is debugging some code]

Maven: Hey Motley. What are you up to?

Motley: Just debugging some code I just wrote. The debugger is every developer's best friend don't you know.

Maven: Actually, when I am writing new code I try and avoid the debugger at all costs. It is my most inefficient time spent as a developer.

Motley: Avoid the debugger?!?! You're insane! How can I expect to get my job done without the debugger. It lets me see the current state of my application and figure out why things went awry. Do you really have any experience in this industry?

Maven: How long does it take you to debug a typical problem using the debugger?

Motley: It depends on the problem of course. On average I would say it takes me about 15 minutes to come up with a decent repro and then another 15 minutes to get to the bottom of the issue. From there the fix is usually just a couple of lines and I'm off and running to the next issue. I suppose now you are going to tell me that your code is bug-free?

Maven: Of course not. Writing bug-free new code is next to impossible. Even "Hello World" typically has bugs. BUT, when I am writing new code I like to think the debugger doesn't even exist.

Motley: Boy, I bet you are really efficient. Can you sense my sarcasm?

Maven: You bet - I AM efficient. By incorporating the right level of unit tests, debug assertions, tracing, logging, and embedded self-tests, I can get to the bottom of an issue in no time. In fact, I can usually nail down the exact lines of code that caused the issue in a few seconds flat.

Motley: Did you say "unit testing"? That's the test team's job! I am not paid to write tests.

Maven: Oh jeez. One thing at a time - we'll have the unit testing conversation next. In the meantime, try inserting some defensive programming techniques into your code, like debug assertions and tracing. Not only will they help you get to root of an issue more quickly, but they will also help the support team once your software ships in the unlikely event a problem occurs. An assertion that fires provides an immediate indication that something is wrong and gives some great information about the error - as long as you take care writing your assertions.

Motley: By assertions you mean something like System.Diagnostics.Debug.Assert in my .NET code, right? I supposed I could sprinkle a few more in there. Including more debug traces would be easy. Unit tests? Not a snowball's chance in a high temperature furnace!

Maven: Okay, at least that's a start. The support team will thank you, and your time spent in the debugger will be minimized. I guarantee that the little bit of extra effort expended to incorporate those things into your code will still give you back lots of the time you would have otherwise spent debugging.

______________________________

Maven's Pointer: Use every tool in your arsenal to tell you about problems in your code before they appear. That involves running tools like Visual Studio's built-in code analysis (FxCop and PREfast) and cranking up compiler warnings to the highest level.

Maven's Resources: Code Complete 2nd Edition, by Steve McConnell, Microsoft Press, ISBN: 0735619670, 2004.