Testing Compiler Diagnostics...and a Cool IDE Tip!

Testing Compiler Diagnostics...and a Cool IDE Tip!

  • Comments 11

Hello everyone, this is Jamie Eckman.  The last time I posted I was an SDET (tester) on the Libraries team.  I’ve since moved to the Compiler team, still as an SDET.  One thing we see a lot of on both teams is compiler diagnostics: warnings and errors.

One of our Libraries developers, Stephan T. Lavavej, wrote this last year on one of our internal discussion lists: Figuring out the mapping from “what the compiler is complaining about” to “what’s actually wrong with my code, if anything” is like 75% of learning how to program. Depending on how you learned to program, 75% may or may not be an overstatement, but the point is clear.  Everyone writes bad code sooner or later, and in the case of new programmers it’s sooner (and often).  For this reason it’s important that the errors the compiler emits are correct and clear enough to diagnose the problem.  Clarity is something we’re making a significant investment into improving over the next few releases of Visual Studio.  For now I’ll mostly stick to talking about correctness.

When I was still on the Libraries team the only direct contact with diagnostics I had was some maintenance of the ATL Server attributes error tests.  But for the most part I looked at compiler errors from the end-user perspective:  This test I wrote doesn’t compile, why?  Now that I’m on the Compiler team, which owns the bulk of the error text you’re likely to see while writing C or C++ code, I’ve had a great opportunity to look at up close how our diagnostics work and how we test them.

The compiler has on the order of 1,000 errors and warnings it can emit.  We test these like you might expect, each warning or error has one or more test cases associated with it.  The test cases verify that the diagnostic that’s expected is actually emitted and is emitted on the correct line.  Of course, the compiler is a hugely complex mechanism and problems are bound to slip through the cracks.

A colleague on our UE team (User Education, responsible for the content on MSDN, among other things) recently came to me with a small list of code snippets from the C/C++ Build Errors section of MSDN that weren’t producing the promised errors.  These snippets are pretty useful when you’re trying to figure out what a particular error or warning means.  Oftentimes the simplified code in the snippet can help you spot the problem in your own code.

So, Tim from the UE team asked me to look at these snippets that weren’t working and figure out what was wrong.  A compiler bug, an expected change we forgot to tell UE about, or something else?  As an example, here’s the first one I looked at:

// C2062_b.cpp

// compile with: /c

class DataSent {};


struct MyDataController {

   MyDataController(int age, DataSet* j) {}   // C2062

   // try the following line instead

   // MyDataController(int age, DataSent* j) {}


This is a simple error, although perhaps not easy to spot.  The class name “DataSent” has been mistyped “DataSet”.  The expected error is C2062, “type ‘DataSet’ unexpected”.  The error that actually gets emitted (with a recent Orcas compiler) is C2061, “syntax error : identifier ‘DataSet’”.  Here we see that correctness and clarity often have an interesting relationship when it comes to error messages.  Although we didn’t get the error we expected, the error we did get nonetheless points us directly at the problem.  I might be able to make the case that this is a compiler bug, but because the new error is functionally just as good as the old error, it’s unlikely to be fixed at this stage in the Orcas release cycle.

Now, here’s a tip that has absolutely nothing to do with the rest of this post.  Some of you may already know this, but I haven’t seen it posted on vcblog yet.  I recently started working in a new codebase with a pretty strict set of coding conventions.  One of the conventions is that nothing should go past column 79.  It’s pretty annoying to watch the column display in the status bar every time you get close to the end of a line, but fortunately there’s a nifty hidden feature of Visual Studio that can help.  Column guides.  This involves editing the registry, so the standard disclaimer applies: Modify the registry at your own risk!

Find this key in your registry:

HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\9.0\Text Editor

For Visual Studio 2005 you’ll want to use ���8.0” instead of “9.0”.  Create a new String Value in the Text Editor key and call it “Guides”.  Modify the data for Guides and add something like this:

RGB(0, 128, 0) 79

This will put a green line at column 79.  You can have multiple guides:

RGB(0, 128, 0) 79, RGB(128, 0, 0) 49

Make sure you restart Visual Studio after you add these values for the changes to take effect.


- Jamie

  • Very nice trick! In machines without Visual Assist this is one of the most missed features to me.

    How is that this is not documented or integrated in the Options menu?

    Thanks for the trick. :)

  • Would like to know too why it's not in the options menu in VS2005 *and* Orcas?

  • Do you think that a convention that nothing should go past column 79 is still appropriate in 2007?

    Most developers have 19" or larger screens today and probably don't look at their code in a 80x25 dos box.

    Surely the columns should be limited in the coding styles but more to a number like 100.

    Often a statement is longer than 79 characters, do you break the statement into multiple lines than? Does that make the code more readable?

    Could you share the coding styles with us?

    P.S.: If a line does not fit on the screen, just use a smaller font ;)

  • >> Surely the columns should be limited in the coding styles but more to a number like 100..

    << We use 98 for many years - it works well with all commonly accepted tab widths: 2,3,4,8. Anyway, the bottom line surely stays: whatever the number is, it is very benefitial to have it declared and keep wathing for the right margin.

  • > Everyone writes bad code sooner or later, and

    > in the case of new programmers it’s sooner

    > (and often).

    Everyone writes bad code sooner AND later, and in the case of all programmers it's sooner and often.

    The difference comes in changing the bad code to good.  Bad programmers change bad code to other bad code and leave it bad.  Good programmers change bad code to other bad code and then change it to good code.  Notice that this extra step occurs during development time instead of debugging time, so it slows down development and it doesn't win a prize during debugging.  Bad programming has its rewards.

    > I might be able to make the case that this is

    > a compiler bug

    Either that or a documentation bug.  If I were to call a situation like this, I'd probably change the documentation to say that a diagnostic is produced but not specify what the diagnostic is.  The actual diagnostic is clear enough, and modified documentation would match it.

  • Regarding the errors and warnings of the compiler, how come that the warnings are not ordered by their line number?

    This is the output I just got from VC6:

    x.cpp(144) : warning C4101: '__todo_*' : unreferenced local variable

    x.cpp(390) : warning C4101: '__todo_*' : unreferenced local variable

    x.cpp(308) : warning C4101: '__todo_*' : unreferenced local variable

    x.cpp(383) : warning C4101: '__todo_*' : unreferenced local variable

    x.cpp(541) : warning C4101: '__todo_*' : unreferenced local variable

    x.cpp(668) : warning C4101: '__todo_*' : unreferenced local variable

    Has this behaviour changed in VS2005 or Orcas?

  • Why isn't this in a menu somewhere?  Well, looking at our bug database, it's because this is an unsupported feature to be completed at some unspecified point in the future.  That bug is from Whidbey, though, so I'll open a new one and see if they say the same thing. :)

    Is 79 columns too short?  Maybe a little bit.  It's mostly an issue when I'm writing output diagnostics (if a test fails, say why the test failed) because these strings usually have lots of embedded data with all the gory details of the failure, so they tend to grow quite large with all the scope resolution for the variables and whatnot.  But I'd run into this problem with any reasonable column restriction, so I don't stress out about it.

    I'm not sure how much detail I can go into about the specific coding style we're using on this particular codebase, but it covers everything from identifier naming to the alignment of comments for member variable declarations.  The idea is to have a very uniformly styled codebase so that future generations of compiler devs don't go insane trying to read it.

    Errors and warnings are emitted from the compiler in the order that they're encountered.  This is usually top-down, but things like templates can complicate matters.  However, the IDE's Error List (Ctrl+\, Ctrl+E) can be sorted by line number.

  • Thanks for the trick, it is useful.

    Just one thing, the green line appear in output window as well, is there any workaround to this?

  • Unfortunately, I don't know of a way to apply the guides only to certain editor windows, so you may be stuck with them in the output pane.

  • I tried the column guide feature on Visual Studio 2005 and it seems that having multiple guides works, but they cannot be different colors.  When you make the key value something like:

    RGB(0, 128, 0) 79, RGB(128, 0, 0) 49

    Both of the lines will be the color set by the last entry.

    I'm not trying to complain or request it be fixed.  Just information dissemination.

  • >>  If I were to call a situation like this, I'd probably change the documentation to say that a diagnostic is produced but not specify what the diagnostic is.  <<

    The problem is that the examples are part of the diagnostics documentation - the whole point of them is to help explain the diagnostic.

    So, you look up a diagnostic and see an example that says it might produce this diagnostic, but might produce another one instead? That's probably not too helpful.

Page 1 of 1 (11 items)