Writing Code On Whiteboards Is Hard

Writing Code On Whiteboards Is Hard

  • Comments 87

Work has been crazy lately and I haven't had much time to work on SimpleScript.  It's a lot of work starting from scratch! Right now I'm writing a templatized hash table for the binders and other lookup tables.  I haven't written templatized C++ for a loooong time and its slow going.  I hope to have the named item logic done over the weekend, so that I can explain how the module system works next week.

Until then, here's an article I wrote a while back on interviewing that I never got around to putting up on the blog before now.

Writing Code On Whiteboards Is Hard

As I've mentioned before,  I occasionally interview candidates for development positions on my team and occasionally other Visual Studio teams.  Plenty of people have written plenty of web pages on interviewing at Microsoft, so I won't rehash the whole story here.  What I wanted to mention today was some words of advice for candidates for development positions. This is by no means complete -- I want to concentrate on one important aspect of the interview.

Dev candidates: if you've done any reading at all, you know that most of your interviews will involve writing some code on a whiteboard.  A word of advice: writing code on whiteboards is HARD.  Practice!

It's hard in part because you don't have intellisense or syntax colouring or any of the other tools at your disposal that you normally do when writing code.  I know that, and I'll take that into account.  I don't care if you write memset(&b, cb, 0x00) when you mean memset(&b, 0x00, cb) -- I don't remember that stuff either.  I don't care if you forget semis or make other syntactical gaffes.  I'm not looking for people who can spit out syntactically perfect code at the drop of a hat; that's not the point of the coding exercise at all.   I'm trying to find out how you solve problems.  Do you:

  • rush right in and start coding without thinking the problem through first? 
  • find areas where the problem is ambiguous and clarify them, or do you make a bunch of assumptions?
  • break the problem down into pieces?
  • do the easy pieces, paint yourself into a corner, and attempt to handwave your way out of the situation, or work on the hard stuff first?
  • have any confidence that your solution is correct?
  • do anything to prove to yourself and me that it is correct?

It would be wise to make it easy for me to see that you're a well-organized person with good problem solving skills.  Leave lots of space -- you might need to stick in some extra lines.  Write slowly and clearly.  Explain what you're doing as you do it.  If the code is clear, well-organized, legible, and at least vaguely syntactically correct, it will be a lot easier on both of us to tell whether the code is well designed and bug free.  Walk through some simple test cases -- don't just dash down some code and say “yep, that's correct!“

The vast majority of the coding problems people pose do not require any "aha!" insights or rocket-science algorithms.  No one is going to ask you to implement a 4-5 Runge-Kutta differential equation solver.  We're going to ask you to implement a hash table or replace every instance of "foo" in a string with "bar". Eric Carter has a copy of Schaum's Programming With C++ in his office because that thing is a gold mine for interview questions.  Got a technical interview coming up?  Pick up a copy of any introductory programming text, pick a few problems at random, and solve them on a whiteboard.  Heck, I'll flip through it at random right now.  Here are some sample problems taken from various places in the book:

  • implement a method which determines how many characters the string pointer must be incremented to point to the trailing null.
  • implement the less-than operator for a class representing rational numbers as (numerator / denominator) pairs of integers
  • implement function that takes an array of integers and returns the maximum, minimum and average.
  • implement a template for generating stack classes
  • implement a program that shuffles an array into a random order

Any of those would be a highly typical coding question.   And before you say "that's easy!" about any of them, think about what's missing from those problem statements before you write the code.

That string pointer: What encoding is it pointing to? 7-bit ASCII chars, UTF-8, UTF-16, some ANSI encoding?  Are you going to ask the interviewer, or are you going to assume that you're writing C code for some operating system that existed in the Before Time when there were no strings that contained Chinese characters?  Hint:  Microsoft writes very little C code targetting PDP-11 machines.  Ditto for programs that never have to deal with non-Roman character sets.

That less-than operator: Do you have to handle nonreduced fractions like 2/4 ?  Illegal fractions like 2/0?  Can you assume anything about the size of the integers?

That array you're shuffling: do we care if a hacker can predict the sequence? Do we need crypto-strength randomness, or reproducible pseudo-randomness?  Is this for an online poker game for real money or a test case for a sort algorithm?

And so on.  None of these problems is well-defined enough that I'd feel comfortable writing production quality code without at least some clarification.

Candidates often make the mistake of concentrating on the code, like the code exists in a vacuum for its own sake.  That's a very unrealistic assumption!  Code not only has to be well engineered and correct, it has to be maintainable, testable, and solve a real customer problem.  In your interview, when you're done writing the code, think about:

  • how would a tester attack it?  Is the design even testable?
  • does it handle stuff that hostile/buggy callers are going to throw at it?  null pointers, large denominators, huge arrays? 
  • does it work well with other technologies?  that is, does it use the conventions of COM or ATL, or does it work against them?
  • is it correct, robust, maintainable, debuggable, portable, extensible? 
  • how would you know whether this was the code the customer wanted or not?

Now, this is not to say that someone who automatically thinks char * when they hear “string“ is an automatic no-hire.  But I am a lot more inclined to believe that “experienced in COM programming“ on your resume if you acknowledge the existence of BSTRs!  Also, I recognize that fresh-out-of-school candidates often have very little experience with these sorts of “real world“ considerations; I cut them some slack in those areas and concentrate on their raw intellectual horsepower, coding talent and long-term potential.

Finally, let me reiterate that technical interviews are hard, and even bright people screw up on them sometimes.  Two teams no-hired me when I interviewed for full-time positions here!  But that's another story.

  • I find all this stuff v. interesting. Thanks.

    One of the things that intrigues me though, as an experienced (12 years+) developer is being asked to code 'common' algorithms (like quicksort) on the spot. A good portion of the MS coding questions I've seen seem to revolve around this. Surely the fact that these algorithms are so common, or rather, well known, means that they can be 'remembered' in which case this isn't a test of your coding and reasoning per se, but more on your memory ability (I would guess that's why you're more interested in whether people recognise BSTRs etc). For any standard algorithm, I (hopefully) tend to recognise that it's been written a thousand times before so I can just look it up.

    The skill then becomes recognising a good one and reusing it. The best way to improve your code is to study others' - not write as much as you humanly can. I think this is a good test of an experienced programmer. The college grad will try and write everything (I think that's what people do when they start anything) whereas the experienced coder will be aiming for much higher quality & productivity IMHO.
  • So, to get to my point, I would guess that your comment to Mr NotSelected above is an indication that just 'cos you can't recall an implementation of every classic CS algorithm (on demand, in an interview) doesn't 'prove' you're not any good?! I hope this is right as I would hate to judge people myself on such narrow criteria.
  • Coming up with good coding questions for candidates isn't easy (thus I appreciate Eric's mention of Schaum's Programming With C++). At two companies I worked for we used a written test to filter candidates before the technical interview. I wrote a significant portion of these tests, and had a hard time coming up with good questions.

    The test I finally came up with had two segments:
    1. Data structures/algorithms where qsort-like questions were asked (though nothing as common).
    2. Technical know-how questions about specific fields (interviewee could pick area of expertise from a list of Win32, communication, C++, Java, ...)

    In both sections I tried to give the candidates the ability to choose questions, say 4 out of 6. This is because I appreciate that a candidate can have a "mental block".

    I can't say the tests were perfect, but they weren't meant to be. All they were meant to do was save us time.

    Here is one example of an algorithmic/coding question: come up with the fastest possible code that returns the number of "on" bits in an 8 bit number.
  • > I would guess that your comment to
    > Mr NotSelected above is an
    > indication that just 'cos you
    > can't recall ... every classic CS
    > algorithm ... doesn't 'prove'
    > you're not any good?!

    The comment was _intended_ to ask a question -- the candidate thinks that he was no-hired because of something and I don't understand what he thinks that something is.

    However, I would certainly agree with the general point of your question. I certainly couldn't tell you off the top of my head what Dijkstra's Algorithm for finding the shortest path in a graph is. I don't expect that of candidates. What I do expect is (a) a basic knowledge of fruitful problem solving tools -- recursion, depth-first-search, all the stuff that should be in any experienced programmer's box of tools, and (b) ability to apply those tools to a simple problem.

    That's why we ask a mix of straight-up computer-science textbook problems and more offbeat problems. People who have "stock" solutions memorized might have a slight advantage, but its really not hard to vary a problem slightly to see if the candidate has any flexibility. (And also, people who spend a lot of time preparing, understanding algorithms, etc, SHOULD have an advantage, no?)

    I personally like asking problems that appear to be offbeat, but actually are stock computer-sciency problems at a deeper level. Candidates who can look at a problem, ask questions, clarify it, think about it for a bit, explore around, try out some designs, and deduce the deeper structure of the problem are likely to be strong hires.
  • > For any standard algorithm ... it's been written a thousand times before so I can just look it up.

    I take your point, but two things come to mind. First, an expert is, according to my working definition, "someone who doesn't need to look up answers to easy questions."

    Second, on the other hand, I _do_ appreciate and give full credit to the "I'd look it up" answer. Recognizing when a problem has already been solved and thereby knowing how to use the work of others to further your own is a useful skill.

    However, the questions that I ask are technically at a low enough level that you shouldn't have to look it up.
  • > Have you ever considered using a computer for technical interview coding questions?

    It's crossed my mind. But like a commenter said above, what that ends up testing is ability to use particular tools.

    Anyone can learn how to use our build system, our compilers, editors, etc. That's easy, so I don't want to test that. I want to test something that's hard to learn: general problem solving skills.

    Also, while I'm at it, I want to point out that yes, writing code on a whiteboard is in a sense "unnatural" but it is also a skill I use all the time in my job. We often get together in meetings and end up writing scraps of code on whiteboards to brainstorm algorithms for various problems. If it gets too complex we find a computer of course, but when its four devs in a room everyone can grab a different colour marker and go nuts writing stuff up and annotating other people's efforts. Four devs in a room with one computer, it's a lot harder.

    Interviews are often very much like those ad hoc code design sessions.
  • Dan: that was an excellent idea to give two types of questions.

    Of course software developers must be able to memorize a lot of information. If you are specialized in C++ then you better know the syntax and the in's and out's of ever aspect of object oriented programming (without a doubt).

    When a programmer says he specializes in a certain language, I assume that he knows those things (stated above). But for me, a programmer does not just have to be a text-book coder, but the ability to think OUT-SIDE-THE-BOX!

    This is a very crucial aspect of programming now days. People want innovation, not a repeat of some other companies program at a lower price.

    Microsoft has shown that they want programmers with imagination. This is an excellent quality found in a lot of programmers, but unfortunately some programmers just do not have that quality.

    And that’s something recruiters cannot see from a piece of paper.


  • "Thinking outside the box" is certainly a useful skill. But too often I see it overemphasized at the expense of the more useful "mastery of the stuff inside the box"!

    We're not in that box because we're a bunch of bozos, we're in the box because by and large, the box is a good place to be.

    Which is why the questions I ask in interviews are not "aha!" questions, but rather realistic questions that can be solved by straightforward application of good design and coding principles.

    I can't stand those brain teasers about manhole covers and weighing airplanes. Not only do they tell me nothing about whether the candidate can code, often the canonical answers are ridiculous.
  • I saw on Eric Lippert's site his post called Writing Code On Whiteboards Is Hard . I have mixed feelings about that style of interview. I've used the code-on-paper as well as the code-on-whiteboard style, occasionally combined with mathematical puzzles....
  • FWIW, my handwriting is not something you want to deal with. On top of that, I'm left-handed, so my handwriting on a whiteboard is smudged even where it happens to be legible.

    I understand all of the reasons you don't want to test someone's skill at using a particular development environment, but an interviewer would learn a whole lot more from sitting me in front of Notepad than standing me in front of a whiteboard. Plus, they'd be able to read it, too.
  • > It's crossed my mind. But like a commenter said above, what that ends up testing is ability to use particular tools.

    Which is exactly what I want to test for in anyone whom I hire, even if they are fresh out of college.

    If I ask a candidate to implement something for me (I'm in java-land) and point her at a dev machine that has various environments available, I'm certainly going to notice if she opens up notepad or gvim vs. eclipse or intellij. I'm certainly going to notice if she can quickly find/configure junit so she can start writing testcases. If she can have a red-bar testcase up and running in intellij within 5 minutes of hearing the problem...well, she's probably going to be a good developer. But one shouldn't make absolute judgements based on limited evidence. :)

    All of which runs I guess runs counter to some of the earlier comments. Coding is a spectator sport: though I want the spectating to happen on a screen...not a whiteboard.
  • ...And I think an even better coding test is giving the candidate a group of red-bar tests and having him make them green; OR giving him a long method with some obvious refactorings and wathching him discover them while learning what the method does.
  • thanks for your comments Eric, you've set my mind at rest. A lot of what's been written here really boils down to identifying talents. Obviously, having a good memory is an essential talent. Active questioning and thinking clearly (esp. under pressure) also are. I completely agree with you that one of the *key* talents is how flexible a person is in applying and bending what they *know* to any particular problem you ask them to solve.

    I would also say that I think whiteboarding is an essential skill. I don't care whether I'm writing code or drawing pictures on a board; in a room full of people it's one of the best ways to show people what you're talking about and to collaborate.

    BTW I hate most brain-teasers too.
  • John Lindsey: If what you're looking for are experienced people who can be immediately productive in your environment, then clearly it's a good idea to test for that in an interview!

    But that's not what _I'm_ looking for. I care very little about _particular_ skills and very rarely am I looking for someone who needs to be firing on all cylinders cranking out code tomorrow.

    I'm interviewing people for the long term, people who are going to work on a huge number of technologies over their careers. I can afford a couple of weeks to ramp someone up on our toolset.
  • So Eric are you syaing your looking for an adaptablity measure of a persons apptitudes more than specific skill knowledge when interviewing candidates?

    I would say to others who have comment on this topic, offers of employment are always a risky venture and that is why Microsoft is so rigorous and hard during their interviews.

    Nothing worse on an organization than a incorrect hire.

    Eric.
    What specific advice might you have for long time programmers other than what you just given?
Page 3 of 6 (87 items) 12345»