September, 2008

  • The Old New Thing



    The command interpreter cmd.exe has a concept known as the error level, which is the exit code of the program most recently run. You can test the error level with the IF ERRORLEVEL command:

    IF ERRORLEVEL 1 ECHO error level is 1 or more

    The IF ERRORLEVEL n test succeeds if the error level is n or more. This was presumably because there were programs that expressed different degrees of failure with higher and higher exit codes. For example, the diff program has three exit codes: 0 means the files are the same; 1 means the files are different; 2 means that something terrible happened. There are also programs that use an exit code of zero to mean success and anything else to mean failure.

    In addition to this internal state, you can, if you wish, create an environment variable with the name ERRORLEVEL, in the same way that you can create an environment variable called FRED. But, as with FRED, that variable won't have any effect on the error level.

    rem this next command sets the error level to zero
    CMD /C EXIT 0
    set ERRORLEVEL=1
    if ERRORLEVEL 1 echo Does this print?

    The message is not printed because the ERRORLEVEL environment variable has no effect on the error level. It's just a variable whose name happens to coincide with a command processor concept.

    set BANKBALANCE=$1,000,000.00

    "Hey, when I tried to withdraw the money, I got an insufficient funds error. What am I doing wrong?"

    Now, it does happen to be the case that if command extensions are enabled and you say %ERRORLEVEL%, then the command processor first looks for an environment variable called ERRORLEVEL, and if it can't find one, then it replaces %ERRORLEVEL% with the current value of the internal error level value. It's a fallback step, in the same way that your neighbor is a fallback delivery location if you aren't home. If you file a change-of-address form for yourself, that doesn't affect packages sent to your neighbor.

    The same behavior can be seen with %CD%: If you did not explicitly set an environment variable called CD, then %CD% expands to the command processor's current directory. But you can't change directories by saying set CD=C:\Windows.

    I can think of a few reasons why this feature may have been added.

    • So you can include the error level in a log file:
      ECHO error level is %ERRORLEVEL%>logfile
    • So you can perform other types of tests against the error level, for example, to perform an equality test:
      IF %ERRORLEVEL% EQU 1 echo Different!

    But I'm digressing. My point for today is that the error level is not the same as the ERRORLEVEL environment variable.

  • The Old New Thing

    Tales from the interview: Can you rotate this two-dimensional array?


    My colleague jeffdav told me about one job interview with a graduating college senior that didn't go well because the candidate simply gave up.

    He offered a simple programming task: Write a function that takes a 4×4 two-dimensional array and rotates it clockwise by 90 degrees. For example, the entry in the upper left corner of the array goes to the upper right corner.

    The interview candidate simply gave up without even writing so much as a function prototype. "I can't do it."

    — Okay, well, let's take it a step at a time. Maybe take a specific example and see what we can learn from it.

    "It's some sort of matrix multiplication."

    — All right, well how would you start working that out?

    "I don't know. I can't do it."

    That's really not a good way to demonstrate your problem-solving skills: By responding to a problem with a simple "I can't do it." All of us are faced with things we can't do. The important thing is that we are willing to learn what we need in order to be able to do them.

    I wonder if this particular person thought that attitude would fly at a real job. If your boss gives you a difficult job, would you just say, "I can't do it"? Heck, how do you even graduate from college with that attitude? Your professor gives you an assignment, and you just say, "I can't do it"?

    (The punch line for people who actually know matrix algebra: Matrix multiplication doesn't solve the problem anyway.)

    Bonus commentary: I reprint JeffDav's comment which he posted below, since it is after all his story.

    This was a question reserved for intern candidates and fresh out of college hires. I was usually the first one on the loop and used this as a warm-up question. Once they got it, we'd move on to something more interesting. This one guy just refused to believe it was even possible.

    Also, I would phrase it as "rotate an N x N matrix where N >= 1, and you are given N along with the matrix A." This makes it super easy. If you allow an N x M matrix (i.e. non-square) the question gets much harder.

    I don't ask this question as often anymore. I get bored with asking the same questions over and over. Furthermore, I think after about the 100th time you ask a question you have lost perspective on it. Once you can write the answer on the whiteboard by heart without even thinking, you get annoyed by anyone who takes more than a few seconds thinking about it. It may not be a conscious annoyance but it's there if you think about it, and I think it gives you a bit of a negative bias at times. At least it does for me. It's good to find new questions so you have to solve them yourself and you have that feeling of approaching the problem for the first time fresh in your memory.

  • The Old New Thing

    QueryPerformanceCounter counts elapsed time, not CPU cycles


    An anonymous coward asks whether the QueryPerformanceCounter function counts elapsed time or CPU cycles.

    It counts elapsed time. It has to, since its value is governed by the QueryPerformanceFrequency function, which returns a number specifying the number of units per second, and the frequency is spec'd as not changing while the system is running.

    For CPUs that can run at variable speed, this means that the HAL cannot use an instruction like RDTSC, since that does not correlate with elapsed time. Commenter "A" appears to have found a buggy HAL that failed to take this into account and returns values that do not correlate with elapsed time.

    What would it take to create a counter that was tied to CPU cycles? Well, first, you'd have to come up with some definition of "CPU cycles" that is architecture-neutral. Maybe you'd say that it's a 64-bit value that increments at a rate proportional to the amount of work the CPU has done. And then you have to come up with some sort of definition of what should happen on multi-processor machines. What if you have two CPUs, one of which has gone into a HLT state (not running), while the other is busy doing work? Should the "cycle counter" run at half speed, since only half of the CPUs are running at full speed? What about hyperthreaded processors? It's all so confusing.

    As a final remark, commenter Ulric wanted to know what I meant when I wrote, "Throw in a handful of workarounds for known buggy hardware." What I meant was that the people who write HALs are aware of various types of buggy hardware and added code to detect that buggy hardware and work around the problems.

  • The Old New Thing

    Even if a function doesn't do anything, you still have to call it if the documentation says so, because it might do something tomorrow


    If the documentation says that you have to call a function, then you have to call it. It may be that the function doesn't do anything, but that doesn't prevent it from doing something in the future.

    Today's example is the function GetEnvironmentStrings, which returns you all the environment variables of the current process in a single block, which you can then study at your leisure. When you're finished, you're supposed to call FreeEnvironmentStrings. That's what the documentation says, and if you did that, then you're in good shape.

    However, some people noticed that on Windows NT 4, the Unicode version of the FreeEnvironmentStrings function didn't do anything. In other words, the Unicode environment block didn't need to be freed. When you called GetEnvironmentStrings, the kernel just returned you a raw pointer to the real live environment strings (which, since this is Windows NT, are kept in Unicode internally). Since nothing was allocated, there was nothing to free.

    The problem with this technique was that if somebody called SetEnvironmentVariable in the meantime, the environment block changed out from under the caller of GetEnvironmentStrings.


    To fix this, the GetEnvironmentStrings function was changed to return a copy of the environment block even if you call the Unicode version. The corresponding Unicode FreeEnvironmentStrings function frees that environment copy.

    Programs that followed the specification and called FreeEnvironmentStrings (even though it didn't do anything up until now) were in good shape. Their call to FreeEnvironmentStrings now frees the memory, and all is right with the world.

    Programs that coded to the implementation rather than the specification are now in a world of hurt. If they simply skipped the "useless" call to FreeEnvironmentStrings, they will now find themselves leaking memory. On the other hand, if they gave lip service to FreeEnvironmentStrings by calling it, but using the memory anyway, they will find themselves accessing invalid heap memory, and all sorts of havoc can ensue.

    There's sometimes a reason for the rules that seem stupid at first glance. ("Call this function that doesn't do anything.") Changes to the implementation may make them less stupid in the future.

    (Credit goes to my colleague Neill Clift for providing the information that led to today's article.)

  • The Old New Thing

    The purpose of charts is normally to make information easier, not harder, to understand


    In a presentation a few years ago, there was a pie chart, but not just any pie chart, but a pie chart that appeared to be specifically designed to convey no information whatsoever. (The presenter didn't realize this, of course, and probably thought it was awesomely cool.)

    The pie chart consisted of five wedges, each of which was a slightly different shade of green. No, wait, that's not right. Each wedge was a gradient from a common shade of green to a slightly different shade of green for each wedge. Accompanying the chart was a legend that described what each shade of green represented. It was completely useless. (Inspired by this story, my friend :: Wendy :: created her own monochromatic pie chart.)

    If you're going to make a chart with a legend, then the items labelled by the legend should be colors which are unlikely to be confused with each other. And for goodness' sake, don't make them all subtly different shades of the same base color. (For bonus points, consider the color-blind members of your audience. And for double bonus points, also consider the blind members of your audience. But that's another topic for another day.)

  • The Old New Thing

    I've seen why people steal the foreground window for their dialog box


    Now, it may very well be true that many people who use GetForegroundWindow() as the owner for a dialog box because they don't know any better, but I'm not convinced that everyone who does so did it out of ignorance; I'm sure there's some malice in there, too. Here's how it may have gone down:

    Bug: I start the product setup, and then I go and work on something else, and then setup hits a problem, but the error message can't be seen because it's hidden behind another window.

    Result: Setup fails to complete.

    Expected: Error dialog is visible so user can address the problem and allow setup to complete.

    The programmer who is told to fix the bug, of course, tries to figure out how to fix the bug, and since the bug report said that the error dialog needs to be visible even when other programs are running, the programmer gives a shrug and does what the bug report says should happen.

    And that's how the installation program ends up disabling Task Manager.

    (Commenter Jamie has another very plausible scenario.)

  • The Old New Thing

    When computer programmers dabble in economics: Buying prepaid lunch vouchers


    Several years ago, the Microsoft cafeterias phased out their prepaid lunch voucher system. Under the new system, you "load up" your security badge with money (either by paying cash at any register or by having the amount deducted from your paycheck) and then swipe your badge at card readers installed at each register. But in the old days, the system operated manually.

    Under the old system, you paid $25 cash for a voucher which, when folded in half, was about the size of a business card. On the card were boxes with various denominations ($1, 25¢, 10¢, 5¢), and when you bought something with your voucher card, the person at the register used a pen to X out the various boxes to void them. For example, if you bought something for $3.50, the person at the register would cross off three $1 boxes and two 25¢ boxes. (And since I know people will ask even though it's totally irrelevant to the story: Yes, there were provisions for "making change", for example, if you had four 25¢ boxes remaining and you bought something for 90¢, but they were rarely needed.)

    Yes, this system was easily hacked, but nobody did. Partly because the fact that it was so easy to hack removed the challenge, but mostly because we try not to hire dishonest people.

    When you bought one of these voucher cards, you receive $27 in credit for your $25 prepayment. The security badge payment system doesn't have this "cash bonus" feature, and when the cafeteria people announced that voucher card system was being phased out, one of my colleagues sat down and, making various assumptions on the time value of money, how much money a typical lunch cost, and possibly even the current price of energy on the spot market for all I know, calculated the optimum number of voucher cards to purchase in order to maximize the benefit from the cash bonus.

    As I recall, the calculations determined that seventeen voucher cards was the optimum.

    [Raymond is currently away; this message was pre-recorded.]

  • The Old New Thing

    If you use an absurd signature, I might end up sending it back to you


    Despite my previous rant, absurdly elaborate email signatures are still common at Microsoft, and I'm not just talking about the ones that contain information that may be required by department policy. I'm talking about signatures that use bright colors, large fonts, maybe a bitmap or two, sometimes even a photo of the sender! I will sometimes mention in my reply, "Please consider making your signature less eye-catching. It distracts from the text of your message." But other times, I just incorporate it in to the reply more directly:

    From: John Doe

    Blah blah blah question blah blah blah.

    John Doe
    Technical Liaison

    telephone: 425-555-1212
    mobile: [available upon request]
    Make sure to visit my blog!

    My reply might go like this:

    To: John Doe

    Hi, John Doe.

    Blah blah blah answer blah blah blah.

  • The Old New Thing

    What were ShellExecute hooks designed for?


    Windows 95 introduced (and Windows Vista removed) the concept of ShellExecute hooks. These are objects which implemented the IShellExecuteHook interface. That interface had just one method: IShellExecuteHook::Execute, which took a SHELLEXECUTEINFO structure and returned S_OK to indicate that the item was executed, S_FALSE to allow processing to continue, and an error code to halt processing.

    The intended purpose of the shell execute hook was to allow you to extend the set of strings that can be executed. For example, Internet Explorer 1 used a shell execute hook so that you could type into the Run dialog and invoke the Web browser. This was necessary because the original version of Windows 95 had no special knowledge of URLs. Without the hook, typing a URL into the Run dialog would have generated an error.

    Let's look at those return values again. The simplest return value is S_FALSE, which means that you didn't recognize the item and wish to continue normal processing. Returning S_OK means that you recognized the item and successfully executed it. Returning an error code means that you recognized the item but an error occurred when you tried to execute it. Returning an error stops further processing, which is important in this case, because you do want to stop processing: You recognized what the user was trying to do and you couldn't do it. If you had returned S_FALSE, then the shell (and other shell execute hooks) would take their shot at executing the item. Since it's something only you recognize, they would fail, and the user would get some sort of error like "File not found" instead of an error that accurately describes why you couldn't execute the item.

    That's the theory, but I've seen a few applications that use shell execute hooks for completely different purposes. And they are doomed to failure. One application installs a shell execute hook in order to implement some sort of weird concept of "security". When their hook is called, they check whether the user is "allowed" to run the program; if allowed, they return S_FALSE and if blocked, they return E_ACCESSDENIED. Another application uses their hook to log all the programs that the user runs, presumably so the logs can be inspected for auditing purposes. After logging the action, they return S_FALSE to allow normal execution to continue.

    Both of these methods are really an abuse of the shell execute hook model since they aren't extending the set of items that can be executed by the shell. They're just using the hook model to sneak some code into the execute path. But that's also the problem.

    First, they are intercepting only actions that go through the ShellExecuteEx function. If somebody just calls CreateProcess directly, then the shell execute hooks won't run, and whoever it is just snuck past your "security" and "auditing" system.

    Second, these hooks both assume that they are the only shell execute hook in the system. (Remember, there can be more than one.) Suppose there are three hooks installed, the hook installed by Internet Explorer to handle URLs, the "auditing" hook, and the "security" hook, and suppose that the shell decides to invoke them in that order. (The order in which hooks are executed is unspecified because, if you use them as intended, the order doesn't matter.) If the user types a URL into the Run dialog, the URL hook launches the Web browser and returns S_OK to say, "I took care of this one." Your "auditing" hook never got to see the URL, and your "security" hook never got a chance to reject the operation. Hooks that sit ahead of the "security" hook not only get to bypass security, but they also elude the "auditing" hook.

    So much for "auditing" and "security".


    Since shell execute hooks effectively become part of the ShellExecute function, they are unwittingly subject to all the application compatibility constraints that ShellExecute must follow. Welcome to the operating system. Your world has just gotten a lot more complicated.

    There is no "list of application compatibility constraints" (oh how life would be simple if there were); basically, anything that ShellExecute did in the past is a de facto compatibility constraint. Some programs crash if you create a worker thread inside ShellExecute; other programs crash if you call PeekMessage inside ShellExecute. Some programs call ShellExecute and immediately exit, which means that your shell execute hook had better not return before its work is done (or at least before the work has been given to another process to finish). Some programs call ShellExecute on an unusually small stack, so you have to watch your stack usage. And even though it is called out in the documentation that programs should not call shell functions from inside DllMain, that doesn't stop people from trying, anyway. Your shell execute hook had better behave in a sane manner when called under such "impossible" conditions.

  • The Old New Thing

    Suggestion Box 3, short answers (part 2 of who knows how many)


    Another round of short answers to questions in the suggestion box.

    How does Windows choose the monitor a window maximizes to?

    The window maximizes to the monitor selected by MonitorFromWindow. The algorithm the MonitorFromWindow uses to select the monitor is documented in MSDN.

    How do you make your Win32 application safe from keyloggers?

    You can't. In the battle between kernel mode and user mode, user mode loses. That's why we have a distinction between kernel mode and user mode in the first place.

    Why should you ever call MessageBox from inside a message handler that is re-entrant? "I'm having a hard time explaining to people that it isn't a blocking call."

    I don't even understand the question. And it is a blocking call. The function doesn't return until the user responds to the message box.

    What happens when you pass NULL to ShellExecute, like you did in one of your articles? "Is UI suppressed if I pass NULL?"

    The window handle is used as the parent for any dialogs or messages that the ShellExecute function may need to display. If you pass NULL, then you get the same effect as if you had passed NULL to, say, MessageBox. In the article, the thread has no UI active, so any error messages displayed by the ShellExecute function will appear as top-level unowned windows. I discussed this issue in detail a few years ago. If you want to suppress UI, then pass the SEE_MASK_FLAG_NO_UI flag to ShellExecuteEx.

    Is there some way to use Explorer's filename sort function?

    It's called StrCmpLogical. Michael Kaplan discussed this function two years ago with a follow-up last year. Of course, if you want to mimic Explorer exactly, you also need to respect the REST_NOSTRCMPLOGICAL policy.

    Is there a way to sleep for less than one millisecond?

    I don't know either. I've never needed to do that. (I try to get eight hours each night.)

    Why are notification icons limited to 16 colors in Windows 95 and 2000?

    They were limited to 16 colors on Windows 95 to conserve memory. Windows 2000 inherited that code and by default, each version of Windows works the same as the previous one. There are millions of lines of code in the shell. It's not like somebody goes through every single one of them and says, "Gosh, should we change this line in the next version of Windows?"

    Does Raymond use _alloca?

    Nope. It wasn't in K&R C, so it's not part of my world view. And incautious use of _alloca can result in security vulnerabilities.

    Windows can only handle 64 threads.

    This statement is patently false. We've seen that even without taking any special precautions, we were able to pack about 2000 threads into a process before running out of address space. What the person probably meant to write is that 32-bit Windows supports a maximum of 32 processors. The reason is that the functions that manage sets of processors (such as SetThreadAffinity) use a 32-bit value to represent processor masks. Note that for 64-bit Windows, these masks were expanded to 64-bit integers, so 64-bit Windows supports up to 64 processors in principle.

    How does that Temporary Internet Files thing work?

    It's a shell namespace extension junction point.

    Why are some count parameters declared as signed integers?

    I don't think there's a profound reason for it. Each API designer is empowered to decide how their functions will work. After all, the original strlen function returned a signed integer, too. You might want to ask Brian Kernighan; he was doing it before Windows. (Signed integers do have the slight advantage of being resilient to integer underflow. If a and b are both non-negative integers, then a - b will never underflow.)

    Why does the desktop lose track of icons, so it has to refresh them?

    I'm not sure what you mean by "lose track of icons". Maybe you're asking about lost change notifications (in which case redirected folders can cause problems with lost network notifications). Or maybe you're talking about icon repositioning (maybe the previous icon locations weren't saved).

    Questions about DLL imports, exports, and cross-module memory

    I inadvertently answered this question in a rather long series of articles on DLL imports and exports, and a discussion of cross-module memory allocation.

    I want to see your blog stats.

    I thought that too, until I saw them. When you get over a million hits a month, a list of all the referrals is just a giant pile of noise. I haven't bothered analyzing the referrals because I have other things to do in my spare time that are more interesting.

    I'd love to see a series of things that are obvious to you but not obvious to everyone.

    How do I know what is obvious to me and not obvious to everyone?

    The next category is, of course, the people who ask questions on things that I listed as topics I am not inclined to cover.

    I have a problem (describes problem).

    I don't think your problem really is of general interest. But it's clear that you're not respecting the modality of the window.

    I'm trying to improve the performance of my specific scenario.

    This doesn't strike me as a topic of general interest.

    What are your thoughts on this research project? What are your thoughts on this Microsoft product?

    I think you confused me with Robert Scoble.

    I have a problem (describes problem).

    This doesn't strike me as a topic of general interest.

    Why does Internet Explorer do X?

    Internet Explorer is explicitly on the list of things I am unlikely to cover.

    Or the people who ask questions I've already answered or questions I've chosen to answer elsewhere.

    Help me modify files that I didn't write.

    Doing this breaks servicing.

    What is the long pause after listing a directory the first time?
    Answered in A brief and incomplete history of FAT32.

    And then there are the questions I can't or choose not to answer.

    Why does the window manager force windows fully on-screen when the workstation is unlocked?

    I don't know either. It makes sense to force windows on-screen after a change in resolution, but if the resolution didn't change between the lock and the unlock, there's really no need to go around "fixing up" window positions.

    What did Apple copy from Microsoft and vice versa?

    I'm not going to touch this one. I don't see any upside to my answering it, only downside, and I don't welcome the flamewar that will doubtless ensue.

    Do you know anything about this?


    Something about defragmenting.

    I have no special knowledge about defragmentation. Try The Filing Cabinet. They answered a few questions in one blog entry and even have a Defrag FAQ. Personally, I've been happy with Dave Whitney's defragmenter to defragment specifically-targeted files. (I don't defragment my entire drive because it seems like a waste of time.)

    A question about Aero glass

    I have no special knowledge about Aero glass.

    How about an under-the-hood look at the Windows Vista Start menu?

    I didn't work on the Windows Vista Start menu, so I don't know how it works.

    Do you have any insights into the evolution of WinHelp?

    Sorry, I'm an outsider just like you when it comes to help technologies.

    Long rambling question about ACCESS_DENIED

    I quickly lost interest.

    Please explain the subtleties of the ScrollWindowEx function when scrolling child windows.

    I don't know the answer and I don't feel like doing the research necessary to find out. Sorry. The fact that I left it unanswered from the previous suggestion box should have been a clue.

    That's all for this year.

Page 1 of 4 (33 items) 1234