• The Old New Thing

    We've traced the call and it's coming from inside the house: Grid lines in list view report mode


    A customer wanted to know how to remove the grid lines from a list view control in report mode. The customer was kind enough to include the source code for the relevant part of the program and drew our attention to the line in the resource file that he believed to be the source of the problem:


    The customer didn't know it, but that line in the resource file was of no help at all in diagnosing the problem. Fortunately, we found the root cause in the source code provided:

    void CListPage::OnInitDialog()
      ListView_SetExtendedListViewStyleEx( hCtrl, 0,
         LVS_EX_LABELTIP |

    The grid lines are there because you explicitly asked for them!

    The customer accepted this answer without response. One of my colleagues suspected how this situation came about:

    If I may guess, the initial developer of the code did what the spec required at the time, which was to have grid lines. The initial developer then left the company, and a new developer inherited the code.

    Now the design team changed their mind and asked for the grid lines to be removed, but the new developer doesn't know the history of the code and therefore doesn't know where in the code the grid lines are turned on or even know that the default state for grid lines is off and that you have to turn them on explicitly.

    Chances are within the next two months, the design team will change their mind again and the developer will have to put the grid lines back, but at least this time he knows how to do it.

  • The Old New Thing

    Kindergarten writing exercise from my niece


    When my niece was in kindergarten, a regular classroom assignment was for students to write a few sentences about something happening in their lives. For one of the assignments, my niece wrote the following:

    My auntie has a baby in her tumy. If it is a boy I will call him Kevin. If it is a grl I will call her Alula.

    We have no idea where the name "Alula" came from.

    The baby turned out to be a girl, but her aunt chose a different name.

    My niece did not hold up her end of the bargain and call her new cousin "Alula".

    Update: Upon further reflection, I think the proposed boy's name was "Derik", not Kevin. Not important to the story, but included for completeness.

  • The Old New Thing

    It rather involved being on the other side of this airtight hatchway: Invalid parameters from one security level crashing code at the same security level


    In the category of dubious security vulnerability, I submit the following (paraphrased) report:

    I have discovered that if you call the XYZ function (whose first parameter is supposed to be a pointer to a IUnknown), and instead of passing a valid COM object pointer, you pass a pointer to a random hunk of data, you can trigger an access violation in the XYZ function which is exploitable by putting specially-crafted data in that memory blob. An attacker can exploit the XYZ function for remote execution and compromise the system, provided an application uses the XYZ function and passes a pointer to untrusted data as the first parameter instead of a valid IUnknown pointer. Although we have not found an application which uses the XYZ in this way, the function neverless contains the potential for exploit, and the bug should be fixed as soon as possible.
    The person included a sample program which went something like this (except more complicated):
    // We can control the behavior by tweaking the value
    // of the Exploit array.
    unsigned char Exploit[] = "\x01\x02\x03...";
    void main()

    Well, yeah, but you're already on the other side of the airtight hatchway. Instead of building up a complicated blob of memory with exactly the right format, just write your bad IUnknown:

    void Pwnz0r()
      ... whatever you want ...
    class Exploit : public IUnknown
      STDMETHODIMP QueryInterface(REFIID riid, void **ppv)
      { Pwnz0r(); return E_NOINTERFACE; }
      STDMETHODIMP_(ULONG) AddRef() { Pwnz0r(); return 2; }
      STDMETHODIMP_(ULONG) Release() { Pwnz0r(); return 1; }
    void main()

    Wow, this new "exploit" is even portable to other architectures!

    Actually, now that you're on the other side of the airtight hatchway, you may as well take XYZ out of the picture since it's just slowing you down:

    void main()

    You're already running code. It's not surprising that you can run code.

    There's nothing subtle going on here. There is no elevation of privilege because the rogue activity happens in user-mode code, based on rogue code provided by an executable with trusted code execution privileges, at the same security level as the original executable.

    The people reporting the alleged vulnerability do say that they haven't yet found any program that calls the XYZ function with untrusted data, but even if they did, that would be a data handling bug in the application itself: Data crossed a trust boundary without proper validation. It's like saying "There is a security vulnerability in the DeleteFile function because it is possible for an application to pass an untrusted file name and thereby result in an attacker deleting any file of his choosing." Even if such a vulnerability existed, the flaw is in the application for not validating its input, not in DeleteFile for, um, deleting the file it was told to delete.

    The sad thing is that it took the security team five days to resolve this issue, because even though it looks like a slam dunk, the issue resolution process must be followed, just to be sure. Who knows, maybe there really is a bug in the XYZ function's use of the first parameter that would result in elevation of privilege. All supported versions of Windows need to be examined for the slim possibility that there's something behind this confused vulnerability report.

    But there isn't. It's just another dubious security vulnerability report.

    Exercise: Apply what you learned to this security vulnerability report. This is also paraphrased from an actual security report:

    There is a serious denial-of-service vulnerability in the function XYZ. This function takes a pointer to a buffer and a length. If the function is passed malformed parameters, it may encounter an access violation when it tries to read from an invalid buffer. Any application which calls this function with bad parameters will crash. Here is a sample program that illustrates the vulnerability:

    int main(int argc, char **argv)
     // crash inside XYZ attempting to read past end of buffer
     XYZ("x", 9999999);
     return 0;

    Credit for discovering this vulnerability goes to ABC Security Research Company. Copyright© 20xx ABC Security Research Company. All Rights Reserved.

  • The Old New Thing

    Creative naming in pursuit of subverting the no-fun zone


    For a time, the Information Technology department at Microsoft cracked down on what it believed to be frivolous mailing lists. All mailing lists inside the company had to have a valid business purpose.

    The nascent wireless networking team found a way to circumvent this rule: They created a mailing list for discussion of non-business topics but officially said that it was for discussing Wireless Networking Interference.

  • The Old New Thing

    TrackMouseEvent tracks mouse events in your window, but only if the events belong to your window


    Greg Williams wonders why Track­Mouse­Event fails to detect mouse hover events when responding to Do­Drag­Drop callbacks. "My suspicion is that Do­Drag­Drop monopolizes the window so that a WM_MOUSEHOVER message is never posted, so it won't end up being useful."

    That's basically it, for the appropriate sense of the word "monopolize."

    The Track­Mouse­Event monitors mouse events that take place in your window and notifies your window when events of interest occur. But this requires that the events actually take place in your window!

    The Do­Drag­Drop function calls Set­Capture so that it can carry out the task of following the mouse anywhere on the screen. Recall that mouse events normally are delivered to the window beneath the mouse, but Set­Capture lets you say, "No, I want all mouse events to go to me for as long as the mouse button is held down. Mine! All mine!" This function is typically called when you are performing some sort of mouse drag operation so that the window can respond to mouse events even after the use has dragged the mouse out of the window. (Which, in many cases, is the only interesting case.)

    That's why Track­Mouse­Event has no effect when you try to use it to detect mouse hovering during a drag/drop operation: The Track­Mouse­Event function is not seeing any mouse events! They're all being stolen by Do­Drag­Drop.

    The unfortunate consequence of this is that if you want to have a special behavior during drag/drop hover, you'll have to detect the hover manually by remembering the mouse position and timestamp and waiting for the appropriate amount of time to elapse without a significant mouse motion.

    Exercise: But wait, since I don't get drag/drop events when the mouse is not inside my window, how can I simulate WM_MOUSELEAVE?

  • The Old New Thing

    ZOMG! This program is using 100% CPU!1! Think of the puppies!!11!!1!1!eleven


    For some reason, people treat a program consuming 100% CPU as if it were unrepentantly running around kicking defenseless (and cute) puppies. Calm down already. I get the impression that people view the CPU usage column in Task Manager not as a diagnostic tool but as a way of counting how many puppies a program kicks per second.

    While a program that consumes 100% CPU continuously (even when putatively idle) might legitimately be viewed as an unrepentant puppy-kicker, a program that consumes 100% CPU in pursuit of actually accomplishing something is hardly scorn-worthy; indeed it should be commended for efficiency!

    Think of it this way: Imagine if your CPU usage never exceed 50%. You just overpaid for your computer; you're only using half of it. A task which could have been done in five minutes now takes ten. Your media player drops some frames out of your DVD playback, but that's okay, because your precious CPU meter never went all the way to the top. (Notice that the CPU meter does not turn red when CPU usage exceeds 80%. There is no "danger zone" here.)

    Consider this comment where somebody described that they want their program to use less CPU but get the job done reasonably quickly. Why do you want it to use less CPU? The statement makes the implicit assumption that using less CPU is more important than getting work done as fast as possible.

    You have a crowd of people at the bank and only ten tellers. If you let all the people into the lobby at once, well, then all the tellers will be busy—you will have 100% teller utilization. These people seem to think it would be better to keep all the customers waiting outside the bank and only let them into the lobby five at a time in order to keep teller utilization at 50%.

    If it were done when 'tis done, then 'twere well / It were done quickly.

    Rip off the band-aid.

    Piss or get off the pot.

    Just do it.

    If you're going to go to the trouble of taking the CPU out of a low-power state, you may as well make full use of it. Otherwise, you're the person who buys a bottle of water, drinks half of it, then throws away the other half "because I'm thinking of the environment and reducing my water consumption." You're making the battery drain for double the usual length of time, halving the laptop's run time because you're trying to "conserve CPU."

    If the task you are interested in is a low priority one, then set your thread priority to below-normal so it only consumes CPU time when there are no foreground tasks demanding CPU.

    If you want your task to complete even when there are other foreground tasks active, then leave your task's priority at the normal level. Yes, this means that it will compete with other foreground tasks for CPU, but you just said that's what you want. If you want it to compete "but not too hard", you can sprinkle some Sleep(0) calls into your code to release your time slice before it naturally expires. If there are other foreground tasks, then you will let them run; if there aren't, then the Sleep will return immediately and your task will continue to run at full speed.

    And cheerfully watch that CPU usage go all the way to 100% while your task is running. Just make sure it drops back to zero when your task is complete. You don't want to be a task which consumes 100% CPU even when there's nothing going on. That'd just be kicking puppies.

    [Raymond is currently away; this message was pre-recorded.]

    Clarification: Many people appear to be missing the point. So let's put it more simply: Suppose you have an algorithm that takes 5 CPU-seconds to complete. Should you use 100% CPU for 5 seconds or 50% CPU for 10 seconds? (Obviously, if you can refine your algorithm so it requires only 2 CPU-seconds, that's even better, but that's unrelated to the issue here.)

  • The Old New Thing

    The alignment declaration specifier is in bytes, not bits


    Explicit object alignment is not something most people worry about when writing code, which means that when you decide to worry about it, you may be a bit rusty on how the declarations work. (After all, if it's something you worried about all the time, then you wouldn't have trouble remembering how to do it!)

    I was looking at some customer code, and there was a class who had a data member with an explicit alignment declaration.

    class Whatever {
        __declspec(align(32)) LONG m_lSomething; // Must be DWORD-aligned to make writes atomic

    I pointed out that the comment didn't match the code. The comment says that the variable needs to be DWORD-aligned (which in Windows-speak means aligned on a 32-bit boundary), but the code aligns it on a 32-byte boundary, eight times as generous as required. On the other hand, maybe they really did want the member aligned on a 32-byte boundary (say to put it on its own cache line).

    Turns out that in this case, the comment was correct and the code was wrong. To force a variable to align on a DWORD boundary, you want to say __declspec(align(4)). Save yourself a bunch of unnecessary padding bytes.

    But in fact, in this case, the customer was simply trying too hard. The code was compiled with default alignment, which aligns integer types on their natural boundaries anyway. The compiler was going to align the variable even if you didn't specify anything.

    [Raymond is currently away; this message was pre-recorded.]

  • The Old New Thing

    I will be speaking at TechEd China 2010 today


    As I've mentioned a few times by now, the way to get me to show up at your event is to invite me. The easiest (i.e. cheapest) way is to hold your event in the Seattle area so that my travel expenses are effectively zero; I just use my bus pass. The folks at TechEd China 2010, on the other hand, had to fly me all the way out to Beijing, which is where I've been this week.

    Preparing these talks is a lot of work, because each one is different. It's not like I have a "stock presentation" that I give over and over. So even if you invite me, I may decline because I simply don't have the time to write up a new presentation for your event, and if your event is outside the Seattle area, I may decline because the travel would take me away for too long. (The TechEd China folks actually invited me last year, but I already had accepted an out-of-town invitation and couldn't really afford two out-of-town presentations in one year.)

    Anyway, if you can read Chinese, then you can check out the TechEd China 2010 Web site — nobody told me that crossed arms was the official speaker pose of TechEd China 2010 — my speaker biography and the talk description. I can't read Chinese, so I'm trusting the translators to have done a good job.

    I've found that a good way to salvage content that I've deleted from the queue (because the topic draws out flame comments) is to use them in talks and presentations instead. It turns out that the people who actually show up for a talk are much more polite than those who just post snarky comments from their mother's basement. I think the fact that you have to be snarky in person removes the veil of anonymity and makes you more accountable for what you say.

    And since I know you're going to ask:

    • I'm giving the talk in English.
    • I don't know whether the talk is going to be recorded.
    • The talk is being given on December 2, even though this entry is dated December 1. That's because it will already be December 2 in China.

    [Raymond is currently away, but he wrote this entry just now.]

  • The Old New Thing

    How do I delete bytes from the beginning of a file?


    It's easy to append bytes to the end of a file: Just open it for writing, seek to the end, and start writing. It's also easy to delete bytes from the end of a file: Seek to the point where you want the file to be truncated and call SetEndOfFile. But how do you delete bytes from the beginning of a file?

    You can't, but you sort of can, even though you can't.

    The underlying abstract model for storage of file contents is in the form of a chunk of bytes, each indexed by the file offset. The reason appending bytes and truncating bytes is so easy is that doing so doesn't alter the file offsets of any other bytes in the file. If a file has ten bytes and you append one more, the offsets of the first ten bytes stay the same. On the other hand, deleting bytes from the front or middle of a file means that all the bytes that came after the deleted bytes need to "slide down" to close up the space. And there is no "slide down" file system function.

    One reason for the absence of a "slide down" function is that disk storage is typically not byte-granular. Storage on disk is done in units known as sectors, a typical sector size being 512 bytes. And the storage for a file is allocated in units of sectors, which we'll call storage chunks for lack of a better term. For example, a 5000-byte file occupies ten sectors of storage. The first 512 bytes go in sector 0, the next 512 bytes go in sector 1, and so on, until the last 392 bytes go into sector 9, with the last 120 bytes of sector 9 lying unused. (There are exceptions to this general principle, but they are not important to the discussion, so there's no point bringing them up.)

    To append ten bytes to this file, the file system can just store them after the last byte of the existing contents. leaving 110 bytes of unused space instead of 120. Similarly, to truncate those ten bytes back off, the logical file size can be set back to 110, and the extra ten bytes are "forgotten."

    In theory, a file system could support truncating an integral number of storage chunks off the front of the file by updating its internal bookkeeping about file contents without having to move data physically around the disk. But in practice, no popular file system implements this, because, as it turns out, the demand for the feature isn't high enough to warrant the extra complexity. (Remember: Minus 100 points.)

    But what's this "you sort of can" tease? Answer: Sparse files.

    You can use an NTFS sparse file to decommit the storage for the data at the start of the file, effectively "deleting" it. What you've really done is set the bytes to logical zeroes, and if there are any whole storage chunks in that range, they can be decommitted and don't occupy any physical space on the drive. (If somebody tries to read from decommitted storage chunks, they just get zeroes.)

    For example, consider a 1MB file on a disk that uses 64KB storage chunks. If you decide to decommit the first 96KB of the file, the first storage chunk of the file will be returned to the drive's free space, and the first 32KB of the second storage chunk will be set to zero. You've effectively "deleted" the first 96KB of data off the front of the file, but the file offsets haven't changed. The byte at offset 98,304 is still at offset 98,304 and did not move to offset zero.

    Now, a minor addition to the file system would get you that magic "deletion from the front of the file": Associated with each file would be a 64-bit value representing the logical byte zero of the file. For example, after you decommitted the first 96KB of the file above, the logical byte zero would be 98,304, and all file offset calculations on the file would be biased by 98,304 to convert from logical offsets to physical offsets. For example, when you asked to see byte 10, you would actually get byte 98314.

    So why not just do this? The minus 100 points rule applies. There are a lot of details that need to be worked out.

    For example, suppose somebody has opened the file and seeked to file position 102,400. Next, you attempt to delete 98,304 bytes from the front of the file. What happens to that other file pointer? One possibility is that the file pointer offset stays at 102,400, and now it points to the byte that used to be at offset 200,704. This can result in quite a bit of confusion, especially if that file handle was being written to: The program writing to the handle issued two consecutive write operations, and the results ended up 96KB apart! You can imagine the exciting data corruption scenarios that would result from this.

    Okay, well another possibility is that the file pointer offset moves by the number of bytes you deleted from the front of the file, so the file handle that was at 102,400 now shifts to file position 4096. That preserves the consecutive read and consecutive write patterns but it completely messes up another popular pattern:

    off_t oldPos = ftell(fp);
    fseek(fp, newPos, SEEK_SET);
    ... do stuff ...
    fseek(fp, oldPos, SEEK_SET); // restore original position

    If bytes are deleted from the front of the file during the do stuff portion of the code, the attempt to restore the original position will restore the wrong original position since it didn't take the deletion into account.

    And this discussion still completely ignores the issue of file locking. If a region of the file has been locked, what happens when you delete bytes from the front of the file?

    If you really like this simulate deleting from the front of the file by decommitting bytes from the front and applying an offset to future file operations technique, you can do it yourself. Just keep track of the magic offset and apply it to all your file operations. And I suspect the fact that you can simulate the operation yourself is a major reason why the feature doesn't exist: Time and effort is better-spent adding features that applications couldn't simulate on their own.

    [Raymond is currently away; this message was pre-recorded.]

  • The Old New Thing

    Microspeak: Take-away


    At Microsoft, the take-away is the essential message of a presentation or the conclusion that you are expected to draw from a situation. It is something you are expected to remember when the whole thing is over, a piece of information you take away with you as you leave the room.

    XYZ demo take away (title of a document)

    The preferred intensifier is key, and you probably see it attached to the phrase take-away more often than not. This example comes from a presentation on the results of a user study:

    Results: XYZ Tough to Use
    • ...
    • Key take-away:
      • Migration to XYZ will be difficult
      • Need to show value of using the power of DEF

    In fact, every single slide in this presentation had a bullet point at the bottom called Key take-away. (And, as you may have noticed, the heading is the singular take-away even though multiple take-aways were listed.)

    Another use of the term take-away follows in the same spirit as the "essential message" usage, but the idea of "taking away" is taken literally: A take-away is a small information card that sales and marketing people give to potential customers. Think of it as the business card for a service rather than for a person.

    [Raymond is currently away; this message was pre-recorded.]

Page 127 of 436 (4,356 items) «125126127128129»