• The Old New Thing

    Raymond's highly scientific predictions for the 2011 NCAA men's basketball tournament


    Once again, it's time for Raymond to come up with an absurd, arbitrary criterion for filling out his NCAA bracket.

    This year, I look at the strength of the school's football team, on the theory that a school with a strong football team and a strong basketball team has clearly invested a lot in its athletics program. My ranking of football teams is about as scientific as my ranking of basketball teams:

    • If the school ended last season with a BCS ranking, I used that.
    • If a school wasn't ranked but received votes in the AP ranking, then I gave it a rank of 30 (and if two such schools faced each other, I looked at who got more votes).
    • If a school still isn't ranked, then I looked to see if it had been ranked at any time earlier in the season; if so, then I gave it a rank of 40.
    • If a school still isn't ranked, but it appeared on the equally-scientific ESPN Fan Rankings, then I gave it a rank of 50.
    • If a school still isn't ranked, but it has a Division I FBS football team, then I gave it a rank of 80. If two such schools faced each other, then I gave what appeared to be the weaker school a rank of 90.
    • If a school still isn't ranked, but it has a Division I FCS football team, then I gave it a rank of 100. If two such schools faced each other, then I gave what appeared to be the weaker schools a rank of 101. (Why 101 instead of 110? Who cares!)
    • If a school still isn't ranked, but it has a football team in some other division, then I gave it a rank of 150.
    • If a school still isn't ranked because its football team is new, then I gave it a rank of 200.
    • If a school still isn't ranked because it doesn't have a football team, but it had one in the past, then I gave it a rank of 300.
    • If a school still isn't ranked because it never had a football team, then I gave it a rank of 400.

    (As a special case, USC received its rank of 22 from two years ago, because it was forced to sit out the 2010 season as part of its punishment for "several major rules violations." Now that's what I call dedication to athletics!)

    I made up all these rules on the fly, which is why the spacing is so uneven and why they were not necessarily applied fairly across the board, but that's what makes it highly scientific.

    As before, once the field has been narrowed to eight teams, the results are determined by a coin flip.


    • Correct predictions are in green.
    • Incorrect predictions are in red.
    • (!) marks upsets correctly predicted.
    • (*) marks upsets predicted but did not take place.
    • (x) marks actual upsets not predicted.

    Opening Round Games

    Texas-San Antonio(200) Alabama State
    Alabama State(80)
    UAB(90) Clemson
    UNC-Asheville(400) Arkansas-Little Rock
    Arkansas-Little Rock(300)
    USC(22*) USC

    East bracket

    1Ohio State(6) Ohio State
    Ohio State
    Ohio State Ohio State
    16Alabama State(80)
    8George Mason(400) Villanova
    (100) (*)
    5Kentucky(80) Kentucky
    West Virginia
    (30) (*)
    4West Virginia(30) West Virginia
    6Syracuse(80) Syracuse
    (80) (x)
    11Indiana State(90)
    3Xavier(300) Xavier
    (300) (x)
    7Washington(30) Washington
    2North Carolina(50) North Carolina
    15Long Island(400)

    West bracket

    1Duke(90) Duke
    (80) (x)
    Arizona Arizona
    8Michigan(80) Michigan
    5Texas(40) Texas
    4Arizona(40) Arizona
    6Connecticut(30) Connecticut
    (12) (*)
    3Cincinnati(80) Missouri
    (12) (*)
    7Temple(80) Penn State
    (40) (*)
    San Diego State
    10Penn State(40)
    2San Diego State(30) San Diego State
    15Northern Colorado(200)

    Southeast bracket

    1Pittsburgh(80) Pittsburgh
    (80) (x)
    Wisconsin Michigan State
    16Arkansas-Little Rock(300)
    8Butler(100) Butler
    9Old Dominion(101)
    5Wisconsin(4) Wisconsin
    4Kansas State(40) Kansas State
    13Utah State(80)
    6BYU(39) BYU
    Michigan State
    3St. John's(200) St. John's
    (200) (x)
    7UCLA(80) Michigan State
    (7) (*)
    Michigan State
    10Michigan State(7)
    2Florida(30) Florida

    Southwest bracket

    1Kansas(90) Kansas
    (80) (*)
    Illinois Texas A&M
    16Boston University(300)
    8UNLV(90) Illinois
    (80) (!)
    5Louisville(82) Louisville
    (82) (x)
    12Morehead State(100)
    4Vanderbilt(90) Vanderbilt
    (90) (x)
    6Purdue(90) Purdue
    Texas A&M
    11Saint Peter's(300)
    3Georgetown(100) USC
    7Texas A&M(18) Texas A&M
    (18) (x)
    Texas A&M
    10Florida State(23)
    2Notre Dame(30) Notre Dame


    Ohio State Ohio State Michigan State
    Michigan State Michigan State
    Texas A&M
  • The Old New Thing

    Why can't Explorer decide what size a file is?


    If you open Explorer and highlight a file whose size is a few kilobytes, you can find some file sizes where the Explorer Size column shows a size different from the value shown in the Details pane. What's the deal? Why can't Explorer decide what size a file is?

    The two displays use different algorithms.

    The values in the Size column are always given in kilobytes, regardless of the actual file size. File is 15 bytes? Show it in kilobytes. File is 2 gigabytes? Show it in kilobytes.

    The value shown in the Size column is rounded up to the nearest kilobyte. Your 15-byte file shows up as 1KB. This has been the behavior since Explorer was first introduced back in Windows 95, Why? I don't know; the reasons may have been lost to the mists of time. Though I suspect one of the reasons is that you don't want a file to show up as 0KB unless it really is an empty file.

    On the other hand, the value shown in the Details pane uses adaptive units: For a tiny file, it'll show bytes, but for a large file, it'll show megabytes or gigabytes or whatever. And the value is shown to three significant digits.

    The result is that a file which is, say, 19465 bytes in size (19.0088 kilobytes) shows up in the Size column as 20KB, since the Size column rounds up. On the other hand, the Details pane shows 19.0KB since it displays the value to three significant digits.

    It looks like Explorer can't make up its mind, and perhaps it can't, but the reason is that the two places on the screen which show the size round in different ways.

  • The Old New Thing

    The old DEBUG program can load COM files bigger than 64KB, but that doesn't mean they actually load as a program


    Some times ago, I described why a corrupted binary sometimes results in the error "Program too big to fit in memory". Commenter Neil was under the impression that nonrelocatable programs files could be larger than 64KB and used the DEBUG command to verify this assertion.

    While it's true that DEBUG can load files bigger than 64KB, that doesn't mean that they will load as a program. If DEBUG decide that you didn't give it a program (the file extension is not EXE or COM),¹ then it treats the file on the command line as a data file and loads it into memory in its entirety, provided it fits in memory in its entirety. When it does this, the BX register contains the upper 16 bits of the file size, and CX contains the lower 16 bits. This is also the format that is used when writing files back out: Use the n command to set the name of the output file and set BX:CX to the file size.

    Even though DEBUG has been obsolete for over a decade, it is still useful for exactly this purpose: You can use it as a hex editor for files less than around 512KB.

    But don't deceive yourself into thinking that you created a COM file that is bigger than 64KB.

    ¹There is another extension which has special meaning to DEBUG, but it's not relevant to the discussion.

  • The Old New Thing

    Why does my TIME_ZONE_INFORMATION have the wrong DST cutover date?


    Public Service Announcement: Daylight Saving Time begins in most parts of the United States this weekend. Other parts of the world may change on a different day from the United States.

    A customer reported that they were getting incorrect values from the GetTimeZoneInformationForYear function.

    I have a program that calls GetTimeZoneInformationForYear, and it looks like it's returning incorrect DST transition dates. For example, GetTimeZoneInformationForYear(2010, NULL, &tzi) is returning March 2nd as the tzi.DaylightDate value, instead of the Expected March 14th date. The current time zone is Pacific Time.

    The value returned by GetTimeZoneInformationForYear (and GetTimeZoneInformation) is correct; you're just reading it wrong.

    As called out in the documentation for the TIME_ZONE_INFORMATION structure, the wDay field in the StandardDate and DaylightDate changes meaning depending on whether the wYear is zero or nonzero.

    If the wYear is nonzero, then the wDay has its usual meaning.

    But if the wYear is zero (and it is for most time zones), then the wDay encodes the week number of the cutover rather than the day number.

    In other words, that 2 does not mean "March 2nd". It means "the second week in March".

  • The Old New Thing

    How do I create a topmost window that is never covered by other topmost windows?


    We already know that you can't create a window that is always on top, even in the presence of other windows marked always-on-top. An application of the What if two programs did this? rule demonstrates that it's not possible, because whatever trick you use to be on-top-of-always-on-top, another program can use the same trick, and now you have two on-top-of-always-on-top windows, and what happens?

    A customer who failed to understand this principle asked for a way to establish their window as "super-awesome topmost". They even discovered the answer to the "and what happens?" rhetorical question posed above.

    We have overridden the OnLostFocus and OnPaint methods to re-assert the TopLevel and TopMost window properties, as well as calling BringToFront and Activate. The result is that our application and other applications end up fighting back and forth because both applications are applying similar logic. We tried installing a global hook and swallowing paint and focus events for all applications aside from our own (thereby preventing the other applications from having the opportunity to take TopMost ahead of us), but we found that this causes the other applications to crash. We're thinking of setting a timer and re-asserting TopMost when the timer fires. Is there a better way?

    This is like saying, "Sometimes I'm in a hurry, and I want to make sure I am the next person to get served at the deli counter. To do this, I find whoever has the lowest number, knock them unconscious, and steal their ticket. But sometimes somebody else comes in who's also in a hurry. That person knocks me unconscious and steals my ticket. My plan is to set my watch alarm to wake me up periodically, and each time it wakes me up, I find the person with the lowest number, knock them unconscious, and steal their ticket. Is there a better way?"

    The better way is to stop knocking each other unconscious and stealing each other's tickets.

    The customer (via the customer liaison) provided context for their question.

    This is not a general-purpose application. This application will be run on dedicated machines which operate giant monitors in retail stores. There are already other applications running on the computer which rotate through advertisements and other display information.

    The customer is writing another application which will also run on the machine. Most of the time, the application does nothing, but every so often, their application needs to come to the front and display its message, regardless of whatever the other applications are displaying. (For example, there might be a limited-time in-store promotion that needs to appear on top of the regular advertisements.)

    Unfortunately, all of these different programs were written by different vendors, and there is no coordination among them for who gets control of the screen. We were hoping that there was some way we could mark our window as "super topmost" so that when it came into conflict with another application running on the machine, it would win and the other application would lose.

    I'm thinking of recommending that the vendors all come up with some way of coordinating access to the screen so they can negotiate among themselves and not get into focus fights. (Easier said than done, since all the different applications running on the machine come from different vendors...)

    Since there is no coordination among the various applications, you're basically stuck playing a game of walls and ladders, hoping that your ladder is taller than everybody else's wall. The customer has pretty much found the tallest ladder which the window manager provides. There is no "super topmost" flag.

    Sure, you can try moving to another level of the system, like say creating a custom desktop, but all that does is give you a taller ladder. And then one of the other applications is going to say, "I need to display a store-wide page (manager to the deli please, manager to the deli), overriding all other messages, even if it's a limited-time in-store promotion." And they'll try something nastier, like enumerating all the windows in the system and calling ShowWindow(SW_HIDE).

    And then another application will say, "I need to display an important store-wide security announcement (Will the owner of a white Honda Civic, license plate 037-MSN, please return to your vehicle), overriding all other messages, even if it's a store-wide page." And it'll try something nastier, like setting their program as the screen saver, disabling the mouse and keyboard devices, and then invoking the screen saver on the secure desktop.

    And then another application will say, "I need to display a critical store-wide announcement (Fire in the automotive department. Everybody evacuate the building immediately), overriding all other messages, even if it's an important store-wide security announcement." And it'll try something nastier, like enumerating all the processes on the system, attaching to each one with debug privilege, and suspending all the threads.

    Stop the madness. The only sane way out is to have the programs coöperate to determine who is in control of the screen at any particular time.

    In response to my hypothetical game of walls and ladders, one of my colleagues wrote, "Note to self: Do not get into a walls-and-ladders contest with Raymond."

  • The Old New Thing

    How to rescue a broken stack trace: Recovering the EBP chain


    When debugging, you may find that the stack trace falls apart:

    ChildEBP RetAddr
    001af118 773806a0 ntdll!KiFastSystemCallRet
    001af11c 7735b18c ntdll!ZwWaitForSingleObject+0xc
    001af180 7735b071 ntdll!RtlpWaitOnCriticalSection+0x154
    001af1a8 2f6db1a9 ntdll!RtlEnterCriticalSection+0x152
    001af1b4 2fe8d533 ABC!CCriticalSection::Lock+0x12
    001af1d0 2fe8d56a ABC!CMessageList::Lock+0x24
    001af234 2f6e47ac ABC!CMessageWindow::UpdateMessageList+0x231
    001af274 2f6f040e ABC!CMessageWindow::UpdateContents+0x84
    001af28c 2f6e4474 ABC!CMessageWindow::Refresh+0x1a8
    001af360 2f6e4359 ABC!CMessageWindow::OnChar+0x4c
    001af384 761a1a10 ABC!CMessageWindow::WndProc+0xb31
    00000000 00000000 USER32!GetMessageW+0x6e

    This can't possible be the complete stack. I mean, where's the thread procedure? That should be at the start of the stack for any thread.

    What happened is that the EBP chain got broken, and the debugger can't walk the stack any further. If the code was compiled with frame pointer optimization (FPO), then the compiler will not create EBP frames, permitting it to use EBP as a general purpose register instead. This is great for optimization, but it causes trouble for the debugger when it tries to take a stack trace through code compiled with FPO for which it does not have the necessary information to decode these types of stacks.

    Begin digression: Traditionally, every function began with the sequence

            push ebp      ;; save caller's EBP
            mov ebp, esp  ;; set our EBP to point to this "frame"
            sub esp, n    ;; reserve space for local variables

    and ended with

            mov esp, ebp  ;; discard local variables
            pop ebp       ;; recover caller's EBP
            ret n

    This pattern is so common that the x86 has dedicated instructions for it. The ENTER n,0 instruction does the push / mov / sub, and the LEAVE instruction does the mov / pop. (In C/C++, the value after the comma is always zero.)

    if you look at what this does to the stack, you see that this establishes a linked list of what are called EBP frames. Suppose you have the following code fragment:

    void Top(int a, int b)
     int toplocal = b + 5;
     Middle(a, local);
    void Middle(int c, int d)
    void Bottom(int e)
     int bottomlocal1, bottomlocal2;

    When execution reaches the ... inside function Bottom the stack looks like the following. (I put higher addresses at the top; the stack grows downward. I also assume that the calling convention is __stdcall and that the code is compiled with absolutely no optimization.)

    Top's stack frame  
    0040F8F8 parameter b passed to Top During execution of Top,
    EBP = 0040F8EC
    0040F8F4 parameter a passed to Top
    0040F8F0 return address of Top's caller
    0040F8EC EBP of Top's caller
    0040F8E8 toplocal
    Middle's stack frame  
    0040F8E4 parameter d passed to Middle During execution of Middle,
    EBP = 0040F8D8
    0040F8E0 parameter c passed to Middle
    0040F8DC return address of Middle's caller
    0040F8D8 0040F8EC = EBP of Middle's caller
    Bottom's stack frame  
    0040F8D4 parameter e passed to Bottom During execution of Bottom,
    EBP = 0040F8CC
    0040F8D0 return address of Bottom's caller
    0040F8CC 0040F8D8 = EBP of Bottom's caller
    0040F8C8 bottomlocal1
    0040F8C4 bottomlocal2

    Each stack frame is identified by the EBP value which the function uses during its execution.

    The structure of each stack frame is therefore

    [ebp+n]Offsets greater than 4 access parameters
    [ebp+4]Offset 4 is the return address
    [ebp+0]Zero offset accesses caller's EBP
    [ebp-n]Negative offsets access locals

    And the stack frames are all connected to each other in the form of a linked list threaded through the EBP values. This linked list is known as the EBP chain. End digression.

    To recover from the broken EBP chain, start dumping the stack a little before things go bad (in this case, I would start at 001af384-80) and then look for something that looks like a valid stack frame. Since the parameters and locals to a function can be pretty much anything, all you have left to work with is the EBP and the return address. In other words, you are looking for pairs of values of the form

    «pointer a little higher up the stack».
    «code address»

    In this case, I got lucky and didn't have to go very far:

      001af474  00000000
     -001af478  001af494
    / 001af47c  14f4fba8 DEF!SubclassBase::CallOriginalWndProc+0x1a
    | 001af480  2f6e4317 ABC!CMessageWindow::WndProc
    | 001af484  00970338
    | 001af488  0000000f
    | 001af48c  00000000
    \ 001af490  00000000
     >001af494  001af4f0
      001af498  14f4fcd6 DEF!SubclassBase::ForwardMessage+0x23
      001af49c  00970338
      001af4a0  0000000f
      001af4a4  00000000
      001af4a8  00000000
      001af4ac  00000000
      001af4b0  2f6e4317 ABC!CMessageWindow::WndProc
      001af4b4  ed758311
      001af4b8  00000000
      001af4bc  15143f70
      001af4c0  00000000
      001af4c4  14f4fb8e DEF!CView::SortItems+0x96
      001af4c8  00000000
      001af4cc  2f6e4317 ABC!CMessageWindow::WndProc
      001af4d0  00000000

    At stack address 001af478, we have a pointer to memory higher up the stack followed by a code address. if you follow that pointer, it points to another instance of the same pattern: A pointer higher up the stack followed by the code address.

    Once you find where the EBP chain resumes, you can ask the debugger to resume its stack trace from that point with the =n option to the k command.

    0:000> k=001af478
    ChildEBP RetAddr
    001af478 14f4fba8 ntdll!KiFastSystemCallRet
    001af494 14f4fcd6 DEF!SubclassBase::CallOriginalWndProc+0x1a
    001af4f0 14f4fc8b DEF!SubclassBase::ForwardMessage+0x23
    001af514 14f32dd1 DEF!SubclassBase::ForwardChar+0x59
    001af530 14f4fcd6 DEF!SubclassBase::OnChar+0x3c
    001af58c 14f4fd76 DEF!HelpSubclass::WndProc+0x51
    001af5e4 761a1a10 DEF!SubclassBase::s_WndProc+0x1b
    001af610 761a1ae8 USER32!GetMessageW+0x6e
    001af688 761a1c03 USER32!GetMessageW+0x146
    001af6e4 761a3656 USER32!GetMessageW+0x261
    001af70c 77380e6e USER32!OffsetRect+0x4d
    001af784 761a2a98 ntdll!KiUserCallbackDispatcher+0x2e
    001af794 698fd0aa USER32!DispatchMessageW+0xf
    001af7a4 2f7bf15c ABC!CThread::DispatchMessageW+0x23
    001af7e0 2f7befc9 ABC!CMessageWindow::MessageLoop+0x3a2
    001af808 2ff56d20 ABC!CMessageWindow::ThreadProc+0x9f
    001af898 75c2384b ABC!CMessageWindow::s_ThreadProc+0x10
    001af8a4 7735a9bd kernel32!BaseThreadInitThunk+0x12
    001af8e4 00000000 ntdll!LdrInitializeThunk+0x4d

    When you do this, make sure to ignore the first line of the resumed stack trace, since that is based on your current EIP, not the return address stored in the stack frame.

    Today was really just a warm-up for another debugging technique that I haven't finished writing up yet, so you're just going to be in suspense for another two years or so, though if you attended my TechEd China talk, you already know where I'm going.

    Bonus reading: In Ryan Mangipano's two-part series on kernel mode stack overflows, the second part does a bit of EBP chain chasing. (Feel free to read the first part, as well as earlier discussion on the subject of stack overflows.)

  • The Old New Thing

    Microspeak: Cadence


    Originally, the term cadence meant the rate at which a regular event recurs, possibly with variations, but with an overall cycle that repeats. For example, the cadence for team meetings might be "Every Monday, with a longer meeting on the last meeting of each month."

    Project X is on a six-month release cadence, whereas Project Y takes two to three years between releases.

    Q: What was the cadence of email requests you sent out to drive contributions?

    A: We started with an announcement in September, with two follow-up messages in the next month.

    In what I suspect is a case of I want to use this cool word other people are using, even though I don't know exactly what it means, the term has been applied more broadly to mean schedule or timeline, even for nonrecurring events. Sample usage: "What is our cadence for making this available outside the United States?"

  • The Old New Thing

    What's the difference between FreeResource and, say, DestroyAcceleratorTable


    MaxMax asks a number of resource-related questions, starting with "How do you Unlock a Lock­Resource?" and culminating in "What are the differences between Free­Resource and Destroy­Accelerator­Table, Destroy­Object, etc.? It would be much easier to use a single function instead of a collection of five."

    It helps if you understand the history of resources, because the functions were designed back when resources were managed very differently from how they are today. The usage pattern is still the same:

    • Find­Resource
    • Load­Resource
    • Lock­Resource
    • use the resource
    • Unlock­Resource
    • Free­Resource

    You unlock a resource by calling, um, Unlock­Resource.

    Although the usage pattern is the same, the mechanism under the covers is completely different. In 16-bit Windows, loading a resource entailed allocating a chunk of memory, then filling that memory block from the disk image. In Win32, resources are mapped into the address space as part of the image; there is no memory allocation and no explicit loading.

    The next thing to understand is that resources are just blobs of binary data. They are not live objects. It's not like there's a HBITMAP sitting in there just waiting to be found.

    Think of resource data as a set of blueprints. If you call Find­Resource + Load­Resource + Lock­Resource, you wind up with the blueprints for a radio, but that's not the same as actually having a radio. To do that, you need to hand the radio blueprints to somebody who knows how to read electronic schematic diagrams and who knows how to solder wires together in order to convert the potential radio into an actual radio.

    If you've been following the sporadic series on the format of resources, you'll know that these schematic diagrams can often be quite complicated. The Load­Bitmap function first does the Find­Resource + Load­Resource + Lock­Resource dance to locate the bitmap blueprint, but then it needs to actually make the bitmap, which is done by parsing the raw resource data and trying to make sense of it, calling functions like Create­Bitmap and Set­DI­Bits to convert the blueprint into an actual bitmap.

    That's why, if you use these helper functions like Load­Accelerators to convert the blueprint into an object, you need to use the corresponding cleanup function like Destroy­Accelerator­Table when you want to destroy the object. You have to use the correct cleanup function, of course. You can't destroy a bitmap with Destroy­Accelerator­Table any more than you can put a radio in the clothing drop bin.

    Just like when the radio guy returns the original blueprints plus a brand new radio, you return the blueprints to the library, but if you want to destroy the radio, you have to take it to the electronics recycling facility.

  • The Old New Thing

    News flash: Companies change their product to appeal to their customers


    There was some apparent uproar because there was an industry which "changed the flavoring of their product depending on which market segment they were trying to appeal to."

    Well duh, don't all industries do this?

    The reason why this even remotely qualified as news didn't appear until the last five words of the article!

  • The Old New Thing

    The window manager needs a message pump in order to call you back unexpectedly


    There are a bunch of different ways of asking the window manager to call you when something interesting happens. Some of them are are in response to things that you explicitly asked for right now. The enumeration functions are classic examples of this. If you call EnumWindows and pass a callback function, then that callback is called directly from the enumerator.

    On the other hand, there is a much larger class of things that are in response either to things that happen on another thread, or in response to things that happen on your thread, but not as a direct result of an immediate request. For example, if you use the SendMessageCallback function, and the window manager needs to trigger your callback, the window manager needs a foot in the door of your thread in order to get control. It can't just interrupt code arbitrarily; that way lies madness. So we're looking for some way the window manager can regain control of the CPU at a time when the program is in a stable, re-entrant state.

    That foot in the door for the window manager is the message pump. That's the one component that the window manager can have some confidence that the program is going to call into periodically. This solves the first problem: How do I get control of the CPU.

    Furthermore, it's a known quantity for programs that when you call GetMessage or PeekMessage, incoming sent messages are dispatched, so your program had better be in a stable, re-entrant state when you call those functions. That solves the second problem: How do I get control of the CPU when the program is in a stable state?

    Take-away: When you register a callback with the window manager, you need to pump messages. Otherwise, the window manager has no way of calling you back.

    Related: The alertable wait is the non-GUI analog to pumping messages.

Page 121 of 439 (4,382 items) «119120121122123»