• The Old New Thing

    The extern "C" specifier disables C++ mangling, but that doesn't mean it disables mangling

    • 15 Comments

    The MSDN documentation on dllexport contains the following enigmatic paragraph, or at least did at the time I wrote this article:

    dllexport of a C++ function will expose the function with C++ name mangling. If C++ name mangling is not desired, either use a .def file (EXPORTS keyword) or declare the function as extern "C".

    I've seen this sentence misinterpreted as follows:

    dllexport of a C++ function will expose the function with C++ name mangling. To disable name mangling either use a .def file (EXPORTS keyword) or declare the function as extern "C".

    This is an understandable misinterpretation, but it is still a misinterpretation.

    The root cause of the misinterpretation is that the author of this documentation was wearing C++-colored glasses. In the author's mind, there are only two interesting cases:

    1. C++ name mangling, where all the cool people are, and
    2. everything else, for all the lamers.

    Here is a precise formulation of the paragraph:

    dllexport of a C++ function will expose the function with C++ name mangling. If C++ name mangling is not desired, either use a .def file (EXPORTS keyword), which will expose the name without mangling, or declare the function as extern "C", which will expose the name with C mangling.

    Here's a version of the paragraph that tries to take away the C++-colored glasses.

    dllexport exposes the function as it is decorated by the compiler. For example, if the function is a C++ function, it will be exposed with C++ name mangling. If the function is a C function, or has been declared as extern "C", it will be exposed with C name mangling. To expose the function under its unmangled name (or to expose it via an alternate name), use use a .def file (EXPORTS keyword).

    Behind the scenes: To forestall nitpickers, I had to go back to my copy of the C++ standard to make sure I filled in the blank in "The extern "C" _________" correctly. Officially, extern "C" is a storage class specifier.

  • The Old New Thing

    How do I prevent unchecked checkboxes in my listview control from disappearing?

    • 21 Comments

    A customer asked, "I have a listview control in report view with the LVS_EX_CHECK­BOXES extended style. I noticed that unchecked checkboxes are not visible until I hover over the corresponding item. Is it possible to get the checkboxes to be visible all the time?"

    I was kind of puzzled by this question because the default behavior of the list view control is to show the checkboxes all the time. I could have sat down and written a test program to prove it, but that would have taken too much time, and it wouldn't have advanced the story any. (The customer would merely have written back, "Well, that's not what I'm seeing.")

    This appeared to be a case of a customer providing incomplete information, forcing me to invoke my psychic powers to fill them in.

    "My psychic powers tell me that you have also set the LVS_EX_AUTO­CHECK­SELECT extended style. When LVS_EX_AUTO­CHECK­SELECT is set, then unchecked checkboxes are hidden."

    Remember, when you ask a question about a component and you have done any customization to that component, please remember to mention that when you ask your question. Otherwise nobody will be able to reproduce your problem, because they will assume you left everything at the defaults.

    It's like calling the Ikea customer service line, saying "My Ikea Frusträt rolling cabinet doesn't roll properly on my hardwood floor when I put more than about 25 pounds of stuff on it." The person on the phone looks up the specifications for the rolling cabinet and sees that that is well within the design limits. "I paid for you guys to assemble and install it, so it can't be an assembly or installation error." The person on the other end scratches their head for a while. And then you mention, "Oh, but I didn't like the color of the wheels, so I replaced them with some other wheels I bought at Home Depot."

    Oh, yeah, thanks for not mentioning that.

  • The Old New Thing

    GUIDs are designed to be unique, not random

    • 45 Comments

    A customer liaison asked, "My customer is looking for information on the GUID generation algorithm. They need to select N items randomly from a pool of M (jury selection), and their proposed algorithm is to assign each item a GUID, then sort the items by GUID and take the first N." (I've seen similar questions regarding using GUIDs for things like passwords or other situations where the programmer is looking for a way to generate a value that cannot be predicted.)

    The GUID generation algorithm was designed for uniqueness. It was not designed for randomness or for unpredictability. Indeed, if you look at an earlier discussion, you can see that so-called Algorithm 1 is non-random and totally predictable. If you use an Algorithm 1 GUID generator to assign GUIDs to candidates, you'll find that the GUIDs are assigned in numerically ascending order (because the timestamp increases). The customer's proposed algorithm would most likely end up choosing for jury duty the first N people entered into the system after a 32-bit timer rollover. Definitely not random.

    Similarly, the person who wanted to use a GUID for password generation would find that the passwords are totally predictable if you know what time the GUID was generated and which computer generated the GUID (which you can get by looking at the final six bytes from some other password-GUID). Totally-predictable passwords are probably not a good idea.

    Even the Version 4 GUID algorithm (which basically says "set the version to 4 and fill everything else with random or pseudo-random numbers") is not guaranteed to be unpredictable, because the algorithm does not specify the quality of the random number generator. The Wikipedia article for GUID contains primary research which suggests that future and previous GUIDs can be predicted based on knowledge of the random number generator state, since the generator is not cryptographically strong.

    If you want a random number generator, then use a random number generator.

    Bonus reading: Eric Lippert's GUID Guide, part 1, part 2, and part 3.

  • The Old New Thing

    Why is the Close button in the upper right corner?

    • 46 Comments
    Chris wants to know how the close button ended up to the right of the minimize and maximize/restore buttons. "In OS/2, it is on the left, which left the two other buttons in place."

    I don't know why the Close button went to the upper right instead of going to the left of the other buttons, but I'm going to guess. (That's what I do around here most of the time anyway; I just don't usually call it out.)

    Two words: Fitts's Law.

    The corners of the screen are very valuable, because users can target them with very little effort. You just slam the mouse in the direction you want, and the cursor goes into the corner. And since closing a window is a much more common operation than minimizing, maximizing, and restoring it, it seems a natural choice to give the close button the preferred location.

    Besides, maximizing and restoring a window already have very large targets, namely the entire caption. You can double-click the caption to maximize, and double-click again to restore. The restore even gets you a little bit of Fitt's Law action because the top of the screen makes the height of the caption bar effectively infinite.

  • The Old New Thing

    What was the registry like in 16-bit Windows?

    • 21 Comments

    Commenter Niels wonders when and how the registry was introduced to 16-bit Windows and how much of it carried over to Windows 95.

    The 16-bit registry was extremely simple. There were just keys, no values. The only hive was HKEY_CLASSES_ROOT. All it was used for was COM objects and file associations. The registry was stored in the REG.DAT file, and its maximum size was 64KB.

    It is my recollection that the registry was introduced in Windows 3.1, but Niels says it's not in a plain vanilla install, so I guess my memory is faulty.

    None of the 16-bit registry code was carried over to Windows 95. Windows 95 extended the registry into kernel mode, added support for values and non-string data types, increased the maximum registry size (though if some people are to be believed, not by enough), and added a bunch of other hives, like added the HKEY_CURRENT_USER, HKEY_LOCAL_MACHINE, and the HKEY_DYN_DATA, The old 16-bit registry code was woefully inadequate for all these new requirements (especially the kernel mode part), so it was all thrown out and a brand new registry written.

    In the early days of the Windows 95 registry, the in-memory signature value to identify the data structures which represent an open registry key were four bytes which corresponded to the ASCII values for the initials of the two programmers who wrote it.

  • The Old New Thing

    How to view the stack of threads that were terminated as part of process teardown from user mode

    • 25 Comments

    Last time we saw how to view the stack of threads that were terminated as part of process teardown from the kernel debugger. You can do the same thing from a user-mode debugger, and it's actually a bit easier there. (The user-mode debugger I'm using is the one that comes with the Debugging Tools for Windows, the debugging engine that goes by a number of different front-ends, such as ntsd, cdb, and windbg.)

    A direct translation of the kernel-mode technique from last time would involve using the !vadump command and picking through for the memory blocks with candidate size and attributes. But there's an easier way.

    Now would be a good point for me to remind you that this information is for debugging purposes only. The structures and offsets are all implementation details which can change from release to release.

    Recall that the TEB begins with some pointers which bound the stack, and the seventh pointer is a self-pointer. What's even more useful is the thirteenth pointer (offset 0x30 for 32-bit TEBs, offset 0x60 for 64-bit TEBs), because that is where the PEB is stored.

    Each process has a single global PEB, so all the TEBs will have the same PEB value at offset 0x30/0x60. And you can figure out the address of the current process's PEB either by using the !peb command or by simply looking at the TEB you already have.

    0:000> dd fs:30 l1
    0053:00000030  7efde000
    

    Now you can search through memory looking for that value. If you see any hits at offset 0x30/0x60, then that's a candidate TEB.

    The debugger normally limits memory scans to 256MB.

    0:001> s 00000000 L 80000000 00 e0 fd 7e
                               ^ Range error in 's 00000000 l 80000000 00 e0 fd 7e'
    

    Therefore, you have to issue the search eight times (for 32-bit processes) to cover the 2GB user-mode address space.

    0:001> s 00000000 L 10000000 00 e0 fd 7e
    0009e01c  00 e0 fd 7e 00 d0 fd 7e-44 e0 09 00 7b ef 17 77  ...~...~D...{..w
    0009fdc0  00 e0 fd 7e 44 00 00 00-f0 ee 3a 00 10 ef 3a 00  ...~D.....:...:.
    0009fe34  00 e0 fd 7e 78 fe 09 00-02 9f 18 77 00 e0 fd 7e  ...~x......w...~
    0:001> s 10000000 L 10000000 00 e0 fd 7e
    0:001> s 20000000 L 10000000 00 e0 fd 7e
    0:001> s 30000000 L 10000000 00 e0 fd 7e
    0:001> s 40000000 L 10000000 00 e0 fd 7e
    0:001> s 50000000 L 10000000 00 e0 fd 7e
    0:001> s 60000000 L 10000000 00 e0 fd 7e
    0:001> s 70000000 L 10000000 00 e0 fd 7e
    7486af70  00 e0 fd 7e 00 00 00 00-b8 00 16 77 28 00 16 77  ...~.......w(..w
    7efda030  00 e0 fd 7e 00 00 00 00-00 00 00 00 00 00 00 00  ...~............
    7efdd030  00 e0 fd 7e 00 00 00 00-00 00 00 00 00 00 00 00  ...~............
    

    Alternatively, you can use the "length sanity check override" by inserting a question mark after the L:

    0:001> s 00000000 L?80000000 00 e0 fd 7e
    0009e01c  00 e0 fd 7e 00 d0 fd 7e-44 e0 09 00 7b ef 17 77  ...~...~D...{..w
    0009fdc0  00 e0 fd 7e 44 00 00 00-f0 ee 3a 00 10 ef 3a 00  ...~D.....:...:.
    0009fe34  00 e0 fd 7e 78 fe 09 00-02 9f 18 77 00 e0 fd 7e  ...~x......w...~
    7486af70  00 e0 fd 7e 00 00 00 00-b8 00 16 77 28 00 16 77  ...~.......w(..w
    7efda030  00 e0 fd 7e 00 00 00 00-00 00 00 00 00 00 00 00  ...~............
    7efdd030  00 e0 fd 7e 00 00 00 00-00 00 00 00 00 00 00 00  ...~............
    

    From the above output, we see that we can quickly reject all but the last two entries because the offset within the page is not the magic value 0x30. (This is a 32-bit process.) Hooray, two debugger commands reduce the search space to just two pages!

    At this point, you can continue with the debugging technique from last time, looking at each candidate TEB to see if there's a valid stack in there.

  • The Old New Thing

    Charles Petzold is back with another edition of Programming Windows

    • 32 Comments

    Back in the day (and perhaps still true today), Charles Petzold's Programming Windows was the definitive source for learning to program Windows. The book is so old that even I used it to learn Windows programming, back when everything was 16-bit and uphill both ways. The most recent edition is Programming Windows, 5th Edition, which was published way back in 1998. What has he been doing since then? My guess would have been "sitting on a beach in Hawaiʻi," but apparently he's been writing books on C# and Windows Forms and WPF and Silverlight. Hey, I could still be right: Maybe he writes the books while sitting on a beach in Hawaiʻi.

    It appears that Windows 8 has brought Mr. Petzold back to the topic of Windows progarmming, and despite his earlier claims that he has no plans to write a sixth edition of Programming Windows, it turns out that he's writing a sixth edition of Programming Windows specifically for Windows 8. (Perhaps he could subtitle his book The New Old Thing.)

    Here's where it gets interesting.

    Before the book officially releases (target date November 15), there will be two pre-release versions in eBook form, one based on the Consumer Preview of Windows 8 and one based on the Release Preview.

    Now it gets really interesting: If you order the Consumer Preview eBook, it comes with free upgrades to the Release Preview eBook as well as the final eBook. (If you order the Release Preview eBook, then it comes with a free upgrade to the final eBook.)

    Can it get even more interesting than that? You bet! Because the price of getting in on the action increases the longer you wait. Act now, and you can get the Consumer Preview eBook (and all the free upgrades that come with it) for just $10. Wait a few weeks, and it'll cost you $20. Wait another few months, and it'll cost you $30; after another few weeks the price goes up to $40, and if you are a lazy bum and wait until the final eBook to be released, it'll cost you $50.

    But in order to take advantage of this offer, you have to follow the instructions on this blog entry from Microsoft Press (and read the mandatory legal mumbo-jumbo, because the lawyers always get their say).

    Bonus chatter: One publisher asked me if I wanted to write a book on programming Windows 8, but I told them that I was too busy shipping Windows 8 to have any extra time to write a book about it. And it's a good thing I turned them down, because imagine if I decided to write the book and found that Charles Petzold was coming out of retirement to write his own book. My book would have done even worse than my first book, which didn't even have any competition!

    Bonus disclaimer: Charles Petzold did not pay me to write this, nor did he offer me a cut of his royalties for shilling his book. But that doesn't mean I won't accept it! (Are you listening, Charles?)

  • The Old New Thing

    How to view the stack of threads that were terminated as part of process teardown from the kernel debugger

    • 6 Comments

    As we saw some time ago, process shutdown is a multi-phase affair. After you call ExitProcess, all the threads are forcibly terminated. After that's done, each DLL is sent a DLL_PROCESS_DETACH notification. You may be debugging a problem with DLL_PROCESS_DETACH handling that suggests that some of those threads were not cleaned up properly. For example, you might assert that a reference count is zero, and you find during process shutdown that this assertion sometimes fires. Maybe you terminated a thread before it got a chance to release its reference? How can you test this theory if the thread is already gone?

    It so happens that when all the threads are terminated during the early phase of process shutdown, the kernel is a bit lazy and doesn't free their stacks. It figures, hey, the entire process is going away soon, so the stack memory is going to be cleaned up as part of process termination. (It's sort of the kernel equivalent of not bothering to sweep the floor of a building that's about to be demolished.) You can use this to your advantage by grovelling the stacks that were left behind.

    Hey, this is why you get called in to debug the hard stuff, right?

    Before continuing, I need to emphasize that this information is for debugging purposes only. The structures and offsets are all implementation details which can change from release to release.

    The first step is to identify where all the stacks are. The direct approach is difficult because the stacks can be all different sizes, so it's not easy to pick them out of a line-up. But one thing does come in a consistent size: The TEB.

    From the kernel debugger, use the !process command to dump the process you are interested in, and from the header information, extract the VadRoot.

    1: kd> !process -1
    PROCESS 8731bd40  SessionId: 1  Cid: 0748    Peb: 7ffda000  ParentCid: 0620
        DirBase: 4247b000  ObjectTable: 96f66de0  HandleCount: 104.
        Image: oopsie.exe
        VadRoot 893de570 Vads 124 Clone 0 Private 518. Modified 643. Locked 0.
        DeviceMap 995628c0
    

    Dump this VAD root with the !vad command, and pay attention only to the entries which say 1 Private READWRITE.

    1: kd> !vad 893de570
    VAD     level      start      end    commit
    ... ignore everything except "1 Private READWRITE" ...
    8730a5f0 ( 6)         50       50         1 Private      READWRITE
    9ab0cb40 ( 5)         60       7f         1 Private      READWRITE
    893978b0 ( 6)         80       9f         1 Private      READWRITE
    87302d30 ( 5)        110      110         1 Private      READWRITE
    889693f8 ( 6)        120      121         1 Private      READWRITE
    872f3fb8 ( 6)        170      170         1 Private      READWRITE
    87089a80 ( 6)        1a0      1a0         1 Private      READWRITE
    8cbf1cb0 ( 5)        1c0      1df         1 Private      READWRITE
    88c079d0 ( 6)        1e0      1e0         1 Private      READWRITE
    9abc33e0 ( 6)        410      48f         1 Private      READWRITE
    873173b0 ( 7)        970      970         1 Private      READWRITE
    8ca1c158 ( 7)      7ffd5    7ffd5         1 Private      READWRITE
    88c02a78 ( 6)      7ffd6    7ffd6         1 Private      READWRITE
    872f9298 ( 5)      7ffd7    7ffd7         1 Private      READWRITE
    8750d210 ( 7)      7ffd8    7ffd8         1 Private      READWRITE
    87075ce8 ( 6)      7ffda    7ffda         1 Private      READWRITE
    87215da0 ( 4)      7ffdc    7ffdc         1 Private      READWRITE
    872f2200 ( 6)      7ffdd    7ffdd         1 Private      READWRITE
    8730a670 ( 5)      7ffdf    7ffdf         1 Private      READWRITE
    

    (If you are debugging from user mode, then you can use !vadump but the output format is different.)

    Each of these is a candidate TEB. In practice, TEBs tend to be allocated at the high end of memory, so the ones with a low start value are probably red herrings. Therefore, you should investigate these candidates in reverse order.

    For each candidate, take the start address and append three zeroes. (Each page on x86 is 4KB, which conveniently maps to 1000 in hex.) Dump the first seven pointers of the TEB with the dp xxxxx000 L7 command.

    1: kd> dp 7ffdf000 L7
    7ffdf000  0016fbb0 00170000 0016b000 00000000
    7ffdf010  00001e00 00000000 7ffdf000 ← hit
    

    If the TEB is valid, then the seventh pointer points back to the start of the TEB. In a valid TEB, the second and third values are the stack limits; in this case, the candidate stack lives between 0016b000 and 00170000. (As a double-check, you can verify that the upper limit of the stack, 00170000 in this case, matches up with the end of a VAD allocation in the !vad output above.)

    Now that you know where the stack is, you can dps it and look for EBP frames. (I usually start about two to four pages below the upper limit of the stack.) Test out each candidate EBP frame with the k= command until you find one that seems to be solid. Record this candidate stack trace in a text file for further study.

    Repeat for each candidate TEB, and you will eventually reconstruct what each thread in the process was doing at the moment it was terminated. If you're really lucky, you might even see the code that incremented the reference count but was terminated before it could release it.

    The above discussion also applies to debugging 64-bit processes. However, instead of looking for 1 Private READWRITE pages, you want to look for 2 Private READWRITE pages. As an additional wrinkle, if you are debugging ia64, then converting a page frame to a linear address is sadly not as simple as appending three zeroes. Pages on ia64 are 8KB, not 4KB, so you need to shift the value left by 25 bits: Add three zeroes and then multiply by two.

    And finally, if you are debugging a 32-bit process on x64, then you want to look for 3 Private READWRITE pages, but add 2 before appending the three zeroes. That's because the TEB for a 32-bit process on x64 is really two TEBs glued together: A 64-bit TEB followed by a 32-bit TEB.

    Note: I did not come up with this debugging technique on my own. I learned it from an even greater debugging genius.

    Next time, we'll look at debugging this issue from a user-mode debugger.

    Trivia: The informal term for these terminated-but-not-yet-completely-destroyed threads is ghost threads. The term was coined by the Exchange support team, because they often have to study server failures that require them to do this type of investigation, and they needed a cute name for it.

  • The Old New Thing

    Sure, we do that: Context menu edition

    • 17 Comments

    A customer reported a problem that occurred only when they installed a particular application. If they uninstalled it, then the problem went away. After installing the application, the "Run As" context menu option stopped working. The customer didn't provide any other details, but we were able to make an educated guess as to what was going on.

    A common programming error in context menu extensions occurs in extensions which add only one menu item. These extensions ignore the parameters to the IContextMenu::InvokeCommand and simply assume that the only reason the method can be called is if the user selected their menu item. After all, if you have only one invokable item, there's no need to figure out which one the user selected, because you have only one to begin with!

    The problem is that a context menu extension can be invoked not because the user selected an item under its control but because a verb is being invoked programmatically, and each handler is being asked, "Do you know how to do this?"

    The result is that the context menu host calls the extension to say, "If you know how to do runas, then please do so," and the the extension says "Sure, we do that" and starts doing its thing. If you are unlucky and the grabby extension is asked the question before the actual runas extension, the runas command winds up being hijacked by the grabby extension.

    (This is the same mistake that causes the Copy To and Move To commands to behave strangely if you add them to the context menu: They assume that the only reason they are invoked is that the user invoked their command, because they weren't designed to be hosted by context menus to begin with! They were designed to go into the toolbar, and the toolbar hosting code never invoked commands by name. It's like taking a ladder and using it as a bridge between two tall buildings. Sure, you can now cross from one building to another, but you also run a serious risk of falling to your death.)

    A variation on the initial problem is "I found that after installing a particular program, I can't run anything from the Start menu." I know of at least two programs which install context menu extensions which steal the "open" command on executables.

    This problem is sufficiently prevalent that there is a special compatibility flag that can be set on a shell extension to say, "This is a grabby shell extension that steals commands. Never ask it if it supports anything, because it will always say yes!"

    Notice that the "MoveTo CopyTo Context Menu" is on the list, which I find interesting because MoveTo/CopyTo was never meant to go on the context menu in the first place. Going back to our analogy, it'd be as if the ladder company issued a safety bulletin to warn people of problems that can occur if you use it as a bridge between two tall buildings!

  • The Old New Thing

    Microspeak: The parking lot

    • 18 Comments

    Mike Dunn wonders what the Microspeak term parking lot means.

    I'm not familiar with this term either, and the first document I turned up during my search was a PowerPoint presentation that said "Avoid using Microsoft jargon terms, such as parking lot and dogfood."

    Yeah, that wasn't much help.

    From what I can gather, the term parking lot started out as a term used during brainstorming sessions. You've got a bunch of people in a conference room tossing out all sorts of ideas. The traditional way of organizing the ideas is to write each one on a Post-It® note and stick it on the whiteboard. As more and more notes appear, you start to organize them by grouping together similar ideas.

    Every so often, you'll run into an idea that, while good, isn't really relevant to the problem you're trying to solve. You don't want to throw it away, so instead, you designate a corner of the whiteboard to be the place to "park" those ideas for later consideration. That corner of the whiteboard is nicknamed the parking lot.

    The term parking lot then began to be applied to the document that collected all of these "parked" ideas, so they could be circulated to a more appropriate audience.

    The term then expanded to refer to any document which served as the official repository of assorted suggestions for future work or discussion. (Known to some people simply as The List.) For example, there is a SharePoint List titled Active Issues and the subtitle parking lot for discussion topics in weekly XYZ meeting. Each item on the list is assigned to a particular person and assigned a priority.

    I can't find any citations for parking lot being used as a way to say something like "we'll talk about this after the meeting is over," but I can see how it could be related to the sense of parking lot I was able to turn up: The parking lot is the list of things that aren't really relevant to the topic at hand but which are still worth discussing. We just won't discuss them here.

Page 1 of 355 (3,548 items) 12345»