November, 2007

  • The Old New Thing

    Amusing bug reports: On poetry and prose


    Back in Windows 95, there was one bug that was originally filed against the window manager team. Upon closer investigation, the window manager team determined that the bug was really in the video driver and added their analysis to the bug as they reassigned it. I don't know what overcame them, but the analysis was written in iambic pentameter. (It may have been rhymed couplets; my memory is kind of hazy. Gosh, I hope it wasn't haiku.)

    At any rate, the device driver team added one comment to the bug immediately upon receiving it: "We ain't much fer book lernin'."

  • The Old New Thing

    I don't know what the Lotus internal clipboard formats are either


    Apparently some people think my psychic powers extend to being able to steal trade secrets by sheer force of concentration.

    I've been trying to write an application which allows to drag and drop documents from lotus notes. If I just drag and drop a document from lotus notes to Windows explorer, it creates a .shb file (regular windows document link file). Therefore I think it is possible to simulate this in a custom application as well.

    BTW: I found that lotus notes uses following list of clipboard formats for OLE drag drop.
    Notes Private Data
    Notes Private Link
    Link Source Descriptor
    Link Source

    I also think only the last two formats will be enough for the task I want to accomplish. In fact they are more likely to be windows OLE clipboard formats. However, these clipboard formats are not documented.

    I would be grateful if you could send me the definitions of data structures I could use to access data from these CFs.

    (Here's another example.)

    I'm not sure why somebody who works for Microsoft is expected to be able to produce documentation on a Lotus Notes data structure.

    When you drag a COM object into an Explorer window on Windows XP, the resulting scrap file doesn't understand the object you dropped. It just makes a copy of it. When you drag the scrap and drop it back into a document, the scrap merely spits back the data it was cloned from. But it doesn't know what that data means.

    If you stick a piece of paper in a photocopier, it will produce a copy of the original document. But there's no point asking the manufacturer of the copier, "I put this document written in French into your copier, and it made a copy. Can you tell me what it says? Obviously, you understand French because you were able to copy the document."

    Reading the question again, perhaps the person is asking for documentation on Link Source Descriptor and Link Source. Don't be helpless. I don't know what they do either, but a little searching turns up OBJECTDESCRIPTOR for the link source descriptor. Link source is a little trickier, but from this page it appears to be a serialized moniker. I have never worked with OLE embeddings; I don't know any more than the next guy. Why don't you go read about it and write a blog entry summarizing what you've learned?

  • The Old New Thing

    You just have to accept that the file system can change


    A customer who is writing some sort of code library wants to know how they should implement a function that determines whether a file exists. The usual way of doing this is by calling GetFileAttributes, but what they've found is that sometimes GetFileAttributes will report that a file exists, but when they get around to accessing the file, they get the error ERROR_DELETE_PENDING.

    The lesser question is what ERROR_DELETE_PENDING means. It means that somebody opened the file with FILE_SHARE_DELETE sharing, meaning that they don't mind if somebody deletes the file while they have it open. If the file is indeed deleted, then it goes into "delete pending" mode, at which point the file deletion physically occurs when the last handle is closed. But while it's in the "delete pending" state, you can't do much with it. The file is in limbo.

    You just have to be prepared for this sort of thing to happen. In a pre-emptively multi-tasking operating system, the file system can change at any time. If you want to prevent something from changing in the file system, you have to open a handle that denies whatever operation you want to prevent from happening. (For example, you can prevent a file from being deleted by opening it and not specifying FILE_SHARE_DELETE in your sharing mode.)

    The customer wanted to know how their "Does the file exist?" library function should behave. Should it try to open the file to see if it is in delete-pending state? If so, what should the function return? Should it say that the file exists? That it doesn't exist? Should they have their function return one of three values (Exists, Doesn't Exist, and Is In Funky Delete State) instead of a boolean?

    The answer is that any work you do to try to protect users from this weird state is not going to solve the problem because the file system can change at any time. If a program calls "Does the file exist?" and the file does exist, you will return true, and then during the execution of your return statement, your thread gets pre-empted and somebody else comes in and puts the file into the delete-pending state. Now what? Your library didn't protect the program from anything. It can still get the delete-pending error.

    Trying to do something to avoid the delete-pending state doesn't accomplish anything since the file can get into that state after you returned to the caller saying "It's all clear." In one of my messages, I wrote that it's like fixing a race condition by writing

    // check several times to try to avoid race condition where
    // g_fReady is set before g_Value is set
    if (g_fReady && g_fReady && g_fReady && g_fReady && g_fReady &&
        g_fReady && g_fReady && g_fReady && g_fReady && g_fReady &&
        g_fReady && g_fReady && g_fReady) { return g_Value; }

    The compiler folks saw this message and got a good chuckle out of it. One of them facetiously suggested that they add code to the compiler to detect this coding style and not optimize it away.

  • The Old New Thing

    Proto-Microspeak: Pre-envisioning


    I have only one citation, so it may not become proper Microspeak. Too early to tell.

    Further discussion will definitely generate a lot of good ideas and help drive them for pre-envisioning.

    Established Microspeak or not, I still don't know what it means.

  • The Old New Thing

    VirtualLock only locks your memory into the working set


    When you lock memory with VirtualLock it locks the memory into your process's working set. It doesn't mean that the memory will never be paged out. It just means that the memory won't be paged out as long as there is a thread executing in your process, because a process's working set need be present in memory only when the process is actually executing.

    (Earlier versions of the MSDN documentation used to say this more clearly. At some point, the text changed to say that it locked the memory physically rather than locking it into the working set. I don't know who changed it, but it was a step backwards.)

    The working set is the set of pages that the memory manager will keep in memory while your program is running, because it is the set of pages that the memory manager predicts your program accesses frequently, so keeping those pages in memory when your program is executing keeps your program running without taking a ridiculous number of page faults. (Of course, if the prediction is wrong, then you get a ridiculous number of page faults anyway.)

    Now look at the contrapositive: If all the threads in your process are blocked, then the working set rules do not apply since the working set is needed only when your process is executing, which it isn't. If all your threads are blocked, then the entire working set is eligible for being paged out.

    I've seen people use VirtualLock expecting that it prevents the memory from being written to the page file. But as you see from the discussion above, there is no such guarantee. (Besides, if the user hibernates the computer, all the pages that aren't in the page file are going to get written to the hibernation file, so they'll end up on disk one way or another.)

    Even if you've magically managed to prevent the data from being written to the page file, you're still vulnerable to another process calling ReadProcessMemory to suck the data out of your address space.

    If you really want to lock memory, you can grant your process the SeLockMemoryPrivilege privilege and use the AWE functions to allocate non-pageable memory. Mind you, this is generally considered to be an anti-social thing to do on a paging system. The AWE functions were designed for large database programs that want to manage their paging manually. And they still won't prevent somebody from using ReadProcessMemory to suck the data out of your address space.

    If you have relatively small chunks of sensitive data, the solution I've seen recommended is to use CryptProtectData and CryptUnprotectData. The encryption keys used by these functions are generated pseudo-randomly at boot time and are kept in kernel mode. (Therefore, nobody can ReadProcessMemory them, and they won't get captured by a user-mode crash dump.) Indeed, this is the mechanism that many components in Windows 2003 Server to reduce the exposure of sensitive information in the page file and hibernation file.

    Follow-up: I've been informed by the memory manager folks that the working set interpretation was overly conservative and that in practice, the memory that has been virtually locked won't be written to the pagefile. Of course, the other concerns still apply, so you still have to worry about the hibernation file and another process sucking the data out via ReadProcessMemory.

  • The Old New Thing

    Why do we even have the DefWindowProc function?


    Some time ago, I looked at two ways of reimplementing the dialog procedure (method 1, method 2). Commenter "8" wondered why we have a DefWindowProc function at all. Couldn't window procedures have followed the dialog box model, where they simply return FALSE to indicate that they want default processing to occur? Then there would be no need to export the DefWindowProc function.

    This overlooks one key pattern for derived classes: Using the base class as a subroutine. That pattern is what prompted people to ask for dialog procedures that acted like window procedures. If you use the "Return FALSE to get default behavior" pattern, window procedures would go something like this:

    BOOL DialogLikeWndProc(HWND hwnd, UINT uMsg, WPARAM wParam, LPARAM lParam)
     switch (uMsg) {
     ... handle messages and return TRUE ...
     // We didn't have any special processing; do the default thing
     return FALSE;

    Similarly, subclassing in this hypothetical world would go like this:

    BOOL DialogLikeSubclass(HWND hwnd, UINT uMsg, WPARAM wParam, LPARAM lParam)
     switch (uMsg) {
     ... handle messages and return TRUE ...
     // We didn't have any special processing; let the base class try
     CallDialogLikeWindowProc(PrevDialogLikeWndProc, hwnd, uMsg, wParam, lParam);

    This works as long as what you want to do is override the base class behavior entirely. But what if you just want to augment it? Calling the previous window procedure is analogous to calling the base class implementation from a derived class, and doing so is quite common in object-oriented programming, where you want the derived class to behave "mostly" like the base class. Consider, for example, the case where we want to allow the user to drag a window by grabbing anywhere in the client area:

    LRESULT CALLBACK CaptionDragWndProc(
        HWND hwnd, UINT uMsg, WPARAM wParam, LPARAM lParam)
     LRESULT lres;
     switch (uMsg) {
     case WM_NCHITTEST:
      lres = DefWindowProc(hwnd, uMsg, wParam, lParam);
      if (lres == HTCLIENT) lres = HTCAPTION;
      return lres;
     return DefWindowProc(hwnd, uMsg, wParam, lParam);

    We want our hit-testing to behave just like normal, with the only exception that clicks in the client area should be treated as clicks on the caption. With the DefWindowProc model, we can do this by calling DefWindowProc to do the default processing, and then modifying the result on the back end. If we had use the dialog-box-like model, there would have been no way to call the "default handler" as a subroutine in order to make it to the heavy lifting. We would be forced to do all the work or none of it.

    Another avenue that an explicit DefWindowProc function opens up is modifying messages before they reach the default handler. For example, suppose you have a read-only edit control, but you want it to look like a normal edit control instead of getting the static look. You can do this by modifying the message that you pass to DefWindowProc:

      if (GET_WM_CTLCOLOR_HWND(wParam, lParam) == m_hwndEdit)
       // give it the "edit" look
       return DefWindowProc(hwnd, WM_CTLCOLOREDIT, wParam, lParam);

    Another common operation is changing one color attribute of an edit control while leaving the others intact. For this, you can use DefWindowProc as a subroutine and then tweak the one attribute you want to customize.

      if (GET_WM_CTLCOLOR_HWND(wParam, lParam) == m_hwndDanger)
       // Start with the default color attributes
       LRESULT lres = DefWindowProc(hwnd, uMsg, wParam, lParam);
       // Change text color to red; leave everything else the same
       SetTextColor(GET_WM_CTLCOLOR_HDC(wParam, lParam), RGB(255,0,0));
       return lres;

    Getting these types of operations to work with the dialog box model would be a significantly trickier undertaking.

  • The Old New Thing

    Staying on top of things with timely updates in separator pages

    Public Service Announcement
    This weekend marks the end of Daylight Saving Time in most parts of the United States, the first "return to standard time" cutoff under the new transition rules.

    For a month prior to the transition this past spring, the separator pages generated by the shared company printers contained a handy reminder that Daylight Saving Time began early this year.

    The reminder remained on the separator pages as late as the summer solstice.

    Maybe the people who design the separator pages didn't have anything more interesting to say, so they let the old message ride.

  • The Old New Thing

    Buy an island, or a country, or at least a replica of one


    Off the coast of Dubai, Al Nakheel Properties is building a chain of 300 islands in the shape of the world. Learn more about the project by watching their video.

    The growing artificiality and commercialization of Dubai concerns some people, however.

    Locals joke that Dubai's call to prayer is "Attention shoppers!"

    The naysayers can cluck their tongues all they want. That won't prevent the construction of an indoor ski slope. (Warning: Link has loud, annoying music.)

  • The Old New Thing

    Why does GetDiskFreeSpaceEx return the wrong amount of free space?


    One customer discovered that viewing the properties of a drive from Explorer shows the amount of disk free space significantly lower than the actual amount of free disk space as resported by the disk management snap-in. The command prompt agrees with Explorer and shows the incorrect result. What's going on?

    Not surprisingly, both Explorer and the command prompt use the GetDiskFreeSpaceEx function to get the amount of free disk space. (It's not surprising because the only two convenient options are GetDiskFreeSpace and GetDiskFreeSpaceEx, and the former exists only for compatibility with older programs who don't support drives bigger than 2GB.) If you go to the documentation for the GetDiskFreeSpaceEx function, you'll see that the amount of free disk space reported is not the actual amount of physical free disk space but rather the amount of available physical free disk space as constrained by the user's quota.

    Just saying the word "quotas" to the customer was enough to satisfy them. I guess that was enough of a nudge that they could figure out the rest for themselves.

Page 3 of 3 (29 items) 123