October, 2011

  • The Old New Thing

    The question mark lets you make up anything you like

    • 38 Comments

    A trend I've noticed in journalism is to make some sort of outrageous statement, but then stick a question mark at the end to disavow any responsibility for the statement. By changing it to a question, you're avoiding actually having to back up what you write. "I'm not saying this is actually true. I'm just raising the question."

    For example, a headline might read "The sign of something new?" The author doesn't want to actually back up the claim that the subject is the sign of something new, so he'll just say it with a question mark. Now the responsibility to support or refute the claim has been shifted to you, the reader.

    The question itself doesn't need to have any merit whatsoever. In fact, you can just make up the craziest stuff imaginable; as long as you put a question mark after it, you're home free. And it doesn't even need to take the form of a question!

    Made-up examples:

    Perhaps I should've titled this article "Journalists are just making up stuff and printing it as if it were news?"

    Bonus journalistic head-scratcher: The word "official" appears to have taken on a meaning I was previously unaware of.

    It's official: The Microsoft Zune has gone the way of the Kin, the Courier, and Bob.

    If you actually click through to the article's source, and then click through to that article's source, you'll see that the source is a person "who declined to be identified because the decision hasn't been announced."

    So let me see if I understand this. They're saying that a statement is "official" because it comes from an anonymous person who doesn't wish to be identified because no official statement has yet been made.

    (I think my copy of Gödel, Escher, Bach just exploded.)

  • The Old New Thing

    If the shell is written in C++, why not just export its base classes?

    • 22 Comments

    ton suggested that since the shell is written in C++, IShell­Folder should have been an abstract class, and then it could have used techniques like exceptions and Inversion of Control.

    Okay, first of all, I'm not sure how Inversion of Control is something that requires C++, so I'm going to leave that aside.

    Second of all, who says the shell is written in C++? As it happens, when IShell­Folder was introduced in Windows 95, the entire shell was written in plain C. That's right, plain C. Vtables were built up by hand, method inheritance was implemented by direct replacement in the vtable, method overrides were implemented by function chaining, multiple inheritance was implemented by manually moving the pointer around.

    const IShellFolderVtbl c_vtblMyComputerSF =
    {
     MyComputer_QueryInterfaceSF,
     MyComputer_AddRefSF,
     MyComputer_ReleaseSF,
     MyComputer_ParseDisplayName,
     ... you get the idea ...
    };
    
    const IPersistFolderVtbl c_vtblMyComputerPF =
    {
     MyComputer_QueryInterfacePF,
     MyComputer_AddRefPF,
     MyComputer_ReleasePF,
     MyComputer_Initialize,
    };
    
    struct MyComputer {
     IShellFolder sf;
     IShellFolder pf;
     ULONG cRef;
     ... other member variables go here ...
    }
    
    MyComputer *MyComputer_New()
    {
     MyComputer *self = malloc(sizeof(MyComputer));
     if (self) {
      self->sf.lpVtbl = &c_vtblMyComputerSF;
      self->pf.lpVtbl = &c_vtblMyComputerPF;
      self->cRef = 1;
      ... other "constructor" operations go here ...
     }
     return self;
    }
    
    // sample cast
    MyComputer *pThis;
    IPersistFolder *ppf =  &pThis->pf;
    
    // sample method call
    hr = IShellFolder_CompareIDs(psf, lParam, pidl1, pidl2);
    // which expands to
    hr = psf->lpVtbl->CompareIDs(psf, lParam, pidl1, pidl2);
    
    // sample forwarder for multiply-derived method
    HRESULT STDCALL MyComputer_QueryInterfacePF(
        IPersistFolder *selfPF, REFIID riid, void **ppv)
    {
     MyComputer *self = CONTAINING_RECORD(selfPF, MyComputer, pf);
     return MyComputer_QueryInterfaceSF(&self->sf, riid, ppv);
    }
    

    So one good reason why the shell didn't export its C++ base classes was that it didn't have any C++ base classes.

    Why choose C over C++? Well, at the time the Windows 95 project started, C++ was still a relatively new language for systems programming. While there were certainly people on the shell team capable of writing code in C++, the old-timers grew up with C as their native language, and the newcomers weren't taught C++ in their computer science classes. (Computer science departments still taught primarily C or Pascal, with maybe some Lisp if you took an AI class.) Also, the C++ compilers of the day did not provide fine control over automatic code generation,¹ and since even saving 4KB of memory had a perceptible impact on overall system performance, manually grouping rarely-used functions into the same region of memory of memory (so they could all remain paged out) was still a common practice.

    But even if the shell was originally written in C++, exporting the base classes wouldn't have been a good idea. COM is a language-neutral platform. People have written COM objects in C, C++, Visual Basic, C#, Delphi, you-name-it. If IShell­Folder were an exported C++ base class, then you have effectively said, "Sorry, only C++ code can implement IShell­Folder. Screw off, all you other languages!"

    But wait, it's worse than just that. Exporting a C++ base class ties you to a specific compiler vendor, because name decoration is not standardized. So it's not just "To implement IShell­Folder you must use C++" but "To implement IShell­Folder you must use the Microsoft Visual Studio C++ compiler."

    But wait, it's worse than just that. The name decoration algorithm can even change between compiler versions. Furthermore, the mechanism by which exceptions are thrown and caught is not merely compiler-specific but compiler-version specific. If an exception is thrown by code compiled by one version of the C++ compiler and reaches code compiled by a different version of the C++ compiler, the results are undefined. (For example, the older version of the C++ compiler may not have supported RTTI.) So it's not just "To implement IShell­Folder you must use C++" but "To implement IShell­Folder you must use Microsoft Visual C++ 2.0." (So maybe Bjarne was right after all.)

    But wait, it's worse than just that. Exporting a C++ base class means that the base class can never change, because various properties of the base class become hard-coded into the derived classes. The list of interfaces implemented by the base class becomes fixed. The size of the base class is fixed. Any inline methods are fixed. The precise layout of member variables is fixed. Exporting a C++ base class for IShell­Folder would have meant that the base class could never change. You want support for IShell­Folder2? Sorry, we can't add that without breaking everybody who compiled with the old header file.

    Exercise: If exporting base classes is so horrible, why does the CLR do it all over the place?

    Footnote

    ¹ Actually, I don't think even the C++ compilers of today give you fine control over automatic code generation, which is why Microsoft takes a conservative position on use of C++ in kernel mode, where the consequences of a poorly-timed page fault are much worse than simply poor performance. It will bluescreen your machine.

  • The Old New Thing

    Why can't I move the Program Files directory via the unattend file?

    • 53 Comments

    We saw last time that the unattend file lets you change some Windows configuration settings that cannot be changed after Setup is complete. But one of the things you can't change is the location of the Program Files directory. Many people wish they could relocate their Program Files directory to another drive in order to relieve disk space pressure on the system partition. Why won't Windows let them do this?

    Now that NTFS is mandatory for the system volume (it took only 13 years to get there!), Windows itself can start taking advantage of NTFS features.

    For example, Windows Update can take advantage of transactions: When updates are applied, they are done as part of a transaction. That way, if something horrific happens in the middle of an update, the entire transaction becomes abandoned and you don't get stuck with some Frankenstein configuration.

    Windows Setup takes advantage of hard links. A large percentage of the files installed by Windows are hard-linked to copies in the C:\Windows\WinSxS directory for reasons I do not understand, but the phrase "component store" may be part of it. (This is why asking Explorer for the size of the C:\Windows directory gives a misleading view of the actual amount of disk space occupied by Windows, because Explorer uses a naive algorithm which counts each hard link as a separate file.) Oh, and in Windows 7, the two copies of Notepad are now hard links to each other.

    Ah, but one of the limitations of hard links is that they cannot span volumes. Some of the hard links out of the WinSxS directory point into places like C:\Program Files\Windows NT\Accessories\wordpad.exe, and this in turn requires that the Program Files directory be on the same volume as your Windows directory.

    Sorry for the inconvenience.

  • The Old New Thing

    The unattend file lets you configure Windows while it's installing, and in some cases it's your only chance

    • 32 Comments

    Some Windows settings can only be established as part of the installation process. This is done with a so-called unattend file. (Remember, no matter where you put an advanced setting, somebody will tell you that you are an idiot.) In earlier versions of Windows, the unattend file took the form of an INI file, but Windows Vista hopped aboard the XML bandwagon, and the unattend file format changed to XML. The nice thing about using XML is that you can publish a schema so people can validate their unattend file without having to perform a test install (only to discover twenty minutes later that a typo resulted in an entire section of the unattend file being ignored, say).

    If you spend a lot of time setting up computers, you can use an unattend file to answer all the Setup questions (like "enter your product key") so all you have to do is type "setup /unattend:myconfiguration.xml" and go out to lunch. When you come back, your machine will be installed and ready.

    Here are two of the most popular unattend settings which must be set during installation. (There are a bunch of popular unattend settings for things that can also be changed post-install; for those other settings, the unattend file is not your only chance.)

    Wait, the C:\Program Files directory isn't on the list of directories that can be relocated. There's a reason for that, which we'll look at next time.

  • The Old New Thing

    Why isn't my transparent static control transparent?

    • 27 Comments

    A customer reported that their application uses transparent static controls positioned over a bitmap image control, but even though they set the Transparent property on the static control, the static control isn't transparent. The customer was kind enough to provide clear steps to illustrate the problem:

    • Open Visual Studio 2005 or 2008.
    • From the menu, select File, New File, Visual C++, Resource Template File (RCT).
    • Right-click on the RCT file, select Add Resource, and add a bitmap named IDB_BITMAP1.
    • Open the dialog box (IDD_DIALOG1) and add a "Picture Control", specifying IDC_BITMAP_1 as its ID.
    • Change the IDC_BITMAP_1 type to Bitmap and change the value of the Image property to IDB_BITMAP1.
    • Add a "Static Text" control IDC_TEST_STATIC and set its caption to "This is a test".
    • Reposition the static control so it overlaps the IDC_BITMAP_1 control.
    • On the IDC_TEST_STATIC control, set the Transparent property to True.
    • Type Ctrl+T to test the dialog and observe that the static control is not transparent.
    Dialog
    This is a test

    The Transparent property in Visual Studio corresponds to the WS_EX_TRANSPARENT window style, and the documentation explains that

    WS_EX_TRANSPARENT: The window should not be painted until siblings beneath the window (that were created by the same thread) have been painted. The window appears transparent because the bits of underlying sibling windows have already been painted.

    The observed behavior, therefore, matches the documentation: The control underneath (the bitmap control) paints first, and then the static control paints on top of it. And a static text control paints by filling with the background brush and drawing the text on top of it. You can customize this behavior by responding to the WM_CTL­COLOR­STATIC message:

    HBRUSH CTestDlg::OnCtlColor(CDC* pDC, CWnd* pWnd, UINT nCtlColor)
    {
     HBRUSH hbr = __super::OnCtlColor(pDC, pWnd, nCtlColor);
     if (pWnd->GetExStyle() & WS_EX_TRANSPARENT) {
      pDC->SetBkMode(TRANSPARENT);
      hbr = GetStockBrush(HOLLOW_BRUSH);
      // even better would be to use a pattern brush, if the background is fixed
     }
     return hbr;
    }
    

    The customer appreciated the explanation but was puzzled as to why the Transparent is available if it doesn't work. "We understand that we can use the WS_EX_TRANSPARENT window style to create a transparent window and it will be painted after its underlying siblings, but the window style by itself doesn't make the static control transparent. But if we have to write the code above, why is the Transparent property available in the Properties box?" They included a screen shot from Visual Studio where the built-in help text for the Transparent property reads "Specifies that the control will have a transparent background."

    The WS_EX_TRANSPARENT style doesn't mean "transparent"; it means "paint over siblings." The style is called "transparent" not because it makes the window transparent but because it makes transparency possible. It is one of the steps (but not the only one) for making child controls render transparently. Another important step is ensuring that the control does not erase its background in its WM_ERASE­BKGND, and that's the step that the On­Ctl­Color override performs.

    Why is the Transparent property offered for static controls when it doesn't actually work? Shouldn't it be disabled for static controls?

    The reason why it is offered is that it is a general window style that can be set on any control. Visual Studio doesn't know which controls can render transparently and which ones don't, or what extra steps are necessary to get the ones who can render transparently to actually do so. It just exposes the WS_EX_TRANSPARENT style and hopes that you know what you're doing.

    In retrospect, it was a poor chose of name for the style. And the incorrect online help doesn't make things any better.

    Bonus chatter: Note that the WS_EX_TRANSPARENT extended style is overloaded. In addition to affecting painting, it also affects hit-testing.

  • The Old New Thing

    Why does copying a file to my USB thumb drive say that the parameter is incorrect?

    • 40 Comments

    Consider the following sequence of operations, assuming that F: is a USB thumb drive with plenty of disk space.

    C:\Users\Bob\Downloads> copy readme.txt F:\
            1 file(s) copied.
    C:\Users\Bob\Downloads> copy Update.iso F:\
    The parameter is incorrect.
    

    Why is the second file copy failing?

    The hint is the file extension: *.iso, which suggests that this is a CD or DVD image, and DVD images have the feature that they tend to be really big.

    Like more than 4GB big.

    USB thumb drives tend to be formatted with the FAT32 file system rather than with NTFS. And FAT32 has a maximum file size of 4GB minus one byte.

    The user confirmed that the Update.iso file was larger than 4GB and that the USB thumb drive was formatted as FAT32.

    Mind you, the error message doesn't help at all in identifying that this is what's going on. I don't know where it's coming from, but my guess is that somewhere inside the copy command, it tries to create the destination file and set its file size. Since the file size is out of range for FAT32, the call fails with the error ERROR_INVALID_PARAMETER, and that's what ends up bubbling out to the user.

    But at least now you know what the confusing error message is trying to tell you.

  • The Old New Thing

    When your vice president tells you to stop replying to a mail thread, you probably should stop replying to the mail thread

    • 11 Comments

    Some time in the early part of this century, somebody sent a message to the Windows NT Development Announcements mailing list at Microsoft. It went something like, "My car was parked in «location X» and somebody ran into it and didn't leave a note. Does anybody have any information about this?"

    Now, one thing you need to know is that the Windows NT Development Announcements mailing list has the entire Windows division as members. We're talking thousands of people. And the sort of announcements sent to the alias are not the "somebody dinged my car" type of announcements. They are announcements like "We will be reformatting the scratch server on Tuesday." Important stuff that everybody on the team needs to know. And it's most certainly not for discussions.

    Anyway, somebody replied to the message saying something like "Yeah, my car was parked in the same area and it got damaged, too." And then somebody else decided to be silly and wrote, "I parked my car in Germany once and it got damaged. I wonder if that's related."

    Before things could spiral out of control, Jim Allchin, vice president of Windows, stepped in and sent a message to everybody. "Stop replying to this thread."

    That shut the thread down really quickly. Nothing like your vice president telling you to shut up to get you to shut up.

    There was one reply that came through, though. It was a reply to Jim's message saying to stop replying. The person wrote, "Yes sir, captain sir. Saluting as I type, sir."

    I have no confirmation that anybody ever saw that person alive again.

    Bonus history: Today is the 14th anniversary of the infamous Bedlam Incident.

  • The Old New Thing

    How do I access a file without updating its last-access time?

    • 29 Comments

    The first problem with discussing file last-access time is agreeing what you mean by a file's last-access time.

    The file system folks have one definition of the file last-access time, namely the time the file was most recently opened and either read from or written to. This is the value retrieved by functions like Get­File­Attributes­Ex, Get­File­Time, and Find­First­File.

    The problem with this definition is that it doesn't match the intuitive definition of last-access time, which is "the last time I accessed the file," emphasis on the I. In fact, the intuitive definition of access is more specific: It's "the last time I opened, modified, printed, or otherwise performed some sort of purposeful action on the file."

    This discrepancy between the file system definition and the intuitive definition means that a lot of operations trigger a file system access but shouldn't count as an access from the user interface point of view. Here are some examples:

    Whenever some shell extension violates this rule, the shell team gets a bug report from some customer saying, "The last-access time shown in Explorer is wrong. A document which hasn't been accessed in months shows a last-access time of today. After closer investigation, we found that the last-access time updates whenever we insert seemingly-innocuous operation here."

    If you're writing a program which needs to access the file contents but you not want to update the last-access time, you can use the Set­File­Time function with the special value 0xFFFFFFFF in both members of the FILETIME structure passed as the last-access time. This magic value means "do not change the last-access time even though I'm accessing the file."

    BOOL DoNotUpdateLastAccessTime(HANDLE hFile)
    {
     static const FILETIME ftLeaveUnchanged = { 0xFFFFFFFF, 0xFFFFFFFF };
     return SetFileTime(hFile, NULL, &ftLeaveUnchanged, NULL);
    }
    

    As the documentation notes, you have to call this function immediately after opening the file.

    Going back to that linked comment: The reason why viewing the Summary tab causes the last-access time to be updated is that the Summary tab retrieves its information by calling Stg­Open­Storage, and there's no way to tell that function, "Hey, when you open the file in order to see if it has any document properties, do that Do­Not­Update­Last­Access­Time thing so you don't update the last access time."

    Bonus chatter: Starting in Windows Vista, maintaining the last-access time is disabled by default. In practice, this means that the number of bugs related to altering the last-access time accidentally will multiply unchecked, because the mechanism for detecting the error is disabled by default.

  • The Old New Thing

    Microspeak: Bug jail

    • 20 Comments

    Bug jail is not a place where bugs are sent as punishment for their crimes. Rather, it's a (virtual) place that developers are sent when they have too many bugs.

    Project management establishes some maximum number of bugs (known as a bug cap) each developer is permitted to have on his or her plate, and developers whose bug count exceeds the specified maximum are placed in bug jail. The precise triggers for bug jail vary from team to team, and it may vary based on the nature of the bug. For example, one trigger might be that any Priority 0 bugs more than 24 hours old will land you in bug jail.

    Once you land in bug jail, you are not allowed to do feature work, be it coding, writing specifications, whatever. The precise triggers for getting out of bug jail also vary from team to team, but one rule might be that you need to get your bug count back down to 50% of the bug jail trigger level before you are allowed to exit.

    Staying on top of your bug count is an important part of the software development process, and the primary motivation behind bug jail is not to punish developers, although I'm sure that's how most developers perceive it. Rather, it serves as an early-warning system to highlight things that are not going smoothly. There may be a bug farm developing in that feature area. Or the team needs to revise its idea of what it means to be "done" with the feature. And it's a signal to project management that they may need to scale back their plans so as not to compromise quality and the ship schedule.

    When everybody understands the reasoning behind bug jail, you avoid lengthy discussions over boundary cases ("What if a developer is about to check in a feature but a low-priority bug comes in that pushes them over the limit?") and avoid discovering that people have been "gaming the system" (for example, by reclassifying bugs as feature requests so they don't count toward the bug cap). Like bug hugging, these sorts of games prevent project management from truly understanding how close the project is to being finished.

  • The Old New Thing

    No, modifying the DLLs that come with Windows is not supported

    • 47 Comments

    From the I can't believe I had to write that file comes this question from a customer:

    Our customer is modifying the ABC.DLL file that comes with Windows in order to accomplish XYZ. Is this supported?

    No, of course this isn't supported. I can't believe I had to write that. if you modify a system file, then the thing you're running isn't Windows any more but is rather some sort of operating system that resembles Windows in many significant ways.

    (Imagine the extreme case of this: The customer modifies NTOSKRNL.EXE, KERNEL32.DLL, USER32.DLL, etc. so that they happen to be byte-for-byte identical to the files that shipped with Windows 2000. Does this mean that Microsoft supports Windows 2000?)

    The customer is just looking for an official Microsoft statement that this is not supported. They know that it's a bad idea, but their client wanted to have this feature, so when the customer tells the client "No, we can't do this," they need some sort of justification.

    Oh, I see now. The customer is chicken and wants Microsoft to be the bad cop when they deliver unfavorable news to their client.

    My fear is that the company, by being spineless, is now subject to manipulation from their client. "What, you're not going to let big old Microsoft tell you what you can and cannot do, are you? Be a man! Implement the feature!" And then the company will buckle under the pressure and implement the feature anyway.

    And then when the feature stops working at the next security hotfix (or worse, when security hotfixes stop working because you modified the file), the client will complain back to the customer, who will defensively say, "Um, yeah, stupid Microsoft broke the XYZ feature of our product. Bad! Bad Microsoft!" Or maybe the customer's plan is to change their name and move to another country so the client can't find them when everything falls apart.

Page 1 of 3 (27 items) 123