Tips/Support

  • The Old New Thing

    What is the real maximum length of a DNS name?

    • 18 Comments

    The maximum length of a DNS name is 255 octets. This is spelled out in RFC 1035 section 2.3.4. A customer didn't understand why the DnsValidateName was rejecting the following string:

    (63 letters).(63 letters).(63 letters).(62 letters)

    The documentation says

    Returns ERROR_INVALID_NAME if the DNS name

    • Is longer than 255 octets.
    • Contains a label longer than 63 octets.
    • ... other criteria not relevant here...

    The length of the domain name passed in is 63+1+63+1+63+1+62=254 characters, just under the length limit of 255. Why is it rejecting this name that is under the limit?

    Because the limit isn't the number of characters; it's the number of octets.

    Section 3.3 says that a domain-name is represented as a series of labels, and is terminated by a label of length zero. (The label of length zero represents the root label.) A label consists of a length octet followed by that number of octets representing the name itself. Therefore, the domain name www.microsoft.com is encoded as follows:

    3 'w' 'w' 'w' 9 'm' 'i' 'c' 'r' 'o' 's' 'o' 'f' 't' 3 'c' 'o' 'm' 0

    Technically, www.microsoft.com is shorthand for www.microsoft.com. with a trailing period, and the trailing zero byte encodes that implied period.

    If you sit down and do the math, you'll see that the the readable maximum length of an ASCII DNS name is 253 characters: You don't encode the dots, but you do encode the length bytes, so they cancel out, except for the length byte of the first label and the length byte of the root label, for an additional cost of two bytes. (On the off chance that you explicitly specified the root label, don't count it towards the 253-character limit.)

    If you use UTF-8 encoding, then the maximum length is harder to describe since UTF-8 is a variable-length encoding.

  • The Old New Thing

    Why does Explorer ignore seconds when sorting by Date Modified?

    • 39 Comments

    A customer reported that Explorer appears to be ignoring the seconds when sorting by Date Modified. The customer was kind enough to include detailed steps to reproduce the problem.

    Start with a folder with several files, sorted by Date Modified.

    Name Date modified Type
    TPS 06-05-2012 Report 6/12/2012 7:00 AM Contoso Document
    TPS 05-29-2012 Report 6/5/2012 11:30 AM Contoso Document
    TPS 05-22-2012 Report 5/29/2012 10:17 AM Contoso Document
    TPS 05-15-2012 Report 5/22/2012 2:35 PM Contoso Document
    TPS 05-09-2012 Report 5/15/2012 11:26 AM Contoso Document
    TPS 05-02-2012 Report 5/9/2012 10:31 AM Contoso Document

    Right-click on the newest file, select Copy.

    Right-click on the blank column on the right, select Paste. This will create a file with the same name, but with "- Copy" appended.

    Press F5 to refresh the view and note the sort order. The copy appears at the top of the list.

    Name Date modified Type
    TPS 06-05-2012 Report - Copy 6/12/2012 7:00 AM Contoso Document
    TPS 06-05-2012 Report 6/12/2012 7:00 AM Contoso Document
    TPS 05-29-2012 Report 6/5/2012 11:30 AM Contoso Document
    TPS 05-22-2012 Report 5/29/2012 10:17 AM Contoso Document
    TPS 05-15-2012 Report 5/22/2012 2:35 PM Contoso Document
    TPS 05-09-2012 Report 5/15/2012 11:26 AM Contoso Document
    TPS 05-02-2012 Report 5/9/2012 10:31 AM Contoso Document

    Highlight the newly-created file, hit F2, and give the document a different name, and also remove the "- Copy" suffix. Hit Enter to accept the operation.

    Press F5 to refresh the view again. Notice that the file that you just renamed, which is the newest file in the folder (it having just been created seconds ago) appears second in the list.

    Name Date modified Type
    TPS 06-05-2012 Report 6/12/2012 7:00 AM Contoso Document
    TPS 06-12-2012 Report 6/12/2012 7:00 AM Contoso Document
    TPS 05-29-2012 Report 6/5/2012 11:30 AM Contoso Document
    TPS 05-22-2012 Report 5/29/2012 10:17 AM Contoso Document
    TPS 05-15-2012 Report 5/22/2012 2:35 PM Contoso Document
    TPS 05-09-2012 Report 5/15/2012 11:26 AM Contoso Document
    TPS 05-02-2012 Report 5/9/2012 10:31 AM Contoso Document

    It appears that Explorer is ignoring the seconds in the Date Modified column and sorting only by the hour and minute.

    It's an interesting theory the customer came up with, but the customer was fooled by the fact that he ran the experiment shortly after modifying the TPS 06-05-2012 Report document, so that the real behavior was masked.

    When you copy a file, the system preserves the date-modified timestamp. The Date modified column is not ignoring the seconds; in fact, it's comparing them quite carefully, but since the timestamps of the original and the copy are the same, the timestamps compare equal. And when the items compare equal according to the sort criteria, Explorer falls back to sorting by name, and the fallback sort is always ascending.

    The confusion would have been cleared up if the Date modified column used the long time format instead of the short time format, but that only pushes the problem to files whose timestamps are fractional seconds apart. You have two files which show up as "6/12/2012 7:00:12 AM" and don't realize that one of them is "6/12/2012 7:00:12.02 AM" and the other is "6/12/2012 7:00:12.89 AM". My guess is that geeks won't be satisfied until Explorer shows the full 64-bit FILETIME so you can see the difference down to the 100-nanosecond interval.

    (If you want to see the seconds, you can look on the file's property sheet.)

  • The Old New Thing

    How can I customize which notification icons are displayed by default on a new installation?

    • 22 Comments

    Windows 7 provides a new Unattended Windows Setup setting known as NotificationArea. This setting lets you replace the default Action Center and Battery icons with two icons of your choosing.

    To specify the replacement icons, you need to provide the path to the application which is providing the replacement icon, and you need the GUID for the replacement icon.

    The path you know how to get, because it's where the application was installed. (Note that the application must also be signed.)

    But where do you get the GUID from?

    The GUID is provided by the application as part of the programmatic interface to the notification area. When the application creates its notification icon, it passes a structure known as NOTIFY­ICON­DATA, and one of the fields in that structure is the guidItem. If an application fills in the guidItem and sets the NIF_GUID flag, then that is telling the taskbar (among other things), "Hey, if an unattended setup file specifies this GUID as a replacement icon, that's me."

    Okay, that says where GUIDs are programmatically specified, but where you do you, the system administrator, get the GUID from?

    You get them by asking the application author, "Hi, we want to specify that your notification icon is displayed by default in Windows 7. Can you please tell us what GUID you are using for your notification icon?"

    Given that application authors are always angling for a bonus, they will probably be more than happy to tell you how to give their icon more guaranteed face-time with the user.

    This setting was originally designed for customization by computer manufacturers, and computer manufacturers will probably have a pretty close relationship with the companies that provide shov^H^H^H^Hvalue-added software for their systems.

  • The Old New Thing

    Why don't music files show up in my Recent Items list?

    • 19 Comments

    If you double-click a music file, it doesn't show up in your Recent Items list. What's so special about music files?

    The technical reason is that the file types are registered with the FTA_No­Recent­Docs flag, which means that they don't show up in the Recent Items list (formerly known as Recent Documents), and they don't show up in the Recent or Frequent section of Windows Media Player's Jump List.

    Okay, fine, but that's like answering "Why is there a door here?" with "Because the blueprints said that there should be a door there." You really want to know why the architect decided to put a door there.

    The reason why music files are not placed in the Recent Items list is that individual songs are not that interesting there. If you spend an hour listening to music, that will fill up your Recent Items list and push out everything else. The list ends up simply telling you about the thing you just finished doing. Not all that useful.

    Therefore, Windows Media Player excludes individual music files from the Recent Items list. Instead, it puts artists and albums in the Jump List, higher-level information that they figure is more interesting than just a list of the songs you just heard.

  • The Old New Thing

    Why was there a font just for drawing symbols on buttons?

    • 19 Comments

    Henke37 wonders why the Marlett font was introduced. Why use a font for drawing symbols on window buttons?

    Using a font was a convenient way to have scalable graphics.

    It's not like Windows could've used VML or SVG since they hadn't been invented yet. EMFs would have been overkill as well. Fonts were very convenient because the technology to render scalable fonts already existed and was well-established. It's always good to build on something that has been proven, and TrueType scalable font technology proved itself very nicely in Windows 3.1. TrueType has the added benefit of supporting hinting, allowing tweaks to the glyph outlines to be made for particular pixel sizes. (A feature not available in most vector drawing languages, but also a feature very important when rendering at small font sizes.)

  • The Old New Thing

    Why don't ZIP files have the FILE_ATTRIBUTE_COMPRESSED attribute?

    • 25 Comments

    A customer reported that when they called Get­File­Attributes on a ZIP file, the FILE_ATTRIBUTE_COMPRESSED attribute was not returned. But ZIP files are compressed. Why isn't the FILE_ATTRIBUTE_COMPRESSED attribute being set?

    Because FILE_ATTRIBUTE_COMPRESSED tells you whether the file was compressed by the file system. It is not a flag which describes the semantics of the bytes stored in the file. After all, the file system doesn't know that this particular collection of bytes is a ZIP file and contains data that was compressed externally. Who knows, maybe it's just some uncompressed file that just happens to look superficially like a ZIP file (but isn't)?

    If a text file consists of the string "ADTUR ADKUH", is this a compressed file? Maybe it's somebody's product key, in which it isn't compressed. Or maybe it is short for "Await instructions before taking further action. Acknowledge receipt of this telegram by wire." That's an example of a commercial code, used to save telegram transmission costs by compressing frequently-used business phrases into five-letter pseudo-words.

    The file system doesn't try to figure out whether a particular sequence of bytes it has been asked to store was externally compressed. It just stores the bytes on disk, perhaps after performing its own internal compression, and if that internal compression was performed (even if it didn't actually result in any compression), the FILE_ATTRIBUTE_COMPRESSED attribute is set.

    Similarly, the FILE_ATTRIBUTE_ENCRYPTED attribute is set if the file contents were encrypted by the file system. If encryption took place externally, then the attribute is not set because the file system doesn't know that the byte sequence it was asked to store represented encrypted data.

    (Note that many special-purpose file formats, such as DOCX, JAR, JPG, and PNG, are internally compressed, even though they are not advertised as such.)

  • The Old New Thing

    Why is the file size reported incorrectly for files that are still being written to?

    • 28 Comments

    The shell team often gets questions like these from customers:

    Attached please find a sample program which continuously writes data to a file. If you open the folder containing the file in Explorer, you can see that the file size is reported as zero. Even manually refreshing the Explorer window does not update the file size. Even the dir command shows the file size as zero. On the other hand, calling Get­File­Size reports the correct file size. If I close the file handle, then Explorer and the dir command both report the correct file size. We can observe this behavior on Windows Server 2008 R2, but on Windows Server 2003, the file sizes are updated in both Explorer and dir. Can anybody explain what is happening?
    We have observed that Windows gives the wrong file size for files being written. We have a log file that our service writes to, and we like to monitor the size of the file by watching it in Explorer, but the file size always reports as zero. Even the dir command reports the file size as zero. Only when we stop the service does the log file size get reported correctly. How can we get the file size reported properly?
    We have a program that generates a large number of files in the current directory. When we view the directory in Explorer, we can watch the files as they are generated, but the file size of the last file is always reported as zero. Why is that?

    Note that this is not even a shell issue. It's a file system issue, as evidenced by the fact that a dir command exhibits the same behavior.

    Back in the days of FAT, all the file metadata was stored in the directory entry.

    The designers of NTFS had to decide where to store their metadata. If they chose to do things the UNIX way, the directory entry would just be a name and a reference to the file metadata (known in UNIX-land as an inode). The problem with this approach is that every directory listing would require seeking all over the disk to collect the metadata to report for each file. This would have made NTFS slower than FAT at listing the contents of a directory, a rather embarrassing situation.

    Okay, so some nonzero amount of metadata needs to go into the directory entry. But NTFS supports hard links, which complicates matters since a file with multiple hard links has multiple directory entries. If the directory entries disagree, who's to say which one is right? One way out would be try very hard to keep all the directory entries in sync and to make the chkdsk program arbitrary choose one of the directory entries as the "correct" one in the case a conflict is discovered. But this also means that if a file has a thousand hard links, then changing the file size would entail updating a thousand directory entries.

    That's where the NTFS folks decided to draw the line.

    In NTFS, file system metadata is a property not of the directory entry but rather of the file, with some of the metadata replicated into the directory entry as a tweak to improve directory enumeration performance. Functions like Find­First­File report the directory entry, and by putting the metadata that FAT users were accustomed to getting "for free", they could avoid being slower than FAT for directory listings. The directory-enumeration functions report the last-updated metadata, which may not correspond to the actual metadata if the directory entry is stale.

    The next question is where and how often this metadata replication is done; in other words, how stale is this data allowed to get? To avoid having to update a potentially unbounded number of directory entries each time a file's metadata changed, the NTFS folks decided that the replication would be performed only from the file into the directory entry that was used to open the file. This means that if a file has a thousand hard links, a change to the file size would be reflected in the directory entry that was used to open the file, but the other 999 directory entries would contain stale data.

    As for how often, the answer is a little more complicated. Starting in Windows Vista (and its corresponding Windows Server version which I don't know but I'm sure you can look up, and by "you" I mean "Yuhong Bao"), the NTFS file system performs this courtesy replication when the last handle to a file object is closed. Earlier versions of NTFS replicated the data while the file was open whenever the cache was flushed, which meant that it happened every so often according to an unpredictable schedule. The result of this change is that the directory entry now gets updated less frequently, and therefore the last-updated file size is more out-of-date than it already was.

    Note that even with the old behavior, the file size was still out of date (albeit not as out of date as it is now), so any correctly-written program already had to accept the possibility that the actual file size differs from the size reported by Find­First­File. The change to suppress the "bonus courtesy updates" was made for performance reasons. Obviously, updating the directory entries results in additional I/O (and forces a disk head seek), so it's an expensive operation for relatively little benefit.

    If you really need the actual file size right now, you can do what the first customer did and call Get­File­Size. That function operates on the actual file and not on the directory entry, so it gets the real information and not the shadow copy. Mind you, if the file is being continuously written-to, then the value you get is already wrong the moment you receive it.

    Why doesn't Explorer do the Get­File­Size thing when it enumerates the contents of a directory so it always reports the accurate file size? Well, for one thing, it would be kind of presumptuous of Explorer to second-guess the file system. "Oh, gosh, maybe the file system is lying to me. Let me go and verify this information via a slower alternate mechanism." Now you've created this environment of distrust. Why stop there? Why not also verify file contents? "Okay, I read the first byte of the file and it returned 0x42, but I'm not so sure the file system isn't trying to trick me, so after reading that byte, I will open the volume in raw mode, traverse the file system data structures, and find the first byte of the file myself, and if it isn't 0x42, then somebody's gonna have some explaining to do!" If the file system wants to lie to us, then let the file system lie to us.

    All this verification takes an operation that could be done in 2 + N/500 I/O operations and slows it down to 2 + N/500 + 3N operations. And you're reintroduced all the disk seeking that all the work was intended to avoid! (And if this is being done over the network, you can definitely feel a 1500× slowdown.) Congratulations, you made NTFS slower than FAT. I hope you're satisfied now.

    If you were paying close attention, you'd have noticed that I wrote that the information is propagated into the directory when the last handle to the file object is closed. If you call Create­File twice on the same file, that creates two file objects which refer to the same underlying file. You can therefore trigger the update of the directory entry from another program by simply opening the file and then closing it.

    void UpdateFileDirectoryEntry(__in PCWSTR pszFileName)
    {
        HANDLE h = CreateFileW(
            pszFileName,
            0,                  // don't require any access at all
            FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE,
            NULL,               // lpSecurityAttributes
            OPEN_EXISTING,
            0,                  // dwFlagsAndAttributes
            NULL);              // hTemplateFile
        if (h != INVALID_HANDLE_VALUE) {
            CloseHandle(h);
        }
    }
    

    You can even trigger the update from the program itself. You might call a function like this every so often from the program generating the output file:

    void UpdateFileDirectoryEntry(__in HANDLE hFile)
    {
        HANDLE h = ReOpenFile(
            hFile,
            0,                  // don't require any access at all
            FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE,
            0);                 // dwFlags
        if (h != INVALID_HANDLE_VALUE) {
            CloseHandle(h);
        }
    }
    

    If you want to update all file directory entries (rather than a specific one), you can build the loop yourself:

    // functions ProcessOneName and EnumerateAllNames
    // incorporated by reference.
    
    void UpdateAllFileDirectoryEntries(__in PCWSTR pszFileName)
    {
        EnumerateAllNames(pszFileName, UpdateFileDirectoryEntry);
    }
    

    Armed with this information, you can now give a fuller explanation of why Read­Directory­ChangesW does not report changes to a file until the handle is closed. (And why it's not a bug in Read­Directory­ChangesW.)

    Bonus chatter: Mind you, the file system could expose a flag to a Find­First­File-like function that means "Accuracy is more important than performance; return data that is as up-to-date as possible." The NTFS folks tell me that implementing such a flag wouldn't be all that hard. The real question is whether anybody would bother to use it. (If not, then it's a bunch of work for no benefit.)

    Bonus puzzle: A customer observed that whether the file size in the directory entry was being updated while the file was being written depended on what directory the file was created in. Come up with a possible explanation for this observation.

    Bonus reading:

  • The Old New Thing

    A feature I didn't even know existed much less had a name: Color hot-track

    • 38 Comments

    I hadn't even noticed this until somebody pointed it out: When you hover your mouse over a button in the Windows 7 taskbar which corresponds to a running application, the taskbar button lights up in a color that matches the colors in the icon itself. (And even more subtly, the lighting effect is centered on the mouse.)

    This feature even has a name: Color hot-track. (Gentlemen, start your photocopiers.)

    Some people ask how it's done. It's really nothing special. The code just looks for the predominant color in the icon. (And, since visual designers are sticklers for this sort of thing, black, white, and shades of gray are not considered "colors" for the purpose of this calculation.)

  • The Old New Thing

    Why do some infotips repeat the name of the item as well as the infotip?

    • 17 Comments

    A customer noticed that when the user hovered over their application name in the Start menu, the infotip that pops up includes their product name:

    ♫ 
    Contoso Professional Music Studio Deluxe 2010
    Record and mix studio-quality music from your computer.
    Contoso Professional Music Studio De...
    ◕  Fabrikam Chart 2.0
    ℒ  Litware 2010

    ... but no other program on the Start menu included the product name in the description:

    ◕ 
    Design and print charts and graphs with ease.
    Fabrikam Chart 2.0
    ♫  Contoso Professional Music Studio De...
    ℒ  Litware 2010

    The customer compared their shortcut with the other ones but couldn't find anything that was telling Explorer, "Include the program name in the pop-up infotip, please."

    Because the reason for the name being included in the infotip had nothing to do with the properties stored in the shortcut. The reason the name was included in the infotip is that the name was being truncated in the main display.

    When an infotip is about to be displayed for a listview item, the listview sends a LVN_GET­INFO­TIP notification with a NMLV­GET­INFO­TIP structure. If the LVGIT_UNFOLDED flag is not set, then the infotip is being displayed for a truncated item, and the pszText is pre-filled with the full name. The program should then append its information to the existing text so that the full name is the first line of the infotip. On the other hand, if the LVGIT_UNFOLDED flag is set, then the item text is fully-visible and you should just copy your desired description text into the pszText buffer.

    The customer was happy to get this information. Their designer wanted only the description to appear in the infotip, and now they know that they need to shorten the program name to make the name disappear from the infotip.

    Bonus chatter: Microsoft® WinFX™ Software Development Kit for Microsoft® Pre-Release Windows Operating System Code-Named "Longhorn", Beta 1 Web Setup.

  • The Old New Thing

    Why does copying a file to my USB thumb drive say that the parameter is incorrect?

    • 40 Comments

    Consider the following sequence of operations, assuming that F: is a USB thumb drive with plenty of disk space.

    C:\Users\Bob\Downloads> copy readme.txt F:\
            1 file(s) copied.
    C:\Users\Bob\Downloads> copy Update.iso F:\
    The parameter is incorrect.
    

    Why is the second file copy failing?

    The hint is the file extension: *.iso, which suggests that this is a CD or DVD image, and DVD images have the feature that they tend to be really big.

    Like more than 4GB big.

    USB thumb drives tend to be formatted with the FAT32 file system rather than with NTFS. And FAT32 has a maximum file size of 4GB minus one byte.

    The user confirmed that the Update.iso file was larger than 4GB and that the USB thumb drive was formatted as FAT32.

    Mind you, the error message doesn't help at all in identifying that this is what's going on. I don't know where it's coming from, but my guess is that somewhere inside the copy command, it tries to create the destination file and set its file size. Since the file size is out of range for FAT32, the call fails with the error ERROR_INVALID_PARAMETER, and that's what ends up bubbling out to the user.

    But at least now you know what the confusing error message is trying to tell you.

Page 7 of 25 (244 items) «56789»