March, 2013

  • The Old New Thing

    How can I see what files and shares are being accessed remotely, and the general usage pattern for the NetXxx functions

    • 4 Comments

    Today's Little Program is a command line version of the Shared Folders MMC snap-in. Why? Because it illustrates the usage pattern for the Net­Xxx family of functions. (It's also a clone of the networking portion of the openfiles tool.)

    The Net­Xxx family of functions generally work like this:

    • You pass in some parameters that describe what you want. Server name, that sort of thing.
    • You pass a "level" parameter that describes what information you want.
    • The function allocates memory to hold the results you requested, and it returns a pointer to that memory through a bufptr parameter.
    • If the function returns an array, then
      • You can tell the function the maximum number of results you want.
      • The function tells you how much information it returned.
      • If the function did not retrieve all the results (because it exceeded your maximum), it tells you how to get the rest of them.
    • When you are finished, you free the memory with Net­Api­Buffer­Free.

    We'll start with the non-array case, since that is much simpler. Suppose you want to get the level 123 information for a Thing.

    THING_INFO_123 *pinfo123;
    if (NetThingGetInfo(pszThing,
                        123, (LPBYTE*)&pinfo123) == NERR_Success)
    {
        DoSomethingWith(pinfo123);
        NetApiBufferFree(pinfo123);
    }
    

    You call the function, passing the desired information level and a pointer to the variable you want to receive the results. You then use the results, and then free them. Let's try it with a simple function to get information about a user.

    #define UNICODE
    #define _UNICODE
    #define STRICT
    #include <windows.h>
    #include <lm.h>
    #include <stdio.h>
    
    void PrintProperty(PCWSTR pszProperty, PCWSTR pszValue)
    {
     wprintf(L"%ls: %ls\n", pszProperty,
                            pszValue ? pszValue : L"<none>");
    }
    
    int __cdecl wmain(int argc, wchar_t **argv)
    {
     USER_INFO_10 *pinfo10;
     if (NetUserGetInfo(NULL, L"Administrator", 10,
                        (LPBYTE*)&pinfo10) == NERR_Success) {
      PrintProperty(L"Name", pinfo10->usri10_name);
      PrintProperty(L"Comment", pinfo10->usri10_comment);
      PrintProperty(L"User comment", pinfo10->usri10_usr_comment);
      PrintProperty(L"Full name", pinfo10->usri10_full_name);
      NetApiBufferFree(pinfo10);
     }
     return 0;
    }
    

    The trickier case is the functions that return arrays of data. In that case, you need to call the functions in a loop, similar to Find­Next­File, in order to read all the data. But unlike Find­Next­File, the functions return chunks of data rather than just one entry at a time.

    The general pattern goes like this:

    THING_INFO_123 *pinfo123;
    NET_API_STATUS status;
    DWORD_PTR resumeHandle = 0;
    do {
     DWORD actual, estimatedTotal;
     status = NetThingEnum(pszThing, 123,
                           (LPBYTE*)&pinfo123,
                           MAX_PREFERRED_LENGTH,
                           &actual,
                           &estimatedTotal,
                           &resumeHandle);
     if (status == NERR_Success ||
         status == ERROR_MORE_DATA) {
      for (DWORD i = 0; i < actual; i++) {
       DoSomethingWith(&pinfo123[i]);
      }
      NetApiBufferFree(pinfo123);
     }
    } while (status == ERROR_MORE_DATA);
    

    The general pattern is to start by calling the data retrieval function. If the function returns with NERR_Success, then it means that it was able to get all the information you requested. If the function returns with ERROR_MORE_DATA, then it means that it was able to get some of the information you requested. In either of those two cases, it returns the actual number of items retrieved in the actual parameter, which you use to read the values out of the results. (It also returns an estimate of the total number of items remaining in the estimated­Total variable, but very few people use that.)

    If the return value was ERROR_MORE_DATA, then you go back and call the function again to get the next batch of results.

    The way the functions can tell whether you're starting a new operation or continuing an old one is via the resume­Handle parameter, which must be a pointer to a DWORD_PTR variable which the function updates. On the first call, set the DWORD_PTR to zero. If the function returns partial results, then it puts an opaque value into the resume­Handle so it can remember where it needs to continue. (By comparison, the Find­First­File passes the resume handle as its return value.)

    Note that there is no equivalent to Find­Close when you are finished with the function. If you don't want to retrieve all the results, you just abandon the handle.

    int __cdecl wmain(int argc, wchar_t **argv)
    {
     FILE_INFO_3 *pinfo3;
     NET_API_STATUS status;
     DWORD_PTR resumeHandle = 0;
     do {
      DWORD actual, estimatedTotal;
      status = NetFileEnum(NULL, NULL, NULL, 3,
                           (LPBYTE*)&pinfo3,
                           MAX_PREFERRED_LENGTH,
                           &actual,
                           &estimatedTotal,
                           &resumeHandle);
      if (status == NERR_Success ||
          status == ERROR_MORE_DATA) {
       for (DWORD i = 0; i < actual; i++) {
        PrintProperty(L"Path", pinfo3[i].fi3_pathname);
        PrintProperty(L"User", pinfo3[i].fi3_username);
        if (pinfo3[i].fi3_permissions & PERM_FILE_READ) {
         PrintProperty(L"Access", L"READ");
        }
        if (pinfo3[i].fi3_permissions & PERM_FILE_WRITE) {
         PrintProperty(L"Access", L"WRITE");
        }
        if (pinfo3[i].fi3_permissions & PERM_FILE_CREATE) {
         PrintProperty(L"Access", L"CREATE");
        }
       }
       NetApiBufferFree(pinfo3);
      }
     } while (status == ERROR_MORE_DATA);
     return 0;
    }
    

    I've been ignoring the parameter known as prefmaxlen because you pretty much always pass MAX_PREFERRED_LENGTH. The parameter lets you limit how much information is returned at a time, but you nearly always want as much as possible (which is why you nearly always pass MAX_PREFERRED_LENGTH). If, for some reason, you want to retrieve only a little bit at a time, you can pass a smaller value as the prefmaxlen. Note that prefmaxlen is in bytes, not elements, and the size in bytes needs to include the auxiliary data (like the strings), not just the structures. If you pass a custom prefmaxlen, then you also have to be prepared to handle the NERR_Buf­Too­Small error code, which means "The value you passed in prefmaxlen wasn't big enough to hold even one item. You'll have to try again with a bigger buffer size." If you're advanced enough to use a custom buffer size, then you're advanced enough to figure out how to tweak the algorithm to handle it properly.

    Note that I have no special knowledge of the Net­Xxxx family of functions. I figured this out by reading the documentation.

  • The Old New Thing

    Microsoft-internal Chuck Norris facts

    • 11 Comments

    A colleague of mine forwarded me some status mail from his team's internal bug push. (This is a push to fix bugs, not a push to introduce new bugs.) Apparently, one of the ways the developers lifted the tension was to discover some new Chuck Norris Facts, two of which were shared in the status mail:

    Chuck facts today:

    Happy birthday, Chuck!

    (Remember, don't let the facts get in the way of a good story.)

    Bonus: Google-internal Chuck Norris facts.

  • The Old New Thing

    Why do Explorer and the command prompt interpret file times differently?

    • 27 Comments

    A customer observed that if they use Explorer to view the timestamp on a file, it is not always in agreement with the value shown if they run a plain DIR in a command prompt. They are sometimes off by an hour. Why is that?

    Whenever you hear the phrase "off by an hour" you should immediately think "Daylight Saving Time".

    The formatting of file timestamps shown by Explorer has changed over time. The most recent algorithm (at the time of this writing) is to use the time zone that was in effect at your current location at the time the timestamp was created. For example, a file created at noon in June 22 will show its timestamp as noon, even if you view it in the middle of December. That's because Explorer says, "Well, on June 22, Daylight Saving Time was not in effect, even though it's in effect now, so I will interpret that time zone as if Daylight Saving Time were not active." (Hey, Raymond, didn't you get that backward? Answer: The customer who asked this question is in New Zealand.)¹

    The old-style function for converting a UTC timestamp into a local timestmap is File­Time­To­Local­File­Time. The documentation for that function points you at the sequence of operations you need to perform if you want to use the time zone of the timestamp instead of the current time zone.

    Explorer switched to using the time zone of the timestamp, but the command processor continues using the old-style conversion.

    Why doesn't the command processor get with the program?

    Well, for one thing, the people who decide what Explorer does are not the same people who decide what the command processor does. (The Explorer folks can certainly make suggestions, but they can't force the command processor to do anything.) It's like asking why Taco Bell puts the men's room on the left, but Pizza Hut puts it on the right.

    The command processor is an old and cranky program. The command processor yells at Explorer to get off his lawn. The command processor gets upset when his Internet connection flakes out while he's watching Matlock online. The command processor doesn't care about your fancy-pants localized file names; it shows the raw file system names. The command processor has hundreds of thousands of scripts, and there's no way of knowing how many of them depend on the exact way it formats dates.

    You may be able to wheedle the command processor into making some changes for you, but you'd better have a really good reason, and he's going to be really grumpy about it. The command processor was once cajoled into changing its date format to four-digit years back in the late 20th century, and he did it only because everybody insisted that it was soooooooo important. But he was so grumpy about it, he had an option to go back.

    ¹ Actually, that's not true. The customer who asked the question was in Texas, but I moved him to New Zealand for the purpose of the story. People in the Southern Hemisphere always have to put up with us Northerners assuming that summer starts in June, so I figured I'd turn the tables for once.

  • The Old New Thing

    What are the conventions for managing standard handles?

    • 31 Comments

    Consider this function:

    void ChangeConsoleColor(WORD wColor)
    {
     HANDLE h = GetStdHandle(STD_OUTPUT_HANDLE);
     if (h != INVALID_HANDLE_VALUE) {
      SetConsoleTextAttribute(h, wColor);
      CloseHandle(h);
     }
    }
    

    "When I call this function, it works the first time, but when I call it a second time, Get­Std­Handle returns a handle numerically identical to the one returned by the first call, but the handle is now invalid, presumably because I closed it. I closed it because I was taught to clean up after myself. Is this a case where I shouldn't?"

    Yes, you should clean up after yourself, but you should also have been taught to be respectful of community property. In this case, you walked into the TV room of your dormitory, watched an episode of Friends, and then smashed the television with a baseball bat. Later, you came back to the room to watch another episode of Friends and said, "Hey, what happened to our television?" (You can tell I'm old because I'm talking about the TV room of a dormitory.)

    The standard handle values are sort of like a global variable for your process. Anybody can call Get­Std­Handle to read the variable, and anybody can call Set­Std­Handle to set it. But as with any other global handle variable, you need to observe certain rules to ensure that the value is always valid.

    Suppose you had a global variable called HANDLE hSomeFile. What invariants would you want to apply?

    • If the value is INVALID_HANDLE_VALUE, then there is no active file. (You might also have decided to use NULL as your special value, but INVALID_HANDLE_VALUE works better here because that is the conventional sentinel value for file handles.)
    • If the value is not the special value above, then it refers to a valid file handle.

    That second invariant above already establishes a rule:

    • If you close the handle held in the global variable, you must also set the global variable to a new valid value.

    As I noted some time ago, programming is a game of stepping-stone from one island of consistency to another. You start with a consistent system, you perturb it (temporarily violating consistency), and then you re-establish consistency. Closing the handle makes the value invalid, so you need to follow up by making the value valid again. Otherwise you left your system in an inconsistent state.

    Okay, now instead of talking about that global variable hSomeFile, let's talk about the global handle hidden behind Get­Std­Handle and Set­Std­Handle. Congratulations, we just established the rules for managing standard handles.

    • If Get­Std­Handle returns INVALID_HANDLE_VALUE, then there is no active file.
    • If the value is not the special value above, then it refers to a valid file handle. (Note that file handles can refer to things that aren't files. In our case, it often will refer to a console.)
    • If you call Close­Handle on a standard handle, then you must also call Set­Std­Handle to set a new value for the standard handle.

    Note that these rules are just conventions. If you want to violate them by, say, closing the handle and then leaving a garbage handle in the hidden global variable for the next guy to trip over, then that's your problem. For example, you might choose to violate the rules temporarily, and then fix things up before anybody notices.

  • The Old New Thing

    What are the dire consequences of not selecting objects out of my DC?

    • 10 Comments

    The convention when working with device contexts is to restore them to the way you found them. If a drawing function selects a bitmap into a device context, then it should select the original bitmap into the device context before returning. Same for fonts, pens, all that stuff.

    But what if you decide to violate that convention? For example, maybe you create a memory DC, select a bitmap into it, and just leave the bitmap selected there, selecting it out only when you get around to destroying the DC. Is that really so bad?

    It sort of depends.

    The danger of leaving objects selected into a DC for an extended period of time is that the owner of the object won't be able to destroy the object, because you can't destroy objects while they are selected into a DC. For example, if you select a font into a DC, and somebody tries to destroy the font, the Delete­Object call will fail, and you end up leaking a font.

    Bitmaps can be selected into only one DC at a time. If you select the bitmap into your DC and just forget about it, then the owner of that bitmap won't be able to select it into any other DC.

    Now, if the objects you are selecting into the DC are all under your control, then you can leave them selected into your private DC, because you will know how to get them out if you need to.

    Remember that this "leave it lying around, I'll clean it up later" technique requires you to control both the vertical and the horizontal. We've been discussing what happens if you select an object that somebody else controls into your private DC and leave it there. Conversely, if you have a bitmap that you control and leave it selected into a DC that you don't control, then you've got the same sort of problem in reverse: You won't be able to select the bitmap back out of that DC when you need to, because you lost control of the DC.

    Bonus chatter: "I've noticed that sometimes, Delete­Object claims to succeed even though it actually failed because the object is still selected in a DC." The GDI folks found that a lot of people mess up and try to destroy objects while they are still selected into DCs. Failing the call caused two categories of problems: Some applications simply leaked resources (since they thought they were destroying the object, but weren't). Other applications checked the return value and freaked out if they saw that Delete­Object didn't actually delete the object.

    To keep both of these types of applications happy, GDI will sometimes (not always) lie and say, "Sure, I deleted your object." It didn't actually delete it, because it's still selected into a DC, but it also ties a string around its finger, and when the object is finally deselected, GDI will say, "Oh, wait, I was supposed to delete this object," and perform the deletion. So the lie that GDI made wasn't so much a lie as it was an "optimistic prediction of the future."

  • The Old New Thing

    Space Mountain as if the lights were on, and other Disneyland/World secrets

    • 5 Comments

    Many years ago, some friends of mine went to Disneyland and got to experience Space Mountain in an unusual way.

    They went through the ride the normal way, but when the car returned to the load/unload area, the cast member (because that's what Disney calls them) asked, "Do you want to go again?"

    Everybody in the car enthusiastically shouted, "Yes!"

    (Presumably they were falling behind on the loading and unloading, and waving a car through saved them a bit of time they could use to catch up.)

    The second time through the ride was qualitatively different: Their eyes had already acclimated to the dark, so even though they were going through the ride like everybody else, they could see the room as if the lights were on.

    There are a number of videos of Disneyworld Space Mountain with the lights on, captured from the PeopleMover, but that's not the same as riding it with the lights on.

    Other Disneyland secrets:

    I'm leaving for vacation today, so the blog will be on autopilot for a while. I might visit a certain theme park. You never know.

  • The Old New Thing

    Inadvertently creating dress-like-Steve day

    • 14 Comments

    Even if you haven't been paying much attention, you may have noticed that Steve Sinofsky has developed a bit of a uniform for himself. You can pretty much count on him wearing a T-shirt with a V-neck sweater. (There appears to be some sort of alphabetic theme there, but I'm not going to check if he's also wearing a G-string.)

    Some time ago, I realized as I was heading in to work that for the second day in a row, I happened to be wearing the Steve Sinofsky uniform. Was I subconsciously mirroring the boss?

    As fate would have it, later that morning, I happened to run into Steve as we both headed from one building to the next. I remarked on my accidental fashion choice and emphasized that I wasn't trying to stalk him or anything. Steve quipped, "Funny you should say that. Just this morning, I was thinking of wearing a suit."

    Bonus chatter: As we reached our destination, I opened the door to let Steve into the building. Steve declined. "I have to let myself into the building. The security folks track my access key, and if they see anything unusual, like if I beep into a building without having beeped out of the previous building, they call my cell phone to confirm my whereabouts and to verify that my access key hasn't been stolen."

  • The Old New Thing

    Around and around and back and somewhere else

    • 20 Comments

    I dreamed that I was navigating through an enormous house, with lots of twists and turns. I eventually became convinced that the house was only locally Euclidean.

    Yes, I sometimes have topology dreams.

  • The Old New Thing

    Marking a shortcut to say that it should not be placed on the Windows 8 Start page upon installation or treated as a newly-installed application

    • 15 Comments

    Today's Little Program creates a shortcut on the Start menu but marks it as "Do not put me on the front page upon installation." This is something you should do to any secondary shortcuts your installer creates. And while you're at it, you may as well set the "Don't highlight me as a newly-installed program" attribute used by Windows 7. (Remember, Little Programs do little to no error checking.)

    #define UNICODE
    #define _UNICODE
    #define STRICT
    #include <windows.h>
    #include <shlobj.h>
    #include <atlbase.h>
    #include <propkey.h>
    #include <shlwapi.h>
    
    int __cdecl wmain(int, wchar_t **)
    {
     CCoInitialize init;
    
     CComPtr<IShellLink> spsl;
     spsl.CoCreateInstance(CLSID_ShellLink);
    
     wchar_t szSelf[MAX_PATH];
     GetModuleFileName(GetModuleHandle(nullptr), szSelf, ARRAYSIZE(szSelf));
     spsl->SetPath(szSelf);
    
     PROPVARIANT pvar;
     CComQIPtr<IPropertyStore> spps(spsl);
    
     pvar.vt = VT_UI4;
     pvar.ulVal = APPUSERMODEL_STARTPINOPTION_NOPINONINSTALL;
     spps->SetValue(PKEY_AppUserModel_StartPinOption, pvar);
    
     pvar.vt = VT_BOOL;
     pvar.boolVal = VARIANT_TRUE;
     CComQIPtr<IPropertyStore> spps(spsl);
     spps->SetValue(PKEY_AppUserModel_ExcludeFromShowInNewInstall, pvar);
    
     spps->Commit();
    
     wchar_t szPath[MAX_PATH];
     SHGetSpecialFolderPath(nullptr, szPath, CSIDL_PROGRAMS, FALSE);
     PathAppend(szPath, L"Awesome.lnk");
     CComQIPtr<IPersistFile>(spsl)->Save(szPath, FALSE);
    
     return 0;
    }
    

    First, we create a shell link object.

    Next, we tell the shell link that its target is the currently-running program.

    Now the fun begins. We get the property store of the shortcut and set two new properties.

    We then commit those properties back into the shortcut.

    Finally, we save the shortcut.

  • The Old New Thing

    The source of much confusion: "backed by the system paging file"

    • 42 Comments

    Perhaps one of the most misunderstood sentences in the Win32 documentation is this little bit in the documentation for Create­File­Mapping:

    If hFile is INVALID_HANDLE_VALUE, the calling process must also specify a size for the file mapping object in the dwMaximum­Size­High and dwMaximum­Size­Low parameters. In this scenario, Create­File­Mapping creates a file mapping object of a specified size that is backed by the system paging file instead of by a file in the file system.

    When people read the underlined portion, they interpret this to mean "The data in the file mapping object will be written to the system paging file." But that's not what it says. It says that it is backed by the system paging file. In other words, "If I need to page this memory out, I will store it in the system paging file."

    Note the word "if".

    Usually, people get all worked up about the description because "I don't want this data to be written to disk by the creator, and then read from the disk by the consumer. I want this to be stored in RAM, just like the memory I allocate with Heap­Allocate or Virtual­Alloc." Of course, what they didn't realize is that memory allocated with Heap­Allocate and Virtual­Alloc is also backed by the system paging file. If memory allocated by Heap­Allocate and Virtual­Alloc needs to be paged out, the memory manager will write it to the paging file.

    In other words, "backed by the system paging file" just means "handled like regular virtual memory."

    If the memory is freed before it ever gets paged out, then it will never get written to the system paging file. Just like you wanted.

    The documentation was written with kernel-colored glasses. They figured that you knew that paging file-backed memory was just a way of saying "normal pageable memory."

    Exercise: What happens if paging is disabled? Where is the memory backed if there is no paging file?

Page 3 of 3 (30 items) 123