September, 2011

  • The Old New Thing

    Why doesn't the Disk Management snap-in incorporate S.M.A.R.T. data?

    • 27 Comments

    My article a while back on Why the Disk Management snap-in reports my volume as Healthy when the drive is dying gave the low-level explanation of why the Disk Management snap-in does not incorporate SMART information: because the Disk Management snap-in is concerned with volume partitioning. DWalker59 noted that the use of the word "Healthy" carries more meaning than the authors of the snap-in intended. The authors of the snap-in assumed that everybody knew what the Disk Management snap-in was for, and therefore everybody know that the word "Healthy" applied to the state of the file system.

    I never said that this was a good situation, and commenter Dog interpreted that since I didn't say whether this was a good situation or a bad situation, I must be saying that it's a good situation. Actually, since I didn't say whether this was a good situation or a bad situation, this means that I'm not saying whether this is a good situation or a bad situation. The article was posted in the Tips/Support category, which is about helping you cope with the frustrations of using Windows, not about passing value judgements on what is good or bad. The point was not to say what is good and what is bad, but merely to say what is.

    Dog thinks that the blog would be far more interesting if I shared my opinion on things. Actually, I try not to share my opinion on things, because the Web site isn't about opinionating on Windows; it's about practical programming on Windows. Practicality means that you have to set aside whether something is good or bad, because it's there and you have to deal with it regardless. If you want opinionated writing, check out Robert Scoble or Michael Kaplan. Dog also assumes that Microsoft's PR department has told me not to opinionate on things. In fact, they haven't told me anything one way or the other (yet, and I hope it stays that way).

    (I found it interesting that Dog claims that "the act of reporting on [something] gives the appearance of support unless otherwise stated." I wonder if people who cover armed conflicts have to add an explicit statement along the lines of "killing is bad" so Dog won't think they support people shooting at each other.)

    From a historical standpoint, the situation is a bit more understandable. After all, the Disk Management snap-in was written long before support for S.M.A.R.T. information showed up in Windows Vista. You can't fault the original authors of the Disk Management snap-in for not taking into account data which didn't exist yet.

    As for why the Disk Management snap-in didn't incorporate this information when it became available, this assumes that there were resources available to do the work. Disk Management is a very old snap-in that hasn't changed much since it was first written. My suspicion is that maintenance of the Disk Management snap-in is assigned to a group which has as its primary goal some other part of the system; they were just given Disk Management because it has to belong to somebody. Consequently, that group has very little incentive to make any changes to Disk Management at all, and certainly has very little incentive to add features to it.

  • The Old New Thing

    Why is the registry a hierarchical database instead of a relational one?

    • 33 Comments

    Commenter ton asks why the registry was defined as a hierarchical database instead of a relational database.

    Heck, it's not even a hierarchical database!

    The original registry was just a dictionary; i.e., a list of name/value pairs, accessed by name. In other words, it was a flat database.

    .txt txtfile
    txtfile Text Document
    txtfile\DefaultIcon notepad.exe,1
    txtfile\shell open
    txtfile\shell\open\command notepad %1

    If you turned your head sideways and treated the backslashes as node separators, you could sort of trick yourself into believing that this resulted in something vaguely approximating a hierarchical database, and a really lame one at that (since each node held only one piece of data).

    When you choose your data structures, you necessarily are guided by the intended use pattern and the engineering constraints. One important engineering constraint was that you have to minimize memory consumption. All of the registry code fit in 16KB of memory. (Recall that Windows 3.1 had to run on machines with only 1MB of memory.)

    Okay, what is the usage pattern of the registry? As originally designed, the registry was for recording information about file types. We have the file types themselves (txtfile), properties about those file types (DefaultIcon), verbs associated with those file types (open), and verb implementations (command or ddeexec). Some verb implementations are simple (command involves just a single string describing the command line); others are complex (ddeexec requires the execute string, the application, and the topic, plus an optional alternate execute string).

    • Given a file type and a property, retrieve the value of that property.
    • Given a file type and a verb, retrieve information about how to perform that verb.
    • The set of properties can be extended.
    • The set of property schemata can be extended.
    • The set of verbs can be extended.
    • The set of verb implementations can be extended.

    Since the properties and verb implementations can be extended, you can't come up with a single schema that covers everything. For example, over the years, new file type properties have been added such as ContentType, OpenWithList, and ShellNew. The first one is a simple string; the second is a list of strings, and the third is a complex key with multiple variants. Meanwhile, additional verb implementations have been added, such as DropTarget.

    Given the heterogeneity of the data the registry needs to keep track of, imposing some sort of uniform schema is doomed to failure.

    "But you can just update the schemata each time the registration is extended."

    That creates its own problems. For example, to support roaming user profiles, you need a single registry hive to work on multiple versions of the operating system. If version N+1 adds a new schema, but then the profile roams to a machine running version N, then that registry hive will be interpreted as corrupted since it contains data that matches no valid schema.

    "Well, then include the schemata with the roaming profile so that when the older operating system sees the hive, it also sees the updated schemata."

    This is trickier than it sounds, because when the profile roams to the newer operating system, you presumably want the schemata to be upgraded and written back into the user profile. It also assumes that the versioning of the schemata is strictly linear. (What if you roam a user profile from a Windows XP machine to a Server 2003 machine? Neither is a descendant of the other.)

    But what kills this proposal is that it makes it impossible for a program to "pre-register" properties for a future version of the operating system. Suppose a new schema is added in version N+1, like, say, the IDropTarget verb implementation. You write a program that you want to run on version N as well as on version N+1. If your installer tries to register the version N+1 information, it will fail since there is no schema for it. But that means that when the user upgrades to version N+1, they don't get the benefit of the version N+1 feature. In order to get the version N+1 feature to work, they have to reinstall the program so the installer says, "Oh, now I can register the version +1 information."

    "Well, then allow applications to install a new schema whenever they need to."

    In other words, make it a total free-for-all. In which case, why do you need a schema at all? Just leave it as an unregulated collection of name/value pairs governed by convention rather than rigid rules, as long as the code which writes the information and the code which reads it agree on the format of the information and where to look for it.

    Hey, wow, that's what the registry already is!

    And besides, if you told somebody, "Hi, yeah, in order to support looking up four pieces of information about file types, Windows 3.1 comes with a copy of SQL Server," they would think you were insane. That's like using a bazooka to kill a mosquito.

    What are you planning on doing with this relational database anyway? Are you thinking of doing an INNER JOIN on the registry? (Besides, the registry is already being abused enough already. Imagine if it were a SQL server: Everybody would store all their data in it!)

    ton explains one way applications could use this advanced functionality:

    An application would have a table or group of tables in relational style registry. A group of settings would be a row. A single setting would be a column. Is it starting to become clearer now how SQL like statements could now be used to constrain what gets deleted and added? How good is your understanding of SQL and DBMS?

    You know what most application authors would say? They would say "Are you mad? You're saying that I need to create a table with one column for each setting? And this table would have a single row (since I have only one application)? All this just so I can save my window position? Screw it, I'm going back to INI files." What'll happen in practice is that everybody will create a table with two columns, a string called name and a blob called value. Now we've come full circle: We have our flat database again.

    And how would they make sure the name of their table doesn't collide with the name of a table created by another application? Probably by encoding the company name and application name into the name of the table, according to some agreed-upon convention. Like say, the Settings table used by the LitSoft program written by LitWare would be called LitWare_LitSoft_Settings. So querying a value from this table would go something like

    SELECT value FROM PerUser.LitWare_LitSoft_Settings
        WHERE name = "WindowPosition"
    

    Hey, this looks an awful lot like

    Registry.CurrentUser.OpenSubKey(@"LitWare\LitSoft\Settings")
            .GetValue("WindowPosition");
    

    One of ton's arguments for using a relational database is that it permits enforcement of referential integrity. But I would argue that in the general case, you don't want strict enforcement of referential integrity. Suppose you uninstall a program. The uninstaller tries to delete the program registration, but that registration is being referenced by foreign keys in other tables. These references were not created by the application itself; perhaps the shell common dialog created them as part of its internal bookkeeping. If the registry blocked the deletion, then the uninstall would fail. "Cannot uninstall application because there's still a reference to it somewhere." And that reference might be in Bob's user profile, from that time Bob said, "Hey can I log onto your machine quickly? I need to look up something." Bob is unlikely to come back to your machine any time soon, so his user profile is just going to sit there holding a reference to that application you want to uninstall for an awfully long time. "Hi, Bob, can you come by my office? I need you to log on so I can uninstall an app."

    So let's assume it goes the other way: The registry automatically deletes orphaned foreign key rows. (And for hives that are not currently available, it just remembers that those foreign key rows should be deleted the next time they are loaded. Nevermind that that list of "foreign key rows that should be deleted the next time Bob logs on" is going to get pretty long.)

    Now suppose you're uninstalling a program not because you want to get rid of it, but because you're doing an uninstall/reinstall troubleshooting step. You uninstall the program, all the orphaned foreign key rows are automatically deleted, then you reinstall the program. Those orphaned foreign key rows are not undeleted; they remain deleted. Result: You lost some settings. This is the reason why you don't clean up per-user data when uninstalling programs.

    Enforcing referential integrity also means that you can't create anticipatory references. One example of this was given earlier, where you register something on version N even though the feature doesn't get activated until the user upgrades to version N+1. More generally, Program X may want to create a reference to Program Y at installation, even if program Y isn't installed yet. (For example, X is a Web browser and Y is a popular plug-in.) The Program Y features remain dormant, because the attempt by Program X to access Program Y will fail, but once the user installs Program Y, then the Program Y features are magically "turned on" in Program X.

    Consider, as an even more specific example, the "kill bit" database. There, the goal isn't to "turn on" features of Program Y but to turn them off. Imagine if referential integrity were enforced: You couldn't kill an ActiveX control until after it was installed!

  • The Old New Thing

    What happened to that suspicious-looking guy hanging around the entrance?

    • 14 Comments

    One of the fun parts of attending a conference is swapping stories with other professionals. Today's story is in honor of Global Security Week. (And retroactively in honor of the upcoming //build conference.)

    One of the attendees (let's call him Bob) shared with me a story of the time they had to make a change in one of their data centers. This particular change required physical presence at the facility, and to minimize impact on customers, the change was made at night (presumably because that's when demand was lowest).

    When Bob arrived at the data center, he walked past a suspicious-looking guy on his way to the door. Bob said hello to the security guard and swiped his access card through the card reader.

    Access denied.

    He tried again. Still no luck.

    He asked the security guard, "Hi, I'm supposed to be here to do a system upgrade, but my access card isn't working. Maybe they just put my name on a list instead of reprogramming the access system?"

    The security guard said, "Let's go down to the main office and find out."

    The two of them walked down to the main security office, and after some checking, everything got straightened out, and Bob and the security guard headed back to the entrance. As they left, the security guard asked the person at the main security desk, "Oh, there's a suspicious-looking guy hanging around the entrance. Did you see him?"

    The person at the main security desk said, "Yeah, he's hiding in the bushes."

    Bob (and I) found it interesting, amusing, and reassuring that the security guards were all fully aware of the suspicious guy and had been keeping tabs on him the whole time.

  • The Old New Thing

    Thanks for letting me know what my ideal career and company are

    • 7 Comments

    When it's performance review season, all of a sudden you start getting mail about career management. What a coincidence. There are a variety of career management tools available, some mandatory, some optional. I gave one of the optional ones a shot, since it claimed to help me "manage my career and professional development", and as I already noted, I appear to have been promoted by mistake all these years, so maybe I should figure out how to get promoted for real.

    This particular tool sends me to the Web site of an external company that was contracted by Microsoft to provide career guidance services. I went through the sign-up process and answered what seemed like a bazillion questions. You know you're in trouble when you're getting tired and the progress bar says that you're currently filling out questionnaire number 1 (of 3) and you're on page 4 (of 19).

    Anyway, I make it through to the end of all the questions and the site offers suggestions as to what my ideal career would be, based on the personality characteristics I demonstrated in the questionnaire.

    It says that I would do well working in the field of information technology and that the best company for me is one with well-established processes and procedures, where decisions are guided by practicality and pragmatism, a characteristic common to companies that are market leaders.

    I don't know whether I should be disappointed that I didn't learn anything new or whether I should be relieved that I'm not missing out on my secret calling to be a topiarist or something.

    (And I guess that having me work at Microsoft plays right into Google's hands, so it's a win-win.)

  • The Old New Thing

    What's the story with the parameters to the WM_INPUT_DEVICE_CHANGE message?

    • 13 Comments

    A customer found these strange macros in winuser.h:

    #if (_WIN32_WINNT >= 0x0601)
    #define GET_DEVICE_CHANGE_WPARAM(wParam)  (LOWORD(wParam))
    #elif (_WIN32_WINNT >= 0x0501)
    #define GET_DEVICE_CHANGE_LPARAM(lParam)  (LOWORD(lParam))
    #endif /* (_WIN32_WINNT >= 0x0601) */
    

    According to the documentation for the WM_INPUT_DEVICE_CHANGE message, the wParam is the operation code and the lParam is a handle to the device that changed. Given that definition, the correct macro would be GET_DEVICE_CHANGE_WPARAM. What's up with the bogus GET_DEVICE_CHANGE_LPARAM macro?

    The macro was incorrectly defined in Windows Vista. In the Windows 7 version of the Platform SDK, the correct macro was added, but in order to avoid introducing a breaking change to existing code, the old broken macro remains in place in order to retain bug-for-bug compatibility with existing code.

    Even though the macro didn't work, there is a good possibility that code exists which relied on it anyway. For example, people may have read the documentation, read the macro, realized that the macro was wrong, and worked around the bug like this:

    case WM_INPUT_DEVICE_CHANGE:
     return OnInputDeviceChange(GET_DEVICE_CHANGE_LPARAM(wParam),
                                (HANDLE)lParam);
    

    To avoid breaking this code, the old broken definition remains in the header file. But at least it's defined only if you say that you want the Windows Vista version of the header file, so at least people won't use the bad macro going forward.

  • The Old New Thing

    Invoking commands on items in the Recycle Bin

    • 14 Comments

    Once you've found the items you want in the Recycle Bin, you may want to perform some operation on them. This brings us back to our old friend, IContextMenu. At this point, you're just snapping two blocks together. You have one block called Retrieving properties from items in the Recycle Bin and you have another block called Invoking verbs on items.

    For the first block, let's assume you've written a function called WantToRestoreThisItem which studies the properties of a Recycle Bin item and determines whether you want to restore it. I leave this for you to implement, since I don't know what your criteria are. Maybe you want to restore files only if they were deleted from a particular directory. Maybe you want to restore files that were deleted while you were drunk. (This assumes you have some other computer program that tracks when you're drunk.)¹ Whatever. It's your function.

    For the second block, we have a helper function which should look awfully familiar.

    void InvokeVerb(IContextMenu *pcm, PCSTR pszVerb)
    {
     HMENU hmenu = CreatePopupMenu();
     if (hmenu) {
      HRESULT hr = pcm->QueryContextMenu(hmenu, 0, 1, 0x7FFF, CMF_NORMAL);
      if(SUCCEEDED(hr)) {
       CMINVOKECOMMANDINFO info = { 0 };
       info.cbSize = sizeof(info);
       info.lpVerb = pszVerb;
       pcm->InvokeCommand(&info);
      }
      DestroyMenu(hmenu);
     }
    }
    

    And now we snap the two blocks together.

    int __cdecl _tmain(int argc, PTSTR *argv)
    {
     HRESULT hr = CoInitialize(NULL);
     if (SUCCEEDED(hr)) {
      IShellItem *psiRecycleBin;
      hr = SHGetKnownFolderItem(FOLDERID_RecycleBinFolder, KF_FLAG_DEFAULT,
                                NULL, IID_PPV_ARGS(&psiRecycleBin));
      if (SUCCEEDED(hr)) {
       IEnumShellItems *pesi;
       hr = psiRecycleBin->BindToHandler(NULL, BHID_EnumItems,
                                         IID_PPV_ARGS(&pesi));
       if (hr == S_OK) {
        IShellItem *psi;
        while (pesi->Next(1, &psi, NULL) == S_OK) {
         if (WantToRestoreThisItem(psi)) {
          IContextMenu *pcm;
          hr = psi->BindToHandler(NULL, BHID_SFUIObject,
                                  IID_PPV_ARGS(&pcm));
          if (SUCCEEDED(hr)) {
           InvokeVerb(pcm, "undelete");
           pcm->Release();
          }
         }
         psi->Release();
        }
       }
       psiRecycleBin->Release();
      }
      CoUninitialize();
     }
     return 0;
    }
    

    One annoyance of the Recycle Bin is that, at least up until Windows 7, it ignores the CMIC_MASK_FLAG_NO_UI flag. It always displays a confirmation dialog if something dangerous is about to happen (like overwriting an existing file). To mitigate this problem, we can at least reduce the number of confirmations from one-per-file to just one by batching up all the objects we want to operate on into a single context menu. For this, it's easier to go back to the classical version of the program.

    int __cdecl _tmain(int argc, PTSTR *argv)
    {
     HRESULT hr = CoInitialize(NULL);
     if (SUCCEEDED(hr)) {
      IShellFolder2 *psfRecycleBin;
      hr = BindToCsidl(CSIDL_BITBUCKET, IID_PPV_ARGS(&psfRecycleBin));
      if (SUCCEEDED(hr)) {
       IEnumIDList *peidl;
       hr = psfRecycleBin->EnumObjects(NULL,
         SHCONTF_FOLDERS | SHCONTF_NONFOLDERS, &peidl);
       if (hr == S_OK) {
        // in a real program you wouldn't hard-code a fixed limit
        PITEMID_CHILD rgpidlItems[100];
        UINT cpidlItems = 0;
        PITEMID_CHILD pidlItem;
        while (peidl->Next(1, &pidlItem, NULL) == S_OK) {
         if (WantToRestoreThisItem(psfRecycleBin, pidlItem) &&
             cpidlItems < ARRAYSIZE(rgpidlItems)) {
          rgpidlItems[cpidlItems++] = pidlItem;
         } else {
          CoTaskMemFree(pidlItem);
         }
        }
        // restore the items we collected
        if (cpidlItems) {
         IContextMenu *pcm;
         hr = psfRecycleBin->GetUIObjectOf(NULL, cpidlItems,
                         (PCUITEMID_CHILD_ARRAY)rgpidlItems,
                         IID_IContextMenu, NULL, (void**)&pcm);
         if (SUCCEEDED(hr)) {
          InvokeVerb(pcm, "undelete");
          pcm->Release();
         }
         for (UINT i = 0; i < cpidlItems; i++) {
          CoTaskMemFree(rgpidlItems[i]);
         }
        }
       }
       psfRecycleBin->Release();
      }
      CoUninitialize();
     }
     return 0;
    }
    

    In the course of the enumeration, we save the ITEMIDLISTs of all the items we want to restore, then create one giant context menu for all of them. This is the programmatic equivalent of multi-selecting the items from the Recycle Bin and then right-clicking. We then invoke the undelete verb on the entire group.

    Okay, so now suppose you want to restore the files, but instead of restoring them to their original locations, you want to restore them to a special folder. Like, say, C:\Files I deleted while I was drunk.¹ No problem. We just need a different block to snap into: The drag/drop block.

    void DropOnRestoreFolder(IDataObject *pdto)
    {
     IDropTarget *pdt;
     if (SUCCEEDED(GetUIObjectOfFile(NULL,
            L"C:\\Files I deleted while I was drunk",
            IID_PPV_ARGS(&pdt)))) {
      POINTL pt = { 0, 0 };
      DWORD dwEffect = DROPEFFECT_MOVE;
      if (SUCCEEDED(pdt->DragEnter(pdto, MK_LBUTTON,
                                   pt, &dwEffect))) {
       dwEffect &= DROPEFFECT_MOVE;
       if (dwEffect) {
        pdt->Drop(pdto, MK_LBUTTON, pt, &dwEffect);
       } else {
        pdt->DragLeave();
       }
      }
      pdt->Release();
     }
    }
    

    And now it's just a matter of snapping out the undelete block and snapping in the drag/drop block.

    int __cdecl _tmain(int argc, PTSTR *argv)
    {
     HRESULT hr = CoInitialize(NULL);
     if (SUCCEEDED(hr)) {
      IShellFolder2 *psfRecycleBin;
      hr = BindToCsidl(CSIDL_BITBUCKET, IID_PPV_ARGS(&psfRecycleBin));
      if (SUCCEEDED(hr)) {
       IEnumIDList *peidl;
       hr = psfRecycleBin->EnumObjects(NULL,
         SHCONTF_FOLDERS | SHCONTF_NONFOLDERS, &peidl);
       if (hr == S_OK) {
        // in a real program you wouldn't hard-code a fixed limit
        PITEMID_CHILD rgpidlItems[100];
        UINT cpidlItems = 0;
        PITEMID_CHILD pidlItem;
        while (peidl->Next(1, &pidlItem, NULL) == S_OK) {
         if (WantToRestoreThisItem(psfRecycleBin, pidlItem) &&
             cpidlItems < ARRAYSIZE(rgpidlItems)) {
          rgpidlItems[cpidlItems++] = pidlItem;
         } else {
          CoTaskMemFree(pidlItem);
         }
        }
        // restore the items we collected
        if (cpidlItems) {
         IDataObject *pdto;
         hr = psfRecycleBin->GetUIObjectOf(NULL, cpidlItems,
                         (PCUITEMID_CHILD_ARRAY)rgpidlItems,
                         IID_IDataObject, NULL, (void**)&pdto);
         if (SUCCEEDED(hr)) {
          DropOnRestoreFolder(pdto);
          pdto->Release();
         }
         for (UINT i = 0; i < cpidlItems; i++) {
          CoTaskMemFree(rgpidlItems[i]);
         }
        }
       }
       psfRecycleBin->Release();
      }
      CoUninitialize();
     }
     return 0;
    }
    

    Footnotes

    ¹ If being drunk isn't your thing, then substitute some other form of impaired judgment.

Page 3 of 3 (26 items) 123