January, 2013

  • The Old New Thing

    The somewhat misguided question of whether MapViewOfFile or WriteProcessMemory is faster

    • 25 Comments

    A customer asked, "Which is faster for copying data from one process to another; Map­View­Of­File or Write­Process­Memory?"

    This is one of those "Huh?"-type questions, where the customer has solved half of a problem and is looking for help with the other half, the half that makes no sense.

    First of all, the question is malformed because Map­View­Of­File does not copy any data at all. It takes existing data and maps it into a process. No copy takes place; both processes are seeing the same memory block, and if the memory is modified via one mapping, the change will be visible in other mappings.

    It's like asking "Which company has better wireless telephone coverage: FedEx or Sprint?"

    Okay, so maybe the question is really "Which is a faster way of transferring data between two processes?" (In the same way the FedEx/Sprint question might really be asking "Which company has a larger service area for me to communicate with my customers?")

    But before you choose one or the other based on performance, you need to make the decision be appropriate from a correctness standpoint.

    "We want to transfer some data from a client process to a server process. We found that mapping the shared memory causes more virtual memory to be consumed by the server, which is already constrained by the large number of components loaded into the server process. We were hoping we could switch to Write­Process­Memory, assuming the performance characteristics are acceptable."

    Okay, now you sort of go "Whoa."

    The Write­Process­Memory function requires PROCESS_VM_WRITE permission on the target process. That permission allows you to write to any byte in the process. If you allow a client process full write access to the complete memory space of a server, then the server is pwned!

    It doesn't matter how much faster (if any) Write­Process­Memory is compared to Map­View­Of­File, because you can't use it if you want to maintain any shred of security.

    "I don't want my packages to get wet, so I taped the key to the front door with instructions to the FedEx delivery person to put the package in the living room."

    That key is not a key to the living room. That is a key to the entire house.

    If you want to reduce the amount of address space consumed by a Map­View­Of­File, you can pass a nonzero value for dwNumber­Of­Byte­To­Map and map a sliding window into the data rather than mapping it all at once.

    And then looking at the question again, it's not clear that the Write­Process­Memory function will help in the first place: If the problem is address space exhaustion, then switching to Write­Process­Memory won't change your address space profile: The memory written by Write­Process­Memory must all be committed in the target process. Whereas Map­View­Of­File lets the server control how much address space is consumed by the view, the Write­Process­Memory function requires it all to be committed up front.

    We never did hear back from the customer, so it's not clear whether they understood that their question was confused and misguided, or whether they were just frustrated with us for "not answering their question."

  • The Old New Thing

    Why doesn't HeapValidate detect corruption in the managed heap?

    • 17 Comments

    A customer had a program that was corrupting the managed heap by p/invoking incorrectly. The problem didn't show up until the next garbage collection pass, at which point the CLR got all freaked-out-like. "According to Knowledge Base article 286470, the GFlags tool is supposed to catch heap corruption, but it doesn't catch squat."

    Depending on your point of view, this is either a case of the customer not understanding what things mean in context or of the KB article author looking at the world through kernel-colored glasses.

    The GFlags tool, pageheap, full pageheap, and the Heap­Validate function all operate on heaps, but the sense of the word heap here is "heaps created by the Heap­Create function." If your program does a Virtual­Alloc and then carves out sub-allocations from it, well, it's not like GFlags and Heap­Validate are psychic and can magically reverse-engineer your code in order to understand your custom heap implementation and be able to determine whether your custom heap is corrupted.

    Clearly no such function could be written, because that's even harder than the Halting Problem! One property of a non-corrupted heap is that it will not send the heap manager into an infinite loop. Therefore, proving that the heap is not corrupted, given no information about the heap implementation other than the code itself, would require proving that the next heap call will return. And that's just one of the things the imaginary ValidateAnyHeap function would have to do. (We try to limit ourselves to one impossible thing at a time.)

    The Heap­Validate function only knows how to validate heaps created by the Heap­Create function. It does not have magic insight into custom heap implementations. The GFlags program modifies the behavior of heaps created by the Heap­Create function, because it naturally does not know what debugging features you've added to your custom heap implementation, so it doesn't know what it needs to do to turn them on and off.

    As far as the kernel folks are concerned, "heap" means "something created by the Heap­Create function." Anything else is just an imposter.

    If you are looking for corruption in a custom heap implementation, then you need to go ask the authors of that custom heap implementation if they provided any debugging facilities for that heap.

  • The Old New Thing

    Poisoning your own DNS for fun and profit

    • 28 Comments

    When you type a phrase into the Windows Vista Start menu's search box and click Search the Internet, then the Start menu hands the query off to your default Internet search provider.

    Or at least that's what the Illuminati would have you believe.

    A customer reported that when they typed a phrase into the Search box and clicked Search the Internet, they got a screenful of advertisements disguised to look like search results.

    What kind of evil Microsoft shenanigans is this?

    If you looked carefully at the URL for the bogus search "results", the results were not coming from Windows Live Search. They were coming from a server controlled by the customer's ISP.

    That was the key to the rest of the investigation. Here's what's going on:

    The ISP configured all its customers to use the ISP's custom DNS servers by default. That custom DNS server, when asked for the location of search.live.com, returned not the actual IP address of Windows Live Search but rather the IP address of a machine hosted by the ISP. (This was confirmed by manually running nslookup on the customer machine and seeing that the wrong IP addresses were being returned.) The ISP was stealing traffic from Windows Live Search. It then studied the URL you requested, and if it is the URL used by the Start menu Search feature, then it sent you to the page of fake search results. Otherwise, it redirected you to the real Windows Live Search, and you're none the wiser, aside from your Web search taking a fraction of a second longer than usual. (Okay, snarky commenters, and aside from the fact that it was Windows Live Search.)

    The fake results page does have an About This Page link, but that page only talks about how the ISP intercepts failed DNS queries (which has by now become common practice). It doesn't talk about redirecting successful DNS queries.

    I remember when people noticed widespread hijacking of search traffic, and my response to myself was, "Well, duh. I've know about this for years."

    Bonus chatter: It so happens that the offending ISP's Acceptable Use Policy explicitly lists as a forbidden activity "to spoof the URL, DNS, or IP addresses of «ISP» or any other entity." In other words, they were violating their own AUP.

    Related

  • The Old New Thing

    Please hold your head perfectly still while you write up that memo

    • 6 Comments

    I dreamed that the original Volkswagen Beetle factory was so cramped that the office workers had to move their desks to the factory floor, with heavy equipment swinging around just inches from their faces. To prevent serious injuries, a template passed through every so often to make sure nothing was out of place.

  • The Old New Thing

    Finding a printer, and then creating a shortcut to that printer

    • 6 Comments

    Today's "Little Program" does two things: It looks for a printer in the Printers folder, and then once it finds it, it creates a shortcut to that printer.

    As is common with "Little Programs", I don't bother with error checking. I'll leave you to do that.

    Second part first, since it is handy on its own: Creating a shortcut to an arbitrary item in the shell namespace, provided either in the form of an ID list or a shell item. (The ID list is the thing that identifies an item in the shell namespace.)

    void CreateShortcutToIDList(PCWSTR pszName, PCUIDLIST_ABSOLUTE pidl)
    {
     CComPtr<IShellLink> spsl;
     spsl.CoCreateInstance(CLSID_ShellLink);
     spsl->SetIDList(pidl);
     CComQIPtr<IPersistFile>(spsl)->Save(pszName, TRUE);
    }
    
    void CreateShortcutToItem(PCWSTR pszName, IShellItem *pitem)
    {
     CComHeapPtr<ITEMIDLIST_ABSOLUTE> spidl;
     CComQIPtr<IPersistIDList>(pitem)->GetIDList(&spidl);
     CreateShortcutToIDList(pszName, spidl);
    }
    

    Neither of these is particular complicated. To create a shortcut given an ID list:

    • Create a brand new Shell­Link object.
    • Tell that shell link object to point to our desired ID list.
    • Save the shell link.

    To create a shortcut given a shell item:

    • Ask the IShell­Item for its ID list.
    • Create a shortcut to that ID list.

    Okay, now the first half: Finding the printer. That is a matter of binding to the Printers folder and enumerating its contents. When we find the one whose name matches the target printer, we declare victory.

    int __cdecl wmain(int argc, wchar_t **argv)
    {
     CCoInitialize init;
     CComPtr<IShellItem> spPrinters;
     SHGetKnownFolderItem(FOLDERID_PrintersFolder, KF_FLAG_DEFAULT,
                          nullptr, IID_PPV_ARGS(&spPrinters));
     CComPtr<IEnumShellItems> spEnum;
     spPrinters->BindToHandler(nullptr, BHID_EnumItems,
                                  IID_PPV_ARGS(&spEnum));
     for (CComPtr<IShellItem> spPrinter;
          spEnum->Next(1, &spPrinter, nullptr) == S_OK;
          spPrinter.Release()) {
      CComHeapPtr<wchar_t> spszName;
      spPrinter->GetDisplayName(SIGDN_NORMALDISPLAY, &spszName);
      wprintf(L"Found printer \"%ls\"\n", spszName);
      if (lstrcmpiW(spszName, argv[1]) == 0) {
       wprintf(L"Creating shortcut as \"%ls\"\n", argv[2]);
       CreateShortcutToItem(argv[2], spPrinter);
      }
     }
     return 0;
    }
    

    This is a little trickier, but not by much.

    • Initialize COM.
    • Get the IShell­Item for the Printers folder.
    • Get the enumerator for the Printers folder.
    • For each item in the Printers folder:
      • Get its name and print it just for diagnostic purposes.
      • If the name matches the one we're looking for, then create the shortcut.

    Here are the header files I used:

    #define STRICT_TYPED_ITEMIDS // enable stricter type checking on ITEMIDs
    #include <windows.h>
    #include <atlbase.h>        // for CComPtr, CComQIPtr
    #include <atlalloc.h>       // for CComHeapPtr
    #include <shlobj.h>         // for shell interfaces
    #include <knownfolders.h>   // for FOLDERID_PrintersFolder
    #include <stdio.h>          // for wprintf
    

    For those of you who prefer to work with CSIDLs, the change is relatively minor. Replace the SHGet­Known­Folder­Item call with

     BindToCsidlItem(CSIDL_PRINTERS, &spPrinters);
    

    You may have noticed that I used the for-if anti-pattern. I could've gone for the item directly by using SHCreate­Item­From­Relative­Name:

    int __cdecl wmain(int argc, wchar_t **argv)
    {
     CCoInitialize init;
     CComPtr<IShellItem> spPrinters;
     SHGetKnownFolderItem(FOLDERID_PrintersFolder, KF_FLAG_DEFAULT,
                          nullptr, IID_PPV_ARGS(&spPrinters));
     CComPtr<IShellItem> spPrinter;
     SHCreateItemFromRelativeName(spPrinters, argv[1], nullptr,
                                  IID_PPV_ARGS(&spPrinter));
     CreateShortcutToItem(argv[2], spPrinter);
     return 0;
    }
    

    As before, if you're the sort of person who prefers to do things old-school, you can parse the name yourself, at which point you may as well give up on shell items, hike up your pants, and do it with an onion on your belt:

    HRESULT BindToIDList(PCUIDLIST_ABSOLUTE pidl,
                         REFIID riid, void **ppv)
    {
     *ppv = nullptr;
     CComPtr<IShellFolder> spsfDesktop;
     HRESULT hr = SHGetDesktopFolder(&spsfDesktop);
     if (SUCCEEDED(hr)) {
      if (pidl->mkid.cb) {
       hr = spsfDesktop->BindToObject(pidl, nullptr, riid, ppv);
      } else {
       hr = spsfDesktop->QueryInterface(riid, ppv);
      }
     }
     return hr;
    }
    
    int __cdecl wmain(int argc, wchar_t **argv)
    {
     CCoInitialize init;
     CComHeapPtr<ITEMIDLIST_ABSOLUTE> spidlPrinters;
     SHGetSpecialFolderLocation(nullptr,
                      CSIDL_PRINTERS, &spidlPrinters);
     CComPtr<IShellFolder> spsfPrinters;
     BindToIDList(spidlPrinters, IID_PPV_ARGS(&spsfPrinters));
    
     ULONG cchEaten;
     DWORD dwAttributes = 0;
     CComHeapPtr<ITEMIDLIST_RELATIVE> spidl;
     spsfPrinters->ParseDisplayName(nullptr, nullptr, argv[1],
                            &cchEaten, &spidl, &dwAttributes);
    
     CComHeapPtr<ITEMIDLIST_ABSOLUTE> spidlPrinter;
     spidlPrinter.Attach(ILCombine(spidlPrinters, spidl));
     CreateShortcutToIDList(argv[2], spidlPrinter);
     return 0;
    }
    

    The Bind­To­ID­List function is nothing special; we already saw the guts of it when we wrote Bind­To­Csidl­Item.

    The main program proceeds in three steps:

    • Get the ID list for the Printers folder and bind to it.
    • Parse the printer name, producing a printer ID list.
    • Create a shortcut to the ID list for the printers folder combined with the printer ID list.
  • The Old New Thing

    When you have a SAFEARRAY, you need to know what it is a SAFEARRAY *of*

    • 5 Comments

    A customer had a problem with SAFEARRAY, or more specifically, with CComSafeArray.

    CComSafeArray<VARIANT> sa;
    GetAwesomeArray(&sa);
    
    LONG lb = sa.GetLowerBound();
    LONG ub = sa.GetUpperBound();
    
    for (LONG i = lb; i <= ub; i++) {
     CComVariant item = sa.GetAt(i);
     ... use the item ...
    }
    

    The GetAt method returns a VARIANT&, and when it is copy-constructed into item, the DISP_E_BAD­VAR­TYPE exception is raised.

    On the other hand, if the offending line is changed to

    CComQIPtr<IAwesome> pAwesome = sa.GetAt(i).punkVal;
    

    then the problem goes away.

    Your initial reaction to this code would be that there is an off-by-one error in the loop control, but it turns out that there isn't because SAFEARRAY uses inclusive upper bounds rather than exclusive.

    The first step in debugging this is seeing what is in the bad variant that makes the copy constructor think it's not a valid variant type.

    Inspecting in the debugger shows that the variant returned by GetAt has a valid punk, but the vt is 0x1234. Well, that's not a valid variant type, so that's the proximate cause of the problem.

    How did an invalid variant type get into your SAFEARRAY?

    At this point the customer realized that maybe their code to create the array was faulty, so they offered to share it.

    void GetAwesomeArray(SAFEARRAY **ppsa)
    {
     SAFEARRAY *psa = SafeArrayCreateVector(VT_UNKNOWN, 0, m_count);
     for (LONG i = 0; i < m_count; i++) {
      CComPtr<IAwesome> spAwesome;
      CreateAwesomeThing(i, &spAwesome);
      SafeArrayPutElement(psa, &i, spAwesome);
     }
     *ppsa = psa;
    }
    

    Okay, now all the pieces fell into place.

    The Get­Awesome­Array function is creating an array of VT_UNKNOWN, but the code fragment that calls Get­Awesome­Array treats it as an array of VT_VARIANT.

    Your array of IUnknown* is being misinterpreted as an array of VARIANT. That explains all the symptoms: The vt is wrong, because it's really just the low-order word of the first IUnknown*. Ignoring the vt and going straight for the punk seems to work because that's where the second IUnknown* happens to be. (Or third, if you are compiling as 32-bit.)

    In other words, it's as if you did a reinterpret_cast<VARIANT&>(punkArray[0]).

    If you had used regular C-style arrays or a C++ collection, then the compile-time type checking would have told you that you mismatched the producer and consumer. But since you went through a SAFEARRAY, that compile-time type information is lost, since a SAFEARRAY is a polymorphic array. It now becomes your job to keep track of what you have an array of, and its dimensions and bounds.

    You can keep track of this information via documentation, "This function returns a 1-dimensional SAFEARRAY of VT_IUNKNOWN, with lower bound 0 and variable upper bound." Or you can check at runtime, by calling Safe­Array­Get­Vartype to see what the base type is, and Safe­Get­Dim to see how many dimensions the array has, and Safe­Array­Get­LBound and Safe­Array­Get­UBound to obtain the upper and lower bounds for those dimensions.

    The code above seemed not to be sure which model it wanted to use. It trusted the base type and the dimension, but checked the upper and lower bounds.

    Anyway, assuming we are going with the "keep track via documentation" approach, the solution for the original problem is to have the producer and consumer agree on exactly what kind of SAFEARRAY is being handed around. Either produce an array of VT_UNKNOWN and consume it as a CComSafeArray<IUnknown*> or produce an array of VT_VARIANT and consume it as a CComSafeArray<VARIANT>.

  • The Old New Thing

    STRICT_TYPED_ITEMIDS is the shell namespace version of the STRICT macro used by USER and GDI

    • 12 Comments

    Starting with the Windows Vista PlatformSDK, defining the symbol STRICT_TYPED_ITEM­IDS before including shell header files changes declarations that previously had simply used ITEM­ID­LIST now use one of various types which are more clear about what type of ID list is being used.

    Think of it as the STRICT macro for the shell.

    The more precise names emphasize the form of the ID list:

    • ITEM­ID_CHILD represents an item ID relative to some implied shell folder. The item ID is followed by a null terminator and is therefore exactly one SH­ITEM­ID long. In file-system speak, this is a "file name."
    • ID­LIST_RELATIVE represents an item ID list relative to some implied shell folder. It can consist of any number of SH­ITEM­ID structures concatenated together, followed by a null terminator. This item ID list must be used in conjunction with the IShell­Folder it is associated with. In file-system speak, this is a "relative path."
    • ID­LIST_ABSOLUTE represents an item ID list relative to the desktop root folder. It too can be any length. This item ID list must be used in conjunction with the IShell­Folder returned by SH­Get­Desktop­Folder. In file-system speak, this is a "fully-qualified absolute path." (An absolute ID list is the special case of a relative ID list where the implied shell folder is the desktop root folder.)
    • ITEM­ID_CHILD_ARRAY represents an array of pointers to ITEM­ID_CHILD objects, where all of the ITEM­ID_CHILD objects are children of the same shell folder parent. The array must be used in conjunction with that parent shell folder.

    These new types were introduced to help catch common programming errors when using the shell namespace. For example, if you try to pass an array of absolute pidls to IShell­Folder::Get­UI­Object­Of, you will get a type mismatch compiler error because that method takes an ITEM­ID_CHILD_ARRAY, and the thing you passed is not an array of ITEM­ID_CHLD pointers.

    You are encouraged to turn on strict mode when compiling code that uses the shell namespace, but to preserve source code backward compatibility, the setting is off by default.

  • The Old New Thing

    Eliot Chang's list of things Asians hate

    • 54 Comments

    One time, somebody asked me, "What nationality are you?"

    I answered, "American."

    "No, I mean what nationality are your parents?"

    "They're also American."

    "No, I mean where are your parents from?"

    "They're from New Jersey."

    "No, I mean before that."

    "North Carolina."

  • The Old New Thing

    Why was WHEEL_DELTA chosen to be 120 instead of a much more convenient value like 100 or even 10?

    • 35 Comments

    We saw some time ago that the nominal mouse wheel amount for one click (known as a "detent") is specified by the constant WHEEL_DELTA, which has the value 120.

    Why 120? Why not a much more convenient number like 100, or even 10?

    Because the value 120 made it easier to create higher-resolution mouse wheels.

    As noted in the documentation:

    The delta was set to 120 to allow Microsoft or other vendors to build finer-resolution wheels (a freely-rotating wheel with no notches) to send more messages per rotation, but with a smaller value in each message.

    Suppose the original wheel mouse had nine clicks around its circumference. Click nine times, and you've made a full revolution. (I have no idea how many actual clicks there were, but the actual number doesn't matter.) Therefore, each click of the wheel on the original mouse resulted in 120 wheel units.

    Now, suppose you wanted to build a double-resolution wheel, say one with eighteen clicks around the circumference instead of just nine. If you reported 120 wheel units for each click, then your mouse would feel "slippery", because it scrolled twice as fast as the original mouse. The solution: Have each click of your double-resolution mouse report 60 wheel units instead of 120.

    That's why the number chosen was 120. The number 120 has a lot more useful factors than 100. The number 100 = 2² × 5² can be evenly divided by the small integers 2, 4, 5, and 10. On the other hand, the number 120 = 2³ × 3 × 5 can be evenly divided by 2, 3, 4, 5, 6, 8, and 10.

    If you wanted to build a triple-resolution mouse, and the MOUSE_WHEEL value were 100, then you would have difficulty reporting each click, because you couldn't just report 33 for each one. (After three clicks, you would have reported only 99 units, and applications which waited for a full MOUSE_WHEEL would still be waiting.) Your mouse driver would have to report 33, 33, 34, 33, 33, 34, 33, 33, 34, and so on. And then it gets messy if the user changes scrolling direction.

    On the other hand, if MOUSE_WHEEL were 120, then the triple-resolution mouse could simply report 40 units per click.

    Okay, so why 120 instead of just 12?

    As noted in the documentation, the value was chosen so that it would be possible to build a mouse with no clicks at all. The wheel simply spun smoothly, and you could stop it at any point. Such a wheel would report one wheel unit for every one-third of one degree of rotation. If the detent were only 12 units, then the wheel would report one unit for every 3 1/3 degrees of rotation, which wouldn't be as smooth.

    I don't know if anybody has developed such a mouse, but at least the possibility is still there. (There are free-spinning mouse wheels, but I don't know whether they are normal WHEEL_DELTA wheels just without the mechanical detents, or whether they really do report fine rotational information.)

    Bonus reading: The History of the Scroll Wheel, written by its inventor, Eric Michelman.

    Mouse wheel trivia: The code name for the mouse wheel project was Magellan. The code name still lingers in error messages that pop up from the original wheel mouse driver.

  • The Old New Thing

    Heads-up: Phone scammers pretending to be JPMorgan Chase MasterCard security

    • 21 Comments

    Recently, a round of phone scammers have been dialing through our area with a caller-ID of (000) 000-0000, which should already raise suspicions.

    When you answer, a synthesized voice says that they are calling from JPMorgan Chase MasterCard security. They claim that your credit card has been disabled due to suspicious activity, and in order to reactivate it, you need to enter your 16-digit credit card number.

    I decided to see how far I could take the robot voice for a ride, so I entered 16 random digits. No luck, the robot voice knew about the checksum and asked me to enter it again. I did a quick search for fake credit card number and landed on this blog entry and entered the first fake credit card number on that page.

    The robot voice accepted the credit card and proceeded to ask me for the card's expiration date (I made one up) and the PIN (I made one up). At that point, it reported that it had successfully verified the information and that my card had been re-enabled.

    Of course, the joke's on them: The fake credit card I used wasn't even a fake MasterCard number. It was a fake VISA credit card!

    By the way, don't read the comments on that blog entry if you want to retain any faith in humanity.

Page 1 of 3 (29 items) 123