November, 2011

  • The Old New Thing

    We've traced the call and it's coming from inside the house: A function call that always fails

    • 64 Comments

    A customer reported that they had a problem with a particular function added in Windows 7. The tricky bit was that the function was used only on very high-end hardware, not the sort of thing your average developer has lying around.

    GROUP_AFFINITY GroupAffinity;
    ... code that initializes the GroupAffinity structure ...
    if (!SetThreadGroupAffinity(hThread, &GrouAffinity, NULL));
    {
     printf("SetThreadGroupAffinity failed: %d\n", GetLastError());
     return FALSE;
    }
    

    The customer reported that the function always failed with error 122 (ERROR_INSUFFICIENT_BUFFER) even though the buffer seems perfectly valid.

    Since most of us don't have machines with more than 64 processors, we couldn't run the code on our own machines to see what happens. People asked some clarifying questions, like whether this code is compiled 32-bit or 64-bit (thinking that maybe there is an issue with the emulation layer), until somebody noticed that there was a stray semicolon at the end of the if statement.

    The customer was naturally embarrassed, but was gracious enough to admit that, yup, removing the semicolon fixed the problem.

    This reminds me of an incident many years ago. I was having a horrible time debugging a simple loop. It looked like the compiler was on drugs and was simply ignoring my loop conditions and always dropping out of the loop. At wit's end, I asked a colleague to come to my office and serve as a second set of eyes. I talked him through the code as I single-stepped:

    "Okay, so we set up the loop here..."

    NODE pn = GetActiveNode();
    

    "And we enter the loop, continuing while the node still needs processing."

    if (pn->NeedsProcessing())
    {
    

    "Okay, we entered the loop. Now we realign the skew rods on the node."

     pn->RealignSkewRods();
    

    "If the treadle is splayed, we need to calibrate the node against it."

     if (IsSplayed()) pn->Recalibrate(this);
    

    "And then we loop back to see if there is more work to be done on this node."

    }
    

    "But look, even though the node needs processing «view node members», we don't loop back. We just drop out of the loop. What's going on?"

    Um, that's an if statement up there, not a while statement.

    A moment of silence while I process this piece of information.

    "All right then, sorry to bother you, hey, how about that sporting event last night, huh?"

  • The Old New Thing

    How to insert a large number of items into a treeview efficiently

    • 18 Comments

    Just a quick tip today.

    If you need to insert a large number of items into a treeview, like tens of thousands, then it's much more efficient to insert them "backwards". (I'm ignoring for now the usability question of having a treeview that large in the first place.) In other words, instead of

    for (i = 0; i < array.Count(); i++) {
     TVINSERTSTRUCT tvis;
     tvis.hParent = hParentNode;
     tvis.hInsertAfter = TVIF_LAST;
     tvis.item.mask = TVIF_TEXT;
     item.item.pszText = array[i].Text();
     TreeView_InsertItem(hwndTreeView, &tvis);
    }
    
    do it this way:
    for (i = array.Count() - 1; i >= 0; i--) {
     TVINSERTSTRUCT tvis;
     tvis.hParent = hParentNode;
     tvis.hInsertAfter = TVIF_FIRST;
     tvis.item.mask = TVIF_TEXT;
     item.item.pszText = array[i].Text();
     TreeView_InsertItem(hwndTreeView, &tvis);
    }
    

    Why is backwards-insertion faster?

    It has to do with the annoying hInsert­After parameter. To validate that the hInsert­After parameter is valid, the treeview needs to verify that the hInsert­After is a valid child of the hParent, and this is done by walking the parent's children looking for a match. The sooner you find the match, the faster the validation completes. (And if you pass TVI_LAST, then the treeview needs to walk to the end of the child list.)

    You'd think that you could verify the parent/child relationship by just doing a Tree­View_Get­Parent(hInsert­After), but that turns out not to be strict enough, because hInsert­After might itself not be a valid parameter. If hInsert­After is a bogus value, then you may crash when you try to read its Parent property. That's if you're lucky. If you're not lucky, then the random memory that hInsert­After points to might look just enough like a valid HTREEITEM that you end up inserting the new node after a completely bogus node, and now the treeview has become corrupted.

    Sure, you got the same problem if you passed a garbage HTREEITEM to Tree­View_Get­Parent, but in that case, it's just garbage in garbage out. Nothing gets corrupted; the application just gets a garbage result. But in the case of Tree­View_Insert­Item, the treeview is going to update its internal data structures based on the garbage you passed in, and that means that the treeview winds up corrupted.

    To ensure that the value passed in is valid, the treeview checks it against the list of valid values for hInsert­After. And therefore, you get better performance if the valid value you pass is the one that it checks first.

    (As it happens, a lot of programs pass garbage for hInsert­After, so this defensive validation step is absolutely necessary.)

    You might say that the treeview could have a one-off optimization for the special TVI_LAST value by remembering the last child so it can be located quickly. The question is whether the complexity of adding that optimization is worth the cost: Any tree rearranging function would have to update the cached value, and if you miss a spot, then the treeview ends up corrupted. That's a pretty high-risk optimization you're suggesting there.

    And think about it this way: You're worrying about a tree where a single node has tens of thousands of children, something which (I am no longer ignoring) is a horrible user interface. That's like a list box with tens of thousands of items, or a dialog box with tens of thousands of checkboxes. You'll probably want to consider a better way of presenting the information than in a tree view that goes on for hundreds of screens. The treeview isn't optimized for this case because you don't optimize for the case where somebody is mis-using your system.

  • The Old New Thing

    Stupid Raymond talent: Screaming carrier

    • 109 Comments

    Similar to Mike, I was able to scream (not whistle: scream) a 300 baud carrier tone. This skill proved useful when I was in college and the mainframe system was down. Instead of sitting around waiting for the system to come back, I just went about my regular business around campus. Every so often, I would go to a nearby campus phone (like a free public phone but it can only make calls to other locations on campus), dial the 300 baud dial-up number, and scream the carrier tone. If I got a response, that meant that the mainframe was back online and I should wrap up what I was doing and head back to the lab.

    Mind you, this skill isn't very useful nowadays.

    What stupid computer talent do you have?

  • The Old New Thing

    How can I tell whether a DLL has been registered?

    • 27 Comments

    A customer pointed out that you can use regsvr32 to register a DLL or to unregister it, but how do you query whether a DLL has been registered?

    DLL registration (via regsvr32) is not declarative; it is procedural. A DLL does not provide a manifest of things it would like to happen when installed. Instead, the DLL merely provides two functions for regsvr32 to call, one for registration (DllRegisterServer) and another for unregistration (DllUnregisterServer). All the regsvr32 function does is call those functions.

    How those functions perform their registration and unregistration is not specified. Most of the time, those functions merely write some registry settings, but the DllRegisterServer is not limited to that. For example, the DllRegisterServer function might write some values only conditionally, say, only if the user is running a specific version of Windows. Or it might back up the old value of a registry key before it overwrites it. It might create or modify files as part of its installation or configure your firewall settings or look for and uninstall previous versions of the same DLL.

    By convention, the DllRegisterServer performs whatever operations are necessary for DLL registration, and the DllUnregisterServer reverses those operations, but since those functions are provided by the DLL, there's no guarantee that that's what actually happens. Who knows, maybe DllRegisterServer formats your hard drive. A DllRegisterServer function might just return S_OK without doing anything. How can you tell whether a function with no side effects has been called?

    Given that DLL registration can encompass arbitrary operations, there is no general-purpose way of determining whether registration has taken place for an arbitrary DLL.

    To determine whether a DLL has been registered, you need to bring in domain-specific knowledge. If you know that a DLL registers a COM object with a particular CLSID, you can check whether that CLSID is indeed registered.

  • The Old New Thing

    Why can't I install this DLL via Regsvr32 /i?

    • 26 Comments

    A customer asked for help installing a particular DLL. They ran the command regsvr /i SomeDll.dll but got the error "SomeDll.dll was loaded, but the DllInstall entry point was not found. This file can not be registered."

    A DLL needs to be specifically written to be used with the regsvr32 command. You can't just grab some random DLL and expect regsvr32 to work. As we saw last week, the regsvr32 program merely loads the specified DLL and calls an entry point established by convention. If the DLL was not written to be used with regsvr32 then the conventional entry point will not be found, and you get an error message.

    The /i switch to regsvr32 instructs the program to look for the entry point known as Dll­Install. By convention, the Dll­Install function performs installation and setup of a DLL, but since it's just some function exported by a DLL, it could do anything it wants (or nothing at all).

    You can't just grab a random DLL and expect regsvr32 to do anything meaningful with it. The DLL has to be designed to operate with regsvr32.

    Handing random DLLs to regsvr32 is like dialing a random telephone number, sending a tone at 1170Hz and getting upset when you don't get a 2125Hz tone in response.

    The number one hit for a search on what does regsvr32 do is an old Knowledge Base article which explains what regsvr32 does, and it even contains a sample program which emulates regsvr32 so you can use it to debug your DLL. (The sample program hasn't been updated to support the /i flag, which I leave as an exercise.)

    One day, I received a piece of email from another employee whom I had never met nor had ever heard of. It didn't even begin with an introduction; it just jumped right in as if we'd been friends for years. "I'm trying to debug a problem where regsvr32 cannot register my DLL. It gives the error 'The specified procedure could not be found.' I saw a blog entry written by you and am trying to understand what our problem is."

    This blog thing has backfired. The reasons I write these articles is to get people to stop asking me questions. (The mechanism for that being to give the answer out in public for everyone to see.) Instead, it turns into "Hi, I found an article you wrote about X, which ipso facto makes you not only the world's foremost authority on X, but also the world's leading support technician on X."

    News flash: Posting a blog entry about something on the Internet should not be taken as evidence that the author is an expert on that subject. (One might argue that it in fact is more likely to be the opposite.)

    At the time, I wasn't aware of the knowledge base article that explains what regsvr32 does and how to debug it, so I couldn't point to it. I wrote back, "All regsvr32 does is Load­Library, Get­Proc­Address, and then calls the function. You can write your own test that does the same thing. You do not require any expertise from me."

    Less than an hour later, I received a reply: "Thanks. I figured it out. There was an older version of the DLL in the path ahead of the one I was trying to register."

    And I never did figure out which blog entry I wrote that made them think I was an expert on regsvr32. Maybe the person worked in Microsoft Research and used a prototype of their machine that predicts the future, and used it to predict that I was going to write about regsvr32 two years later.

  • The Old New Thing

    The life story of the SwitchToThisWindow function

    • 36 Comments

    Commenters Mick and Nick (you guys ever considered teaming up and forming a morning radio show?) are interested in the life story of the Switch­To­This­Window function.

    The Switch­To­This­Window was originally added in enhanced mode Windows 3.0 in order to support switching out of fullscreen MS-DOS sessions. Recall that enhanced mode Windows 3.0 was actually three operating systems in one: There was a 32-bit virtual machine manager, and inside one virtual machine ran a copy of standard-mode Windows,¹ and inside all the others ran a copy of MS-DOS. This mean that when you pressed a key when in an MS-DOS session, the keyboard interrupt went to the MS-DOS program and not to Windows.

    When you pressed Alt+Tab, some crazy magic had to happen. The virtual machine manager had to "un-press" the Alt key in the MS-DOS program, then synchronize the shift states of the Windows virtual machine to match the one from the MS-DOS virtual machine. (For example, if you had the shift key down in the MS-DOS virtual machine, it had to simulate pressing the shift key in the Windows virtual machine so they two shift states were back in sync.) And then it could simulate pressing the Tab key, at which point the Windows virtual machine would see the Alt+Tab sequence and put up the Alt+Tab interface.

    That's how things worked if you were running in a windowed MS-DOS session. But if you were in a fullscreen MS-DOS session, things worked differently. Switching back to Windows would mean a display mode reset (which can take a second or longer), and then all the applications on your desktop had to redraw themselves (and probably paging quite a bit in order to do so). This definitely failed to meet the responsiveness people expected from Alt+Tab, so the virtual machine manager pulled a trick: If you pressed Alt+Tab while in a fullscreen MS-DOS session, then instead of switching back to the Windows virtual machine, the virtual machine manager displayed a text-mode version of the Alt+Tab interface.

    I will stop to let the craziness of that sink in: The virtual machine manager had its own Alt+Tab interface built out of text mode.

    Anyway, when you finally released the Alt key and completed the Alt+Tab sequence, the virtual machine manager needed to tell Windows, "Hey, like, pretend that an Alt+Tab thingie just happened, okay?"

    That is what the Switch­To­This­Window function was for. It was the function the virtual machine manager called to tell Windows to switch to a window as if the user had selected it via Alt+Tab (because that is, in fact, what the user did, just via the text-mode interface rather than the graphical one).

    A similar thing happened if you pressed Alt+Esc (or Alt+Shift+Esc in a fullscreen MS-DOS session. That's why there's a second parameter to indicate whether the switch should be done "in the style of Alt+Tab" or "in the style of Alt+Esc."

    The function was undocumented because it existed only for the virtual machine manager to call in order to coordinate its actions with Windows user interface so that you had one big happy Alt+Tab family.

    The text-mode Alt+Tab interface disappeared in Windows 95, but the Switch­To­This­Window function hung around because it wasn't causing anybody any harm, and there was at the time no formal process in place to deprecate and eventually remove an API, not even an internal undocumented one.

    In the Windows XP SP1 timeframe, a bunch of lawyers decided that some functions in Windows needed to be documented. The precise rules for determining which functions needed to be documented and which didn't need to be documented were rather complicated. (Some people applied an algorithm different from the ones those lawyers used and came up with a list of functions that are "missing", when all that they really came up with is a list of functions different from the list those lawyers came up with.)²

    Anyway, the Switch­To­This­Window function got caught in the dragnet, so it got documented. Mind you, like it says right at the top of the documentation, there is no guarantee that the function will continue to exist; it can vanish at any time. Although there is documentation, it has the logical status of an internal function, and internal functions have a tendency to change or vanish entirely. Perhaps someday a new chapter will be added to the life story of Switch­To­This­Window: "The Switch­To­This­Window was removed in Windows Q" for some value of Q.

    Footnotes

    ¹ Not true, but true enough. Don't make me bring back the Nitpicker's Corner.

    ² I will delete any comments on the subject of the algorithm by which those lawyers determined which functions needed to be documented, or on the documentation itself.

    Bonus chatter: As far as I can determine, Switch­To­This­Window just does a Set­Foreground­Window on the window you're switching to, possibly posting it a WM_SYS­COMMAND/SC_RESTORE message, and moving the previous foreground window to the bottom of the Z-order if switched via Alt+Esc. It doesn't provide any special secret sauce for bypassing the normal foreground activation rules. The process that calls Switch­To­This­Window still requires foreground-change permission.

  • The Old New Thing

    If you protect a write with a critical section, you may also want to protect the read

    • 30 Comments

    It is common to have a critical section which protects against concurrent writes to a variable or collection of variables. But if you protect a write with a critical section, you may also want to protect the read, because the read might race against the write.

    Adam Rosenfield shared his experience with this issue in a comment from a few years back. I'll reproduce the example here in part to save you the trouble of clicking, but also to make this entry look longer and consequently make it seem like I'm actually doing some work (when in fact Adam did nearly all of the work):

    class X {
     volatile int mState;
     CRITICAL_SECTION mCrit;
     HANDLE mEvent;
    };
    
    X::Wait() {
     while(mState != kDone) {
       WaitForSingleObject(mEvent, INFINITE);
     }
    }
    
    X::~X() {
     DestroyCriticalSection(&mCrit);
    }
    
    X::SetState(int state) {
     EnterCriticalSection(&mCrit);
     // do some state logic
     mState = state;
     SetEvent(mEvent);
     LeaveCriticalSection(&mCrit);
    }
    
    Thread1()
    {
     X x;
     ... spawn off thread 2 ...
     x.Wait();
    }
    
    Thread2(X* px)
    {
     ...
     px->SetState(kDone);
     ...
    }
    

    There is a race condition here:

    • Thread 1 calls X::Wait and waits.
    • Thread 2 calls X::SetState.
    • Thread 2 gets pre-empted immediately after calling Set­Event.
    • Thread 1 wakes up from the Wait­For­Single­Object call, notices that mState == kDone, and therefore returns from the X::Wait method.
    • Thread 1 then destructs the X object, which destroys the critical section.
    • Thread 2 finally runs and tries to leave a critical section that has been destroyed.

    The fix was to enclose the read of mState inside a critical section:

    X::Wait() {
     while(1) {
       EnterCriticalSection(&mCrit);
       int state = mState;
       LeaveCriticalSection(&mCrit);
       if(state == kDone)
         break;
       WaitForSingleObject(mEvent, INFINITE);
     }
    }
    

    Forgetting to enclose the read inside a critical section is a common oversight. I've made it myself more than once. You say to yourself, "I don't need a critical section here. I'm just reading a value which can safely be read atomically." But you forget that the critical section isn't just for protecting the write to the variable; it's also to protect all the other actions that take place under the critical section.

    And just to make it so I actually did some work today, I leave you with this puzzle based on an actual customer problem:

    class BufferPool {
    public:
     BufferPool() { ... }
     ~BufferPool() { ... }
    
     Buffer *GetBuffer()
     {
      Buffer *pBuffer = FindFreeBuffer();
      if (pBuffer) {
       pBuffer->mIsFree = false;
      }
      return pBuffer;
     }
    
     void ReturnBuffer(Buffer *pBuffer)
     {
      pBuffer->mIsFree = true;
     }
    
    private:
     Buffer *FindFreeBuffer()
     {
      EnterCriticalSection(&mCrit);
      Buffer *pBuffer = NULL;
      for (int i = 0; i < 8; i++) {
       if (mBuffers[i].mIsFree) {
        pBuffer = &mBuffers[i];
        break;
       }
      }
      LeaveCriticalSection(&mCrit);
      return pBuffer;
     }
    private:
     CRITICAL_SECTION mCrit;
     Buffer mBuffers[8];
    };
    

    The real class was significantly more complicated than this, but I've distilled the problem to its essence.

    The customer added, "I tried declaring mIs­Free as a volatile variable, but that didn't seem to help."

  • The Old New Thing

    Why does Internet Explorer not call DLL_PROCESS_DETACH on my DLL when I call ExitProcess?

    • 11 Comments

    A customer asked a question, but as is often the case, the question was much more telling than the answer.

    We have an Internet Explorer plug-in which calls Exit­Process to force Internet Explorer to exit. We found that when we do this, our plug-in does not receive a DLL_PROCESS_DETACH notification. What could be preventing our plug-in from receiving the DLL_PROCESS_DETACH notification?

    As we saw some time ago when we looked at the way processes shut down (plus an important follow-up or two), all a process has to do to thwart proper delivery of DLL_PROCESS_DETACH notifications is to do something untoward during shutdown, at which point the kernel just gives up and calls Terminate­Process.

    But like I said, the answer is much less interesting than the question. What if the user had an unsaved email message at the time you decided to exit Internet Explorer? Recall that plug-ins are a guest in the host process; don't go changing the carpet. When we asked the customer why they were exiting Internet Explorer from their plug-in, we received the explanation, "The reason I am calling Exit­Process is that I do not know another good way to exit Internet Explorer from a plug-in."

    In this case, the guest is doing far more than just changing the carpet. The guest called in a demolition company!

    "Why did you call the demolition company to destroy my house?"
    "I couldn't think of a good way to destroy your house."

    The point isn't that it's bad to use a telephone call to hire a demolition company to destroy somebody's house and that you should use some other method to contact them (like, say, a text message). The point is that it's bad to destroy somebody else's house in the first place.

    Upon further investigation, the customer was writing a test for their plug-in. They open Internet Explorer and navigate to a page that uses the plug-in. When they are satisfied that the plug-in operated correctly, they want to exit the copy of Internet Explorer in order to conclude the test.

    If you want to destroy a house, then destroy your own house. Call Co­Create­Instance(CLSID_Internet­Explorer) to build a house, navigate to your test page with IWeb­Browser2::Navigate, and when you're done, you can destroy the house with IWeb­Browser2::Quit(). There is sample code to do exactly this in the documentation for the IWeb­Browser2 interface.

    Bonus chatter: The IWeb­Browser2 interface is scriptable.

    var ie = new ActiveXObject("InternetExplorer.Application");
    ie.Visible = true;
    ie.Navigate("http://www.microsoft.com/");
    WScript.Sleep(5000); // five seconds, say
    ie.Quit();
    
  • The Old New Thing

    Fontography term or pretentious blather?

    • 25 Comments

    Fontography is like wine. The connoisseurs speak in a language that only superficially resembles English.

    Here's a list of words. Which of them are terms used in fontography, and which are just pretentious blather?

    • aloof
    • assertive
    • cold
    • cuddly
    • humanist
    • international
    • neutral
    • no-nonsense
    • open
    • quirky

    If you aren't familiar with font speak, here's a sample I stole from an old article on the evolution of the Internet explorer logo:

    In The Elements of Typographic Style by Robert Bringhurst, "this typeface is described as a heavy un-modulated line and tiny aperture (which) evoke an image of uncultivated strength, force and persistence."

    Bonus link: Cheese or Font?

  • The Old New Thing

    The power of statistical photography

    • 12 Comments

    Inside Microsoft, there was an employee photography contest to provide images to be included in Windows 7, either in one of the pre-release versions or in the final product. Each subsidiary selected the photos to be included in their localized version of Windows, choosing images which best reflect that region's culture, history, and natural beauty. The employee-submitted photos were in direct competition against the professional photographs; as a result, some regions ended up selecting multiple employee-contributed images and others picked none. The Swiss delegation, in characteristically Swiss fashion, put it to a public vote.

    (As you may recall, Windows Vista included photographs drawn from the community. That article is another great example of No matter what you do, somebody will call you an idiot. Microsoft decides to involve the community in a fun way, and the result is condemnation from the professional photographic community.)

    Around 2000 photographs were submitted by Microsoft employees, and one of the photos selected for the U.S. version of Windows 7 Beta was taken by a member of the user interface team. In response to congratulations, he humbly replied, "I'm a statistical photographer. I rely on sheer quantity to produce occasional quality."

    Bonus reading: A Look Behind the Backgrounds of Windows 7.

Page 1 of 3 (23 items) 123