November, 2011

  • The Old New Thing

    How can I tell whether a COM pointer to a remote object is still valid?

    • 27 Comments

    A customer asked the rather suspicious question, "How do I check whether a pointer is valid in another process?"

    This question should make your head boggle with bewilderment. First of all, we've moved beyond Crash­Program­Randomly to Crash­Some­Other­Program­Randomly. Second of all, what the heck are you doing with a pointer in another process? You can't do anything with it!

    After some back-and-forth¹ we manage to tease the real question out of the customer: How can I tell whether a COM pointer to a remote object is still valid?

    The easy answer is "Don't worry. COM will take care of it." Just call the method on the object. If the remote object is not valid, you will get an error back, like RPC_E_DISCONNECTED or RPC_E_SERVER_DIED or RPC_E_SERVER_DIED_DNE or HRESULT_FROM_WIN32(RPC_S_SERVER_UNAVAILABLE). When you get an error like that, you'll know that the remote object is no longer valid, and you can respond accordingly.

    What if you want your program to be a little proactive and prune dead remote objects instead of just noticing that they're dead the net time you want to use them?

    Some people "solve" this problem by performing a Query­Interface on a newly-generated interface ID. Since the IID has never been seen before, COM cannot consult its cache of previously-queried interfaces and must remote the call, at which point the death of the remote object will be detected. (The second rule for implementing Query­Interface exists in part so that COM can optimize Query­Interface of remote objects.) The problem with this technique is that by subverting the cache, you also end up polluting it. Each time you generate a new IID and do a dummy Query­Interface on it, you add another dummy entry to the Query­Interface cache. This wastes memory keeping track of interfaces that nobody will ever ask for again, and may even push out information about interfaces that your program actually uses!

    The COM folks tell me that your program should just accept the fact that the other process can go away at any time. Instead of making some sort of decision based on whether the other process is still there (since a response of "yeah, it's still here" could be wrong by the time you act on it), you should just call the method and accept that it may fail because the other process vanished while you weren't looking.

    Footnote

    ¹ The customer first explained that their server process created an object and gave a pointer to that object to the client. The client then registered a callback object with the server, and the server wanted to check that the client object was still valid before invoking any methods on it. When asked, "Why not just use COM?" the customer replied, "We are using COM. We create the object on the server via Co­Create­Instance, then register the client object via a method on our interface."

    The customer was under the impression that when a COM pointer refers to an object in another process, you just get that pointer from the other process.

    If you think about it, this makes no sense at all. How could any of your method calls work? You call pRemote­Object->AddRef() and the compiler is going to deference the pRemote­Object pointer, and then crash because the pointer would refer to memory in another process. I guess the customer was under the impression that some magic voodoo happens so that the CPU knows that "Oh wait, this pointer really belongs to another process, let me go fetch the memory from that other process. Okay, and now you want to call a function pointer in another process? Okay, um, let me magically merge the two processes together so the remote code running in that other process can access the objects in your process." Or something.

    When you have a COM pointer to an object in another process, the pointer that you have is a proxy which accepts method calls and marshals the call to the real object somewhere else.

  • The Old New Thing

    Fontography term or pretentious blather?

    • 25 Comments

    Fontography is like wine. The connoisseurs speak in a language that only superficially resembles English.

    Here's a list of words. Which of them are terms used in fontography, and which are just pretentious blather?

    • aloof
    • assertive
    • cold
    • cuddly
    • humanist
    • international
    • neutral
    • no-nonsense
    • open
    • quirky

    If you aren't familiar with font speak, here's a sample I stole from an old article on the evolution of the Internet explorer logo:

    In The Elements of Typographic Style by Robert Bringhurst, "this typeface is described as a heavy un-modulated line and tiny aperture (which) evoke an image of uncultivated strength, force and persistence."

    Bonus link: Cheese or Font?

  • The Old New Thing

    The challenges in changing the way Explorer hosts shell extensions

    • 26 Comments

    Various types of shell extensions such as thumbnail extractors are run in a separate process, and if one of those shell extensions crashes, it takes out the COM Surrogate rather than the main Explorer process.

    Anonymous wondered if this model could be extended to all types of shell extensions, perhaps not all at once, but gradually. The dangers of extending this model to existing shell extensions are compatibility (of course) and re-entrancy.

    The thumbnail extractor interface was lucky in that the only parameter an extractor received was an IShellItem representing the object for which the caller wishes to retrieve a thumbnail. There's no foothold into the user interface, which means that it can be moved to a place that has no user interface.

    Unfortunately, most shell extensions are not so lucky. Many of them receive some user interface object (usually a window handle) as a parameter. If those types of shell extensions have historically been hosted in-process, then the implementors of those shell extensions will do things like take that window handle and subclass it (possibly walking around the window hierarchy for a while until it finds a window it wants to subclass). Moving the shell extension into another process would break it, because you can't subclass windows in another process.

    Even if you manage to find an interface that is given no foothold into the user interface, you still may not be able to move it to a host process due to the danger of re-entrancy. When you invoke a COM method call from a single-threaded apartment (and all UI work is done in single-threaded apartments), and the object that is the recipient of the call lives on another thread or even in another process, COM will send the request to that other thread and pump messages waiting for the reply. This means that a method call which previously never pumped messages now does, opening windows of re-entrancy, and the great thing about windows of re-entrancy is that you never hit them yourself, but your customers somehow manage to find them without any problem.

  • The Old New Thing

    Stupid Raymond talent: Screaming carrier

    • 109 Comments

    Similar to Mike, I was able to scream (not whistle: scream) a 300 baud carrier tone. This skill proved useful when I was in college and the mainframe system was down. Instead of sitting around waiting for the system to come back, I just went about my regular business around campus. Every so often, I would go to a nearby campus phone (like a free public phone but it can only make calls to other locations on campus), dial the 300 baud dial-up number, and scream the carrier tone. If I got a response, that meant that the mainframe was back online and I should wrap up what I was doing and head back to the lab.

    Mind you, this skill isn't very useful nowadays.

    What stupid computer talent do you have?

  • The Old New Thing

    How can I tell whether a DLL has been registered?

    • 27 Comments

    A customer pointed out that you can use regsvr32 to register a DLL or to unregister it, but how do you query whether a DLL has been registered?

    DLL registration (via regsvr32) is not declarative; it is procedural. A DLL does not provide a manifest of things it would like to happen when installed. Instead, the DLL merely provides two functions for regsvr32 to call, one for registration (DllRegisterServer) and another for unregistration (DllUnregisterServer). All the regsvr32 function does is call those functions.

    How those functions perform their registration and unregistration is not specified. Most of the time, those functions merely write some registry settings, but the DllRegisterServer is not limited to that. For example, the DllRegisterServer function might write some values only conditionally, say, only if the user is running a specific version of Windows. Or it might back up the old value of a registry key before it overwrites it. It might create or modify files as part of its installation or configure your firewall settings or look for and uninstall previous versions of the same DLL.

    By convention, the DllRegisterServer performs whatever operations are necessary for DLL registration, and the DllUnregisterServer reverses those operations, but since those functions are provided by the DLL, there's no guarantee that that's what actually happens. Who knows, maybe DllRegisterServer formats your hard drive. A DllRegisterServer function might just return S_OK without doing anything. How can you tell whether a function with no side effects has been called?

    Given that DLL registration can encompass arbitrary operations, there is no general-purpose way of determining whether registration has taken place for an arbitrary DLL.

    To determine whether a DLL has been registered, you need to bring in domain-specific knowledge. If you know that a DLL registers a COM object with a particular CLSID, you can check whether that CLSID is indeed registered.

  • The Old New Thing

    Okay, everybody, it's time for rumors and gossip

    • 11 Comments

    A friend of mine told me one technique his boss used for keeping group meetings on time. The last item on every meeting agenda was called Rumors and gossip.

    A group meeting is sort of like a mandatory watercooler session. Everybody is now in a room sitting around a table, and you naturally start discussing whatever rumors you've heard about what's going on in upper management, that newspaper article about what your competition is up to, or whether you think your supplier is really going to deliver that component on time.

    Whenever the meeting started to drift into rumors and gossip, the boss would simply say, "Save it for rumors and gossip." And then the meeting would reach the Rumors and gossip stage, and everybody (including the boss) could gossip about whatever it was that they heard from a friend of a friend. The boss might confirm some rumors, deny others, say "I don't know either," or even "I don't know, but I'll try to find out." This was also a good time for the employees to raise their concerns about the project.

    From what I could gather, Rumors and gossip was a big hit. Everybody knew to save their rumors and gossip for the end of the meeting, allowing the rest of the meeting to stay on track, and it became the part of the meeting everybody looked forward to.

  • The Old New Thing

    Microspeak: Level-set

    • 18 Comments

    In mathematics, a level set is the set of points at which a function takes a particular value. This has nothing to do with the way the term is used at Microsoft.

    In fact, the way the term is used at Microsoft, I have no idea what it means. But here are citations. The first is from an upper-level executive:

    Before we start the meeting, let me level-set. Here's what we plan to accomplish today.

    The next is from a presentation to a large group on some investigative work a team undertook. After the presenter spent a few minutes discussing the background of the problem:

    That's some level-setting on the hardware we had available for investigation.

    Here are some citations from presentation slide decks:

    Agenda

    • ...
    • Pre-work check and Level Set
    • ...
    Initial Data gathering and Level Setting
    Teams share solution overviews, roadmaps and architecture views with each other to level set.

    (Notice that level-set is as a verb.)

    From what I can gather, level-setting is some sort of blend of expectation-setting and calibration.

  • The Old New Thing

    Percentages may not add up to 100%, but not for the reason you suggest

    • 24 Comments

    I saw a chart which had the disclaimer, "Percentages may not add up to 100%, as they are rounded to the nearest percent."

    The values in the table were 10.4%, 4.0%, 9.4%, 9.3%, 9.2%, 21.2%, 20.0%, and 15.8%.

    This is a use of the phrase "nearest percent" I was previously unfamiliar with.

  • The Old New Thing

    The life story of the SwitchToThisWindow function

    • 36 Comments

    Commenters Mick and Nick (you guys ever considered teaming up and forming a morning radio show?) are interested in the life story of the Switch­To­This­Window function.

    The Switch­To­This­Window was originally added in enhanced mode Windows 3.0 in order to support switching out of fullscreen MS-DOS sessions. Recall that enhanced mode Windows 3.0 was actually three operating systems in one: There was a 32-bit virtual machine manager, and inside one virtual machine ran a copy of standard-mode Windows,¹ and inside all the others ran a copy of MS-DOS. This mean that when you pressed a key when in an MS-DOS session, the keyboard interrupt went to the MS-DOS program and not to Windows.

    When you pressed Alt+Tab, some crazy magic had to happen. The virtual machine manager had to "un-press" the Alt key in the MS-DOS program, then synchronize the shift states of the Windows virtual machine to match the one from the MS-DOS virtual machine. (For example, if you had the shift key down in the MS-DOS virtual machine, it had to simulate pressing the shift key in the Windows virtual machine so they two shift states were back in sync.) And then it could simulate pressing the Tab key, at which point the Windows virtual machine would see the Alt+Tab sequence and put up the Alt+Tab interface.

    That's how things worked if you were running in a windowed MS-DOS session. But if you were in a fullscreen MS-DOS session, things worked differently. Switching back to Windows would mean a display mode reset (which can take a second or longer), and then all the applications on your desktop had to redraw themselves (and probably paging quite a bit in order to do so). This definitely failed to meet the responsiveness people expected from Alt+Tab, so the virtual machine manager pulled a trick: If you pressed Alt+Tab while in a fullscreen MS-DOS session, then instead of switching back to the Windows virtual machine, the virtual machine manager displayed a text-mode version of the Alt+Tab interface.

    I will stop to let the craziness of that sink in: The virtual machine manager had its own Alt+Tab interface built out of text mode.

    Anyway, when you finally released the Alt key and completed the Alt+Tab sequence, the virtual machine manager needed to tell Windows, "Hey, like, pretend that an Alt+Tab thingie just happened, okay?"

    That is what the Switch­To­This­Window function was for. It was the function the virtual machine manager called to tell Windows to switch to a window as if the user had selected it via Alt+Tab (because that is, in fact, what the user did, just via the text-mode interface rather than the graphical one).

    A similar thing happened if you pressed Alt+Esc (or Alt+Shift+Esc in a fullscreen MS-DOS session. That's why there's a second parameter to indicate whether the switch should be done "in the style of Alt+Tab" or "in the style of Alt+Esc."

    The function was undocumented because it existed only for the virtual machine manager to call in order to coordinate its actions with Windows user interface so that you had one big happy Alt+Tab family.

    The text-mode Alt+Tab interface disappeared in Windows 95, but the Switch­To­This­Window function hung around because it wasn't causing anybody any harm, and there was at the time no formal process in place to deprecate and eventually remove an API, not even an internal undocumented one.

    In the Windows XP SP1 timeframe, a bunch of lawyers decided that some functions in Windows needed to be documented. The precise rules for determining which functions needed to be documented and which didn't need to be documented were rather complicated. (Some people applied an algorithm different from the ones those lawyers used and came up with a list of functions that are "missing", when all that they really came up with is a list of functions different from the list those lawyers came up with.)²

    Anyway, the Switch­To­This­Window function got caught in the dragnet, so it got documented. Mind you, like it says right at the top of the documentation, there is no guarantee that the function will continue to exist; it can vanish at any time. Although there is documentation, it has the logical status of an internal function, and internal functions have a tendency to change or vanish entirely. Perhaps someday a new chapter will be added to the life story of Switch­To­This­Window: "The Switch­To­This­Window was removed in Windows Q" for some value of Q.

    Footnotes

    ¹ Not true, but true enough. Don't make me bring back the Nitpicker's Corner.

    ² I will delete any comments on the subject of the algorithm by which those lawyers determined which functions needed to be documented, or on the documentation itself.

    Bonus chatter: As far as I can determine, Switch­To­This­Window just does a Set­Foreground­Window on the window you're switching to, possibly posting it a WM_SYS­COMMAND/SC_RESTORE message, and moving the previous foreground window to the bottom of the Z-order if switched via Alt+Esc. It doesn't provide any special secret sauce for bypassing the normal foreground activation rules. The process that calls Switch­To­This­Window still requires foreground-change permission.

  • The Old New Thing

    How do I generate a unique 32-bit value for a time zone?

    • 8 Comments

    Public Service Announcement: Daylight Saving Time ends in most parts of the United States this weekend. Other parts of the world may change on a different day from the United States.

    A customer asked the following question:

    Given two TIME_ZONE_INFORMATION structures, I would like to compute a LONG for each that I can then compare to determine whether they represent the same time zone. When I say the same, I mean that when the two are passed to System­Time­To­Tz­Specific­Local­Time with the same LPSYSTEM­TIME input, the output is the same.

    A TIME_ZONE_INFORMATION structure contains more information than can be packed into a 32-bit value. (At least there's no obvious way to pack it into a 32-bit value.) You're not going to be able to squeeze the entire structure into a 32-bit value that is unique for each time zone, so that comparing the 32-bit values will tell you whether the time zones are the same or not.

    Fortunately, the customer also provided context for the question, explaining their underlying problem. And as is often the case, the customer had broken down the problem into two parts, one easy and one impossible. The customer solved the easy part and was asking for help with the impossible part.

    But on closer inspection, the problem wasn't so much impossible as it was improperly specified:

    The bigger problem I'm actually trying to solve is that we call System­Time­To­Tz­Specific­Local­Time inside a deeply nested loop. I would like to cache the results for performance, using the time zone as a key to a CAtl­Map which would hold the cached results for each time zone. I'm looking for help coming up with what combinaion of the structure members to use to uniquely identify the time zone.

    Okay, the customer appears to be a bit confused about hash keys. Hash keys do not need to be unique for each time zone. It is perfectly legitimate for two different items to result in the same hash value; that's why we have the term hash collision. Of course, you want to take reasonable steps to minimize collisions, but when you don't control the domain space, hash collisions are a part of life.

    From looking at some time zone data, it looks like (Bias + Standard­Bias) is unique for any time zone, but I know that there are a lot of complicated issues when dealing with time zones so I wanted to check if I could be sure of that.

    LONG CTimeZoneTraits::GetHash(const TIME_ZONE_INFORMATION& tz)
    {
     return tz.Bias + tz.StandardBias;
    }
    
    int CTimeZoneTraits::Equals(const TIME_ZONE_INFORMATION& tz1,
                                const TIME_ZONE_INFORMATION& tz2)
    {
     return tz1.Bias         == tz2.Bias &&
            tz1.StandardBias == tz2.StandardBias &&
            tz1.DaylightBias == tz2.DaylightBias &&
            memcmp(&tz1.StandardDate, &tz2.StandardDate,
                                      sizeof(tz1.StandardDate) &&
            memcmp(&tz1.DaylightDate, &tz2.DaylightDate,
                                      sizeof(tz1.DaylightDate);
    }
    

    If you think it about it, it's clear that (Bias + Standard­Bias) does not always uniquely identify a time zone. Consider two cities at the same longitude in the same hemisphere in the middle of winter: They will have the same Standard­Bias (because they have the same longitude) and the same Bias (because Daylight Saving Time is not applicable during the winter), but if the cities are in different countries (or sometimes, even different parts of the same country), they will transition to/from Daylight Saving Time differently and consequently do not belong to the same time zone.

    On the other hand, since this is being used simply as a hash key, uniqueness is not an absolute requirement, so even a bad hash function will still "work"; it'll just be slower than a good hash function.

    If it were up to me, I would choose as a hash function something like this:

    LONG CTimeZoneTraits::GetHash(const TIME_ZONE_INFORMATION& tz)
    {
     return tz.StandardBias +
            tz.StandardDate.wDay +
            (tz.StandardDate.wDayOfWeek << 16) +
            (tz.StandardDate.wMonth << 24);
    }
    

    I wouldn't use the Bias in the hash code because the Bias changes over time. If the hash table lifetime extends across a daylight saving time transition, then the Bias will change.

    For the hash, I use the Standard­Bias, which is the number of minutes east of UTC. In practice this does not exceed 60 × 25 = 1500, and it's a multiple of 30. (But not necessarily a multiple of 60.) The wDay is typically in the range [0,5], though it can go as high as 31 if the transition is based on a specific day. Therefore, I'll simply add it to the Standard­Bias, taking advantage of the fact that the Standard­Bias is a multiple of 30. The month and day of the week are thrown into the upper 16 bits.

    Now, this hash function will still have collisions: If there are two time zones at the same longitude which transition to Standard time with the same rule, but which transition to Daylight time according to different rules, then we will still have a collision.

    I would like to reduce the number of collisions by understanding how often two equal values of (Bias + Standard­Bias) could represent different time zones.

    How likely is such a collision? You can answer this question yourself: Take all the time zones currently known to the system and hash them all to see what happens. Of course, time zones change all the time, so don't assume that your results will hold true in perpetuity, but if you're just looking for a rough guide, calculating against the current state of affairs is a pretty good one. It's true that time zones change all the time, but they typically don't change by much.

Page 2 of 3 (23 items) 123