Free Associations

I'm Famous!

2009-02-15T03:58:00Z

http://www.microspotting.com/2009/02/zeke

IEProcess.h - IE inproc APIs

2008-11-05T01:01:00Z

IE8 beta1 introduced Loosely Coupled IE (LCIE) which gives tab a level of process isolation from the frame: http://blogs.msdn.com/ie/archive/2008/03/11/ie8-and-loosely-coupled-ie-lcie.aspx.

For the most part this works pretty well. But certain binary extensions (activeX objecst, BHOs, etc) that are already deployed have assumptions about window hierarchy and process model. To get this to work the IE team has been diligently testing extensions and finding ways to fix them. For the most part we do this by using the Detours library to redirect Windows APIs to private wrappers. We have been using this technique since we added tabs. For tabs we were just avoiding tabs contending for window modality by calling functions like MessageBox(). Our hook would effectively created a queue of modal dialogs avoiding confusing dialogs from hidden tabs, etc.

But not all problems can’t be fixed by just detouring an existing API this way. A reasonable example is when an application has a custom modal dialog manager. This means that our hooks on DialogBoxParam() and related functions won’t be hit. Other hooking options are complex and fragile so we opted to expose the "right" way for external developers. IEProcess.h exposes functions for helping with this: http://msdn.microsoft.com/en-us/library/cc994365(VS.85).aspx

This API was designed with the idea that many components that are loaded in IE can actually be loaded elsewhere as well. So instead of linking to IE dlls directly, some level of indirection is required. This avoids the cost of loading a DLL when it won’t be used, and allows the IE dev team a little flexibility for how we expose these APIs. First a caller detects whether any IEProcess exports with the following code:

    HMODULE ieModule = IEProcess:GetProcessModule(); 
    //    if ieModule != NULL then there are exports available

After retrieving the module, the methods are grouped similar to COM interfaces, but are static, and exist for the lifetime of the process. This is similar to doing GetProcAddress() for a set of functions:

    static const IETabWindowExports* ieExports = GetTabWindowExports(ieModule);
    //   ieExports contains the methods we need to call

Now if you look here http://msdn.microsoft.com/en-us/library/cc994365(VS.85).aspx again, you will see how we implemented our detour of DialogBoxParamW(), and this can be fairly easily replicated to a custom dialog manager. WaitForTabWindow() queues your modal call with the tab; when it returns, its your turn. Then take the lock with AcquireModalDialogLockAndParent(), which will create an appropriate top level window if required for security boundary reasons. Then show the modal dialog, and call ReleaseModalDialogLockAndParent() when done. It’s that easy!

This is super straightforward if you always run in IE, but looks a little awkward to handle the error conditions if you run outside IE. So I wrote a wrapper class that does pass through logic when running outside of IE. It has an H and CPP file. This is how MSHTML does its IE synchronization for HTML Dialogs, which have a custom dialog manager. The header is below, and the CPP is attached.

#pragma once

/// 
/// Helper class that does delay load binding to the IEProcess
/// export tables with fallback logic for easy pass through.
/// 
class IEProcessHelper
{
public:

    /// 
    /// Detect whether per-process exports are enabled for this process.
    /// This does it's binding on demand and the result is cached.
    /// 
    static bool IsIEProcess();    

    /// 
    /// Retrieve the tab window related exports for this process.
    /// If they aren't available it returns a table of fallbacks 
    /// with pass through logic so that this never returns NULL.
    /// This does it's binding on demand and the result is cached.
    /// 
    static const IETabWindowExports* TabWindow();

private:
    
    static HRESULT WaitForTabWindowNotImpl(
        __in    bool    allowUnknownThread,  
        __in    HWND    hwndParentProposed,  
        __out   HWND*   phwndParentActual);  

    static HRESULT AcquireModalDialogLockAndParentNotImpl(
        __in        HWND    hwndParentProposed,  
        __out       HWND*   phwndParentActual,
        __deref_out HANDLE* phModalDialogLock);

    static void ReleaseModalDialogLockAndParentNotImpl(
        __in_opt HANDLE hModalDialogLock);

private:
    static const IETabWindowExports _TabWindowExportsNotImpl;
    static const IETabWindowExports* _TabWindowExports;
    static HMODULE _hmoduleIEProcess;
    static bool    _checkedIEProcess;
};

This simplifies our detour so that the following is the new calling pattern, and doesn’t require any kind of fallback logic.

INT_PTR WINAPI Detour_DialogBoxParamW(
        HINSTANCE hInstance, LPCWSTR lpTemplateName, 
        HWND hwndParent, DLGPROC lpDialogFunc, LPARAM dwInitParam)
{
    INT_PTR nRes = -1;

    HRESULT hr = IEProcessHelper::TabWindow()->WaitForTabWindow(
            false, hwndParent, &hwndParent);
    if (SUCCEEDED(hr))
    {
        HANDLE hModalDialogLock;
        hr = IEProcessHelper::TabWindow()->AcquireModalDialogLockAndParent(
                hwndParent, &hwndParent, &hModalDialogLock);
        if (SUCCEEDED(hr))    
        {
            nRes = Real_DialogBoxParamW(hInstance, lpTemplateName, 
                    hwndParent, lpDialogFunc, dwInitParam);

            IEProcessHelper::TabWindow()->ReleaseModalDialogLockAndParent(
                    hModalDialogLock);
        }
    }
    
    return nRes;
}

Check out MSDN related docs for running in Protected Mode: http://msdn.microsoft.com/en-us/library/bb250462.aspx

Watch out for sprintf()!

2008-08-15T20:44:00Z

I was looking at some code the other day that does string concatenation with StringCchPrintf(“%s%s”). I can imagine how this would have near optimal performance, and a hopeful developer could easily assume that in fact it has been tuned for this kind of concatenation. But measuring tells all. Three trivial implementations of concat using strsafe.h:

void concat_printf(PCWSTR str1, PCWSTR str2, PWSTR out, size_t max)
{
    StringCchPrintfW(out, max, L"%s%s", str1, str2);
}

void concat_copy(PCWSTR str1, PCWSTR str2, PWSTR out, size_t max)
{
    StringCchCopyEx(out, max, str1, &out, &max, 0);
    StringCchCopy(out, max, str2);
}

void concat_cat(PCWSTR str1, PCWSTR str2, PWSTR out, size_t max)
{
    StringCchCopy(out, max, str1);
    StringCchCat(out, max, str2);
}

Measurements for short concatenating strings (~10 chars) and long strings (~1000 chars).

method	short	long
concat_printf	187	10530
concat_copy	15	952
concat_cat	15	1264

Unsurprisingly concat_copy() is the most efficient, with concat_cat() overhead of a strlen() only noticeable with longer strings. But wow, concat_printf() is shocking order of magnitude slower than either of the others. Taking a peak under the covers explains why: MSVCRT is using a FILE* stream abstraction that does a lot more work for every character copy. This is so the code for printf(), fprintf() and sprintf() can all be shared. Generalization at the cost of performance!

Performance work is slippery

2008-08-13T20:57:00Z

… like a water snake. Which is why we measure, measure and measure again. Even the most obvious rule of thumb can be wrong. I was optimizing some string parsing code (which is almost always ripe for optimization), and trying to reduce the cost of traversing a string.

When iterating over the elements of an array there are two typical models:
index increment

    while (ptr[index] != 0)
    {
        index++;
        //  do stuff with ptr[index]
    }

pointer increment

    while (*ptr != 0)
    {
        ptr++;
        //  do stuff with *ptr
    }

And while they both generate reasonable code the index increment is typically ~2x the cost of the pointer increment. For iterations that aren’t called much or that have expensive inner operations this is likely not noticeable. This makes sense because it avoids having to recalculate the offset to the current character inside the loop (ptr[index]). The incremented pointer is the current offset. My rule of thumb is to use array indexes when random access is required and use pointer increment for iteration.

As an example for this rule of thumb, I wrote the simplest of string iterations: calculating a string’s length.

size_t slen_index(PCWSTR str)
{
    size_t index = 0;
    while (str[index] != 0)
    {
        index++;
    }
    return index;
}

size_t slen_ptr(PCWSTR str)
{
    PCWSTR ptr = str;
    while (*ptr != 0)
    {
        ptr++;
    }
    return ptr - str;
}

And then I measured to find out how much faster the ptr loop is (lower numbers are faster):

    slen_ptr   = 265
    slen_index = 188

Surprise, the intuitively “faster” version is actually slower. Peeking at the disassembly provides a quick answer. slen_index() keeps index in a register instead of a stack location, whereas slen_ptr()stores the incremented back into the stack variable. This is a pathological case where the compiler is able to make a better optimization because we don’t do anything with the characters we are traversing. Let's check out functions that actually do something in the loop.

#define IS_ALLOWED(ch) (((ch) > 0x20) && (((ch) < 0x80))) 

bool isallowed_index(PCWSTR str)
{
    size_t index = 0;
    while (str[index] != 0)
    {
        index++;
        if (!IS_ALLOWED(str[index]))
            return false;
    }
    return true;
}

bool isallowed_ptr(PCWSTR str)
{
    PCWSTR ptr = str;
    while (*ptr != 0)
    {
        ptr++;
        if (!IS_ALLOWED(*ptr))
            return false;
    }
    return true;
}

And now measure (lower numbers are faster):

    isallowed_ptr    = 250
    isallowed_index  = 390

That’s more like it! My rule of thumb makes more sense, but the stronger rule is to measure, measure, and measure again!

PS - these numbers also show that for most functions it probably won't make a difference in your app. But if the function is called a lot, it can (eg strlen()).

How do explorer context menus sort verbs?

2005-05-20T00:36:00Z

Windows Explorer has a default implementation for IContextMenu commonly called DefCM that has been used since Win95. It has many clients and has evolved over time, of course, but its primary focus has always been to support the most important of shell folders: File System Folder. A big part of this support revolves around extensibility. Raymond has been talking about hosting IContextMenu, I am going to talk about how IContextMenu extensions interact with DefCM. And while I may call out the behavior of older versions of Windows to explain things, I am going to focus on the behavior of WinXP.

The first thing to read is Verbs and File Associations on MSDN. The section on "Static vs. Dynamic Verbs" explains that there are really two extensibility methods. this is important since they will be aggregated together by DefCM. DefCM enumerates verbs from a set of registry hkeys typically generated from file associations. For more details about these keys look here.

DefCM starts by gathering all of the IContextMenu extensions which are registered under the "shellex\contextmenuhandlers" subkeys. There is also one implicit object for file associations that handles static verb registration which is added first. The result of this enumeration is an array of objects starting with the static verb implementation, followed by each of the registered ContextMenuHandlers in order of the keys above. Handlers that are registered under two keys are pruned to a single entry.

In order to get the expected menu order, DefCM calls QueryContextMenu() on each of these objects to insert verbs in reverse order, and has each one insert at the top of the menu (zeroth position). As a result the static verbs are inserted last and end up at the top, which is good since usually these guys are the default.

So the sorting is based on the following elements in decision order:

Key priority (eg, txtfile, *, AFSO)
Registry Enumeration order of shellex\contextmenuhandlers with a special case for static verbs always being first
IContextMenu Implementation order

So if there is any contention for position, there is no consistent way for an extension to guarantee their relative position within the menu.

The Bizarre and Unhappy Story of 'file:' URLs

2005-05-19T19:15:00Z

For my first blog entry, I will start with something I wrote for my team back in 1997 (reedited for this blog) that has been helpful to generations of subsequently bewildered developers:

Go back to 1995 and a small team dreaming an impossible dream of the Internet on Windows! URLs were young, so young they still hadn't been seen in a television ad....

IE1.0 was just a fledgling trying to get some features that Mosaic (or the looming competitor) didn't have. Since we shipped with Win95 and a big feature for Win95 was Long File Names, so IE needed to have support for spaces in names. Since there was no spec for the "file" protocol scheme (heck there was practically no spec for any protocol) we added automagical escaping spaces. We happily escaped spaces all over the place, and since nobody used our browser it didnt make a whits difference.

Now IE began to mature and grew into a precocious 3-year-old, and when the powers that be noticed this, they decided that 3 was such a big number that the browser needed a new image. That image was componentization. The browser was taken and split in three major parts. One part became mshtml (for rendering HTML) another shdocvw (for hosting DocObjects), and the third was wininet and urlmon (two binaries that were basically inseparable for downloading and caching content). Unfortunately a key component from the platform was missing, so it was replicated into each of the other components. That was an URL parsing platform.

No one component could agree with the others about the best way to handle the wild URLs that were so plentiful. So each component came up with slightly different rules, although very logical, based on its personal fantasies of the one true URL. The fantasies were fairly similar but the devil is in the details and these URLs went through a lot of metamorphoses on each trip down the stack to the wire. Now each breed of URL had its own issues with these transformations, but in the end all children must be pushed from the nest to learn to fly on their own.

Enter IE4, Nashville, Win98, the next generation of everything. And with it the new patron saint of URL parsing. Now someone undertook the plight of the underrepresented URLs very seriously and wished to give them the respect they deserved. Some cried with fear, and others with joy, but none could debate the necessity of a savior. So the little URLs were taken and given a safe place to live and be loved: shlwapi.dll. And the saint went amongst the browser and the browser's friends and said, "Let no one but the chosen ones touch or talk or molest these URLs, and there will be a peace among us." Some resisted, but they were put down as heretics should be and a peace came upon the land. Except for the one abused problem URL: file:.

No one had been quite as abused as the the little file: URL. This URL was special because we had always used files and DOS paths (and no one at the time knew about path canonicalization attacks), everyone was quite sure what they looked like , acted like, and even tasted like. It didn't help that the file: protocol remained in RFC limbo as a platform/OS specific protocol. So the browser and the browser's little friends would take turns dressing a DOS path like an URL in a pink bunny suit and undressing the URL with a pair of rusty scissors, pretending it was the same DOS path they started with. Only the simplest of URLs was able to withstand this abuse, and it soon became clear that something would have to be done, lest the little file: URLs go off on their own and be lost forever.

After much praying and meditation and consultation, it was finally decided that file: URLs must have two forms, one being the well formed easily communicable URL form and the other a legacy mutant form of a DOS path. These two forms had different purposes, and soon different personalities. The URL form lived happily inside the browser, but any time the browser needed to talk about the file: URL with its friends, it would clone and mutate the URL into its ugly half DOS form.

What this MEANS:

There are two kinds of file: URLs. The first is the well formed URL style. This is allowed query strings, fragment IDs, escape sequences and all the other goodies that URLs are supposed to support. We call this a healthy file: URL. The other kind is basically a DOS path with "file://" stuck on the front. We use this for legacy communication with outsiders only. We call this the legacy file: URL. Examples:
DOSPATH:         c:\windows\My Documents 100%20\foo.txt 
HEALTHY:         file:///c:/windows/My%20Documents%20100%2520/foo.txt

LEGACY:          file://c:\windows\My Documents 100%20\foo.txt 


DOSPATH:         \\server\share\My Documents 100%20\foo.txt 
HEALTHY:         file://server/share/My%20Documents%20100%2520/foo.txt 
LEGACY:          file://\\server\share\My Documents 100%20\foo.txt 
NOW all of the Url* APIs in shlwapi, and thus everyone in IE4, understand the difference between these two types of URLs and act accordingly. as long as a client of the APIs, doesn’t try to access the data directly, there is no issue whatsoever. in other words as long as you dont have code anywhere that looks like this:
LPTSTR GetFilePath(pszUrl)

{

     if (0 == lstrcmpn("file://", pszUrl, sizeof("file://")))

     {

          return pszUrl + sizeof("file://");
     }
     return NULL;
}
but instead always call shlwapi like this:
PathCreateFromUrl(pszUrl, pszPath, cchPath, dwFlags);
You will always be happy and golden. Straying from the path is straying into unknown lands. There are of course flags that you can pass into some of the APIs that will change the form the file: URLs when they come out, but I highly recommend not using it for any reason, as it just increases the chances of creating more bizarre unsupportable behavior.