This was an interesting scenario one of our customers came up with. They process a lot PSTs. As part of this processing, they create search folders and wait for them to complete building. They found that for many PSTs, they would never get a notification that the search folder had finished building.

Here’s the basic outline of what they were doing:

  1. Their application starts and they load a PST.
  2. Register an Advise Sink for fnevSearchComplete.
  3. Create a search folder using CreateFolder with the FOLDER_SEARCH flag.
  4. Apply search criteria using SetSearchCriteria.
  5. Wait for fnevSearchComplete.

Usually this worked, but occasionally their application would end up stuck in step 5 waiting for the search folder completion. What was really strange is that if we opened MFCMAPI to look at the stuck search folder, after some poking around their application would suddenly get the completion notification!

To understand what’s going on here, we need to know a little bit about how the PST handles searches. There are two basic ways a search can be fulfilled. The most common way currently is that the PST provider leverages Windows Search to handle the search for it. That’s the default configuration, and the customer’s code always worked when it was enabled. But Windows Search can be turned off. This is when the customer’s code would have problems. Without WIndows Search to do the work for us, the PST provider falls back to the mechanism it always used before Windows Search existed: on a background thread it manually does the search and populates the search folder. It’s the handling of this background thread where we were having problems.

When you first load the PST provider, it spins up an idle thread which handles background processing such as search folder generation and updates. Since a PST can be loaded in multiple processes, with only one process actually loading the file and the various processes negotiating reads/writes through shared memory, every process which opens a MAPI profile is a candidate to be the one who manages searches. So every search folder has a process which is considered to be the owner of the search. This owner can go away, in which case the search is orphaned, and a new owner has to be selected. Originally, this was the first process who’s idle thread detected there was a search without an owner which needed to be processed. This, however, caused a problem with short lived processes. Suppose you have a processes that needs to get in and out of MAPI fairly quickly, such as to send an e-mail. This process could unwittingly become the owner of a search, meaning it’s on the hook to stay around and process the search, even though there’s little chance it will benefit from it. This caused problems, so we introduced a delay. If the PST provider has been loaded less than 5 seconds, we won’t try to take ownership of a search.

This is where the customer’s application got in trouble. They started up and got through to the SetSearchCriteria so quickly their idle thread decided it shouldn’t handle the search. And since they were doing nothing else while they were waiting, nothing ever kicked the idle thread back to life to realize there was an unfulfilled search. When MFCMAPI ran, clicking around in the UI was enough for MFCMAPI’s idle thread to spot the search and complete it. This is not to blame their application – they did everything right and nothing wrong. They just managed to walk the narrow tightrope needed to see this problem.

So – how can they avoid the hang? One workaround is obvious – Sleep for 5 seconds before creating search folders. But that’s a performance hit and nobody wants that. Fortunately, there’s a bypass for the delay. If your MAPI session was created with MAPI_BG_SESSION, we assume you’re not just some client sending e-mail, so we grab any search your process is eligible to process the first time we see it. Note that the documentation for MAPI_BG_SESSION indicates “A client application such as an indexing engine or opening a Personal Folders File (PST) for background type access are some examples of where to use MAPI_BG_SESSION”. This is precisely the sort of scenario where we intend this flag to be used.

The customer tried this flag in their code and it worked perfectly. Yay!