This week, we shipped the February Cumulative Update for Outlook 2010. One of the fixes in there relates to the MAPI<->MIME conversion API. In particular, with Outlook 2010 if you use MAPIToMIMEStm to convert a message to EML you’ll see the source message get updated. This didn’t happen with Outlook 2007. The reason for the change in behavior has to do with how we handle conversation threading, particularly for messages that don’t already have some of the conversation properties already stamped on them. In order to preserve threading should the MIME message come back in to MAPI (a scenario which occurs with POP3/SMTP profiles and replies), we would stamp the conversation properties on the source message during the export.
In general, this is fine, but it did cause a problem for some vendors who needed their export to not modify the source. So for them, in the February CU we enable a new flag:
#define CCSF_PRESERVE_SOURCE 0x40000 // don't modify the source message
Pass this new flag in to MAPIToMIMEStm and we’ll bypass the new code that saves changes back to the source message.
One caveat: We may still make changes to the message. This flag skips the SaveChanges call that commits the changes. So if your code subsequently calls SaveChanges, you’ll see the changes made during the export get committed. Of course, if your code is calling SaveChanges, you probably won’t be using this flag in the first place.
Enjoy!
Remember Jason’s article about the reconnect logic we added to Exchange’s MAPI? The idea was that if your GC went down, Exchange’s MAPI should be able to reconnect to a different GC and proceed smoothly. Mostly smoothly though, since the last call sent to the old GC may fail with MAPI_E_END_OF_SESSION and need to be retried. What was left unsaid from that article was that if you’re using Outlook’s MAPI, which supports reconnect without special configuration, there are some other error codes that can bubble up to the client. Development has given me permissions to document a couple that have shown up in the wild:
#define MAPI_E_RECONNECTED MAKE_MAPI_E( 0x125 ) // 0x80040125 #define MAPI_E_OFFLINE MAKE_MAPI_E( 0x126 ) // 0x80040126
#define MAPI_E_RECONNECTED MAKE_MAPI_E( 0x125 ) // 0x80040125
#define MAPI_E_OFFLINE MAKE_MAPI_E( 0x126 ) // 0x80040126
MAPI_E_RECONNECTED, aka 0x80040125, is returned by the Exchange Address Book and Message Store providers when the RPC connection token is discovered to be out of date. The connection token is basically a number tracking the current connection. If the token on our current transaction is different from the token on the connection, this means we have reconnected, so MAPI_E_RECONNECTED is returned. MAPI_E_RECONNECTED can be treated the same as MAPI_E_END_OF_SESSION and the call should be retried.
MAPI_E_OFFLINE, aka 0x80040126, is returned by the Exchange Address Book and Message Store providers when we discover that we are offline. Typically this means that something has happened in the environment, such as a server going down, or loss of network connectivity. This error is most likely to occur when using a cached mode profile and attempting to bypass the cache to talk to the server. If the cache was never able to establish a connection to the server in the first place, we may be in this offline state where MAPI_E_OFFLINE could surface.
Neither one of these errors will be returned in all scenarios where it would appear they would appear to apply. In most cases, MAPI_E_NETWORK_ERROR or MAPI_E_CALL_FAILED will be returned instead. But now if you do get one of these errors you know what it means. Both of these errors are specific to Outlook’s MAPI only. They will not appear with MAPICDO.
This was an interesting scenario one of our customers came up with. They process a lot PSTs. As part of this processing, they create search folders and wait for them to complete building. They found that for many PSTs, they would never get a notification that the search folder had finished building.
Here’s the basic outline of what they were doing:
Usually this worked, but occasionally their application would end up stuck in step 5 waiting for the search folder completion. What was really strange is that if we opened MFCMAPI to look at the stuck search folder, after some poking around their application would suddenly get the completion notification!
To understand what’s going on here, we need to know a little bit about how the PST handles searches. There are two basic ways a search can be fulfilled. The most common way currently is that the PST provider leverages Windows Search to handle the search for it. That’s the default configuration, and the customer’s code always worked when it was enabled. But Windows Search can be turned off. This is when the customer’s code would have problems. Without WIndows Search to do the work for us, the PST provider falls back to the mechanism it always used before Windows Search existed: on a background thread it manually does the search and populates the search folder. It’s the handling of this background thread where we were having problems.
When you first load the PST provider, it spins up an idle thread which handles background processing such as search folder generation and updates. Since a PST can be loaded in multiple processes, with only one process actually loading the file and the various processes negotiating reads/writes through shared memory, every process which opens a MAPI profile is a candidate to be the one who manages searches. So every search folder has a process which is considered to be the owner of the search. This owner can go away, in which case the search is orphaned, and a new owner has to be selected. Originally, this was the first process who’s idle thread detected there was a search without an owner which needed to be processed. This, however, caused a problem with short lived processes. Suppose you have a processes that needs to get in and out of MAPI fairly quickly, such as to send an e-mail. This process could unwittingly become the owner of a search, meaning it’s on the hook to stay around and process the search, even though there’s little chance it will benefit from it. This caused problems, so we introduced a delay. If the PST provider has been loaded less than 5 seconds, we won’t try to take ownership of a search.
This is where the customer’s application got in trouble. They started up and got through to the SetSearchCriteria so quickly their idle thread decided it shouldn’t handle the search. And since they were doing nothing else while they were waiting, nothing ever kicked the idle thread back to life to realize there was an unfulfilled search. When MFCMAPI ran, clicking around in the UI was enough for MFCMAPI’s idle thread to spot the search and complete it. This is not to blame their application – they did everything right and nothing wrong. They just managed to walk the narrow tightrope needed to see this problem.
So – how can they avoid the hang? One workaround is obvious – Sleep for 5 seconds before creating search folders. But that’s a performance hit and nobody wants that. Fortunately, there’s a bypass for the delay. If your MAPI session was created with MAPI_BG_SESSION, we assume you’re not just some client sending e-mail, so we grab any search your process is eligible to process the first time we see it. Note that the documentation for MAPI_BG_SESSION indicates “A client application such as an indexing engine or opening a Personal Folders File (PST) for background type access are some examples of where to use MAPI_BG_SESSION”. This is precisely the sort of scenario where we intend this flag to be used.
The customer tried this flag in their code and it worked perfectly. Yay!