MAPI Store Providers and Outlook 2003

Put this together from a posting I made to the MAPI-L list and a couple cases I worked recently:

The problem is with MAPI store providers that just don't work when loaded under Outlook 2003. The developers of the providers, when they contact us, are usually convinced the problem is a bug in Outlook 2003 since the providers “worked perfectly” under Outlook XP. My experience though, is that the problem is that Outlook 2003 demands so much more of the MAPI spec than previous versions of Outlook that bugs which have always been in these providers are now exposed. What's very interesting though is that I'm seeing the same bugs over and over in these providers. I hope to highlight the source of these bugs and offer some ideas on how to correct them.

There is a book which is very much coveted in the MAPI development community: Inside MAPI. One of the samples in this book is MSLMS, the Microsoft Sample Local Message Store. For what it is, sample code demonstrating how to write a message store provider, it's great. I'm not aware of any other sample message store out there. So it does not suprise me that a number of people choose to base their provider on the code in MSLMS. As such, they tend to inherit the same bugs and design flaws present in the sample.

I recently spent a couple days hacking on the MSLMS sample so that it could load under Outlook 2003. Here's a short list of the problems I found and what I had to do to fix them:

• Two heaps we're being allocated for the Provider object, then used in all objects - the heaps were destroyed when the Provider object was destroyed, meaning we'd crash if any object had a longer lifetime. Fixed this by introducing DLLMain and creating/destroying the heaps on module load and unload. Putting this code in DLLMain may not be safe, but it stopped the crash for now. In production code, I'd find a better solution.
• PR_RECORD_KEY abuse: This bug will stop a provider cold in Outlook 2003.In a number of tables, PR_RECORD_KEY for all objects was
  returning the store's record key instead of the key associated with the individual objects. Since Outlook 2003 depends on PR_RECORD_KEY to cache objects, this error effectively meant nothing in the provider could be rendered. The fix is to make sure PR_RECORD_KEY is unique for all objects and can be accessed through the object and contents tables and hierarchy tables.
• GetNamesFromIDs deviated far from the MAPI spec. Suppose we make this call asking for two IDs, one which does exist and one which doesn't. MSLMS would return an array with one result, leaving the caller with no way to determine which ID got matched to a name. The correct thing to do is return an array with two entries - one being the matched name and the other indicating an error.
• Multivalued string and multivalued binary props - The code which handled property persistance failed both of these cases. These properties can get quite complex, with a number of internal pointers. From what I could tell, it did attempt to address them, but it failed to fix up all of the internal pointers when flattening the data structures on save and unflattening them on load. In addition to failing to persist data, this bug also lead to a number of crashes due to the invalid pointers.
• Named properties were being excluded from contents tables: Don't know why they were being blocked, but this meant Outlook couldn't get to some data on items that it expected to be able to retrieve through the contents table.

Those are just the problems I fixed. Here are some I didn't:

• PR_RECORD_KEY (again): MSLMS assumes it can always use PR_ENTRY_ID when asked for PR_RECORD_KEY. This assumption breaks down on attachments, where objects do have PR_RECORD_KEY, but do not have PR_ENTRY_ID.
• MV_INSTANCE: This somewhat obscure flag is at the heart of Outlook's code for handling the Group By box. MSLMS doesn't support the flag. Another effect is that Contact Address books cannot be hosted in MSLMS because the contab32 address book provider depends on the underlying store provider having support for this flag.

Anyone who has based their message store provider on MSLMS (even if it was “What would MSLMS do?“ and not just borrowing the code) needs to review their code and ensure they've addressed all of the above bugs before they approve their provider for support under Outlook 2003.