July, 2004

I was hoping to post something less controversial for my first blog entry, but this issue came up again recently and I felt I had to address it.

I'm frequently asked why I we don't support putting Outlook and Exchange on the same machine. One response I typically hear is “But I run both on my machine and never have a problem”. In this post, I'll attempt to clarify a few things and explain what kinds of problems you can expect when you put Outlook and Exchange on the same server.

A couple definitions - shorthand mainly:

Admin: The Exchange Administrator (5.5) or the Exchange System Manager (2000 and 2003).

MAPI: Used by iself, refers to Extended MAPI. I'll spell out Simple MAPI when I need to make a distinction.

The main thing to clear up is that the warning against putting the Outlook and Exchange on the same machine applies primarily to servers and other mission critical machines. Basically, if your server hosts an application which uses MAPI, you should never install Outlook on it.

So why do the articles make a point to say not combine the Admin and Outlook? The answer is because there are a large number of server applications out there which rely on MAPI for integration with Exchange. Examples include voicemail, PDA synchronization, workflow, archival, legal discovery and connectors (gateways) to third party mail/database systems. The recommended method for installing MAPI onto a server for one of these applications is to install the Admin.

Here's a (greatly simplified) synopsis of MAPI's architecture. The core file is mapi32.dll. After loading MAPI, applications then open message stores and address books. These are implemented by providers. The providers which allows MAPI to talk to Exchange and the directory are emsmdb32.dll and emsabp32.dll.

MAPI was designed so that anyone could implement the core component and providers. A few companies did choose to implement the Simple MAPI portion of the API in their own mapi32.dll, but AFAIK, the only currently available implementations of the Extended MAPI portion of the API are those shipped by Outlook and Exchange. A key flaw in the original MAPI design is that it did not allow for multiple implementations to coexist.

Life would be very simple if the mapi32, emsmdb32 and emsabp32 shipped by Outlook and Exchange were the same implementation, built out of the same code tree. If that were the case, the only concern would be keeping up with the latest builds.

However, life is not this simple. Outlook and Exchange have different needs from MAPI, and, as such, the code for their implementations has diverged a good deal. Exchange needs high stability and scalability out of MAPI, so a good portion of the design and testing is focused on eliminating potential deadlocks and memory leaks. Outlook needs a strong user experience, so the focus is on features like Cancel RPC, RPC over HTTP, Cached Mode, and server reconnect. Outlook also needed to solve the coexistence problem to allow for other mail clients, so enter the MAPI Stub.

The stub library works by acting as a central dispatcher for MAPI calls, proxying the calls out to the various implementations of MAPI which may be on the box. There's a performance penalty to the stub library's proxying mechanism though. This performance penalty was not acceptable in a server environment, so Exchange declined to support their implementation of MAPI with the stub. The effect of all this is that Exchange's providers expect to be loaded by Exchange's MAPI, and Outlook's providers expect to be loaded by Outlook's MAPI. We can't guarantee this will be the case if both are installed on the same box.

I began this post promising to list some concrete examples of the problems this can cause. These are real issues I've encountered as an Escalation Engineer:

• Missing interfaces - Exchange has recently extended the IExchangeManageStore interface to allow the Admin to provide listings of over 65000 mailboxes and public folders. The new interfaces required for this are not in Outlook's Emsmdb32.
• Memory leaks and deadlocks - Outlook's MAPI trails far behind Exchange's MAPI in shipping fixes for issues which can show in high stress multithreaded MAPI applications. This is not suprising considering the level of testing Exchange's implementation gets, not only internally but by third parties. Quite often, we get calls from our partners saying that “the latest fix didn't work” - that their customer is still seeing a leak/deadlock/transient hang that should have been fixed by a recent hotfix. On investigation, we'll find that some administrator at the customer's site had installed Outlook on the application server “just for testing“, essentially forcing the application to run in an environment where it was never previously tested.
• Stub dependencies Lead to Crashes - Just found this one last week. The Outlook stub library works great with Simple MAPI, which has only 12 functions, each with a well defined DLL entry points. Proxying Extended MAPI calls is much more complex - more places to fail. In the scenario I debugged, Outlook's stub failed to locate the entry points for some functions in Exchange's Emsmdb, so the function pointers were left NULL, leading to a crash when we tried to invoke the functions. I believe the setup was Exchange 2000 with Outlook 2000.
• Simple MAPI - I just mentioned the stub library works great with Simple MAPI, but it can't work at all if Simple MAPI isn't there. As of Exchange 2003, Exchange no longer implements Simple MAPI. The API was removed to reduce the potential attack surface.
• Rpc_Binding_Order - Installing Outlook on an Exchange cluster will prevent mad.exe from logging on. The Exchange release notes only indicate Freebusy won't work, but that's just one effect of mad.exe failing. Any other MAPI based program could fail in this scenario as well.

Of course, most of these problems could be solved (any problem can be solved, right?), and the stub library is a good first step, but committing to support of this configuration would greatly expand the test matrices of both Exchange and Outlook, not to mention the increased test burden for third party server applications using MAPI. Unfortunately, we've not been able to justify the cost this would entail.

So, can you put Outlook and the Exchange Admin on the same box? If you're just talking about an administrator's desktop, one which can be rebooted on a whim or even rebuilt if needed, where downtime isn't a big issue, sure - it still won't be supported, but you might get away with it. If you're talking about your Exchange server or a server hosting an application integrated with Exchange, you do so at your own peril.

[Comments on this post have been closed]

Some of you may know that I wrote the MAPI utility/sample MFCMAPI. Someday I'll write some posts directly about it. This post, however, is about memory management in MAPI.

The latest build of MFCMAPI was just added to the internal dev tools collection for the next version of Office. This means a lot of the developers and testers are running MFCMAPI against their private, debug builds of Outlook. One of the developers e-mailed me last week to let me know that this build of MFCMAPI was causing debug assertions. They took a look at the stack and told me the function throwing the assertion was MAPIFreeBuffer. They also pointed out where I was making the call.

Horror of horrors! Had I somehow attempted to MAPIFreeBuffer memory that hadn't been allocated via MAPI? Or was I freeing the same memory twice? After all the preaching I do to my customers over the importance of good MAPI memory management had I committed one of the sins I caution against?

Well, yes and no. I had indeed made an error, but it wasn't one I had seen before.

MFCMAPI displays a lot of data in list boxes. A row in a list box has a single data pointer. I have a structure that contains pointers to various buffers to hold things like Entry IDs. To simplify cleanup, I allocate the main structure with MAPIAllocateBuffer, then I allocate buffers that hang off of this structure with MAPIAllocateMore. That way, a single call to MAPIFreeBuffer can free the structure and all of the extra buffers. This makes my code very clean.

Sometimes I need to wipe out one of these buffers and replace the contents. This is where I got into trouble. I didn't want to leak any memory, so I called MAPIFreeBuffer on the buffer, set the pointer to NULL, then allocated a new buffer with MAPIAllocateMore. That call to MAPIFreeBuffer is the one that caused the assertion. You cannot free memory allocated with MAPIAllocateMore by calling MAPIFreeBuffer on it. The only way to free that memory is to call MAPIFreeBuffer on the 'parent' memory which you indicated in the call to MAPIAllocateMore.

So, lesson learned. Here are some other common MAPI memory leaks I see in customer's code (heck, I think I've seen every one of them in our code at one point or another):

• Freeing LPSRowSet and LPAdrList pointers with MAPIFreeBuffer: Big mistake, big leak. These special array based structures are not built with MAPIAllocateMore. Each entry in the list is allocated with it's own call to MAPIAllocateBuffer. Special structures need special cleanup functions, FreeProws and FreePadrlist. These functions walk the arrays and call MAPIFreeBuffer on each entry before calling MAPIFreeBuffer on the array itself. (A neat side effect of this is you can 'rip' an entry out of an array and stuff it somewhere else without having to copy anything. Just NULL out the array entry and it's yours. Make sure you free it when you're done!)
• Calling MAPIFreeBuffer on MAPI objects: You wouldn't believe how many times I've seen this.
• Calling UlRelease on MAPI structures: Or this.
• Not calling Release when you're done with MAPI objects: Developers come up with all sorts of crazy reasons to justify this practice. My favorite was a developer who was deleting messages with IMAPIFolder::DeleteMessage. He had an IMessage object pointing at the message he was deleting and wouldn't call Release on it because 'the message had been deleted'. He was suprised to when he started getting MAPI_E_CALL_FAILED when calling OpenEntry after his app had been running a long time.

So how did I fix my bug? The answer was quite simple - don't call MAPIFreeBuffer here! The memory I had allocated will be freed when the parent memory is freed, regardless of whether or not I still have a pointer to it. Since this scenario was rare, I can afford the memory hit of having a few extra buffers allocated for the lifetime of the structure.

[Comments for this post have been closed]

Put this together from a posting I made to the MAPI-L list and a couple cases I worked recently:

The problem is with MAPI store providers that just don't work when loaded under Outlook 2003. The developers of the providers, when they contact us, are usually convinced the problem is a bug in Outlook 2003 since the providers “worked perfectly” under Outlook XP. My experience though, is that the problem is that Outlook 2003 demands so much more of the MAPI spec than previous versions of Outlook that bugs which have always been in these providers are now exposed. What's very interesting though is that I'm seeing the same bugs over and over in these providers. I hope to highlight the source of these bugs and offer some ideas on how to correct them.

There is a book which is very much coveted in the MAPI development community: Inside MAPI. One of the samples in this book is MSLMS, the Microsoft Sample Local Message Store. For what it is, sample code demonstrating how to write a message store provider, it's great. I'm not aware of any other sample message store out there. So it does not suprise me that a number of people choose to base their provider on the code in MSLMS. As such, they tend to inherit the same bugs and design flaws present in the sample.

I recently spent a couple days hacking on the MSLMS sample so that it could load under Outlook 2003. Here's a short list of the problems I found and what I had to do to fix them:

• Two heaps we're being allocated for the Provider object, then used in all objects - the heaps were destroyed when the Provider object was destroyed, meaning we'd crash if any object had a longer lifetime. Fixed this by introducing DLLMain and creating/destroying the heaps on module load and unload. Putting this code in DLLMain may not be safe, but it stopped the crash for now. In production code, I'd find a better solution.
• PR_RECORD_KEY abuse: This bug will stop a provider cold in Outlook 2003.In a number of tables, PR_RECORD_KEY for all objects was
  returning the store's record key instead of the key associated with the individual objects. Since Outlook 2003 depends on PR_RECORD_KEY to cache objects, this error effectively meant nothing in the provider could be rendered. The fix is to make sure PR_RECORD_KEY is unique for all objects and can be accessed through the object and contents tables and hierarchy tables.
• GetNamesFromIDs deviated far from the MAPI spec. Suppose we make this call asking for two IDs, one which does exist and one which doesn't. MSLMS would return an array with one result, leaving the caller with no way to determine which ID got matched to a name. The correct thing to do is return an array with two entries - one being the matched name and the other indicating an error.
• Multivalued string and multivalued binary props - The code which handled property persistance failed both of these cases. These properties can get quite complex, with a number of internal pointers. From what I could tell, it did attempt to address them, but it failed to fix up all of the internal pointers when flattening the data structures on save and unflattening them on load. In addition to failing to persist data, this bug also lead to a number of crashes due to the invalid pointers.
• Named properties were being excluded from contents tables: Don't know why they were being blocked, but this meant Outlook couldn't get to some data on items that it expected to be able to retrieve through the contents table.

Those are just the problems I fixed. Here are some I didn't:

• PR_RECORD_KEY (again): MSLMS assumes it can always use PR_ENTRY_ID when asked for PR_RECORD_KEY. This assumption breaks down on attachments, where objects do have PR_RECORD_KEY, but do not have PR_ENTRY_ID.
• MV_INSTANCE: This somewhat obscure flag is at the heart of Outlook's code for handling the Group By box. MSLMS doesn't support the flag. Another effect is that Contact Address books cannot be hosted in MSLMS because the contab32 address book provider depends on the underlying store provider having support for this flag.

Anyone who has based their message store provider on MSLMS (even if it was “What would MSLMS do?“ and not just borrowing the code) needs to review their code and ensure they've addressed all of the above bugs before they approve their provider for support under Outlook 2003.