November, 2006

I thought it'd be interesting to document the registry values that MAPI Editor uses. Most of them are exposed in the UI through Options/Other.

All of these registry keys are stored under the registry key HKEY_CURRENT_USER\Software\Microsoft\MFCMAPI.

If a registry value is not found, the default is assumed.

True/false values are stored with the usual convention of 0 being false, and any other value being true.

Option: "Debug Output Tags (0 = None, ffff = Verbose. See Help.)"
Reg Value: DebugTag (REG_DWORD)
Default: 0
Meaning: Sets which debug tags are enabled. The list of tags is in the Help (F1) dialog. Note that setting the high order tags (DBGWindowProc and DBGMenu) is liable to slow the process to a crawl.

Option: "Enable Debug Output to File"
Reg Value: DebugToFile (REG_DWORD)
Default: 0
Meaning: Whether debug output is written to a text file. If debug tags are enabled, but file logging is not, debug output is still emitted with OutputDebugString and can be monitored using a debug monitor such as dbgview.

Option: "Debug Output File:"
Reg Value: DebugFileName (REG_SZ)
Default: c:\mfcmapi.log
Meaning: File for debug output. This file is always opened in append mode. If this file cannot be opened, there will be no error dialog, but there should be an error output emitted through OutputDebugString.

Option: "Parse Named Properties"
Reg Value: ParseNamedProps (REG_DWORD)
Default: true
Meaning: If enabled, use GetNamesFromIDs to determine property names.

Option: "Parse Named Properties for All Props (including those below 0x8000)"
Reg Value: GetPropNamesOnAllProps (REG_DWORD)
Default: false
Meaning: Some providers return useful data when GetNamesFromIDs is passed a property ID outside the named property range. When this option is enabled, GetNamesFromIDs is called for all property IDs. Note that some providers will complain heavily.

Option: "Throttle Level: If nonzero, limits the number of rows displayed in tables:"
Reg Value: ThrottleLevel (REG_DWORD)
Default: 0
Meaning: A long time ago, MFCMAPI loaded tables on the main UI thread, so this option was necessary. Now that tables are loaded on a background thread, and table loads can be halted early by hitting Esc, this option isn't as useful. However, there may be times during testing when it is nice to only load the first few rows without having to hit Esc, so this option was left in.

Option: "Register Notifications on Hierarchy Tables During Load"
Reg Value: HierNotifs (REG_DWORD)
Default: true
Meaning: If enabled, when a folder is added to the tree view, load its hierarchy table and register for notifications. This allows for dynamic updates of the tree view as folders are created, deleted, move, or renamed. Folders which are being monitored in this manner show as bold in the tree view.

Option: "Register Notifications on Hierarchy Tables During Node Expansion or Selection"
Reg Value: HierExpandNotifs (REG_DWORD)
Default: true
Meaning: If enabled, when a node is selected or expanded in the tree that was not already registered for notifications, then register for notifications. If the folder isn't already bold, it will turn bold if the registration is successful.

Option: "Allow Notifications on Root Container (may cause infinite loop in some Address Books)"
Reg Value: HierRootNotifs (REG_DWORD)
Default: false
Meaning: Some providers really do not like when MAPI clients register for notifications on the root container. The Exchange Address book is a notable example. This option is disabled by default to avoid problems with such providers.

Option: "Hierarchy Node Load Count: If nonzero, limits the number of child nodes for which the hierarchy table is automatically loaded:"
Reg Value: HierNodeLoadCount (REG_DWORD)
Default: 20
Meaning: Registering for notifications on hierarchy tables is expensive. So when expanding a folder, this option determines how many children of the folder get registered for notification. This option exists primarily for Public Folders where a single folder is likely to have thousands of child folders.

Option: "Perform GetProps call on objects when selecting or displaying. Extra properties will still display if unchecked."
Reg Value: DoGetProps (REG_DWORD)
Default: true
Meaning: When enabled, whenever an row is selected in the upper pane, call GetProps to get a default list of properties for the item associated with the row. Regardless of how this is set, if Extra properties have been set, a GetProps call with the list of extra properties will still be performed.

Option: "Use GetPropList to get property list for GetProps call (otherwise use NULL tag list)"
Reg Value: UseGetPropList (REG_DWORD)
Default: true
Meaning: There are two ways to get a default list of properties. One, the default, is to call GetPropList to get a default list of property tags, then pass this list in to GetProps. The other way, which will happen if this option is disabled, is to call GetProps with NULL for the property tag list. Some providers have bugs which can be easily exposed by calling GetProps with a NULL property tag list.

Option: "Allow duplicate columns of different types."
Reg Value: AllowDupeColumns (REG_DWORD)
Default: false
Meaning: This is used in two places. The first is when SetColumns is called. Some providers allow for multiple columns with the same property tag, but different property types, such as one of PT_STRING8 and another of PT_UNICODE. Other providers treat this as an error. With this option enabled, two columns of the same ID but different types will be treated as different. With this option disable (default), the two columns will be merged, with the first type winning, before SetColumns is called. This option is also used in the Extra properties code for the bottom/right pane, using the same logic in constructing the list of property tags for GetProps.

Option: "Default to using row data for property list instead of calling GetProps."
Reg Value: UseRowDataForSinglePropList (REG_DWORD)
Default: false
Meaning: Normally, when a row is selected in the upper pane or a node is selected in the tree control, if an item which inherits from IMAPIProp can be opened, then result of GetProps calls will be used to fill the property pane. If an item cannot be opened (either due to an error or because the row is not associated with an item, like in the Rules table), then the row used for the upper pane or tree node is displayed. The status pane will indicate which data is being displayed. When this option is enabled, GetProps is never attempted.

Option: "Use nice names for certain column headers and pull them to the front of the view"
Reg Value: DoColumnNames (REG_DWORD)
Default: true
Meaning: For many types of tables, a default "interesting" set of properties has been selected. When these tables are displayed, these properties will be used to call SetColumns on the table. When this option is enabled, nice names are used in the column headers instead of the property tag names. Hovering over column headers will still display the property tag information in a tooltip bubble.

Option: "When loading tables, display SetColumns edit dialog"
Reg Value: EditColumnsOnLoad (REG_DWORD)
Default: false
Meaning: When enabled, after the set of columns to be used for a table is determined, display a dialog allowing the user to modify the set.

Option: "Use MDB_ONLINE when calling OpenMsgStore"
Reg Value: ForceMDBOnline (REG_DWORD)
Default: false
Meaning: See http://support.microsoft.com/kb/834496. If a cached mode profile is suddenly very slow in MAPI Editor, check that this option hasn't been set. Due to the way the underlying providers work, it may be necessary to restart MAPI Editor if the profile has already been opened before this option is toggled.

Option: "Use MAPI_NO_CACHE when calling OpenEntry"
Reg Value: ForceMapiNoCache (REG_DWORD)
Default: false
Meaning: See http://support.microsoft.com/kb/834496. If a cached mode profile is suddenly very slow in MAPI Editor, check that this option hasn't been set. Due to the way the underlying providers work, it may be necessary to restart MAPI Editor if the profile has already been opened before this option is toggled.

Option: "Allow caching of IPersistMessage objects when opening Outlook forms in compose mode, which may prevent shutdown of certain versions of Outlook"
Reg Value: AllowPersistCache (REG_DWORD)
Default: false
Meaning: Unless you're very interested in how form viewers work, don't mess with this option.

Option: "Pass IMAPIProgress implementation to functions that take it"
Reg Value: UseIMAPIProgress (REG_DWORD)
Default: false
Meaning: Many MAPI functions take an optional IMAPIProgress parameter. With this option enabled, MAPI Editor's implementation of this interface is passed to these functions. Output from this interface is displayed on the status pane. Not enabled by default because many providers' support for IMAPIProgress is minimal and buggy.

Option: "Use IID_IMessageRaw when opening messages"
Reg Value: UseMessageRaw (REG_DWORD)
Default: true
Meaning: See http://support.microsoft.com/kb/908074. When enabled, IID_IMessageRaw is used for opening messages. Most providers do not support this interface.

Option: Not list in Other/Options. Instead, this is a check box on the Help (F1) dialog.
Reg Value: DisplayAboutDialog (REG_DWORD)
Default: true
Meaning: When enabled, displays the Help (F1) dialog when MAPI Editor is started. Since this option can be toggled directly from the dialog, it is not listed in Options/Other.

A fellow engineer recently asked me how to set custom properties on a folder object in a PST. I reminded him that folders in the PST provider don't support named properties. He was surprised to hear this. I was surprised to learn that this hadn't actually been documented before. (If anyone can produce a link to previous documentation on this I'd be grateful!)

Neither the Exchange provider (emsmdb32.dll) or the PST provider (mspst32.dll) support named properties on folders. Both will allow you to call GetIDsFromNames and GetNamesFromIDs, but neither will allow you to set a property in the 0x8000 to 0xFFFE range. This is contrary to the MAPI documentation, which suggests that providers which support named properties would support them for both message and folder objects.

If you attempt to set a named property on a folder in a PST or cached mode (which is built on top of the PST provider), then you will get MAPI_E_NO_ACCESS in the SPropProblemArray structure returned by SetProps.

If you attempt to set a named property on an Exchange folder, you will get MAPI_E_NO_SUPPORT in the problem array.

Note that the documentation for Named Properties in the MSDN indicates that providers should return MAPI_E_UNEXPECTED_ID if they do not support named properties, so these error codes would appear to be, well, in error. However, I do not believe this is the case since the providers certainly do support named properties. They just don't support them on certain objects, a case the documentation doesn't allow for.

So - there's a memory leak in Outlook's implementation of MAPI. That shouldn't be too much of a shock. Just about any sizable application is bound to have a leak or two. What's interesting about this leak is that it is intentional.

A little history: We first shipped Simple MAPI back in MS Mail, circa 1992. Later on, when Extended MAPI was being developed for Windows 95, we chose to reimplement Simple MAPI as a wrapper around Extended MAPI. This reduced bug counts, simplified testing, etc. Everyone was happy. (Or so I've been told - I was still in school at the time!)

As we neared RTM however, our testers started finding Simple MAPI applications which had worked fine with MS Mail but now crashed. Investigation showed that these applications were accessing memory which had been freed. Why?

Let's digress a bit and look at a typical Simple MAPI function, MAPIResolveName:

ULONG FAR PASCAL MAPIResolveName(
	LHANDLE lhSession,
	ULONG ulUIParam,
	LPTSTR lpszName,
	FLAGS flFlags,
	ULONG ulReserved,
	lpMapiRecipDesc FAR * lppRecip )

There are two parameters of interest here: lhSession and lppRecip. lhSession can be used to pass in an existing session that was created with MAPILogon. Or it can be left NULL to let Simple MAPI create and use a temporary session for the duration of the call. lppRecip is used to pass back a structure - the resolved name. This structure is to be freed with MAPIFreeBuffer.

How would we implement this function in Extended MAPI? It would look something like this (pseudo-code):

if (!lhSession) 
{
	lhSession = InitializeMAPIAndLogonToProfile();
	bTempSession = true;
}
DoWorkToResolveNames(lhSession,lppRecip);
if (bTempSession) LogoffProfileAndUninitializeMAPI();
return;

So far so good. And as long as all the test harnesses for Simple MAPI pass a session in whenever they call MAPIResolveName, there's no problem. But what happens when we call MAPIResolveName without a session?

It might help here to comment that MAPI handles it's own memory allocations. That's why we have a special function, MAPIFreeBuffer, for freeing memory. When MAPI is initialized, it creates a heap to handle all of it's allocations. Similarly, when MAPI is uninitialized, it destroys this heap.

I'm sure the A students are bored with this lecture so I'll wrap it up now: In the original implementation, as soon as we clean up the temporary session and uninitialize MAPI, the heap from which lppRecip was allocated is destroyed. So any attempt to use it will result in a crash.

To fix this, we were faced with two choices. Either completely rethink how memory is handled in Extended MAPI, or leak the heap whenever we uninitialize MAPI. We chose the leak. The first version of Extended MAPI we shipped back in Windows 95 contained this leak.

Jump to today: That original version of Extended MAPI/Simple MAPI eventually forked to become the basis of both Outlook and Exchange's implementations of MAPI. While both implementations have had their changes over the years, the intentional leak has remained. And as long as Simple MAPI is implemented on top of Extended MAPI, we can't take the leak out. It will even be there in Outlook 2007.

Mitigation

If you're using Outlook's MAPI, or any version of Exchange's MAPI prior to Exchange 2003, the best you can do is minimize the number of times you uninitialize MAPI. If you're using Simple MAPI, this means do an explicit MAPILogon/MAPILogoff, and don't do them in a loop. If you're using Extended MAPI, initialize MAPI on your main thread before you do any MAPI work, and wait to uninitialize it until just before your process exits. Subsequent MAPIInitialize/MAPIUninitialize calls on other threads will act like Addref/Release on the MAPI subsystem so they won't incur the leak.

If you know you'll be using MAPI from Exchange 2003 or from the MAPI download, then you can benefit from the fact that we cut Simple MAPI there. This means we were finally able to put back the code to clean up the heap. It took us a couple times to get it right, but the upshot is that once the hotfix from KB 901014 is applied, you can set the CleanUpMAPIHeap key and no longer worry about this leak.

Incidentally, the reason we wrapped this fix in a registry key is because before we took this fix, any program which got memory from MAPI, uninitialized MAPI, and then used the memory would not appear to have a problem. But like Simple MAPI, as soon as we start cleaning up the heap, this pattern will cause crashes. So do test carefully before deploying this key!