So – there’s a potential hang in the current builds of MAPICDO, but you can avoid it, so it doesn’t have to be a huge problem.
The repro scenario for the hang is this: You’ve got an application which does thread level impersonation and also uses MAPI. Your application starts up multiple threads under different user contexts, each of which log in to MAPI. Eventually, as each of these threads completes its work, you start releasing objects. Your thread hangs while releasing a message store object. If you were to attach a debugger and knew the stacks to look for, this hang would look almost identical to the hang we fixed in MAPICDO 6.5.8244, discussed here.
What’s going on here is related to how MAPI tracks connection lists. Each user context gets its own connection list, stored in shared memory and indexed using what is essentially the CRC of the SID of the user who initiated the connection. When starting, there are no connection lists being tracked. The first MAPI thread to initiate a connection builds a connection list which is stored in this shared memory. This thread then spins up a polling thread using CreateThread. No special effort is made to adjust the thread token of this helper thread, so it ends up running with the thread token of the calling process. At some point, this thread needs to access the connection lists. As long as only one connection list exists, we assume it must be the one we need and use it – everything’s fine.
The problems happen when we have a second MAPI thread under a new user context. We create a second connection list and add this to the list. The polling thread spun up for this thread now has two connection lists to choose from. It concludes that neither is appropriate because the SIDs don’t match, and ends up shutting down without doing any work. Later, the first polling thread encounters the two connection lists and ends up shutting itself down without ever signaling that it has finished its work.
This is where the main MAPI threads are now in trouble, as they will wait for those polling threads to signal completion, which will never happen because those threads are no longer running.
Fortunately, most MAPI server applications don’t work this way. Most MAPI server applications run as a single service account which has appropriate permissions to log in to the various mailboxes it needs to. So either there is no impersonation at all, or all of the MAPI threads impersonate the same user. Either way, there is no opportunity to run in to this problem.
So – if you do happen to run in to this issue, you should look in to creating a single service account to handle all your MAPI work, then reduce/eliminate your thread level impersonation.
I’ve just been given clearance to publicize this since apparently it was discussed at MEC: The MAPICDO download *will* be updated to support Exchange Server 2013. This is important because:
So without this update to MAPICDO, the only MAPI implementation which would be able to connect to Exchange 2013 would be Outlook’s MAPI.
Publication dates, instructions for usage, etc. are all forthcoming.
One of the lesser known tricks in MAPI is that when you open a distribution list (MAPI_DISTLIST) from an address book using OpenEntry, if you don’t specify an interface what you get is not an IMailUser, but is in fact an IDistList, which inherits from IMAPIContainer. This gives you a means to access the members of the distribution list. Just call GetContentsTable and there they are. However, this trick doesn’t always work with MAPICDO when you’re talking to Exchange 2010. When you look at an Exchange 2010 address book using MFCMAPI, you’ll notice that all of the entry IDs are short term entry IDs. They don’t contain the DN of the entry being referred to. Instead, what they contain is an ephemeral ID, which is supposed to be a way to refer to an entry in the address book without constantly sending a DN back and forth. With the introduction of the Address Book Service in Exchange 2010, we split the concept of ephemeral IDs to include those generated by the Active Directory, and those generated by the Address Book service. The ID we find in these short term entry IDs is an Active Directory ID.
The upshot of all this is that when we use this short term entry ID to open an entry in the address book, we don’t get the full complement of properties we would normally expect (and that we would get if the short term Entry ID had used an Address Book ID). A side effect of this is that when we further try to call GetContentsTable, NspiGetMatches, the NSPI function which we’re calling under the covers, doesn’t understand the ID we pass to it and the call fails with the error MAPI_E_CALL_FAILED.
All is not lost, however. One of the properties we DO get using the short term entry ID is a proper long term entry ID with a full DN. And objects opened with this entry ID contain the full set of properties we normally expect to see. Further, we can now call GetContentsTable and get the members of the distribution list.
If you’re playing along in MFCMAPI, you would do this by:
The DL members will now appear in a new window. Back in the original window, the full set of properties for the DL will appear in the lower pane. In code, you would do this by opening the item with the short term entry ID, calling GetProps to get the long term entry ID, then calling OpenEntry again to get the full item.
This trick works for any of the PT_OBJECT type properties you expect to find on any entry in the address book, MAPI_MAILUSER included.
The August 2012 Release (build 184.108.40.2065) is live: http://mfcmapi.codeplex.com.
In honor of the Outlook 2013 Preview, this month I’ve decided to turn MFCMAPI blue. Along the way, I was able to fix a number of nagging flicker issues, as well as fix the system button placement on XP systems. However, the changes I made for Outlook 2013 weren’t just cosmetic. As you may have noticed in the updated MAPI Reference, it’s now possible to have more than one version of Outlook installed on a machine at the same time. This poses a problem for MFCMAPI: which MAPI to load? To solve this problem, MFCMAPI locates all the implementations of MAPI it can and puts them on a menu. You’ll find this under Session/MAPI Initialization/Load MAPI. Of course, if you’re happy with the default MAPI, you don’t to do anything different. If you’re interested in seeing how MFCMAPI looks for MAPI implementations, consult stubutils.cpp and the MAPIPathIterator class.
[Edit] How could I change the UI (again) without posting a screenshot?
Here's a change list - see the Issue Tracker on Codeplex for more details, or look at the code:
As you know, when you install a new provider on a system, you have to update MAPISVC.INF to point it to the new provider. There are a few standard properties set during this configuration which tell MAPI where to find your DLL. One is PR_SERVICE_DLL_NAME, set in the Message Service section, and the other is PR_PROVIDER_DLL_NAME, set in the Service Provider section. For both properties, you are expected to set the name of your provider’s DLL (minus the “32” suffix"). MAPI will then load your provider by looking for it on the path.
What if the path isn’t good enough? What if, like any other modern application, you want to drop your provider over in Program Files and not dirty the path? According to the MAPI documentation, you shouldn’t be able to do that. However, it turns out that, with a few restrictions, Outlook’s MAPI can deal with full paths to MAPI providers. Outlook development has agreed to support this for Outlook 2010 and higher.
Here are the particulars:
To demonstrate this, I’ve updated the Wrapped PST sample over on Codeplex. The magic happens in MergeWithMAPISVC and GenerateProviderPath.
A customer just raised this issue with our Wrapped PST provider sample. They were trying to retrofit the sample to use Unicode paths to the NST and found that the CreateStoreEntryID routine was producing an invalid entry ID after they changed to to use a Unicode path. Through experimentation, they were able to produce a valid entry ID and wondered if I could document the necessary format. It turns out we use two different formats for the entry ID of a wrapped PST. One format is for ASCII paths and the other is for Unicode paths. Here they are, represented as structures:
typedef struct // short format
BYTE rgbFlags; // MAPI-defined flags
MAPIUID uid; // PST provider MUID
BYTE bReserved; // Reserved (must be zero)
CHAR szPath; // Full path to store (ASCII)
// Unicode version of EIDMSW is an extension of EIDMS
// and szPath is always NULL
typedef struct // Long format to support Unicode path and name
CHAR szPath; // ASCII path to store is always NULL
WCHAR wzPath; // Full path to store (Unicode)
The net effect of the difference in these structures is there are two NULL bytes prior to a Unicode path. If you’re interpreting the entry ID, one way to determine how to interpret it would be to cast it as EIDMS first, then check if szPath is NULL. If it is, you need to cast it as EIDMSW instead.
We recently resolved a case here where a customer, as part of their larger application, was using SetReceiveFolder to direct messages of their message class to the correct folder so their application could do further processing on them. However, when they tested their application in cached mode, it didn’t work. The SetReceiveFolder call would return S_OK, but the mail would continue to be delivered to the Inbox.
After some investigation, we found that Outlook does not appear to honor SetReceiveFolder settings made on the OST when it synchronizes mail from Exchange. Fortunately, there’s an easy workaround: make the setting on the server instead! Even though this customer was targeting Office 365, which only allows cached mode profiles, it’s still possible to access the server directly to make the call. The trick is to use MAPI_NO_CACHE, as demonstrated here. Once they opened the folder on the server, they were able to make the SetReceiveFolder call and the server would then honor it during message delivery.
Incidentally, we were able to test all of this using MFCMAPI, which has under Tools/Options the switches needed to bypass the cache, and which has the ability to both set receive folders and view the receive folder table. I knew writing that code would pay off!
The June 2012 Release (build 220.127.116.114) is live: http://mfcmapi.codeplex.com.
This month we ship some pretty cool additions to MrMAPI: the –store flag gets a turbo boost, allowing you to open a store by Entry ID, which helps some of the automated diagnostics out there that use MrMAPI for data gathering. Also new is the ability to look up a specific property for a store, also by request of the diagnostics authors. For the troubleshooters out there who use MrMAPI for property lookup, check out the –flag switch, which allows you to look up flags on the fly.
Finally – Surface!
On this page, talking about the Upload Folder State, we have a note which references the constant UPS_ONE_FOLDER. This constant is not defined in our list of MAPI Constants or in a header file. This is an oversight. Here’s the definition for this constant:
#define UPS_ONE_FOLDER 0x00000004
That is all.
The April 2012 Release (build 18.104.22.1683) is live: http://mfcmapi.codeplex.com.
I ship a new build of MFCMAPI every two months. That’s a personal goal I keep to help motivate me. Somehow I got it in my head that I was due to release a new build of MFCMAPI here in April. So I worked extra hard to track down some bugs and implement some new features to make the new build worth it. Only when that was done did I realize I shipped a new build in March – one month ago. So here you go – bonus build!
When you start this new build, you’ll see a new menu, which is a feature I’m quite proud of: QuickStart. This is a collection of actions that will get you to common destinations in MFCMAPI with a minimum of fuss. For instance, you can now in one action open the Inbox, Calendar, or Contacts folders. Other QuickStarts I implemented in this first batch are one to display the contents of your nickname cache (this one can be slow) or display your mailbox quotas (several properties which are a pain to track down since they’re not visible by default). Over time I plan to add more QuickStarts to the list, so send me any idea you have and I’ll consider them.
If you’ve ever tried to use MFCMAPI to troubleshoot Exchange server mailbox quotas, you’re probably familiar with the properties PR_STORAGE_QUOTA_LIMIT, PR_PROHIBIT_SEND_QUOTA, and PR_PROHIBIT_RECEIVE_QUOTA. These properties live on the message store object and tell you what your various quotas are. The problem with these properties is that they only work with an online profile. If you’re working with a cached mode profile, these properties will not work. Prior to Outlook 2010, the only workaround was to either build an online profile or bypass the cache. As of Outlook 2010, there’s another option. The OST in Outlook 2010 exposes three properties which contain the same quota information as the online only properties. Development just gave me permission to document them:
#define PR_QUOTA_WARNING PROP_TAG( PT_LONG, 0x341A)
#define PR_QUOTA_SEND PROP_TAG( PT_LONG, 0x341B)
#define PR_QUOTA_RECEIVE PROP_TAG( PT_LONG, 0x341C)
#define PR_QUOTA_WARNING PROP_TAG( PT_LONG, 0x341A)
#define PR_QUOTA_SEND PROP_TAG( PT_LONG, 0x341B)
#define PR_QUOTA_RECEIVE PROP_TAG( PT_LONG, 0x341C)
These properties map exactly to the corresponding online properties and should contain the same values, in kilobytes. So PR_QUOTA_WARNING maps to PR_STORAGE_QUOTA_LIMIT, PR_QUOTA_SEND maps to PR_PROHIBIT_SEND_QUOTA, and PR_QUOTA_RECEIVE maps to PR_PROHIBIT_RECEIVE_QUOTA.
BTW, for completeness, if you’re looking at any of the above properties you may also be interested in PR_MAX_SUBMIT_MESSAGE_SIZE, which is the largest a message can be for submission, in kilobytes, and PR_MESSAGE_SIZE_EXTENDED, which is the size of the mailbox in bytes. Both of these properties function in cached as well as online profiles.
A while back I mentioned our MAPI Stub Library. This is a couple of .cpp files you can drop in to your project instead of linking to mapi32.lib, Alternatively, you can build your own version of mapi32.lib. You can read more about the whys and wherefores on the codeplex site or in the MSDN article on MAPI linking.
We just updated the logic used to find the correct MAPI binary to both make it more correct and to simplify it. According to the dev who did the work, we were able to eliminate two thirds of the code from StubUtils.cpp. The code is now more future proof, so if you were using the original version, you should look at incorporating the update. MFCMAPI will use this new logic in its next build.
From time to time, folks ask me what they need to do in their message store provider to support Outlook 2010’s conversation view. I tell them they need to support restrictions on message store tables as discussed in the MAPI documentation. If Outlook hands them a restriction, they can’t return MAPI_E_TOO_COMPLEX and expect the conversation view to work. I also tell them their table needs to support categorization with multivalued columns and MV_INSTANCE. Because they support categorization, they also need to return STORE_CATEGORIZE_OK in PR_STORE_SUPPORT_MASK. Basically, when you turn on a conversation view, Outlook is going to exercise more of the MAPI spec than it has before. If your provider can’t handle it, then the view won’t work.
This is all true, but as usual, there’s more to the story. The more comes about when you decide your store isn’t going to be able to handle conversations. We usually find that categorization is the feature most people have trouble getting to work. So they decide “Ok – we just won’t support conversations” and they don’t return STORE_CATEGORIZE_OK. Their store works fine in Outlook, until a user tries to configure a conversation view. Then the user gets the error “Cannot display information. There is a problem connecting to the server or an error occurred. Try starting Microsoft Outlook again.” They continue to get this error until they change to a different view.
What’s going on? When we implemented the conversation view, we took a dependency on the categorization feature working in order to get the view working quick enough to be acceptable to the user. We also implemented a fallback path in case categories weren’t implemented, but since all of our built in providers did implement categories, this path did not get well tested. In fact, there’s a logic bug in it that causes the error the user sees. We investigated this a while back and even attempted a fix. However, when we did this, we realized why having categories was so important. Without categories, the performance of the conversation view is, to be blunt, horrible, Categories, it turns out, were what allowed us to implement the conversation threads without incurring a high level of post processing on the table.
We decided we couldn’t fix that code path (or we technically, we could, but no one would want the fix), and we looked at alternatives for delivering a better experience. The obvious choice here is to disable the conversation view UI when the store doesn’t support categories. However, this presents a challenge – we didn’t consider in our original design how to handle multiple stores where some stores (like the PST) support the view but others don’t. So while we agreed this would be a good design change, it was deemed too expensive to consider in a hotfix or service pack.
One customer found that, through heavy customization of the ribbon, they could hide the UI themselves when the user was looking at one of their folders, but leave the UI enabled when someone else’s folder was selected. However, it was still possible to set a conversation view to one of their folders. This could happen because of a prompt we display when the user checks “Show as Conversations”:
If the user picks “All folders” here, the conversation view gets applied to the store that couldn’t support it. Fortunately, there’s a way to suppress this dialog. We have a policy, UpgradeToConversations, which controls conversion to/from the conversation view. 0 converts conversation views to date sorts, 1 converts them the other way, and 2 says leave them all alone. If this policy is set to any value, we don’t display the prompt. For the customer that customized the ribbon, setting the policy to 2 allowed their users to have conversation views in the PST and Exchange stores, but not get them set in their store.
The moral of this story is if you have a custom store provider, you really need to support categories if you want your users to have a good experience.
We did it! MFCMAPI is now on Wikipedia!
The March 2012 Release (build 22.214.171.1242) is live: http://mfcmapi.codeplex.com.
No UI changes this round, so no new screenshot. I did fix some interesting bugs and added a few nice features. Probably the nicest feature is new MAPI call logging. I‘ve always had error call logging in MFCMAPI via the Debug Viewer (Ctrl+D), but I never had a way to output exactly which MAPI calls were being made, whether they failed or not. Now you can get that – just turn on tag DBGMAPIFunctions (0x40000000) in the Debug Viewer and you’ll see every MAPI call that gets made. If you want more perspective on an error, I’d also add DBGGeneric (0x1) and DBGHRes (0x40). You can use 0x40000021 to turn on all of these at the same time.
Oh – and perhaps the biggest change of all: All the binaries are now code signed!
We’ve added a couple new C++ only events to Outlook 2010 which I have been asked to document. These events were added per customer request to help fill some gaps in the transition from ECE to Com Add-In. One of the events is available as of the October 2011 Cumulative Update, and the other just shipped in the February 2012 Cumulative Update.
The two new events are:
If you’re familiar with writing Exchange Client Extensions and Com Add-Ins in C++, this is likely sufficient information to handle these events. If not, you might want to review some samples which demonstrate handling events from the IDispatch::Invoke function. For example, I found this add-in on codeproject which handles Application events. In that code, the CAppEventListener::Invoke function is where you would handle the BeforePrint event.