The August 2014 Release (build 15.0.0.1042) is live: http://mfcmapi.codeplex.com.
It’s been a chaotic half year and I haven’t been able to get my builds out as often as I’d like, though I did do a ton of work on MFCMAPI.
This month we get a totally rebuilt MAPI property parsing engine. As it’s an engine change, you shouldn’t notice much different in MFCMAPI itself, but all of my XML exports get an upgrade.
Here's a change list - see the Issue Tracker and ChangeSets on Codeplex for more details:
Enjoy.
For those of you who know what to do with this, here’s the manifest for Message Header Analyzer:
https://mha.azurewebsites.net/manifest.xml
For the rest, I’ll put some instructions together later.
Enjoy!
I thought I’d share a script I’ve been using on some recent cases to generate memory dump files on multiple machines for intermittent crashes. The scenario this script targets is that you have a process, say Outlook.exe, which is crashing for a number of users, but you haven’t yet narrowed down a repro scenario. You’d like to get memory dumps when the process crashes, and you’d also like to get extra debugging information which may help you hone in on the issue. You'd also like to set all of this up so the user doesn't have to do anything themselves. They just run the process and data is generated automatically for you to collect later.
This script, set through Image File Execution Options, attaches Procdump.exe to your process every time it starts, so it can listen for exceptions and write dump files when interesting ones occur. It also captures Procdump’s debug spew in a log file which you can use to identify first chance exceptions that you'd also like to get dumps for. Once enabled, you'll just need to visit the user's machine periodically to collect the dumps and logs for analysis.
To use this script, create a file called DoProcDump.cmd and place it in c:\tools alongside procdump.exe (you can configure these paths differently if you like, but you’ll need to modify the script) and then create an entry under Image File Execution Options. Next time the user starts the process, Procdump will attach and listen for exceptions. There are a number of ways you can change up the data collection, such as enabling first chance exceptions, or getting dumps on process shutdown (useful for silent exits, though this is even better). Look at the comments in the script to see how to enable and configure these.
To disable this script, just remove the reg key you created under Image File Execution Options.
BTW, I got the unique file name code from here: http://blogs.msdn.com/b/myocom/archive/2005/06/03/creating-unique-filenames-in-a-batch-file.aspx
@ECHO off REM *****DoProcDump.cmd***** REM Configure this in Image File Execution Options like so: REM [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\Outlook.exe] REM "Debugger"="cmd /c c:\\tools\\DoProcdump.cmd" REM (Double \\ are how \ are represented in .reg files. If setting this value manually, use single \.) REM Change these paths to reflect the location of Procdump and the location to which the output files should be written. REM Data collection should include .dmp files and all procdumpOutput*.txt files written to the out path. set PDOutPath="C:\tools" set PDExe="C:\tools\Procdump.exe" REM Only enable one of the following. REM Use this filter to catch only second chance exceptions. Set PDFilter=/e REM Enable this filter to catch specific first chance exceptions in addition to second chance exceptions. REM Set PDFilter=/e 1 /f "System.Threading.ThreadAbortException" /f "System.AccessViolationException" REM Configure the maximum number of dumps to gather here. Set PDNumDumps=5 REM Enable this also gather dumps on process shutdown. REM Set PDTerm=/t set PDCfg=/ma /n %PDNumDumps% %PDTerm% %PDFilter% /x @ECHO ************************************************************* @ECHO * * @ECHO * DO NOT CLOSE THIS WINDOW! * @ECHO * * @ECHO * This window will close automatically when Outlook exits. * @ECHO * * @ECHO ************************************************************* @ECHO. @ECHO Running: %PDExe% %PDCfg% %PDOutPath% %* REM This generates a good file name for our log file. for /f "delims=/ tokens=1-3" %%a in ("%DATE:~4%") do ( for /f "delims=:. tokens=1-4" %%m in ("%TIME: =0%") do ( set LOGFILE=%PDOutPATH%\procdumpOutput-%username%-%%c-%%b-%%a-%%m%%n%%o%%p.txt ) ) @ECHO %* > %LOGFILE% @ECHO Current time %TIME: =0% >> %LOGFILE% @ECHO Running: %PDExe% %PDCfg% %PDOutPath% %* >> %LOGFILE% %PDExe% %PDCfg% %PDOutPath% %* >> %LOGFILE% 2>&1
In this article I’ll go over the current state of Outlook 2013 Click To Run (C2R)and support for various programmatic interfaces. Please comment on any areas I’ve missed and I’ll update the article accordingly.
With Outlook 2013, we enabled something we had never supported before, side by side (SxS) installation with other versions of Outlook. Specifically, we support installing Outlook 2013 side by side with Outlook 2010 or Outlook 2007. We do this by installing Outlook 2013 using Click-to-Run. For most MAPI applications and Outlook Add-Ins, C2R does not pose a problem. However, there is a small set of COM interfaces for which C2R creates an issue. We’ll go over them one by one.
This interface gave us the most trouble, but we’ve got it mostly fixed now. Let’s look at how the Outlook Account Manager, or CLSID_OlkAccountManager, caused a problem in the first place. This will also help us understand the remaining interfaces.
On a regular 64 bit installation of Windows with Outlook 2010 32 bit installed, we’ll find this interface registered here:
HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Classes\CLSID\{ED475410-B0D6-11D2-8C3B-00104B2A6676}
The registration will tell COM where to find OUTLMIME.DLL, which actually implements the interface. On the same system, with Outlook 2013 C2R installed, we find this key instead:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office\15.0\ClickToRun\REGISTRY\MACHINE\Software\Classes\Wow6432Node\CLSID\{ED475410-B0D6-11D2-8C3B-00104B2A6676}
COM won’t ever know to look here. But when Outlook needs this interface it finds it. How? That’s part of the Click To Run application bubble. Applications outside the bubble see the normal registry. Applications inside the bubble see the normal registry, plus any registry keys mapped through the ClickToRun key. So when Outlook, which runs inside the bubble, looks for the class {4E3A7680-B77A-11D0-9DA5-00C04FD65685}, it has no trouble finding it. Other applications won’t see it though.
Outlook and Office register every class they expose inside the bubble. A few classes are also registered outside the bubble so that other applications can use them. So why don’t we just register CLSID_OlkAccountManager outside the bubble? We actually tried that. We took a fix to register this interface outside the bubble and found that Outlook 2010 clients started crashing. When Outlook 2010 tried to locate it’s own implementation of the Account Manager, it got Outlook 2013’s implementation, which promptly crashed. We had to pull this fix and be a little smarter about it. Now, C2R will only register this key outside the bubble if it detects that a previous version of Outlook isn’t already installed. This is the only interface we’ve done this for.
The MAPI<->MIME conversion interface, IConverterSession and the support interface it depends on, IMimeMessage, are also only registered inside the bubble. Several customers have chosen to fix this one by hand by copying the registrations from inside the bubble to outside. If the bitness of Outlook matches the bitness of the OS, you could do this as follows:
rem For when the Office bitness matches Windows bitness: rem IConverterSession reg copy HKLM\SOFTWARE\Microsoft\Office\15.0\ClickToRun\REGISTRY\MACHINE\Software\Classes\CLSID\{4E3A7680-B77A-11D0-9DA5-00C04FD65685} HKLM\SOFTWARE\Classes\CLSID\{4E3A7680-B77A-11D0-9DA5-00C04FD65685} /s /f rem IMimeMessage reg copy HKLM\SOFTWARE\Microsoft\Office\15.0\ClickToRun\REGISTRY\MACHINE\Software\Classes\CLSID\{9EADBD1A-447B-4240-A9DD-73FE7C53A981} HKLM\SOFTWARE\Classes\CLSID\{9EADBD1A-447B-4240-A9DD-73FE7C53A981} /s /f
If the bitness differs:
rem For Office 32-bit on Windows 64-bit: rem IConverterSession reg copy HKLM\SOFTWARE\Microsoft\Office\15.0\ClickToRun\REGISTRY\MACHINE\Software\Classes\Wow6432Node\CLSID\{4E3A7680-B77A-11D0-9DA5-00C04FD65685} HKLM\SOFTWARE\Classes\Wow6432Node\CLSID\{4E3A7680-B77A-11D0-9DA5-00C04FD65685} /s /f rem IMimeMessage reg copy HKLM\SOFTWARE\Microsoft\Office\15.0\ClickToRun\REGISTRY\MACHINE\Software\Classes\Wow6432Node\CLSID\{9EADBD1A-447B-4240-A9DD-73FE7C53A981} HKLM\SOFTWARE\Classes\Wow6432Node\CLSID\{9EADBD1A-447B-4240-A9DD-73FE7C53A981} /s /f
Only do this if Outlook 2013 is not installed side by side with another version of Outlook, or you risk breaking the older version of Outlook!
Also, note that this is why many of you have written to complain to me that MFCMAPI can no longer work with EML files with Outlook 2013 – you have Outlook 2013 C2R and it can’t work unless you patch up the registry.
Finally, there’s the little known/understood area of MAPI marshalling. This is an area we almost never see…except that Redemption depends on it. In this case, the problem isn’t that COM objects themselves aren’t registered (MAPI isn’t COM after all). The problem is that the proxy stub maps are hidden in the bubble. Dmitry and I worked together to come up with a .reg file to fix this one: https://groups.yahoo.com/neo/groups/Outlook-Redemption/conversations/topics/9600
The same warning about not doing this in a side by side scenario applies. Dmitry points out that we intend to fix this in the C2R installer. We did intend to fix it at the time Dmitry and I spoke, but after the problems we encountered with the Account Manager it became harder to justify these sorts of changes.
It’s time for what is most likely the last release of the MAPICDO download. Here are the release notes:
Installation details:
Fixes:
The February 2014 Release (build 15.0.0.1041) is live: http://mfcmapi.codeplex.com.
The past month’s experiment of checking in as I go seemed to work well. Since the checkins are now self annotated I’ve decided to stop post creating items in the issue tracker for work I’ve already done. Do please continue to file items for work I need to do.
This month we get the ability to export profile information using MrMAPI. The folks working on our diagnostics tools are salivating over that. :)
Here’s a neat trick:
Since our restriction is “word starts with”, our restriction won’t match and we won’t get results, as expected. Now – turn off indexing for this PST and try it again. Now you start getting matches!
What’s happening? In MAPI search, “word starts with” doesn’t correspond to any of the traditional fuzzy flags. FL_PREFIX will only match the beginning of the block of text, and FL_SUBSTRING will match anything within the text, no matter where it is within a word (or even spanning words). So we had to invent a new fuzzy level: FL_PREFIX_ON_ANY_WORD to capture the intended search semantics.
When searching against Exchange, or when using the index against a PST, we understand this flag. However, we never implemented this flag for the PST (non-indexed), and other stores wouldn’t be aware of it either. So, as a compromise, when dealing with a store which we think won’t understand FL_PREFIX_ON_ANY_WORD, we remove the flag from the search and substitute FL_SUBSTRING. For searches of large blocks of text, such as Subject or Body, this is a good compromise. For searches of From or other fields which usually consist of one word (or, at least, no white space), we’ll see this side effect of the searches matching the middle or ends of the words.
BTW – support for this documentation ultimately got cut, but we did point out this substitution of FL_SUBSTRING for FL_PREFIX_ON_ANY_WORD here:
http://blogs.msdn.com/b/stephen_griffin/archive/2006/05/11/595338.aspx
A couple of customers recently reported this interesting issue with Outlook views and custom form icons:
Expected: The entry in the table uses the custom icon.
Actual: The entry in the table has no icon.
Further, if you narrow the view so that it switches to two line mode, you’ll see the icon appear, and it will continue to be used even if you switch back to the Table view.
The problem stems from the logic we use to determine when we can go back to the forms manager to get the icon. We don’t want to hit the forms manager while we’re rendering the view since this could cause a network call which would slow everything down. So we decide we’ll go get the icon later. Except later never happens for this view. The other views don’t have this flaw – only the Table view. We’re working on a fix for this. I’ll post back here when the fix is ready.
One of the things we’ve seen with Outlook 2013 is that Wrapped PSTs that never used to be indexed by Outlook are indexed now. This can be great if you’ve got a store which can deal with being crawled, but what if your store can’t handle it? The PST provider on which the Wrapped PST is based controls if and when it is indexed. Part of the logic it uses is to call in to the Crawl Scope Manager (CSM) interface to determine if the store has been excluded. Namely, it calls IncludedInCrawlScope, passing in a URL which represents the PST. The CSM will then report whether or not the PST is eligible for indexing.
If you don’t want your store indexed, then the trick is to make sure the CSM returns false here. You can do this by calling AddUserScopeRule and passing the appropriate URL. Here’s the format for the URL:
mapi15://{SID}/StoreDisplayName($HashNumber)/
As you may note, this URL format is very similar to the one described here:
http://msdn.microsoft.com/en-us/library/office/ff960454.aspx
There are a few key differences and caveats:
Here’s an example URL you might generate:
mapi15://{S-1-5-21-123456789-123456789-1234567890-12345}/MyStoreProvider($abcd1234)/
Another issue to be aware of is when you add your URL to the CSM. The PST provider is very sensitive to the indexing state changing while the PST is open. If the state changes, the PST may end up kicking off an installer to repair Outlook. So you need to add the URL before the PST knows anything about it. The best time to do this is when your provider is first loaded, during the IMSProvider::Logon, before you call Logon in the PST. At this point, you have enough information to construct the URL and call in to the CSM, but the PST provider is not aware yet that this PST is being opened.
[Update - 1/16/13] Having registered your store for exclusion, you may want to check that it is, in fact, excluded. If you look at the Indexing Options, you'll see an interesting listing: Microsoft Outlook is both under "Included Locations" and "Exclude". Don't worry - this doesn't mean all of Outlook isn't being indexed. The shell extension that Outlook has registered for the mapi15 namespace simply returns "Microsoft Outlook" as the user readable name for any URL which has been excluded. Development did look at augmenting the extension to provide a better string, but decided most users would never look at this UI to begin with, so it wasn't worth the risk. If you want to check that your code ran correctly, you can look at the registration in the registry under this key:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows Search\CrawlScopeManager\Windows\SystemIndex\WorkingSetRules.
The December 2013 Release (build 15.0.0.1040) is live: http://mfcmapi.codeplex.com.
It’s been a while. I’ve been busy while I was gone. All dialogs got a visual refresh with collapsible panes. New windows cascade now. I finally implemented a viewer for messages. And most important of all, I went out and got myself a snazzy new icon for Christmas:
Here's a change list - see the Issue Tracker on Codeplex for more details, or look at the code:
In Outlook 2013, we took a change to improve how we build search folders when searching against Exchange 2010 and higher for text we’d expect to find in commonly indexed fields. Prior to this change, we’d build a search folder restriction that looks like this:
PR_SUBJECT contains 'text' || PR_BODY contains 'text' || PR_SENDER_NAME contains ‘text' || ...
If you use Outlook 2013 and create an online search folder (say, by just doing a search), then look in MFCMAPI, you’ll find the search criteria is greatly simplified:
0x0eaf001f contains ‘text’
Development has allowed me to document this property:
#define PR_SEARCH_ALL_INDEXED_PROPS_W PROP_TAG(PT_UNICODE, 0x0EAF)
Use of this property in building your searches has a few benefits:
This property will work against Exchange 2010 and higher.
Outlook likes to sort folders alphabetically. When Outlook 2010 came out, some of our custom store provider developers came to us and asked that Outlook not sort the folders they returned. So we gave them a way to advertise to Outlook that the order they returned is the order Outlook should use. However, we didn’t port this fix to Outlook 2013. Why? Because in Outlook 2013 we already had a new feature which allows users to reorder their folders however they like and Outlook would remember the order. The missing piece of the puzzle is to document how Outlook remembers this order so store providers can give a default to their liking.
The key property is PR_SORT_POSITION. Here’s the definition:
#define PR_SORT_POSITION PROP_TAG( PT_BINARY, 0x3020)
Outlook will use this property as part of it’s sort order when requesting folders from a provider, so it’s important that your provider can handle sorting on a binary property – or can fake it when asked to sort by this property. Outlook will also use this property directly when deciding where to insert nodes in the visible tree, so it’s also important that your provider can return this property when Outlook looks for it on a folder.
If you look at some folders with MFCMAPI, you’ll see that Outlook usually only puts only a single byte into this property. Outlook will look at the first byte of this property to do the first pass of the sort. If two folders have the same first byte, Outlook looks at the second byte to break the tie, and so on. You can watch these values change as you continually drag the bottom folder of a list to the top and watch as eventually Outlook has assigned multiple folders a first byte of 00 and has to move to the second byte to break the tie,
There’s a second property Outlook will use for custom sort ordering:
#define PR_SORT_PARENTID PROP_TAG( PT_BINARY, 0x3021)
As the name suggests, this property stores an entry ID which can be used to sort a folder under a different node than it’s natural parent. Normally, a folder will be sorted under the folder represented by PR_PARENT_ENTRYID. This property allows you to suggest a different parent for display.
By presetting these properties appropriately, you can direct Outlook in how you wish your provider’s folders to be sorted. And if you allow Outlook to write to these properties, you can preserve whatever sort order your users desire.
Some of you may have seen this:
http://support.microsoft.com/kb/2849083
The title of the KB is “"Could not complete the operation. One or more parameter values are not valid" error message when an organizer updates a recurring meeting” and it concerns an issue where we find PR_SENT_REPRESENTING_NAME to be missing on various calendar entries, which in turn leads Outlook to complain when you try to update the meeting. We’re working on a fix for Outlook that should be out soon, but I’m still trying to track down why that property went missing in the first place. Here’s where I was hoping the community could help. I know several of you out there have encountered this problem. I need data from a mailbox where the issue has been seen. In particular, I need to get the Calendar Version Store and Calendar Logging that Exchange keeps. In that data I hope to find some clues as to where the property went.
What I want to get is the Calendar Logs for a user who has seen the problem, but I don’t want to get them the usual way, using Powershell. Instead, I want to get them directly using MFCMAPI. Here are the instructions:
You can do this on any user who has seen the problem, regardless of whether you have “fixed” the appointments by restoring the property via MFCMAPI. If you happen to know the subject of an appointment which was broken to help me focus my research, that would be helpful, but not essential.
If you can – great! I can offer my undying gratitude in return. Just contact me through this blog and I’ll arrange a secure means to transfer the logs to me. The data will be kept securely here while I analyze it, then will be deleted. I’ll keep you updated on anything I find in the data that may help us track this problem down.
Thanks in advance to anyone who can help us track this issue down.
Jens has a really good post over on his blog about an issue he faced trying to configure OAuth integration between Lync Server 2013 and Exchange Server 2013:
http://blogs.technet.com/b/jenstr/archive/2012/11/22/getting-internal-server-error-500-when-creating-new-cspartnerapplication-for-exchange-2013.aspx
Why do I point this out? Because the same issue he saw is the one that’s been causing my Message Header Analyzer app for Office to fail in some environments. So far, I’ve only gotten one bad review:
Loads into mail preview grey bar okay but doesn't work. 1. Something went wrong and we couldn't start this app. Try again. then retry 2. This app may not load properly then start 'Message Header Analyser' title appears. thats it.
It turns out, that’s a symptom of Jens’ OAuth problem! The customer actually has an issue they need to correct in their environment. If you’re out there Richard Collins, this article is for you!
Here’s what typically happens:
In the case of Message Header Analyzer – it just stops rendering. I could perhaps handle this case more gracefully, and might in the future. Other apps display an error to the user.
If we use F12 in IE to look at the network traffic, we’ll see the final frame’s request looks something like this:
{"__type":"GetClientAccessTokenJsonRequest:#Exchange", "Header":{"__type":"JsonRequestHeaders:#Exchange", "RequestServerVersion":"Exchange2012", "TimeZoneContext":{"__type":"TimeZoneContext:#Exchange", "TimeZoneDefinition":{"__type":"TimeZoneDefinitionType:#Exchange", "Id":"Eastern Standard Time" } } }, "Body":{"__type":"GetClientAccessTokenRequest:#Exchange", "TokenRequests":[ {"__type":"TokenRequest:#Exchange", "TokenType":"ExtensionCallback", "Id":"62916641-fc48-44ae-a2a3-163811f1c945" }, {"__type":"TokenRequest:#Exchange", "TokenType":"ExtensionCallback", "Id":"d39dee0e-fdc3-4015-af8d-94d4d49294b3" }, {"__type":"TokenRequest:#Exchange", "TokenType":"ExtensionCallback", "Id":"f60b8ac7-c3e3-4e42-8dad-e4e1fea59ff7" } ] } }
This will be met with an error. The exact error appears to depend on the version of Exchange 2013, but I’ve seen both a GetClientAccessTokenResponseMessage with the text “The token for this extension could not be retrieved.” as well as a 503 Service Unavailable. In both cases, when we checked Get-AuthConfig, we found a thumbprint which was not present on any of the certificates found when we ran Get-ExchangeCertificate. Following the steps in Jens’ article to ensure that the thumbprint listed in Get-AuthConfig matched a cert in Get-ExchangeCertificate allowed OAuth to work, in turn allowing the App to work.
Administrators: If you find a mail app can load, but doesn’t function, it might be a problem with your OAuth configuration. Check your F12 trace to see if the GetClientAccessTokenRequest frame is the one that fails. If it is, check your OAuth configuration.
Developers: If your mail app uses EWS, be on the lookout for this sort of error and react appropriately. I suggest directing the user to someone who can help their administrator check the OAuth configuration.
Prior to Exchange 2013, when Outlook’s emsmdb32 provider would log on to the server, it would get various bits of information back from the server, including information needed to connect to the Public Folder store. The provider would use this information to add the Public Folder store to the profile. This work involved pumping messages and this is where the workaround given here was involved:
http://blogs.msdn.com/b/stephen_griffin/archive/2007/05/30/outlook-2007-public-folders-mapi-and-you.aspx
With Exchange 2013, logon no longer returns any information related to Public Folders. Instead, the information needed to connect to Public Folders is handled through Autodiscover. When we do the initial AutoDiscover for profile creation, we get an entry back indicating that there is a Public Folder mailbox. Here’s what it looked like for my test mailbox:
<PublicFolderInformation>
<SmtpAddress>TestPFMailbox@tailspintoys.com</SmtpAddress>
</PublicFolderInformation>
We see the name of the Public Folder mailbox is TestPFMailbox. Outlook uses this information to conduct another Autodiscover conversation to get information on the Public Folder mailbox. Essentially, we send a request that looks like this:
<?xml version="1.0" encoding="utf-8"?>
<Autodiscover xmlns="http://schemas.microsoft.com/exchange/autodiscover/outlook/requestschema/2006">
<Request>
<EMailAddress>TestPFMailbox@tailspintoys.com</EMailAddress>
<AcceptableResponseSchema>http://schemas.microsoft.com/exchange/autodiscover/outlook/responseschema/2006a</AcceptableResponseSchema>
</Request>
</Autodiscover>
We get a response back with information on the PF mailbox. Outlook then uses this information to make a call to CreateProvider to actually add the Public Folder store to the profile. CreateProvider takes a set of MAPI properties, whose values were derived from the information in the Autodiscover response. The properties we pass to CreateProvider are all derived from the AutoDiscover response:
I’ve done a bit of hand waiving above, and I’ve likely missed a number of corner cases, but this should be enough for someone to get started. All of this work actually happens in Outlook.exe, so there is no way for external MAPI processes to trigger it (though it’s possible it might happen while using the Outlook Object Model).
For more information on the Autodiscover process, see [MS-OXDSCLI]: Autodiscover Publishing and Lookup Protocol. Enjoy!