Suppose you want to put the Web Browser control on an Outlook form. You might follow the steps in this article to create a custom form with the Web Browser control embedded on the second page. Suppose then you switch to this page, load a web page in the control, then try to use the keyboard to navigate around. You’ll see that the keyboard shortcuts for selecting (Ctrl+A), copying (Ctrl+C, Ctrl+X), and pasting (Ctrl+V) text all fail. In fact, if you try any of them, you get a dialog that says “The operation failed. An object could not be found.”

Well – if you’ve run into this, it’s now been fixed.

Here’s the fix for Outlook 2007, and for the same issue in Outlook 2003. Note that the Outlook 2007 fix just needs to be applied to take effect, but the Outlook 2003 fix requires that you set the NewAccel registry key as documented in the hotfix article.

Ok- not exactly SP2 itself, but the Outlook Team has announced that the February update that shipped this week includes a number of performance tweaks and fixes that were originally destined for SP2. Notable among these tweaks are a number of improvements to PSTs, and the Fast Shutdown mechanism. I’m working on documentation for developers to take advantage of Fast Shutdown – look for that in this space soon.

Get the update here, and read about what’s in it here. BTW, if you don’t want to install a cumulative update, you can also wait for SP2 to ship, which should be here by April, according to the Office Sustained Engineering Blog. Enjoy!

Tired of waiting on the Internet? Ever wish you could study MAPI while you’re sitting on the beach? Well now you can! The Outlook 2007 MAPI Reference has been conveniently repackaged as a downloadable .chm file you can install on your laptop and take with you on the go. Enjoy!

[Edit] The problems with the download have been fixed. I was able to install on the machines where it failed before. Let me know if you still have problems installing it.

I don’t play with VSTO much, but I had an opportunity recently to debug an issue with tab order in Form Regions that resulted in an interesting workaround.

This issue was this: Suppose you’ve created an Outlook 2007 Add-in project in C# in Visual Studio 2008. To this project, you’ve added an Outlook Form Region, accepting all the defaults. And on this form region, you added a few standard controls: maybe a couple text boxes and a couple combo boxes. Up until now, you haven’t actually written any code – the wizards did all the work for you. You fire up this add-in to test it out:

So far so good. Now click on the first combo box and hit Tab. You’d expect the cursor to jump to the second combo box. But it doesn’t. It jumps to the second edit box. Click back in the first combo box and hit Tab. Again, you expect the cursor to jump to the second combo box. Instead it flickers and lands back in the first combo box.

What’s going on here is for certain controls hosted in this fashion, WM_SETFOCUS is not being received when you click on them. Since they don’t get WM_SETFOCUS, they don’t set the internal properties that indicate to WinForms that they are the current control. So when you hit Tab, we look at the current control, which is not the one with the cursor in it, figure out the next control in the tab order, and set focus there.

It’s not just combo boxes affected by this bug. All sorts of common controls show the same problem. Even regular edit boxes, which appeared to work, are not immune. You can right click in an edit box and hit Esc to dismiss the drop down menu. Now the text box has the cursor, but isn’t the current control.

This has been confirmed to be a bug in WinForms (it can be reproduced without VSTO), and should be addressed in a future version. Fortunately, we don’t have to wait to get this to work. One of our Visual Studio devs gives the following workaround:

Since WM_PARENTNOTIFY is sent to the parent window whenever a child window is clicked, we can hook that message in WndProc and set focus manually. You can drop the following in your FormRegion class and the tabs will start working:

        protected override void WndProc(ref Message m)
        {
            switch (m.Msg)
            {
                case NativeMethods.WM_PARENTNOTIFY:
                    if (NativeMethods.Util.LOWORD(m.WParam) == NativeMethods.WM_LBUTTONDOWN ||
                        NativeMethods.Util.LOWORD(m.WParam) == NativeMethods.WM_RBUTTONDOWN)
                    {
                        // Get x and y coordinate
                        int x = NativeMethods.Util.LOWORD(m.LParam);
                        int y = NativeMethods.Util.HIWORD(m.LParam);
                        // Get a child control according to coordinates
                        Control control = GetChildControl(x, y);
                        if (control != null && !control.Focused)
                            control.Focus();
                    }
                    break;
            }
            base.WndProc(ref m);
        }
        private Control GetChildControl(int x, int y)
        {
            foreach (Control control in this.Controls)
            {
                Rectangle rect = new Rectangle(control.Location, control.Size);
                if (rect.Contains(x, y))
                    return control;
            }
            return null;
        }
        internal class NativeMethods
        {
            public const int WM_PARENTNOTIFY = 0x0210;
            public const int WM_LBUTTONDOWN = 0x0201;
            public const int WM_RBUTTONDOWN = 0x0204;
            public static class Util
            {
                public static int HIWORD(int n)
                {
                    return (n >> 16) & 0xffff;
                }
                public static int LOWORD(int n)
                {
                    return n & 0xffff;
                }
                public static int HIWORD(IntPtr n)
                {
                    return HIWORD(unchecked((int)(long)n));
                }
                public static int LOWORD(IntPtr n)
                {
                    return LOWORD(unchecked((int)(long)n));
                }
            }
        }

Time for another round on referrals. When last we left off, we were able to get the MAPI download to request and receive referrals from Exchange 2007, but the settings didn’t work if MAPI was running locally on the Exchange 2007 server. Let’s take a closer look at why it fails:

The whole point of the Exchange 2007 specific settings was that RPC_C_AUTHN_WINNT was removed as an authentication mechanism for the referral interface, so we had to configure the profile to use a different authentication mechanism, RPC_C_AUTHN_GSS_NEGOTIATE. When we try these settings while running locally, we fail because  RPC_C_AUTHN_GSS_NEGOTIATE isn’t a valid authentication mechanism for local RPC.

If only we didn’t have to use local RPC when running locally! If we were connecting via TCP/IP, these settings would work. Turns out, we can do that. It’s an old trick – so old I had forgotten about it. We can set the RPC_BINDING_ORDER and remove local RPC as an option. Then the connection will be made over TCP/IP and the settings can work. Note that if you’re running on a 64 bit operating system (and you would be, since Exchange 2007 in production is 64 bit only), the reg key is slightly different. Here’s what I tested:

• Key: HKLM\SOFTWARE\Wow6432Node\Microsoft\Exchange\Exchange Provider
• Value: Rpc_Binding_Order
• Setting: ncacn_ip_tcp

With this reg key set, I was able to use my GCReconnect tool to create a profile and log on, obtaining and using a referral to the GC in the process.

A few months ago I documented a bug in the Exchange MAPI download that prevents you from loading ExMAPI32.dll directly. The only way to load it was through the MAPI Stub library. This problem is now fixed with the 6.5.8069 update. A few notes:

• You must uninstall the Exchange MAPI download before installing the update. If you don’t do this, the update will appear to function, but the DLLs won’t be updated.
• The 6.5.8069 update also includes the security fix noted here and here.
• The 6.5.8069 update includes the timezone update noted here.

We had a customer recently who’s application iterated through mailboxes on the Exchange server, advising for notifications on a handful of folders in each mailbox. They were fine with a few thousand mailboxes, but as they tried to scale up, they ran into a couple limitations.

The first limitation was already well known: MAPI uses shared memory to track a number of details between multiple processes, including notification registrations. This memory is allocated by the first MAPI client to log on to a given profile. By default, this memory will be 1 MB. When this customer ran their test with 1 MB of shared memory, they were able to call Advise 7828 times. The total size of the memory needed to track the 7828 notifications, plus all the other data also kept in this shared memory, used up the entire megabyte, so the next attempt at calling Advise failed. The reason the article gives a range of values (7800-7900) is that the practical limit will vary depending on whether other MAPI applications are running, what they’re doing, what needs various providers have for notifications, etc.

Increasing the size of the shared memory got them past this first limitation, but also makes it possible to hit the next limitation. MAPI tracks the notifications which have been registered in a structure similar to an array. This structure, which lives in shared memory, will grow and shrink depending on how many notifications have been registered. However, the largest this array can be is 65528 bytes, as that is the largest block the shared memory routines can allocate. This is a fixed limitation and cannot be altered through any registry keys. Since each notification takes up 4 bytes in the array, we have a theoretical maximum number of notifications of 16382. Again, practical concerns (MAPI’s own use of notifications, providers which may use notifications, and some fixed overhead in the structure) will limit how many notifications a client will actually be able to register. In the customer’s test, they were able to get 16296 before Advise failed with MAPI_E_NOT_ENOUGH_MEMORY. Your own mileage may vary – expect to top out just north of 16000 calls to Advise no matter how large you make the shared memory.

Note that due to their common lineage, both Outlook and Exchange’s implementation of MAPI have the same behavior. Both of them implement the SharedMemMaxSize reg key and both have the same practical limitations on the number of times Advise can be called.

Now – since this limitation is derived from how shared memory is used, running more MAPI clients that use the same shared memory will not work around the problem. So running another process as the same user will just spread the limit across two processes. If you want to work around the problem with a second process, the second process will need to run as a different user altogether.

Alternatively, if the goal is to monitor as many mailboxes as possible, you can try reducing the number of times you call Advise. The article on ignoring notifications may come in handy here.

Finally, this limitation might be enough to get you looking at EWS, in which case you’ll probably want to read Vikas’ overview of notifications in EWS.

Suppose you want to register for notifications on all the visible folders in a user’s mailbox. One option would be to walk the list of folders in the hierarchy tree and register for notifications. That would work, but would be very inefficient. It also wouldn’t be dynamic, since you wouldn’t catch new folders as they’re created. And if you register for too many notifications, you might run into this problem. A more elegant solution is to register for notifications at the message store level. This is what one customer did, but recently they ran into a problem with one of the new features in Outlook 2007.

Before Outlook 2007, if someone gave you permissions to see their Calendar, when you opened their Calendar Outlook would have to make a direct connection to their mailbox server, even if your profile was running in Cached mode. This was never an ideal experience for the user, as they would have to wait while Exchange built the views for the folder. And if too many people shared the same calendar, you could run into a scenario where each time a user looked at the folder, some other user’s views were deleted to make room, essentially meaning access for everyone, including the owner of the mailbox, was slow. This is a nasty scenario that isn’t fun to troubleshoot or resolve. Cached mode lessens the problem for the owner of the mailbox; their access will always be fast. But it doesn’t help the other users.

With Outlook 2007, we added the idea of caching these shared folders. You may have seen the option in your mailbox settings:

This allows you to cache a copy of the other users Calendar, Contacts, Tasks and Notes folders in your own mailbox so you don’t have to connect to the Exchange server every time you want to look at them. The folders get updated periodically through the same incremental change synchronization (ISC) mechanism used to sync your own mailbox. (BTW – if you want to cache mail folders too, there’s a reg key for that.)

Since we’re caching these folders locally, the data has to live somewhere right? And where else but right there in your OST! Taking a look at my cached mode profile using MFCMAPI, we can see I have Jeff Garman’s Calendar cached:

This is where the customer ran into a problem. Remember they were registering for notifications store wide. In cached mode, that would include these folders under Shared Data, which they weren’t particularly interested in. Their question was how to identify which notifications were for these Shared Data folders so they could ignore them. Now, it turns out we haven’t documented many MAPI properties in this area, but there’s a way to approach this that doesn’t require any special knowledge. The key is recognizing that all the folders they do wish to monitor live under IPM_SUBTREE, and that folder is well documented. Here’s the approach we worked out:

1. Maintain a cache of known folder entry IDs. Each entry would be accompanied by a Boolean indicating whether the folder is under IPM_SUBTREE or not.
2. Seed the cache with the entry IDs for IPM_SUBTREE, marked as being under IPM_SUBTREE, and the root folder, marked as being not under IPM_SUBTREE.
3. Implement a function, IsUnderIPMSubtree, which checks if the target entry ID is in the cache, and returns the value it finds. Comparisons should be done with CompareEntryIDs.
4. If the entry ID is not in the cache, IsUnderIPMSubtree obtains the entry ID of the parent folder, and calls IsUnderIPMSubtree with that parent entry ID
5. The result of this recursive IsUnderIPMSubtree call is cached with the target entry ID, and the value is then returned.
6. Whenever a notification comes in, use the function IsUnderIPMSubtree to determine if the notification was interesting or not.

This algorithm makes the most sense to run in Cached mode, where CompareEntryIDs and OpenEntry will both be relatively cheap. Expected recursion level should be rather low, and the total number of folders opened over the lifetime of the algorithm is capped by the total number of folders in the store. In addition to filtering out notifications from the Shared Data folders, you’ll also eliminate any spurious notifications from any of the other non-visible folders.

If you’ve been working with PSTs for a while, you’re probably familiar with the properties PR_PST_PW_SZ_OLD and PR_PST_PW_SZ_NEW, given in mspst.h, which are used to access password protected PSTs and also to change the password. I had a customer recently who needed to know if a PST they had was password protected in the first place, so they could process them appropriately.

We did some research and found that if you added such a PST to a profile, then called OpenMsgStore on it with the MDB_NO_DIALOG flag set, you would get MAPI_E_FAILONEPROVIDER, which is pretty much the most generic error you can get in MAPI. Since we knew there were other reasons you could get this error, the question was whether we could distinguish this particular case. Most interfaces in MAPI support the function GetLastError. I set up a bit of code to try calling OpenMsgStore on a password protected PST, and took a look at the result of GetLastError. Here’s what I saw in the MAPIERROR structure:

ulVersion = 0x00000000
lpszError = 0x00000000
lpszComponent = Personal Folders
ulLowLevelError = 0x00000000
ulContext = 0x30060401

Note the value for ulContext. It turns out it’s a unique constant the PST provider will set when the PST was password protected and we could not display a dialog to prompt for credentials. Development has given permission to document this context, as well as one other:

If the credentials attempted failed with Access Denied and MAPI_NO_DIALOG wasn’t passed, the PST provider will attempt to prompt for credentials. The other context (0x30060402) should be rare, only occurring in out of memory or other unusual circumstances.

It turns out the PST provider is very good about setting errors with error contexts. Usually the top level MAPI error is sufficient to explain why a call failed, but if there are other ambiguous scenarios involving the PST provider, check the ulContext from GetLastError. If it’s unique, let me know the scenario and the context you got, and I’ll see if I can document it.

A scenario recently came up that a couple of customers have hit with Outlook’s version of MAPI. These customers are in the business of processing PST files found on a user’s machine. They may be loading them up to scan the messages for viruses, or to ensure they’re backed up. Since opening a PST requires MAPI, they would create a MAPI profile to do their work, add the PST to the profile, read the data they needed to read, then logoff the profile and delete it. These scans would happen throughout the day without user interaction. The interesting scenario occurs when the user happened to be running Outlook, and they had the PST already loaded in Outlook’s profile. In Outlook 2003, we didn’t get any reports of a problem with this. But with Outlook 2007, both of these customers started seeing failures to open these PST files. Usually they wouldn’t have a problem, but every once in a while, they couldn’t open the file.

Investigation

Let’s look at what happens when you open a PST file using MAPI. First – MAPI itself doesn’t deal with the PST. It’s the provider, mspst32.dll that opens the PST file. The first thing the PST provider tries to do is open the file with exclusive write access, allowing others to read but not write. If this fails, then it assumes another instance of the PST provider has already opened the file, so it requests read access. If both processes accessing the PST are running as the same user, in the same session, the provider is able to coordinate access to the file. This is how it worked in Outlook 2003 and usually there weren’t any problems sharing access, even if the different MAPI sessions were using different profiles.

In Outlook 2007, as part of some optimizations around how we read and write to the PST, we implemented a cache. Access to this cache is controlled by a number of shared objects (a memory mapped file, some events, etc.). These shared objects derived their name from the path of the PST file. And here is where the problem came in.

Investigation using Process Explorer showed that when the problem happened, Outlook’s MAPI session and the customer’s MAPI session were referencing the same file, using the same path, but the case was different. For instance, in one process, the file handle might point to:

C:\TestFiles\MyPST.pst

where in the other process, the file handle points to:

C:\testfiles\mypst.pst

Note that some characters are uppercase in one path, and lowercase in the other. For access to the file, this difference in case doesn’t matter, so everything worked in Outlook 2003. And in Outlook 2007, again, for access to the PST file itself, the difference in case wasn’t a problem. However, when we build the names for the shared memory objects, case does matter. For instance, memory mapped files are created using the function CreateFileMapping. Although it’s not specifically documented as such, this function is case sensitive. The names

C__TestFiles_MyPST_pst_WCINFO

and

C__testfiles_mypst_pst_WCINFO

when used in the lpName parameter of CreateFileMapping will point to two different objects. So our mechanism for synchronizing access to the cache fails, and the second process to try to access the PST ends up returning an error, usually MAPI_E_FAILONEPROVIDER.

I raised this as a bug with development, with the suggested fix that we just lowercase (or uppercase) the paths before building the shared object names. However, in the course of trying to fix it, we realized the problem is actually much bigger than the case of the path. For instance, if one profile uses the path:

C:\TestFiles\MyPST.pst

and the other uses

C:\testfi~1\mypst.pst

both are still accessing the same file. However, this scenario wouldn’t be fixed by simply lowercasing the file name. Also problematic would be sym links, drive mappings, etc. The real fix is to not depend on the path name as part of the synchronization, and instead use some internal characteristic of the PST file itself. This fix is in the works, however, it’s too big to get into a hotfix. The next version of Outlook should handle all of these scenarios much better.

Workaround

As it turns out, the workaround for this is fairly straightforward. It’s based on this fact: As long as all the profiles accessing the PST use the same path, with the same case, then the problem can’t happen. All the shared memory objects will use the same names and there won’t be any problems with synchronization. So, if you have an application that routinely adds PSTs to your own profile, and wish to avoid this problem, all you have to do is check if there are any other profiles, and if they’re using the same PST. If they are, use the same path they’re using, and then you can’t conflict with them! In practice, scanning the profiles looks like this:

1. Use MAPIAdminProfiles to get an IProfAdmin object and IProfAdmin::GetProfileTable to get a table of profiles.
2. For each profile in the list, use IProfAdmin::AdminServices to get an IMsgServiceAdmin object and then IMsgServiceAdmin::GetMsgServiceTable to get a list of services in the profile.
3. For each service where PR_SERVICE_NAME is “MSPST MS” or “MSUPST MS”, use IMsgServiceAdmin::AdminProviders with the value from PR_SERVICE_UID to open an IProviderAdmin object, then use IProviderAdmin::GetProviderTable to get a list of providers.
4. From the provider in the table, use IProviderAdmin::OpenProfileSection to open the profile section, from which you can read PR_PST_PATH.
5. Add this path to your list of “known PSTs”.

This process can be repeated each time you have a new PST to manipulate, or just periodically. Once you’ve got your list of known PSTs, you just need to compare your PST to the ones in the list. As long as you ensure the path you use is the same as the path in the other profiles, you eliminate the possibility of this problem happening.

This comparison can be simple or complex, depending on what scenarios you want to cover. Given that the most common scenario you’re going to hit here is the paths are the same except for casing, you could just do a case insensitive compare and cover most of cases. If you want to cover more cases, you could use GetFullPathName on the paths before you do your comparison. And if you want to cover even more cases, you can use GetFileInformationByHandle to get the volume number and index for each path and compare those.