[This is now documented here: http://msdn.microsoft.com/en-us/library/bb821096.aspx, http://msdn.microsoft.com/en-us/library/bb821097.aspx, http://msdn.microsoft.com/en-us/library/bb821099.aspx, http://msdn.microsoft.com/en-us/library/bb821100.aspx]
This is preliminary documentation for Outlook 2007 Beta 2. It does not apply to earlier versions of Outlook.
Notification Based Indexing Support A store provider supports Notification Based Indexing if it doesn’t require incremental or full crawls and indexes all the items through notifications. Sometimes these providers are referred to as “pusher” stores, since such a store provider will push URLs to the indexer by calling appropriate indexer APIs. These URLs are built based on algorithms documented below. Each URL corresponds to one message, folder or attachment. The indexer will pass the URLs to the MAPI Protocol Handler (MAPI PH) which will parse it open to determine which message/folder/attachment it corresponds to. MAPI PH then opens that object through MAPI and gets the properties to be indexed. The text is returned to the indexer so that it can word-break it and save the terms in the catalog. Some properties like bodies or attachments may require the use of the filter host which is a different process. The filter host will basically use IFilters to crack the documents and get the text so that the indexer can then word-break the text.
New Store Support Mask Flag This flag is set by pusher stores.
Definition
#define STORE_PUSHER_OK ((ULONG) 0x00800000)
If the store provider sets this flag the MAPI PH won’t crawl the store and the store is responsible to push any changes through notifications to the indexer to have messages indexed.
Usage This flag can be retrieved by getting the property PR_STORE_SUPPORT_MASK from the store.
New Properties for the MAPI PH
#define PR_PROVIDER_ITEMID PROP_TAG(PT_BINARY, 0x0EA3) #define PR_PROVIDER_PARENT_ITEMID PROP_TAG(PT_BINARY, 0x0EA4)
These properties are used to identify an item or a folder by the store provider. They are retrieved when the store provider gets the search results from the search engine and are used to identify which items matched the query. Store providers can provide provider specific values for these properties but their values shouldn’t change between sessions otherwise they won’t know how to map the items when getting search results.
MAPI URLs Every time a folder, message or an attachment is to be indexed a unique Unicode URL has to be generated and sent to the indexer. This URL will later be used to identify which object to be indexed in the MAPI PH. Store providers that want to implement notification based indexing are responsible for generating these URLs.
MAPI URLs have the following format:
Mapi://SID/StoreDisplayName ($HashNumber)/StoreType/FolderNameA/…/FolderNameN/[EntryIDEncoded[?at=AttachIDEncoded:FileName]]
Parameters
SID Current user’s SID
HashNumber DWORD in hex calculated based on the store entry id and in some case the file path. This value will be stored in the registry and will be used later to identify the store in the MAPI PH. This number has to be calculated in a way that minimizes collisions between other stores. See below for the algorithm Outlook uses to calculate the hash.
StoreType Number that identifies the type of the store that contains the object to be indexed. Here are the possible values for pusher stores: 0 – Norma store (non public or delegate stores. This includes default stores, PSTs, etc.) 1 – Delegate store (used for delegate items cached locally) For stores which have been crawled by the indexer, this value will always be ‘X’.
FolderNameA/…/FolderNameN The path from the root of the IPM_SUBTREE to the folder or message. For instance, a message in the “Family” folder under “Inbox” will have Inbox/Family for this parameter.
EntryIDEncoded MAPI Entry ID for the item encoded as a Unicode string. See below for how it gets encoded. Note that when viewed as text, this encoded entry ID will appear as random Hangul characters or boxes depending on available fonts.
AttachIDEncoded Attachment ID encoded as a Unicode string.
FileName Attachment file name as it appears in the message.
Here are some examples of MAPI URLs for a folder, message and attachment respectively:
mapi://S-1-5-21-2127521184-1604012920-1887927527-71418/Mailbox – Some User ($be19928f)/2/Office mapi://S-1-5-21-2127521184-1604012920-1887927527-71418/Mailbox – Some User ($484efb89)/0/Calendar/곯가가가걍걝걌곌겷걢곒갑겛개가검걟곔걙곾걤곂갠가 mapi://S-1-5-21-2127521184-1604012920-1887927527-71418/Mailbox – Some User ($484efb89)/0/Inbox/곯가가가걍걝걌곌겷걢곒갑겛개가검걟곔걙곾간곷갦가/at=겅걋각가:somefile.txt
Algorithm to calculate the store hash
DWORD ComputeStoreHash(ULONG cbStoreEID, LPENTRYID pbStoreEID, LPCWSTR pwzFileName) { DWORD dwHash = 0; ULONG cdw = 0; DWORD* pdw = NULL; ULONG cb = 0; BYTE* pb = NULL; ULONG i = 0; // Get the Store Entry ID // pbStoreEID is a pointer to the Entry ID // cbStoreEID is the size in bytes of the Entry ID pdw = (DWORD*)pbStoreEID; cdw = cbStoreEID / sizeof(DWORD); for (i = 0; i < cdw; i++) { dwHash = (dwHash << 5) + dwHash + *pdw++; } pb = (BYTE *)pdw; cb = cbStoreEID % sizeof(DWORD); for (i = 0; i < cb; i++) { dwHash = (dwHash << 5) + dwHash + *pb++; } // You may want to also include the store file name in the hash calculation // Get store FileName // pwzFileName is a NULL terminated string with the path and filename of the store if (pwzFileName) { while (*pwzFileName) { dwHash = (dwHash << 5) + dwHash + *pwzFileName++; } } // dwHash now contains the hash to be used. It should be written in hex when building the URL. return dwHash; }// ComputeStoreHash
Algorithm to encode the Entry ID and the attachment ID The goal of this algorithm is to generate a compact representation of the Entry ID or attachment ID.
const WORD kwBaseOffset = 0xAC00; // Hangul char range (AC00-D7AF) LPWSTR EncodeID(ULONG cbEID, LPENTRYID rgbID) { ULONG i = 0; LPWSTR pwzDst = NULL; LPBYTE pbSrc = NULL; LPWSTR pwzIDEncoded = NULL; // rgbID is the item Entry ID or the attachment ID // cbID is the size in bytes of rgbID // Allocate memory for pwzIDEncoded pwzIDEncoded = new WCHAR[cbEID+1]; if (!pwzIDEncoded) return NULL; for ( i = 0, pbSrc = (LPBYTE)rgbID, pwzDst = pwzIDEncoded; i < cbEID; i++, pbSrc++, pwzDst++) { *pwzDst = (WCHAR) (*pbSrc + kwBaseOffset); } // Ensure NULL terminated *pwzDst = L'\0'; // pwzIDEncoded now contains the entry ID encoded. return pwzIDEncoded; }// EncodeID
New Notification to Notify the MAPI PH of the Process Pushing MAPI URLs to be Indexed A new MAPI notification type has been introduced to facilitate shutdown scenarios for pusher stores. These stores will have to persist what has to be pushed since it may not be able to index everything before a shutdown occurs. When a store provider is a pusher it should send the following notification so that the MAPI PH knows which process it should watch for a given store. This way if the process is shut down or crashes the MAPI PH will immediately detect that and close (stop indexing) the store it opened.
#define fnevIndexing ((ULONG) 0x00010000) /* Indexing notifications (used for FTE related communications) */ /* Shares EXTENDED_NOTIFICATION to pass structures below, */ /* but NOTIFICATION type will be fnevIndexing */ // Stores that are pusher enabled (PR_SUPPORT_MASK contains STORE_PUSHER_OK) // are required to send notifications regarding the process that is pushing. #define INDEXING_SEARCH_OWNER ((ULONG) 0x00000001) typedef struct _INDEX_SEARCH_PUSHER_PROCESS { DWORD dwPID; /* PID for process pushing */ } INDEX_SEARCH_PUSHER_PROCESS;
BLOB structure associated with each URL when indexing When pushing URLs to be indexed pusher stores should also create a blob with some information that will be used by the MAPI PH. This blob will be associated with each URL and will be sent when the URL is pushed to the indexer. The format of the blob is the following:
DWORD dwVersion DWORD dwFlags ULONG cbProfileName WCHAR wszProfileName ULONG cbProviderItemID WCHAR wszProviderItemID
Note that these values have to be written to the blob in the order shown above. The provider item ID should only be sent for folders to prevent opening extra folders to get this information.
dwVersion This is the version of the data being sent. Currently this value is 1.
dwFlags Reserved for future use. Currently this should be 0.
cbProfileName Size of the profile name in bytes. This information will be useful for the MAPI PH to know which profile to use when indexing the item.
wszProfileName Null terminated Unicode string with the profile name.
cbProviderItemID Size of the Provider Item ID in bytes
wszProviderItemID Null terminated Unicode string with the provider item ID that uniquely identifies the item in the store.
Named Properties The following named properties are indexed by the MAPI PH. These properties are documented as read-only. They should not be used to create or modify items. These GUIDs represent the name spaces of the named properties
const GUID PSETID_Appointment = {0x00062002, 0x0000, 0x0000, {0xC0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x46}}; const GUID PSETID_Task = {0x00062003, 0x0000, 0x0000, {0xC0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x46}}; const GUID PSETID_Address = {0x00062004, 0x0000, 0x0000, {0xC0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x46}}; const GUID PSETID_Common = {0x00062008, 0x0000, 0x0000, {0xC0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x46}}; const GUID PSETID_Log = {0x0006200A, 0x0000, 0x0000, {0xC0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x46}}; const GUID PS_PUBLIC_STRINGS = {0x00020329, 0x0000, 0x0000, {0xC0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x46}}; const GUID PS_INTERNET_HEADERS = {0x00020386, 0x0000, 0x0000, {0xC0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x46}};
MNID_ID properties:
// In PSETID_Address #define dispidWorkAddressStreet 0x8045 #define dispidWorkAddressCity 0x8046 #define dispidWorkAddressState 0x8047 #define dispidWorkAddressPostalCode 0x8048 #define dispidWorkAddressCountry 0x8049 #define dispidInstMsg 0x8062 #define dispidEmailDisplayName 0x8080 #define dispidEmailOriginalDisplayName 0x8084 // In PSETID_Task #define dispidTaskStartDate 0x8104 #define dispidTaskDueDate 0x8105 #define dispidTaskActualEffort 0x8110 #define dispidTaskEstimatedEffort 0x8111 #define dispidTaskFRecur 0x8126 // In PSETID_Appointment #define dispidLocation 0x8208 #define dispidApptStartWhole 0x820D #define dispidApptEndWhole 0x820E #define dispidApptDuration 0x8213 #define dispidRecurring 0x8223 #define dispidAllAttendeesString 0x8238 #define dispidToAttendeesString 0x823B #define dispidCCAttendeesString 0x823C // In PSETID_Common #define dispidReminderSet 0x8503 #define dispidSmartNoAttach 0x8514 #define dispidCommonStart 0x8516 #define dispidCommonEnd 0x8517 #define dispidRequest 0x8530 #define dispidCompanies 0x8539 #define dispidReminderNextTime 0x8560 // In PSETID_Log (also known as Journal) #define dispidLogType 0x8700 #define dispidLogStart 0x8706 #define dispidLogDuration 0x8707 #define dispidLogEnd 0x8708
MNID_STRING properties
// In PS_PUBLIC_STRINGS "Keywords" // In PS_INTERNET_HEADERS "return-path"
[4:12 - fixed some truncated prop vals]
[12:10 - 12/20/06 - fixed one-off bug per Robert's comment]
[This is now documented here: http://msdn.microsoft.com/en-us/library/bb820947.aspx]
There was a thread a while back in the microsoft.public.win32.programmer.messaging newsgroup about how Outlook 2003's new Unicode MSG files. The big question was how to create one. One poster even went as far as to reverse engineer the MSG format from scratch. Fortunately, there's an easier way, which I just got clearance to publish.
We start with the canonical sample for MSG files: http://support.microsoft.com/kb/171907
The key is the OpenIMsgOnIStg function. This function takes a ulFlags parameter. All you need to do to create a Unicode MSG file is to pass MAPI_UNICODE in this parameter. Once you do that, the resulting message will show STORE_UNICODE_OK in it's PR_STORE_SUPPORT_MASK and unicode properties will work.
This flag is supported in this function on Outlook 2003 and higher only.
[This is now documented here: http://msdn.microsoft.com/en-us/library/bb820936.aspx]
Blocked Attachments Microsoft Outlook includes a feature that blocks attachments that are considered unsafe. The attachments which are blocked can vary from client to client depending on how Outlook is configured and on policies the administrator may have applied. See http://support.microsoft.com/kb/829982 for more information on how this is configured.
Custom code can query to see if a particular attachment is considered blocked by Outlook using the IAttachmentSecurity interface. This interface exposes a function, IsAttachmentBlocked, which will analyze a file name and report if this attachment is considered blocked by Outlook and won’t be shown in the UI or indexed.
DEFINE_GUID(IID_IAttachmentSecurity, 0xB2533636, 0xC3F3, 0x416f, 0xBF, 0x04, 0xAE, 0xFE, 0x41, 0xAB, 0xAA, 0xE2); #define MAPI_IATTACHMENTSECURITY_METHODS(IPURE) \ MAPIMETHOD(IsAttachmentBlocked) \ (LPCWSTR pwszFileName, BOOL *pfBlocked) IPURE; DECLARE_MAPI_INTERFACE_(IAttachmentSecurity, IUnknown) { BEGIN_INTERFACE MAPI_IUNKNOWN_METHODS(PURE) MAPI_IATTACHMENTSECURITY_METHODS(PURE) };
Usage This interface can be obtained by calling QueryInterface on the MAPI session object, requesting IID_IAttachmentSecurity. IsAttachmentBlocked will return true in pfBlocked if the attachment is considered blocked by Outlook and won’t be shown in the UI or indexed
HRESULT IsAttachmentBlocked(LPMAPISESSION lpMAPISession, LPCWSTR pwszFileName, BOOL* pfBlocked) { if (!lpMAPISession || !pwszFileName || !pfBlocked) return MAPI_E_INVALID_PARAMETER; HRESULT hRes = S_OK; IAttachmentSecurity* lpAttachSec = NULL; BOOL bBlocked = false; hRes = lpMAPISession->QueryInterface(IID_IAttachmentSecurity,(void**)&lpAttachSec); if (SUCCEEDED(hRes) && lpAttachSec) { hRes = lpAttachSec->IsAttachmentBlocked(pwszFileName,&bBlocked); } if (lpAttachSec) lpAttachSec->Release(); *pfBlocked = bBlocked; return hRes; }// IsAttachmentBlocked
Update: Support for these flags has been cut.
Full Text Search Query Support Outlook 2007 introduces full text queries and wordwheel filtering. Store providers wishing to support these queries will need to support new restriction flags and expose their support in their PR_STORE_SUPPORT_MASK.
New Store Support Mask Flags These flags are set by stores which support full text queries and wordwheel.
Definitions
#define STORE_FULLTEXT_QUERY_OK ((ULONG) 0x02000000)
#define STORE_FILTER_SEARCH_OK ((ULONG) 0x04000000)
Usage These flags can be retrieved by getting the property PR_STORE_SUPPORT_MASK from the store.
New Restriction Flags These flags are intended to be used for stores that have full text search support. The PST/OST provider will support these new flags in restrictions.
FL_PREFIX_ON_ANY_WORD: Will match prefix on words instead of the whole prop value. FL_PREFIX_ON_ANY_WORD can be combined with FL_SUBSTRING so the provider can default to substring matching if full text searching has not been enabled.
FL_PHRASE_MATCH: Phrase match means the words have to be exactly matched and the sequence matters. This is different than FL_FULLSTRING because it doesn't require the whole property value to be the same. One term matching another term in the property value is enough for a match even if there are more terms in the property. Word breaker characters will be ignored in the match. FL_PHRASE_MATCH can be combined with FL_SUBSTRING so the provider can default to substring matching if full text searching has not been enabled.
#define FL_PREFIX_ON_ANY_WORD 0x00000010 #define FL_PHRASE_MATCH 0x00000020
Usage These new flags are used in a similar fashion to existing restriction flags in building restrictions. These flags are only supported on stores which have specified STORE_FULLTEXT_QUERY_OK in their PR_STORE_SUPPORT_MASK. Using these flags in stores that do not support them may cause restrictions to fail.
[edit - correcting flag name]
Update – this is now part of the Outlook MAPI Code Samples.
I finally got around to testing the Wrapped PST sample with Outlook 2007 Beta 2. I found that I had hardcoded the path to mspst32.dll and it really didn't like that the path has changed in the beta. So I've gone in and hardcoded the new path in there as well. I'll leave the task of implementing a robust solution to anyone who wants to build a product out of this. :)
Anyway, that was the only change I had to make. Here's the updated code: http://stephengriffin.members.winisp.net/wrappst/wrappst.zip