In the Exchange 2007 SDK, we have an antivirus sample application which really exists to illustrate how to build a Transport Routing Agent and handle events. A few customers had difficulties with this sample, so I wanted to go over how to get is set up and working. At the end of this article you'll find a link to an updated sample which should be much easier to play with.
This sample consists of two projects. The first, AntivirusAgent, is a .Net based DLL that will be installed as the actual transport agent. The second, AntivirusService, is a C++ based EXE which functions as an out of proc COM service to do the real work. The split of work in the sample is arbitrary. It didn't have to use an out of proc COM service. In fact, things would have been much simpler if it didn't. :) Getting the .Net code to speak to the COM service, which has nothing to do with demonstrating how to build a Transport Agent, turned out to be quite tricky to get working.
The first problem I saw with AntivirusService is that it dynamically links to the C++ runtime. This is fine for production code, where you will be deploying the runtime as appropriate with your installer, but for samples it's simpler to statically link. So that's the first thing I changed in the sample.
The next thing I noticed is that the AntivirusService builds for both the Win32 platform and the x64 platform, but only the Win32 platform has a post build event to build the ComInterop.dll that is used for the interprocess communication. So if you were building for and testing with a 64 bit server, you would either not have a ComInterop.dll at all, or you'd have one but it would be 32 bit. So the second thing I changed was to add a post build step for the 64 bit build.
The third thing I changed was to clean up the install/uninstall powershell scripts. They assumed the sample would be built on the machine it would be tested on, a bad assumption, and they also failed to register the COM service correctly. Cleaning these scripts up a bit meant fewer manual steps as I built and rebuilt the project.
The fourth change I made was to the way the sample updated headers. The sample was using GetMimeReadStream and GetMimeWriteStream to read the MIME message and rewrite it. However, GetMimeReadStream apparently doesn't always return the MIME stream of the full message. Maybe that's a bug in GetMimeReadStream - maybe not - I didn't investigate as that wasn't really the point of the sample. So I commented out the code that wrote the stream back and instead used the HeaderList class to add a header.
With these changes, I had a functioning sample that could communicate to an out of proc service and write headers back to the mail. It worked (for me) on both 32 and 64 bit Exchange 2007 servers. So I sent the updated sample off to some customers and awaited their kudos. Unfortunately, none of them could get the updated sample to work either. Every time AntivirusAgent tried to call the BeginVirusScan function, which was implemented by AntivirusService, it would get a 0xC0000005 access violation.
What went wrong? After much back and forth, we finally isolated the problem to the .tlb they were generating from the .idl files included in the sample. ComInterop.idl contained this line:
HRESULT _stdcall BeginVirusScan([in] IComCallback* callback);
But when we use OleView to examine the .tlb being generated by midl.exe, we saw this:
HRESULT _stdcall BeginVirusScan([in] IComCallback** callback);
Note the double star. As Harry said to Lloyd, you can't double star a single star. At least, that's how I remember the line. Anyway, when this .tlb is used to marshal calls from one process to another, this extra level of indirection will result in an AV.
It turns out midl.exe has a problem importing an .idl that looks like this:
library MyInterfaces { interface IMyOtherInterface; interface IMyInterface : IUnknown { HRESULT _stdcall DoSomething1([in] IMyOtherInterface* lpParam); }; interface IMyOtherInterface : IUnknown { HRESULT _stdcall DoSomething2(); }; };
With both Visual Studio 2005 and Visual Studio 2008, if you import that .idl file into anther, then use midl to build a .tlb file, the resulting .tlb (as viewed with OleView) will sometimes have an extra star on the definition of DoSomething1. I've passed this bug along to the Visual Studio team and hope to hear back from them soon. Meanwhile, fixing our sample not to hit this problem is a simple matter of changing the order of the interface definitions.
Overall - getting the sample to work only required a few targeted changes. For those who would like to play with the sample but don't want to have to fix it first, I've put all my changes up for download: http://stephengriffin.members.winisp.net/AntiVirus/antivirus.zip
Enjoy!
[Edit] BTW - I want to give credit where credit is due - Stan Miasnikov at PhatWare.com is the one who tracked the MIDL problem down to the forward declaration. I had to make sure I had his OK before mentioning him here.
[This is now documented here: http://msdn.microsoft.com/en-us/library/ee201126(EXCHG.80).aspx ]
Those of you wrapping PSTs may see something interesting in the restrictions Outlook 2007 will try to set against your folders. There are a couple of new restriction types you won't have seen before. Outlook will only use these restriction types when it's working with stores it knows supports them (like a PST), so in general you can ignore them and just pass the restriction on to the underlying store. However, if you're trying to make a copy of the restriction structure for some later use, then you need to know what they look like in memory.
Here are the definitions of the two new restriction types:
#define RES_COUNT ((ULONG) 0x0000000B) #define RES_ANNOTATION ((ULONG) 0x0000000C)
The layout in memory of the associated restrictions is the same as some existing types, which should make parsing them for the purposes of copying easier. RES_COUNT is identical to RES_NOT and RES_ANNOTATION is identical to RES_COMMENT, so to copy a restriction of type RES_COUNT, you can use the resNot member of the union, and for RES_ANNOTATION you can use the resComment member. As an example, we look at how we would fix up MFCMAPI's HrCopyRestrictionArray function. We can take advantage of our existing code by just adding the new cases to the switch. This works since the structures involved are identical:
HRESULT HrCopyRestrictionArray( LPSRestriction lpResSrc, // source restriction ... ) { ... switch (lpResSrc[i].rt) { ... case RES_NOT: case RES_COUNT: if (lpResSrc[i].res.resNot.lpRes) { ... case RES_COMMENT: case RES_ANNOTATION: if (lpResSrc[i].res.resComment.lpRes) { ...
The next update to MFCMAPI will have these changes (and more) in it.
In case you didn't see this on the Exchange Blog:
Today, we posted the preliminary version of the Exchange Open Protocol docs. Read about them in this press release, or view them directly here. We've also created a forum where these protocols may be discussed and feedback given. I know a bunch of you have been waiting for these. Enjoy!
The April 2008 Release (build 6.0.0.1005) is live: http://www.codeplex.com/MFCMAPI
Here's the change list - see the Issue Tracker on Codeplex for more details, or look at the code:
Enjoy.