Programming Virtual Server from a managed application (Part 1)

One of the coolest things (IMHO) about Virtual Server is that we have a completely documented COM API that allows end users and other developers to create programs that automate and interact with Virtual Server.  Unfortunately doing this from a managed application (VB.Net or C#) is not the easiest thing to do.  The reason for this is that there are two main complications that you have to deal with before you can get going.

The first complication is handling COM security levels. Calling the Virtual Server COM interfaces requires that your application be running with a COM security level of ‘impersonation’ (or higher).  It is possible to do this from a managed application by using CoInitializeSecurity() - however it is only possible for an application to call CoInitializeSecurity() once – and the first call applies to the entire application.  The problem starts when you realize that many components will attempt to call CoInitializeSecurity() for you – with the wrong COM security level (Even using the VMRC ActiveX control in your application will cause this to happen).

Thankfully - this is fairly easy to deal with - if you know what you are doing.  First - there is sample code in the Virtual Server Programmers guide that shows you how to do this (be warned that this sample code has some typographical errors in it - but they are fairly easy to figure out).  You should be able to take this code and use it directly - but if you are programming in VB.Net (my preferred managed language - yes I know that this will divide my reading audience :) I recommend that you put this sample code in a separate class - rather than a separate module (for reasons that I will discuss next week).

Once you have done this - you can simply call this class to initialize COM security as part of a shared main subroutine - ensuring that you get in first.  There are two remaining problems that you should be aware of though:

1. It is possible to have a managed application grow so complicated that it is hard to ensure that your call to CoInitializeSecurity will be the first one made.  If you get into this situation the only real solution is to create an unmanaged wrapper application that calls CoInitializeSecurity first and then launches your application.
2. The Visual Studio 2005 beta actually uses a wrapper application to launch managed application for debugging that will stop calls to CoInitializeSecurity from working (it calls it first).  To stop this from happening you will need to go into your project settings and either enable unmanaged code debugging or disable the use of the 'vshost' process (depending on which beta release you are running).

Finally you should know that calling CoInitializeSecurity will mean that your program will require unmanaged code privileges.

Cheers,
Ben