This article will describe a method for having a locally-installed process automatically launch inside a virtual environment. This method does not require publishing new shortcuts for the local application.
App-V customers often have interesting, advanced scenarios that require locally installed applications to run inside of a virtual environment. For example, let's say that the customer has a web application that requires an ActiveX control in order to function correctly. However, they don't want to allow their users to install / upgrade ActiveX controls for security reasons. And, they don't want other websites to have access to this ActiveX control, in case it has any security vulnerabilities that a malicious website could use to attack their enterprise. What can they do? How can they deploy / service this app's ActiveX control safely? At first glance, it may seem like App-V can't be helpful in this situation, since Internet Explorer is locally installed.
It turns out that App-V can be used to help solve this deployment / servicing problem. This topic has already been covered in other articles. The basic technique is as follows:
When the user clicks on the shortcut for the web application, Internet Explorer will launch inside the virtual environment of the ActiveX control and the web application will be fully functional. This approach has a few key benefits:
With the advent of Dynamic Suite Composition in App-V 4.5, this approach has been pushed even further. Now, instead of having Internet Explorer launch in the Virtual Environment of a single package, some customers have started running IE run in a Virtual Environment that contains components from multiple App-V packages. The approach taken is similar to what was done previously, with a couple of additional steps. The basic technique is as follows:
This technique has all of the same benefits of the previous one, but provides additional entitlement and servicing granularity. However, some customers don't like having to publish a separate shortcut for the local application (in this case, Internet Explorer) in order to leverage it. In some cases, they don't want to have to train their users to click on a different shortcut. Or, in the case of complex applications, like Microsoft Office, there are just too many ways that the application can be launched other than through its shortcuts, such as the myriad file type associations that Office publishes. What they're looking for is the ability to express that a locally-installed application should always run inside of its virtual environment, regardless of how it gets launched.
This article will describe how to accomplish this scenario.
If you've ever been curious enough to look at the shortcuts that App-V publishes for virtual applications, you may have noticed something interesting. The shortcuts don't point directly to the app that is going to be launched. Rather, they point to App-V launcher process (sfttray.exe) and pass a parameter that tells the launcher which application should be launched. So, for example, let's say you have virtualized Microsoft Word 2007. If you look at the properties of the Microsoft Word shortcut, you might expect to see a path to the exe, something like: Q:\office\office12\winword.exe. However, instead, you will see something like: "C:\Program Files\Microsoft Application Virtualization Client\sfttray.exe" /launch "Microsoft Office Word 2007". This is the mechanism that App-V uses to ensure that the Virtual Environment for the application gets set up before the process is launched. It is the only supported way for a non-virtual process to launch a process inside of a virtual environment.
So, we need to find a way to have the launch of a local process trigger this mechanism, so that it will come up inside a virtual environment. To do this, we will need to leverage some developer-focused features of Windows and some custom C/C++ code.
To achieve our goal, the first thing we need to be able to do is to intercept local process creation and change what happens during launch. In order to do this, we can leverage a developer-focused feature of Windows called Image File Execution Options. If you're interested in reading more about Image File Execution Options, read this article. Basically, Image File Execution Options allows control over certain aspects of a process' launch. The option we're interested in using is the Debugger option. This option is designed to allow developers to intercept the creation of a process and start the process in a debugging session. However, there is no reason that we can't leverage this mechanism to achieve our non-debugging goals.
Before I discuss how to use the Debugger option, I need to point out one danger with using it. The Debugger option will catch every process launch for a particular binary name (e.g. winword.exe) and replace the launch with another process. So, if I specify replace.exe as the Debugger for original.exe, when I try to launch original.exe, replace.exe will actually get launched instead. This may seem fine, since it is what we want to happen. But, what happens if replace.exe eventually tries to launch original.exe? You guessed it, replace.exe will actually get launched. And, if that instance of replace.exe eventually tries to launch original.exe, replace.exe will get launched again. In programming, this process of something calling itself is called recursion. If it continues without stopping, it is called infinite recursion. As you can probably guess, infinite recursion is usually a very bad thing.
The reason that I bring up the concept of infinite recursion is that naïve usage of Debugger can result in infinite recursion and recovering from this can be a pain. In fact, the simplest way to recover is typically to reboot (there are more advanced ways of recovering, such as using pskill.exe from Sysinternals, but I'm not going to get into that in this article).
There are several ways to avoid this type of infinite recursion in practice. While most of them require you to write custom code, the way I'm going to demonstrate does not require any custom code to be written. Instead, it relies on App-V's Virtual Registry to hide the Debugger value inside the Virtual Environment, thereby breaking the recursion cycle. To do this, we simply add some elements to the OSD file for the primary package that tell App-V to hide Debugger value. Here is a sample that will hide the Debugger value for notepad.exe, which would be stored under the following registry key: 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\notepad.exe'.
<REGKEY HIVE="HKLM" KEY="SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\notepad.exe">
<REGVALUE NAME="Debugger" REGTYPE="REG_SZ"></REGVALUE>
As you can see in this sample, a <REGISTRY> element was added under the existing <VIRTUALENV> element. Then, REGKEY and REGVALUE elements were added to set the appropriate value in the registry. Remember that the key is specific to the name of the binary being launched. So, if you wanted to change the binary in the local registry, you absolutely need to update the OSD file according and make sure that App-V gets receives the update by triggering a publish refresh operation.
Now that we have guarded against the danger of infinite recursion, it is time to set the Debugger value in the local registry. To set this value, you need two pieces of information:
For our example, let's assume that we are trying to intercept notepad.exe and we want it to run inside the Virtual Environment represented by an app named "Virtual Office". To do this, we would create a key in the local registry named: 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\notepad.exe'. Next, we would create a new String Value and name it Debugger. Then we would set the Debugger value to:
sfttray.exe /exe sftlp.exe /launch "Virtual Office"
I'm not going to go into the details of what this command does. But, basically, it launches notepad.exe in the Virtual Environment of the Virtual Office app.
In summary, the steps you need to take to automatically launch a locally-installed app inside a virtual environment are:
Once you have performed these steps, the app will always get created in the appropriate Virtual Environment.