Image File Execution Options
There is well-known (or not so well-known, depending on what you do) feature in NT family system, called “Image File Execution Options”.
It is really in the registry
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options
Create a sub key, with the name as your executable without the path (i.e. notepad.exe). When your application starts, OS will look for specific registry values under that reg key, and act accordingly.
Regmon (http://www.sysinternals.com/ntw2k/source/regmon.shtml) trace of a helloworld app shows the following registry value is queried:
Debugger,
DisableHeapLookaside,
ShutdownFlags,
MinimumStackCommitInBytes,
ExecuteOptions,
GlobalFlag,
DebugProcessHeapOnly,
ApplicationGoo,
RpcThreadPoolThrottle,
“Debugger” is discussed in many articles, like this one http://support.microsoft.com/default.aspx?kbid=238788. It is a way to automatically launch a debugger when an application starts.
“DisableHeapLookaside” is discussed here http://support.microsoft.com/default.aspx?scid=http://support.microsoft.com:80/support/kb/articles/Q195/0/09.ASP&NoWebContent=1&NoWebContent=1. A way to fall back to ancient heap manager.
“ShutdownFlags” is discussed here http://msdn.microsoft.com/library/default.asp?url=/library/en-us/appendix/hh/appendix/enhancements5_5ppv.asp. A way to detect heap leak.
“RpcThreadPoolThrottle” is discussed here http://support.microsoft.com/default.aspx?scid=%2Fservicedesks%2Fbin%2Fkbsearch.asp%3FArticle%3D267255.
“GlobalFlag” is controlled by a tool called gflags.exe, which is documented in MSDN http://msdn.microsoft.com/library/default.asp?url=/library/en-us/ddtools/hh/ddtools/gflags_00s3.asp. It is bundled with windows debugger (http://www.microsoft.com/whdc/devtools/debugging/default.mspx), which in my opinion, the best debugger ever created.
If you play with gflags.exe more, you will found more interesting registry values under Image File Execution Options.
I can’t find anything for “ApplicationGoo”. But you can see that in a vanilla WindowsXP system, several reg keys are already presented in Image File Execution Options with ApplicationGoo set to random binaries. I suspect this is something to do with AppCompat.
I can’t find anything for “MinimumStackCommitInBytes”, “ExecuteOptions” and “DebugProcessHeapOnly”. But it is very easy to decipher their meaning from their name (except ExecuteOptions, which I suspect is yet another AppCompat goo).
There is another one “BreakOnDllLoad”, which is discussed here http://www.west-wind.com/presentations/iis5Debug.htm. But with the “sxe ld” command in windows debugger, I think it is much less useful. Of course if you are using something other than windows debugger, it will help you to determine why (and when) a certain dll is loaded.
