I have been working on the .NET Framework setup since before v1.0 shipped, and in that time I've seen all sorts of bugs and setup failures.  There is one particularly tricky set of bugs that all show the same symptom but that we haven't been able to track down a common root cause for.  The error that is seen during setup is a failure to register System.EnterpriseServices.dll or a problem running RegSvcs.exe /bootstrapi, which then causes setup to stop and rollback.

Almost none of the cases I have seen where this error appears were actually caused by the file System.EnterpriseServices.dll itself.  It is a bit misleading because the registration of this file is done by a custom action during setup, and that custom action is the first time that the version of the CLR we are trying to install is used to run managed code on the computer.  So that means that if anything is wrong on the machine to cause that version of the CLR to not work properly, the error the user sees is that this DLL failed to register.

When we encounter an error like this, we typically try the following cleanup and troubleshooting steps in the order listed below.

1. Download the .NET Framework 1.0 and 1.1 cleanup tool and run it
2. Verify that the data in the Locale registry value located at HKEY_USERS\.Default\Control Panel\International is valid.  The Locale value should contain one of the values listed at HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Nls\LocaleMapIDs
3. Get a crash dump with Windbg on the custom action that registers System.EnterpriseServices.dll

In many cases, running the cleanup tool is sufficient, but every once in a while we run into a computer in a state that we can't diagnose and we have to try some more advanced debugging techniques like those listed in steps 2 and 3.

We use the following steps to configure the MsiBreak environment variable to cause .NET Framework setup to pause at the beginning of the custom action that registers System.EnterpriseServices.dll, attach a debugger and gather crash dump information:

1. Download and install the Debugging Tools for Windows
2. Right-click My Computer, and then click Properties.
3. On the Advanced tab, click Environment Variables, and then under System Variables, click New.
   Note: You must add MsiBreak to the list of System Variables (and not to the list of User Variables).
4. In the Variable Name field, enter "MsiBreak" (without the quotation marks).
5. In the Variable Value field, enter the name of custom action you want to break on.  The name must match the name listed in the Action column of the CustomAction table of the MSI:
   For .NET Framework 1.0 or 1.1 setup: CA_ComregEnterpriseServices.3643236F_FC70_11D3_A536_0090278A1BB8
   For .NET Framework 2.0 setup: DD_CA_ComregEnterpriseServices_X86.3643236F_FC70_11D3_A536_0090278A1BB8
6. Run .NET Framework setup and start installing
7. During installation, a break message will appear stating "To debug your custom action, attach your debugger to process XXX (0xZZZ) and press OK".  Open WinDBG instead of following the instructions on the dialog that pops up.  If you are running on Windows Vista, you will need to launch WinDBG with elevated privileges.  To do that, right-click on WinDBG.exe and choose Run as administrator.
8. From the File menu, choose Open Executable...
9. In the File name text box, enter the following information:
   For .NET Framework 1.0 setup: %windir%\Microsoft.NET\Framework\v1.0.3705\regasm.exe
   For .NET Framework 1.1 setup: %windir%\Microsoft.NET\Framework\v1.1.4322\regsvcs.exe
   For .NET Framework 2.0 setup: %windir%\Microsoft.NET\Framework\v2.0.50215\regsvcs.exe (or change the build number depending on what build of .NET Framework 2.0 is being installed)
10. In the Arguments text box, enter the following information:
    For .NET Framework 1.0 setup: %windir%\Microsoft.NET\Framework\v1.0.3705\System.EnterpriseServices.dll
    For .NET Framework 1.1 or 2.0 setup: /bootstrapi
11. After entering the above information, choose Yes to save base workspace information
12. Press g in the Windbg window to resume execution until the process crashes, then save the crash log to a file

Variations of the above steps can also be used to gather debugging information for any custom action that fails during an MSI-based setup as long as you are able to consistently reproduce the crash during setup.  You just need to substitute the appropriate custom action name in step 5 (by looking at the verbose MSI log file or looking at the MSI in Orca to get the exact name).  You also need to substitute the custom action name and command line parameters in steps 9 and 10 above.  In the case of this example, the custom actions are executable files that are installed as a part of .NET Framework setup.  If the custom action is a file streamed into the binary table of the MSI, I believe you will need to extract the file from the MSI and run it using rundll32.exe or something like that (but I haven't yet tried to do that to verify that it works like I think it will).

<update date="9/25/2006"> Added a note about running WinDBG with elevated privileges on Windows Vista </update>