VS 2008 SP1 fails to install because of missing packages from the cache

Some customers are reporting that Microsoft Visual Studio 2008 SP1 is failing to install with the following error in the HTML log you can view from the error dialog.

Patch (C:\Users\heaths\AppData\Local\TEMP\WebDesignerCore_KB950278.msp) install failed on product (Microsoft Office Enterprise 2007). Msi Log: Microsoft Visual Studio 2008 SP1_20080816_141317516-Microsoft Office Enterprise 2007-MSP0.txt

Final Result: Installation failed with error code: (0x80070663), This update package could not be opened. Verify that the update package exists and that you can access it, or contact the application vendor to verify that this is a valid Windows Installer update package.

Workaround

Most often this happens because a patch package is missing from the Windows Installer cache and is required to install a new patch. In this case, the Windows Installer cache must be rebuilt.

While the method to determine which packages have to be restored is fairly generic, these following instructions are specific to Office to serve as an example and because this seems to be the most common case given the likely cause described in the next section, Prevention, and the size of Office 2007 SP1 patch packages.

1. Click on the link to the log file right above the line beginning with "Final Result". A file should open where the majority of lines begin with "MSI".
2. Search for "SOURCEMGMT" (without quotes). A few lines above it should note the missing package. For example (highlight added for emphasis):
   Couldn't find local patch 'C:\WINDOWS\Installer\3bb98.msp'. Looking for it at its source.
3. Search ahead from that point for "2203" or "1706" (without quotes). Note the file name within that line. For example (highlight added for emphasis):
   Note: 1: 2203 2: C:\Temp\OWP15A.tmp\MAINWWsp1.msp 3: -2147287037
4. In this example, MAINWWsp1.msp is part of Office 2007 SP1. If you find a different file name, search the web for any references to that file to determine what to download. Be sure you only download packages from their original manufacturers' web sites. In this example, download http://www.microsoft.com/downloads/details.aspx?FamilyID=9EC51594-992C-4165-A997-25DA01F388F5 to %TEMP%.
5. Open an elevated command prompt a change directories to %TEMP%:
   cd %TEMP%
6. Extract the contents of the executable package; note that the file name will differ based on the language you downloaded:
   office2007sp1-kb936982-fullfile-en-us.exe /extract:"%TEMP%\office_2007_sp1"
7. Copy MAINWWsp1.msp to the location where Windows Installer was originally looking. In this example above, that location is C:\WINDOWS\Installer\3bb98.msp. Remember that this location will vary so you need to check your log file as described above.
   copy "%TEMP%\office_2007_sp1\MAINWWsp1.msp C:\WINDOWS\Installer\3bb98.msp

Now try installing VS2008 SP1 again.

Prevention

While there are potentially other reasons packages might be missing from the Windows Installer cache in %WINDIR%\Installer, the most common cause is when those files are deleted to free up disk space. Those files directly under %WINDIR%\Installer are required for Windows Installer to function properly for all maintenance installations, which is everything after initial install including patch install, patch uninstall, repairs, changes to feature selection, and even product uninstalls. These files must not be removed.

If you need to free disk space, be sure to run the Disk Cleanup utility in your your Accessories / System Tools programs folder first. You can also uninstall old and unneeded applications.

Because Windows Installer may require a lot of space on your system drive, be sure to partition your hard disk with plenty of space not just for Windows or Windows Installer, but also because a lot of applications may install to Program Files or Common Files which are, by default, on your system drive.

Details

Windows Installer will always open every patch registered to a product whether or not it has already been obsolesced or superseded when opening a product or package handle (as long as machine state is not ignored). When performing a maintenance installation, a product is opened and all patches registered to that product are opened. If one or more of those patch packages is missing from the Windows Installer cache, Windows Installer attempts to resolve the location as you can see from the log below.

MSI (s) (D4:98) [14:17:19:688]: Opening existing patch 'C:\WINDOWS\Installer\3bb98.msp'.
MSI (s) (D4:98) [14:17:19:688]: Note: 1: 2203 2: C:\WINDOWS\Installer\3bb98.msp 3: -2147287038
MSI (s) (D4:98) [14:17:19:688]: Couldn't find local patch 'C:\WINDOWS\Installer\3bb98.msp'. Looking for it at its source.
MSI (s) (D4:98) [14:17:19:688]: Resolving Patch source.
MSI (s) (D4:98) [14:17:19:688]: User policy value 'SearchOrder' is 'nmu'
MSI (s) (D4:98) [14:17:19:688]: User policy value 'DisableMedia' is 0
MSI (s) (D4:98) [14:17:19:688]: Machine policy value 'AllowLockdownMedia' is 0
MSI (s) (D4:98) [14:17:19:688]: SOURCEMGMT: Media enabled only if package is safe.
MSI (s) (D4:98) [14:17:19:688]: SOURCEMGMT: Looking for sourcelist for product {BEE75E01-DD3F-4D5F-B96C-609E6538D419}
MSI (s) (D4:98) [14:17:19:688]: SOURCEMGMT: Adding {BEE75E01-DD3F-4D5F-B96C-609E6538D419}; to potential sourcelist list (pcode;disk;relpath).
MSI (s) (D4:98) [14:17:19:688]: SOURCEMGMT: Now checking product {BEE75E01-DD3F-4D5F-B96C-609E6538D419}
MSI (s) (D4:98) [14:17:19:704]: SOURCEMGMT: Media is enabled for product.
MSI (s) (D4:98) [14:17:19:720]: SOURCEMGMT: Attempting to use LastUsedSource from source list.
MSI (s) (D4:98) [14:17:19:720]: SOURCEMGMT: Trying source C:\Temp\OWP15A.tmp\.
MSI (s) (D4:98) [14:17:19:720]: Note: 1: 2203 2: C:\Temp\OWP15A.tmp\MAINWWsp1.msp 3: -2147287037
MSI (s) (D4:98) [14:17:19:720]: SOURCEMGMT: Source is invalid due to missing/inaccessible package.
MSI (s) (D4:98) [14:17:19:720]: Note: 1: 1706 2: -2147483647 3: MAINWWsp1.msp
MSI (s) (D4:98) [14:17:19:720]: SOURCEMGMT: Processing net source list.
MSI (s) (D4:98) [14:17:19:720]: Note: 1: 1706 2: -2147483647 3: MAINWWsp1.msp
MSI (s) (D4:98) [14:17:19:720]: SOURCEMGMT: Processing media source list.
MSI (s) (D4:98) [14:17:25:470]: SOURCEMGMT: Resolved source to: 'MAINWWsp1.msp'
MSI (s) (D4:98) [14:18:57:126]: Note: 1: 1314 2: MAINWWsp1.msp
MSI (s) (D4:98) [14:18:57:126]: Unable to create a temp copy of patch 'MAINWWsp1.msp'.
MSI (s) (D4:98) [14:18:57:141]: Note: 1: 1708
MSI (s) (D4:98) [14:18:57:141]: Note: 1: 2729

Failing to resolve the source location for the patch, Windows Installer returns error code 1635 (ERROR_PATCH_PACKAGE_OPEN_FAILED).

Often the simple solution is to find and restore the package to where Windows Installer expected to find it in the first place: back in the Windows Installer cache under %WINDIR%\Installer. You could also put it in the other locations where you see "SOURCEMGMT: Trying source".

All files directly under %WINDIR%\Installer are required. To free up additional space as mentioned in why Windows Installer may require so much space, you can delete subdirectories under or even all of %WINDIR%\Installer\$PatchCache$, or even disable it for future patch installations but at the cost of prompts for original source in many scenarios. The baseline cache files are intended to provide a backup of baseline (ex: RTM) files so that when they are needed they are available. They are needed when applying patches containing binary delta patches to individual files (as opposed to whole-file updates, which are more common) or when uninstalling patches.

You can also try finding and uninstalling superseded patches. Superseded patches are not actually uninstalled when superseded - they just become inactive. This way when all superseding patches are removed with respect to these superseded patches, those superseded patches become active again. You can even install patches in an initially superseded state for this very purpose. I do have an example of detecting superseded patches using PowerShell with the Windows Installer PowerShell Extensions installed but there are other ways by calling the MsiEnumPatchesEx() function or the PatchesEx method for script automation in your own applications or scripts.