Thoughts about setup and deployment issues, WiX, XNA, the .NET Framework and Visual Studio
All postings are provided AS IS
with no warranties, and confer no rights. Additionally, views expressed
herein are my own and not those of my employer, Microsoft.
The most recent version of the VS 2005 cleanup tool has been posted to this location on the Microsoft Download Center. This version matches the version on my WinISP site as of right now, plus it is digitally signed and posted as an EXE instead of a ZIP file so it does not have to be saved to the desktop and unzipped before you can use it. This version on the Download Center will likely not be updated until VS 2005 releases because it has proven to work correctly in beta 1 and CTP uninstall scenarios. I will update the version on my WinISP site if/when any additional beta 1 or CTP issues are found in the future. We are currently working on updating the uninstall instructions on the VS 2005 site to include a link to this tool and more specific instructions, so stay tuned for that.
5/1 UPDATE: The new uninstall known issues page has been posted. You can view it by clicking this link.
I just got word from a couple of friends that there is a nearly hour-long interview with Jim Allchin (Microsoft group vice president, in charge of Windows among other things) posted up on the Channel9 site. I listened to a bunch of it and there are some really interesting insights. It is a big video (~200 megs) but if you have the bandwidth to handle it I highly recommend giving it a listen. The Channel9 page also has a handy index so you can skip directly to questions that sound most interesting if you want to.
Now that Visual Studio 2005 beta 2 has been released, I have gotten a handful of questions about my blog post from last August explaining why VS setup forces the user to install the J# redistributable package when they don't intend to do J# development in the IDE. In that blog post I explained the reasons behind the behavior in VS .NET 2003 setup and also stated that it was being changed in VS 2005. Some folks have noticed that running VS 2005 beta 2 setup and unchecking the J# language tool item in the setup selection tree still results in setup installing the J# redistributable package. A couple of other folks wondered why there wasn't a separate check-box in the setup selection tree to turn off the J# redistributable package.
Back when I wrote that blog post, the code hadn't yet been written to allow us to make the J# redistributable package an optional component, so I couldn't elaborate on exactly how VS 2005 setup would work with respect to that component because it was only a conceptual design. Now that the code has been implemented, I wanted to explain exactly how VS 2005 setup controls the installation of the J# redistributable package.
First of all, the J# redistributable package is one of a group of components that can be thought of as "feature-dependent chained components." Other examples of this type of component include the VS Tools for Office Runtime and the .NET Compact Framework. All of these components are required prerequisites for specific features that the user can select or deselect in the VS setup selection tree. Because they are required, we decided to not show them as items in the tree themselves or provide checkboxes to allow the user to uncheck them. The only indication setup gives that they will be installed is in the UI on the install progress page.
The tricky part about these feature-dependent chained components is that there is no way to tell in setup UI which feature(s) require the component. In the case of the J# redistributable, most people would expect that it is only needed by the J# language tools and are naturally surprised if they unselect J#, click Install Now and still see the J# redistributable package in the list of components to be installed.
What happens behind the scenes is that setup looks at data listed in the file baseline.dat (which is located in the setup subdirectory on the VS 2005 DVD or CD1). That file has a set of attribute/value pairs for each component that setup might need to install. For example, the J# redistributable package is listed as [gencomp68] in baseline.dat in VS 2005 beta 2. Components that are feature-dependent chained components have their ComponentType value set to 3 and also contain an attribute called DependentFeatures. The value of the DependentFeatures attribute is a semi-colon separated list of GUIDs, and each GUID represents a possible FeatureID in the Feature table of the VS MSI (vs_setup.msi). When the user presses Install Now in VS setup UI, the setup engine processes each of those FeatureID values and determines whether or not they were selected in the setup tree. If at least one of those features was selected, setup will add the feature-dependent chained component to the list of items to be installed.
In the case of the J# redistributable, if you use Orca and manually search for each of the DependentFeature GUIDs in vs_setup.msi, you will see that Visual Web Developer also requires the J# redistributable (because you can create web projects using J# in addition to VB and C#).
In the case of the J# redistributable package, there is also a secondary check in the VS IDE. So if you have J# and/or Visual Web Developer installed and then uninstall the J# redistributable package, you will see an error message indicating that the J# redistributable was not found. Dismissing that will give a secondary error message stating that the Visual J# Language Service Package has failed to load. If you press Yes to this package load failure dialog, the IDE should not attempt to load the Visual J# Language Service Package in the future and you should not see these errors when launching the IDE anymore.
In summary and to make a really long story short, if you uncheck both J# and Visual Web Developer during VS 2005 beta 2 setup, you should not see the J# redistributable package installed.
I have posted bits and pieces of information in different blog posts about the issues we have found on customer machines during the process of removing VS 2005 and .NET Framework 2.0 beta 1 and installing VS 2005 and .NET Framework 2.0 beta 2. I wanted to write an overall summary that explained all of the issues and outlined how and when to use the beta cleanup tool in one place now that things have settled down a little bit and we have discovered most of the major issues and added steps to the cleanup tool to address them.
Information about the VS 2005 beta cleanup tool
We have discovered a case where one of the Visual Studio 2005 beta 2 Package Load Failure error dialogs will happen even after running the beta cleanup tool. If you installed VS 2005 beta 2 and unchecked the item named Team Foundation Client in the setup feature selection tree, you will see an error loading the package named Microsoft.VisualStudio.QualityTools.TestCaseManagement.QualityToolsPackage when launching the VS IDE.
To workaround this issue, you can locate the Visual Studio 2005 Team System Beta 2 - English item in the Add/Remove Programs control panel and launch setup again. Then check the item named Team Foundation Client and update your installation.
Please let me know if you have tried the cleanup tool and also installed the Team Foundation Client item in the setup tree and still see any Package Load Failures when using VS 2005 beta 2.
I've posted an updated version of the VS 2005 and .NET Framework 2.0 cleanup tool that will fix Package Load Failure error dialogs that might occur after uninstalling VS 2005 beta 1 or any of the Community Tech Preview (CTP) versions and then installing VS 2005.
You can click on this link to download the latest version of the VS 2005 beta cleanup tool.
One of the steps performed by this version of the cleanup tool removes some of the native images generated for VS 2005 beta 2. These native images are used to improve the startup time when you launch the VS IDE. You can regenerate these native images by running the following from a cmd prompt:
If you are using VS 2005 beta 2 for web development on a local IIS server, you may see that ASP.NET 2.0 extensions for IIS are missing. In order to resolve this, run the following from a cmd prompt after running the cleanup tool:
If the cleanup tool doesn't work for you
The cleanup tool has been tested by several people inside of Microsoft who had been receiving errors when trying to use the VS 2005 beta 2 IDE. However, I'm still not certain that it will correctly clean up 100% of machines that have previously had beta 1 or a CTP installed. If you try to run the tool and still encounter errors while using VS 2005 beta 2, please try to gather the following data and post a comment on my blog or contact me directly and we will take a look:
Why do these errors happen?
If you are interested, I described the root causes of why certain uninstall/reinstall scenarios will encounter these problems in this blog post.
I got a chance to investigate a couple of machines from Microsoft employees who ran into issues after trying to uninstall VS 2005 and .NET Framework beta 1 and installing VS 2005 and .NET Framework beta 2. The machines I looked at had problems because Package Load Failure dialogs appeared when launching the IDE or creating projects in VS 2005 beta 2. I was able to fix the machines by doing the following;
The most common problem I saw was that there were 2 copies of the file Microsoft.VisualStudio.Shell.Interop.8.0.dll in the GAC, one from beta 1 and one from beta 2. For some reason, the beta 1 version takes precedence when devenv.exe launches and the VS IDE package fails to load.
I am working on compiling a full list of assemblies that exhibit this behavior and once I get that, I will update my cleanup tool to automatically remove these files. However, I probably will not be done until sometime tomorrow so I wanted to post a manual workaround in the meantime for anyone who reads this blog post. Hopefully I'll be able to save some drive reformatting and make beta 2 installation and usage a little less painful.
Technical details if you are interested
In both cases that I investigated today, the .NET Framework beta 1 was uninstalled before VS 2005 beta 1, and that caused VS 2005 beta 1 uninstall to fail to remove some assemblies from the GAC because fusion.dll is used to install/uninstall assemblies and since fusion.dll was installed as a part of .NET Framework 2.0 beta 1, it was removed by the uninstall and was therefore not present to perform the GAC uninstall during VS 2005 beta 1 uninstall.
This uninstall issue should cause all VS assemblies to be orphaned in the GAC. However, installing VS 2005 beta 2 should update the versions of the files in the GAC because they have higher file versions in beta 2 than in beta 1 (since the bug I described here has been fixed). But there is one critical issue that causes a few of the assemblies to not be updated. In the case of the file Microsoft.VisualStudio.Shell.Interop.8.0.dll that I listed above, it contained metadata that identifed it as an MSIL assembly in beta 1, but the MSIL tag was removed for beta 2 for some reason. So Fusion treats it as a completely different assembly and uses a different physical location on disk to store it in the GAC, and as a result the machine ends up with 2 copies of that file. I haven't gotten a complete list of orphaned files yet, but my theory is that all of the other orphaned files are caused by similar metadata changes.
Really in-depth technical details if you are really interested
This problem theoretically exists for VS 2002/.NET 1.0 and VS 2003/.NET 1.1, but there is a design change we took in the .NET Framework 2.0 that makes this problem worse. In 1.0/1.1, you will see the .NET Framework create a folder under %windir%\system32 named UrtTemp that contains copies of a few files that Windows Installer uses to install assemblies. These files are needed because the .NET Framework contains the code needed to install assemblies using the MsiAssembly and MsiAssemblyName tables, but it also needs to install assemblies itself (the classic chicken-and-egg problem).
In 2.0, we wrote a custom action to call fusion APIs directly to install assemblies because of a limitation in Windows Installer that would not let us use the DuplicateFile table if we needed to install files to the file system and the GAC. This liimitation had caused us to carry 2 copies of each assembly in dotnetfx.exe, which caused the download size to be unnecessarily high. In order to reduce size of the .NET Framework we had to use this custom action, which is why you see Assembly and AssemblyName tables in netfx.msi and do not see anything in the MsiAssembly and MsiAssemblyName tables.
Because we implemented this custom action, we were also able to eliminate the part of setup that copied files to UrtTemp. In the past, uninstalling .NET Framework would leave behind the files in UrtTemp so that assembly uninstall could be accomplished by using those copies of the fusion files. However, in 2.0 we don't use UrtTemp so after 2.0 is uninstalled, there are no files left behind to use for future GAC uninstalls. Visual Studio setup does not fail and rollback if GAC uninstall fails because we decided that if a user chooses to uninstall and files are left behind, we should not rollback and add all the files back to the machine. For a released product that is probably OK, but for beta versions, that can cause orphaned files that interfere with future releases of the same product as seen above.
The other complicating factor here is that the .NET Framework is a prerequiste for any application that is written in managed code (such as parts of Visual Studio), but we do not have any way of reference-counting the .NET Framework to prevent the user from uninstalling the .NET Framework out from under any applications that need it. Because of this, even though we know that uninstalling .NET Framework 2.0 beta 1 will cause orphaned files when you try to uninstall VS 2005 beta 2 later on, we cannot prevent the user from performing the uninstall. We have iterated over several proposed designs to create a reference-counting scheme but found too many confusing scenarios where we would pop up dialog boxes strongly suggesting that the user not uninstall, or scenarios where we block uninstall when we should not, or scenarios where 3rd party applications failed to increment reference counts because it requires extra work that was too obscure or difficult. So for 2.0, we will have to live with this type of behavior.
I suppose the ultimate answer will be close to what I always wanted to see - the .NET Framework is a system component that is not uninstallable. You see that on Windows Server 2003 where .NET Framework 1.1 is part of the OS and is always present, and you will likely also see this on Longhorn for .NET 2.0.
Hey all, I'm sorry for the delay posting this. I've got a version of the cleanup tool that I wrote that I specifically tailored to automate that long list of removal steps listed here and here.
You can download and try out the VS 2005 beta removal tool by going to this link.
Note that this tool isn't "officially" supported by Microsoft or me. That being said, please let me know if you have any trouble getting this to work on your computers. My next step is to create a version of this tool that will automate all of the manual removal steps for the .NET Framework 2.0. Stay tuned....
As a side note, I was hoping to have it ready when VS 2005 and .NET Framework 2.0 beta 2 went live on Sunday, but obviously that didn't happen. I apologize to the folks who have contacted me with broken machines and wasted days cleaning them up (and also those who have gotten burned by the manual uninstall but haven't contacted me). I know it is not really any consolation now, but we're really working hard to do what we can to fix any broken machines without forcing you to reformat and also to get any of the underlying setup bugs fixed by the time VS 2005 ships. Thank you for your patience and interest in VS 2005 and .NET Framework 2.0.
I stumbled upon a pretty interesting Eweek article while I was reading some of the internet coverage of the Visual Studio 2005 and .NET Framework 2.0 beta 2 release that happened yesterday evening. The author of the article appears to have worked with some Microsoft folks while researching and writing the article, and based on the first few paragraphs it sounds like he attended at least one of the shiproom meetings leading up to the VS 2005 beta 2 release. When I was on the VS and .NET Framework setup team, I attended shiproom towards the end of major shipping milestones for the VS 2002/.NET 1.0 and VS 2003/.NET 1.1 releases, and I can say that the impressions in the article are pretty accurate. The shiproom meetings are not populated exclusively by developers as the article suggests. There are representatives from program management, test, localization, user assistance, and other disciplines. Actually I found that developers are sort of in the minority at these meetings.
Most of the times when I attended shiproom, my role was to present the technical details behind the bugs in our setup features that we wanted to fix before the products shipped. This can be challenging because you have to be able to rapidly get to the point and bring 50 people up to speed who have no knowledge of your features and explain what the defect is, why it was not found until now, what the customer impact is of fixing it (or not fixing it), what the fix is (sometimes down to the exact lines of code, particularly as it gets closer to the final day before sign-off), and the risk that making the fix presents in terms of causing regressions in other known working code. Not only this, but you have to be prepared for basically any random question that might come up from anybody in the room while you explain any of the above. And the tricky part of this is that you need to focus on keeping the focus of the conversation headed in the right direction because Microsoft folks seem to have a tendency to "rat-hole" or drill down really deeply to discuss a sub-point that is really minor in the overall scope of the issue being discussed.
From my experience in shiproom, I found that it became pretty easy to pick out the bugs that the people presenting them did not really understand well. It also became pretty easy to pick out the people who went into too much depth and quickly lost track of the bigger picture and couldn't explain customer impact to justify fixing a bug. This tended to happen more to developers than other disciplines because they spend a lot of their time in the details of the code rather than looking at end-to-end customer scenarios, but everyone was susceptible, particularly those new to presenting at shiproom).
The other interesting dynamic that I saw emerge in shiproom was that it appeared to me that achieving a leadership role in the meetings worked as a kind of meritocracy. Basically, if you cared enough to show up regularly, and then showed good insights in the questions you asked or opinions you offered, others in the room would recognize this and start looking to you to step up and continue to offer opinions and help determine whether to accept or reject bug fixes. Ultimately, there always needs to be a "moderator" (in the words of the author of the article) because sometimes discussions will get particularly heated or hopelessly off-track or folks just won't agree on an issue and the buck has to stop somewhere. But it is really cool to see that individuals really can make a difference in the overall shipping process and it is also really cool to see that not just anyone can, but those that do really earn that privilege.
Of course, all of the above is just my impression of the process and it may be somewhat different for VS 2005 since I haven't been in that group in a while. It also probably varies somewhat from product to product. But I thought it might be interesting for folks out there to see a bit of detail about how the process works.
There is also a blog that the Developer Division Release Team maintains that might be interesting for more insights into the process of shipping VS and the .NET Framework. The lead of that team is the "moderator" of shiproom and I've worked closely with most of the folks on that team.
I know I'm a little late jumping on this particular bandwagon and posting a blog item about it, but just in case you haven't seen yet, Visual Studio 2005 and .NET Framework 2.0 beta 2 are now available for download. You can find more info at the VS 2005 Developer Center. It looks like you can order a DVD for the Visual Studio Team System beta 2, but the VS 2005 Express Edition beta 2 versions can be downloaded from the developer center. Also, if you are only looking for the .NET Framework redist package and/or SDK, you can download them at this .NET Framework site.
Another interesting item listed on the beta 2 download site is a Go Live license that lets you use VS 2005 beta 2 and deploy your applications in a production environment in some cases. Check out this link for more info.
One thing I want to mention again, and I can't stress this enough - I really hope that you'll provide any feedback you have about these beta products (good or bad) at the MSDN Product Feedback Site. I have seen the process for what happens to the issues reported to that site, and they all get reported to the product team's bug tracking tool. And best of all, they all get looked at by real members of the teams. So please let us know how things are looking and report bugs that we need to fix before VS 2005 officially ships or for future versions.
Here are some useful links for setup problems you might encounter and some other known issues in beta 2:
Now that Visual Studio 2005 and .NET Framework 2.0 beta 2 is about to be available, I wanted to communicate a couple of issues we have seen related to uninstalling previous beta and Community Tech Preview (CTP) versions that can cause problems when you try to install beta 2.
Uninstall should be done in the reverse order of installation
I'm sure you've noticed the large number of entries that are added to your Add or Remove Programs control panel after installing Visual Studio. There are a lot of sub-products being installed silently as a part of Visual Studio setup and there are some specific ordering issues that are enforced during install because Visual Studio manages the chaining of the installation. But the ordering is not enforced during uninstall. The biggest problem we have seen is that uninstalling the .NET Framework before trying to uninstall Visual Studio, the J# redistributable, MSDN, or SQL Express will cause those other products to leave behind files in the GAC. This is because the .NET Framework is the ship vehicle for Fusion, which is the technology that manages GAC install/uninstall. So, in general the safest way to uninstall is to treat the list of products as a computer science stack and uninstall them in the reverse order they were installed. Hong Gao, a program manager on the setup team, has posted sets of instructions on her blog for the Visual Studio 2005 Express SKUs and for other Visual Studio 2005 SKUs that I strongly recommend you follow.
Update - Whidbey Beta 2 is now available, and there is now a more official list of uninstall steps and the correct order to uninstall in. Click here to see the uninstall list/order.
Uninstall for previous versions of the .NET Framework 2.0 may not be completely clean
Most often, the way you will notice this issue is if you see a failure while installing a future version of the .NET Framework 2.0 after uninstalling a previous version. You can find a detailed explanation for how to workaround this issue in my previous blog post.
After installing Visual Studio, the IDE may fail to launch or you may see multiple Package Load Failure error dialogs
This is caused by some known issues in previous beta and CTP versions related to installing native Win32 assemblies to the %windir%\WinSxS cache. To workaround these issues please try the following steps:
As always, please let me know if any of these workarounds do not work for you. Also, I strongly encourage you to use the MSDN Product Feedback site to report any bugs that you find with uninstalling previous versions or installing new versions. I'll update my blog with other common issues and suggested workarounds as they arise.
In response to the blog article I posted last week where I provided a link to a .NET Framework manual cleanup tool, I got some questions about whether or not a comparable version is available for cleaning up the .NET Framework 2.0. I am currently working on a couple of small work items in the code for the tool to enable it to work with 2.0, but in the meantime I wanted to post some manual steps. I know there have been a lot of uninstall/reinstall issues because we have released an alpha, a beta and numerous Community Tech Preview (CTP) versions and not all of them will uninstall completely cleanly in order to allow a future beta version of 2.0 to install correctly.
The following steps will help resolve .NET Framework 2.0 installation failures/hangs in most cases. Before proceeding please note these important caveats:
Steps to clean up a machine to fix a failed .NET Framework 2.0 installation:
If you have any trouble getting these steps to work correctly please let me know. Also stay tuned for a future post once I get the cleanup tool updated to work with .NET Framework 2.0 and post it for download.
I wrote an application late last year that is designed to clean up computers that have problems getting the .NET Framework 1.0 or 1.1 to install correctly. I have been working on refining the tool for the past couple of months, working out some bugs, adding additional cleanup features, etc. The .NET Framework setup Product Support team has been using this cleanup tool for the past few months to help resolve many cases, and the internal Microsoft helpdesk has also started using it to solve internal cases where employees cannot get .NET Framework service packs or hotfixes to install correctly. I have also been sending this tool out to individuals who email me via my blog and ask for help resolving setup problems - most commonly this is because of issues installing .NET Framework service packs or hotfixes such as MS05-004.
Since I have been seeing really good success rates for using this cleanup tool and it has proven to speed up the process of resolving issues so customers can get the .NET Framework installed correctly and start using managed code on their computers, I decided to try to get a KB article written up with a copy of the tool that customers could download on their own without needing to contact me directly or call our PSS team. The KB publishing process can sometimes take a while with technical reviews and things like that, so in the meantime I am going to post a link to the tool here on my blog.
You can download the tool by visiting the .NET Framework Cleanup Tool User's Guide and using one of the download links listed there.
There are a couple of very important caveats that you should read before using this tool to cleanup .NET Framework bits on your machine:
The tool itself has been fairly well tested, but I'm sure it is still not perfect. I'm still in the process of fixing bugs as I find them and adding features to make it more effective at cleaning up known issues and to make it more intelligent about identifying root causes so we can fix the underlying bugs in .NET Framework setup for future releases. As I update it, I will post updates to my blogs and update the copy of the tool located at the link above.
I hope this tool will be helpful in resolving problems installing the .NET Framework. Please let me know if you run into any issues while using the cleanup tool or if you are still unable to install the .NET Framework (or any service packs or hotfixes) after running it.
<update date="3/3/2009"> Added a link to the .NET Framework Cleanup Tool User's Guide, which contains the most up-to-date download location for this tool. <update>
Ever since I posted my original instructions for chaining unattended installation of the Visual Studio .NET Prerequisites, Visual Studio and MSDN, I have had a few customers who tried these steps out and had problems getting the steps to work. So as a result I want to write a new post where I give better detail and clarify each of the steps to make this unattended installation process easier to understand and implement.
To start with you should stage Visual Studio bits to a network share so that you can use this as your installation source later on. You can accomplish that with the following steps (also described in the VS readme):
Now that you have a network image, you can create the unattended INI files to install VS Prerequisites, VS and MSDN using the following steps:
There are a couple of gotchas that I have seen that you should keep an eye out for when following these instructions:
There are a couple of advanced tricks that you can use when creating these unattend scripts as well:
As always, let me know if you have questions or problems with any of the above steps....
If you have ever installed the .NET Framework 1.0 or 1.1 package by running dotnetfx.exe directly (as opposed to installing it silently via Windows Update or via an application that redistributes it), I'm sure you've seen strange behavior where it appears to hang for several minutes right before setup completes. Normally when this happens the progress bar is full or very nearly full and the time remaining says "0 seconds" (or possibly it hits one of my favorite Windows Installer bugs and says "1 seconds").
During this time, setup is not actually hung. The .NET Framework setup is running several custom actions to launch ngen.exe to generate native images (also called pre-jitting) for several .NET Framework assemblies. Depending on the speed of the machine that setup is running on, this process can take several minutes.
Unfortunately we also shipped with a couple of problems in the .NET Framework setup package that prevented us from updating progress while this was going on:
So the bad news is that setup sometimes looks hung towards the end for the .NET Framework 1.0 and 1.1. The good news is that there was some custom action work done specifically around how we do native image generation in the .NET Framework 2.0 and partially as a result of this, progress UI now looks and behaves interactively for the entire length of setup and doesn't appear hung at the end anymore.
Ironically, when we install the .NET Framework as part of Visual Studio setup, we are able to give a more realistic sense of progress because of the "fake progress" strategy we implemented there.