Notes on comments.
Welcome to our blog dedicated to the engineering of Microsoft Windows 7
This post is about disk space and the disk space “consumed” by Windows 7. Disk space is the sort of thing where everyone wants to use less, but the cost of using a bit more relative to the benefits has generally been a positive tradeoff. Things have changed recently with the availability of solid-state drives in capacities significantly smaller than the trend in spinning drives. Traditionally most all software, including Windows, would not hesitate to consume a 100MB on a specific (justified) need when looking at a 60GB (or 1,500GB) drive; with desirable machines shipping with 16GB of solid-state storage, we are looking carefully at the disk space used by Windows—both at setup time and also as a PC “ages”. We also had a specific session at WinHEC on solid-state drives that might be interesting to folks. This post is authored by Michael Beck, a program manager in the core OS deployment feature team. --Steven
Let’s talk about “footprint”. For the purposes of this post, when I say “footprint” I’m talking about the total amount of physical disk space used by Windows. This includes not only the Windows binaries, but all disk space consumed or reserved for system operations. Later in this entry, I’ll discuss in detail how the disk footprint is consumed by various Windows technologies.
A number of comments have asked about disk footprint and what to expect in terms of Windows 7’s usage of disk space. Like many of the design issues we have talked about, disk space is also one where there are tradeoffs involved so this post goes into the details of some of those tradeoffs and also discusses some of the feedback we have received. It should be noted, that we are not at the point where we are committing to system requirements for Windows 7, so consider this background and engineering focus.
To structure this post we’ll take two important points of feedback or questions we have received:
We’ll then talk about the focus and engineering of Windows 7.
We definitely get a lot of questions about the new (to Vista) Windows SxS directory (%System Root%\winsxs) and many folks believe this is a big consumer of disk space as just bringing up the properties on a newly installed system shows over 3000 files and over 3.5 GB of disk consumed. Over time this directory grows to even higher numbers. Yikes--below is an example from a Steven's home PC.
“Modularizing” the operating system was an engineering goal in Windows Vista. This was to solve a number of issues in legacy Windows related to installation, servicing and reliability. The Windows SxS directory represents the “installation and servicing state” of all system components. But in reality it doesn’t actually consume as much disk space as it appears when using the built-in tools (DIR and Explorer) to measure disk space used. The fact that we make it tricky for you to know how much space is actually consumed in a directory is definitely a fair point!
In practice, nearly every file in the WinSxS directory is a “hard link” to the physical files elsewhere on the system—meaning that the files are not actually in this directory. For instance in the WinSxS there might be a file called advapi32.dll that takes up >700K however what’s being reported is a hard link to the actual file that lives in the Windows\System32, and it will be counted twice (or more) when simply looking at the individual directories from Windows Explorer.
The value of this is that the servicing platform (the tools that deliver patches and service packs) in Windows can query the WinSxS directory to determine a number of key details about the state of the system, like what’s installed, or available to be installed (optional components, more on those later), what versions, and what updates are on the system to help determine applicability of Windows patches to your specific system. This functionality gives us increased servicing reliability and performance, and supports future engineering efforts providing additional system layering and great configurability.
The WinSxS directory also enables offline servicing, and makes Windows Vista “safe for imaging”. Prior to Windows Vista, inbox deployment support was through “Setup” only. IT professionals would install a single system, and then leverage any number of 3rd party tools to capture the installed state as a general image they then deployed to multiple systems. Windows wasn’t built to be “image aware”. This meant that greater than 80% of systems were deployed and serviced using a technology that wasn’t supported natively, and required IT departments to create custom solutions to deploy and manage Windows effectively. In addition, state stored in the WinSxS directory can be queried “offline”, meaning the image doesn’t have to be booted or running, and patches can be applied to it. These two features of WinSxS give great flexibility and cost reductions to IT departments who deploy Windows Vista, making it easier to create and then service standard corporate images offline.
While it’s true that WinSxS does consume some disk space by simply existing, and there are a number of metadata files, folders, manifests, and catalogs in it, it’s significantly smaller than reported. The actual amount of storage consumed varies, but on a typical system it is about 400MB. While that is not small, we think the robustness provided for servicing is a reasonable tradeoff.
So why does the shell report hard links the way it does? Hard links work to optimize disk footprint for duplicate files all over the system. Application developers can use this functionality to optimize the disk consumption of their applications as well. It’s critical that any path expected by an application appear as a physical file in the file system to support the appropriate loading of the actual file. In this case, the shell is just another application reporting on the files it sees. As a result of this confusion and a desire to reduce disk footprint, many folks have endeavored to just delete this directory to save space.
There have been several blogs and even some “underground” tools that tell you it’s ok to delete the WinSxS directory, and it’s certainly true that after installation, you can remove it from the system and it will appear that the system boots and runs fine. But as described above, this is a very bad practice, as you’re removing the ability to reliably service, all operating system components and the ability to update or configure optional components on your system. Windows Vista only supports the WinSxS directory on the physical drive in its originally installed location. The risks far outweigh the gains removing it or relocating it from the system, given the data described above.
As we all know adding new functionality consumes additional disk space--in Windows or any software. In reality, “code” takes up a relatively small percentage of the overall Windows footprint. The actual code required for a Windows Vista Ultimate install is just over 2GB, with the rest of the footprint going to “data” broadly defined. Let’s dig deeper into the use of storage in a Windows Vista installation and what we mean by "data".
Reliability and security were core considerations during the engineering process that built Windows Vista. Much of the growth in footprint comes from a number of core reliability features that users depend on for system recovery, performance, data protection, and troubleshooting. Some of these include system restore, hibernation, page file, registry back up, and logging. Each of these represent “backup state” that is available to the system to recover from any number of situations, some planned and others not. Because we know that different customers will want to make different tradeoffs of disk space relative to recovery (especially on small footprint devices) with Windows 7 we want to make sure you have more control than you currently do to decide ahead of time how much disk space to use for these mechanisms, and we will also tune our defaults to be more sensitive to overall consumption due to the changing nature of storage.
System restore and hibernation are features that help users to confidently recover their system and prevent data loss, in a number of situations such as low battery (hibernation), bad application installation or other machine corruption (system restore). Combined, these features consume a large percentage of the footprint. Because of the amount of space these use, they are easy to identify and make decisions regarding.
System restore protects users by taking snapshots of the system prior to changes and on regular intervals. In Windows Vista, system restore, is configured to consume 300mb minimally, and up to 15% of the physical disk. As the amount of space fills up with restore points, System Restore will delete older restore points to make room for new ones. The more space you have, the greater the number of restore points you have available to “roll back” to. We have definitely heard the feedback from Windows Vista customers around system restore and recognize that the it takes significant space and is not easy to tune. Some have already seen the pre-beta for Windows 7 provides an interface to manage the space better.
Hibernate is primarily used on mobile PCs and saves your work to the hard disk and puts the computer in an extremely low power state. Hibernate is used on mobile PCs when the battery drains below a certain threshold or when turning the computer off without using Shut Down to extend battery life as much as possible. On Windows Vista, Hibernate is also automatically used with Sleep on desktop PCs to keep a backup copy of open programs and work. This feature is called Hybrid Sleep and is used to save state to the hard disk in case power fails while the computer is sleeping. Hibernate writes all of the content in memory (RAM) to a file on the hard drive named Hiberfil.sys. Therefore, the size of the reserved Hiberfil.sys is equal to the amount of RAM in the machine. In the Windows Vista timeframe, the amount of RAM being built into computers has increased significantly, thus the disk footprint of Hibernate is more noticeable than before. This space must be reserved up front to guarantee that in a critical low battery situation, the system can easily write memory contents to the disk. Any mobile PC user that has experienced their computer automatically entering Hibernate when the battery is critically low can appreciate the peace of mind this footprint growth provides. While we're talking about RAM and disk footprint in the same paragraph, Mark Russinovich has a post this week on virtual memory and how big the swapfile could/should/can be that you might find interesting.
Now it’s clear that in the description above, I don’t account for the entire footprint required by Windows Vista. For instance, we also include many sample files, videos, high resolution backgrounds that allow users to easily customize their experience, and try out new features, but we’ve covered a couple of the more common questions out there.
It’s important that we consider more than just the size of the system once deployed, but we must also look at how the system grows over time as services write logs, updates, and service packs are installed, system snapshots are taken etc. For many, the “growth” over time of the installation proves to be the most perplexing—and we hear that and need to do better to (a) make smarter choices and (b) make it clearer what space is being consumed and can be reclaimed.
The following table provides one view of the installation footprint of a Windows Vista Premium/Ultimate installation. This includes the full installation, but to make it digestible this has been broken down into some logical categories and also highlights some specific features. Part of the reason to highlight specific feature is to illustrate the “costs” for items that have been raised as questions (or questionable).
Here are some items worth calling out:
Windows disk space consumption has trended larger over time. While not desirable, the degree to which it’s been allowed is due in large part to ever-increasing hard drive capacity, combined with a customer need and engineering focus that focused heavily on recoverability, data protection, increasing breadth of device support, and demand for innovative new features. However, the proliferation of Solid State Drives (SSDs) has challenged this trend, and is pushing us to consider disk footprint in a much more thoughtful way and take that into account for Windows 7.
This doesn’t mean that we’re going to stop adding great features or make Windows less reliable or recoverable. As we look to the future, it’s critical that as we innovate, we do so treating the disk space consumed by our work as a valuable resource, and have a clearer design for how Windows uses it. We want to make sure that we are making smart choices for the vast majority of customers and for those desiring more control providing places to fine tune these choices as appropriate. This design goal isn’t about a type of machine, or specific design, all Windows editions benefit from efforts that focus on a reduction of the overall footprint.
For example, as we consider the driver support discussed above, Windows Vista with SP1 installs almost 1GB of drivers on the system to support plug and play of devices. This local cache can get out of date as IHVs release updates to their drivers, and as a result, users are pushed to Windows update to get the latest version once the device is installed.
Why not extend the PnP user experience to include (or only use) the Windows Update cache of drivers and save some disk space? This has several benefits:
With this example it’s easy to see how engineering for a minimal footprint might actually deliver a better experience for people when attaching new devices to their systems. At the same time, we want to be careful about going too far too soon. We get a tremendous amount of feedback regarding the “plug and play” experience or feedback about costly download times (if download is at all possible). For Windows 7 we are going to continue to be deliberate in what we include based on the telemetry of real world devices and reducing the inbox set to cover the most popular devices around the world. At the same time we will continue a very significant effort around having the best available Windows Update site for all devices we can possibly support.
Windows features installed by default make sense in most cases to support many scenarios. We should consider how we make some features/components (such as Media Center) optional when they are not required rather than installing them by default on every system. We’re committed to make more features of Windows optionally installed. As you might notice today in Windows, when you choose to add a feature that was not installed Windows does not require a source (a DVD or network location). This is because the feature is stashed away as part of a complete Windows install—this is itself a feature. We will always keep features available and will always service them even when components are not installed—that way if you add a component later you do not risk adding a piece of code that might have been exploited earlier. This is another important way we keep Windows up to date and secure, even for optional features.
System growth over time is an area where we need to provide more “transparency”. For instance, Windows will archive previous versions of updated system components to allow robust rollback. A new system will install patches as Windows Update makes them available, just as expected by design. As a Service Pack or other large update is installed that contains or supersedes any of the previous patches; we can simply recover the space used by the old updates sometime after the update is successfully installed.
Windows writes logs in many places to aid in troubleshooting and these logs can grow very large. For instance, when an application crashes, Windows will archive a very large dump file to support analysis of the failure. There are many good reasons for this behavior, but as we change our mindset towards footprint, we need to extend our scenarios to include discussions of how to manage the growth, and recover the disk space consumed whenever possible. Other areas where we are considering the default disk space reserved include System restore and hibernation. On a disk constrained system, the 1GB or more reserved to support hibernation is costly and there may be ways to shrink the size of hiberfil.sys. System restore should be configurable, and default in all cases to the minimally useful number of snapshots vs. a blanket 15% of the system disk.
At WinHEC we had several machines on display with 16GB drives/partitions and on those you could see there was plenty of free disk space. Like all the benchmarks, measuring disk space on the pre-beta is not something we’re encouraging at this time.
In conclusion, as we develop Windows 7 it’s likely that the system footprint will be smaller than Windows Vista with the engineering efforts across the team which should allow for greater flexibility in system designs by PC manufacturers. We will do so with more attention to defaults, more control available to OEMs, end-users and IT pros, and will do so without compromising the reliability and robustness of Windows overall.
This means that the OEM Manufacturers can't add software"crapware"?
@Domenico -- The numbers provided represent the installation footprint based on a typical (in hardware) 1GB machine if you were to run setup on a new partition from the Windows 7 Vista DVD.
OK , many thank's MR Steven.
With 805 megabytes reserved for printers, one would think it's time to revise policies for them; that's an awful lot of footprint spent on something most users will only go through one or two over the course of a PC's usable lifespan, or less.
Some systems are in a position where it is preferable to not have every driver for every possible device ready to be called upon at any time. An option to restrict the number of drivers that are installed would be a nice consideration.
Thanks for explaining the WinSxS folder. I have read a lot about it, but this verifies what really is going on. One thing that I am still not sure about, does the OS accounts for this folder as mostly containing hard links when it computes the disk usage? Also, will Windows 7 behave any differently in regards to this folder? I guess, given how hard links are supposed to work, is there really anything wrong with the way Vista reports it?
"As we know, however, fonts are east to delete should you desire."
Have you ever tried uninstalling some of the Arabic, etc. fonts? I've found them FAR from easy to delete. I don't care about the size, but as a designer, I'd like control of what fonts show up on my system, because I don't want to wade through fonts I'll never use.
That said, it seems like Windows 7 has found a good middle ground in allowing you to hide fonts (at least screenshots I see imply this). I don't have the preview build, but if this truly hides fonts from apps like Photoshop, Illustrator, that would basically solve the issue for me.
Can anyone explain why the games use 100MB? I bet you could that get down to 10MB easily. Or even to 1MB if you really wanted to (procedural generation of textures, etc.). I believe most of the "Windows Components" could be shrunken easily as well.
And what about code compression? UPX?
But it's great to hear that you pay attention to this. When you get the system down to some megabytes booting should be faster as well. At least on my notebook with the slow 4200 RPM drive...
certainly for PCs, and in particular laptops, that are bought pre-configured, why would you want to preload so many drivers?
if i think back to all the PCs and laptops i've had since 1994, i can't think of a time i relied on window's cached drivers, simply because the device always comes with its own driver.
nor have i ever used the rollback mechanism. you may call me lucky that i've never needed to which is fair, but then i do backup my data regularly.
i guess it's horses for courses and you need to allow all these decisions to be made configurable in the control panel. then i can choose to turn off all this data bloat for a lighter, less active system which in my case is what i want.
As an owner of an ASUS eee, very expensive SSD Disks and manager of an infrastructure with a lot of virtual disks (Hyper-V) this is a topic very close to my heart.
You make a number of valid points about moving too fast too soon, but as an end user I want to be in control.
Some comments that I am sure you have already considered, but I am going to bother you with anyway:
1. Why not give me the option (via an advanced install) to skip the massive local driver database and always look on-line (Except for network drivers ;-). 'Regular' users who don't specify this option get the local driver db.
2. Give me the option to specify that my native language is English and that I have no interest whatsoever in all those massive Asian / Arabic font files.
3. Give me the option to automatically (or time delayed) clear out the contents of C:\Windows\SoftwareDistribution. If I am not rolling back an update within a few days believe me, I will never roll it back.
4. Have a good look at what ccleaner offers (www.ccleaner.com) and 'embrace and extend' their ideas and integrate it in the disk cleanup wizard.
5. Educate your software partners to not leave all those massive MSI files around after installation (including but not limited to the contents of C:\Windows\Installer). They are probably there for a servicing purpose, but there must be a better way to deal with this. I just did a search for MSI files on my system and found 20 copies of Quicktime.msi. I have no desire whatsoever to roll back to a 2 year old version.
I know you don't have an easy job. You need to please regular users, power users such as myself as well as IT departments. So good luck, any improvements you can make in this area will be much appreciated.
RotoSequence makes a very good point about printer driver space. I for one always go on-line and download printer drivers anyway; they are not core to the operating system so I don't expect them to be there. Wouldn't it be great if there was some kind of de facto standard for basic printing?!
On a separate note, if we are going to start lopping drivers from out of the system, can we have a guarantee from Microsoft now that they won't remove (indeed that they will add) (wireless) networking drivers. The ammount of times I have had to search for these from another machine, just to get a computer on-line for windows update to work.
There was mention in a post quite some time ago about how almost everyone uses a piece of software specific to that user. This is quite applicable to hardware as well.
You can not plan for every type of device that may be plugged in. 7/10 times that I plug in a device and rely on Windows to get the driver, it rarely finds it. On a clean install for example, I should not have to install the Ethernet driver. Network adapter driver support should be a top priority over all others.
Drivers for about any piece of hardware can now be found on the internet if they do not have an installation disk.
When people get a new printer, the instructions in the box intently say "INSTALL OUR SOFTWARE/DRIVER FIRST", even if Windows would already have a driver to support it. Including 800MBs worth of drivers for printers in unnecessary.
Its great you want to rely on windows update for drivers more, but before you do that, windows update needs to become reliable.
Many times I ahve installed network/sound cards and been offered an update by windows, and rebooted only to find my sound/network card has stopped working.
I still get offered a buggy nvidia driver from June of this year for my 8800GTS despite having installed that driver at the time and then upgraded. Infact, it had already been made out of date by nvidia when it was first offered to me!
Sometimes windows update works for drivers, but common consensus is that windows update drivers break things that already work.
Hi folks, Hi steven, I would ask you about the Windows Setup.
At this moment Windows have a good setup, that make ready the system in about 15 minutes, I've tried Windows 7 pre-beta on a Dell XPS 1330, and is very good that all (except finger-printer reader) works, wireless included!
But, with this consideration about windows disk footprint, have you considered the option of a "very custom install" or "dynamic install"?
Actually the users know if they have (a good) internet connection or not. So the setup can check first if all components has newer drivers on Windows Update and install only these? A sort of "Dynamic install" of drivers.
I think that must be a little set of drivers to make working usb, chipset, network, most common devices... but some user want to choose about others devices.
Generally I have 1-2 printers at home or work, not 1 million :D.
Good to see you are working to reduce the footprint, that's one of the points people don't like Vista - because it eats free space.
I have to agree with what was said above about printer drivers. Usually you only have 1-2 printers you need drivers for, and since manufacter drivers have more features, i usually install them, so the pre-installed windows drivers are not really useful to me.
I think making the driver DB optional (or splitting it into categories) + allowing the user to select WHERE to put it would be best.
Same for fonts and other features. I understand that it might be more comfortable to not having to insert the DVD every time you install a new feature for many people, but i prefer to have the possibility to be able to decide whether i want that or not.
I agree with above about the network drivers being important, have installed the 64 bit version of Windows 7 and I had multiple missing drivers but the biggest one was the network driver. I can't check Windows update or Microsoft update for drivers without an internet connection.
Also I have an Asus K8V deluxe motherboard and finding a 64 bit driver is a pain in the but. It has a marvall lom card but I had to the use 64 bit 3com driver to make it work.
Also the detection logic must be flawless. I know this involes getting hardware and drivers that properly repot all plug and play information. For example, I have an ATI remote wonder. It worked fine in XP, one day a driver update popped up on windows update, I donwloaded it hardware stops working. Upon investigation it was for the X10 remote wonder not the regular one. This was in the Microsoft update beta. I bugged it in Connect and to ATI. It was marked as fixed on Connect to this day two years later it shows the X10 version Driver on XP SP 2 and 3, Vista and SP 1 and Windows 7.