Engineering Windows 7

Welcome to our blog dedicated to the engineering of Microsoft Windows 7

Disk Space

Disk Space

This post is about disk space and the disk space “consumed” by Windows 7. Disk space is the sort of thing where everyone wants to use less, but the cost of using a bit more relative to the benefits has generally been a positive tradeoff. Things have changed recently with the availability of solid-state drives in capacities significantly smaller than the trend in spinning drives. Traditionally most all software, including Windows, would not hesitate to consume a 100MB on a specific (justified) need when looking at a 60GB (or 1,500GB) drive; with desirable machines shipping with 16GB of solid-state storage, we are looking carefully at the disk space used by Windows—both at setup time and also as a PC “ages”. We also had a specific session at WinHEC on solid-state drives that might be interesting to folks. This post is authored by Michael Beck, a program manager in the core OS deployment feature team. --Steven

Let’s talk about “footprint”. For the purposes of this post, when I say “footprint” I’m talking about the total amount of physical disk space used by Windows. This includes not only the Windows binaries, but all disk space consumed or reserved for system operations. Later in this entry, I’ll discuss in detail how the disk footprint is consumed by various Windows technologies.

A number of comments have asked about disk footprint and what to expect in terms of Windows 7’s usage of disk space. Like many of the design issues we have talked about, disk space is also one where there are tradeoffs involved so this post goes into the details of some of those tradeoffs and also discusses some of the feedback we have received. It should be noted, that we are not at the point where we are committing to system requirements for Windows 7, so consider this background and engineering focus.

To structure this post we’ll take two important points of feedback or questions we have received:

  • What does the WinSxS directory contains and why is it so big, and can I just delete it?
  • Where does all the disk space go for Windows components?

We’ll then talk about the focus and engineering of Windows 7.

WinSxS directory

We definitely get a lot of questions about the new (to Vista) Windows SxS directory (%System Root%\winsxs) and many folks believe this is a big consumer of disk space as just bringing up the properties on a newly installed system shows over 3000 files and over 3.5 GB of disk consumed. Over time this directory grows to even higher numbers. Yikes--below is an example from a Steven's home PC.

Example properties sheet for WinSxS directory.

“Modularizing” the operating system was an engineering goal in Windows Vista. This was to solve a number of issues in legacy Windows related to installation, servicing and reliability. The Windows SxS directory represents the “installation and servicing state” of all system components. But in reality it doesn’t actually consume as much disk space as it appears when using the built-in tools (DIR and Explorer) to measure disk space used. The fact that we make it tricky for you to know how much space is actually consumed in a directory is definitely a fair point!

In practice, nearly every file in the WinSxS directory is a “hard link” to the physical files elsewhere on the system—meaning that the files are not actually in this directory. For instance in the WinSxS there might be a file called advapi32.dll that takes up >700K however what’s being reported is a hard link to the actual file that lives in the Windows\System32, and it will be counted twice (or more) when simply looking at the individual directories from Windows Explorer.

The value of this is that the servicing platform (the tools that deliver patches and service packs) in Windows can query the WinSxS directory to determine a number of key details about the state of the system, like what’s installed, or available to be installed (optional components, more on those later), what versions, and what updates are on the system to help determine applicability of Windows patches to your specific system. This functionality gives us increased servicing reliability and performance, and supports future engineering efforts providing additional system layering and great configurability.

The WinSxS directory also enables offline servicing, and makes Windows Vista “safe for imaging”. Prior to Windows Vista, inbox deployment support was through “Setup” only. IT professionals would install a single system, and then leverage any number of 3rd party tools to capture the installed state as a general image they then deployed to multiple systems. Windows wasn’t built to be “image aware”. This meant that greater than 80% of systems were deployed and serviced using a technology that wasn’t supported natively, and required IT departments to create custom solutions to deploy and manage Windows effectively. In addition, state stored in the WinSxS directory can be queried “offline”, meaning the image doesn’t have to be booted or running, and patches can be applied to it. These two features of WinSxS give great flexibility and cost reductions to IT departments who deploy Windows Vista, making it easier to create and then service standard corporate images offline.

While it’s true that WinSxS does consume some disk space by simply existing, and there are a number of metadata files, folders, manifests, and catalogs in it, it’s significantly smaller than reported. The actual amount of storage consumed varies, but on a typical system it is about 400MB. While that is not small, we think the robustness provided for servicing is a reasonable tradeoff.

So why does the shell report hard links the way it does? Hard links work to optimize disk footprint for duplicate files all over the system. Application developers can use this functionality to optimize the disk consumption of their applications as well. It’s critical that any path expected by an application appear as a physical file in the file system to support the appropriate loading of the actual file. In this case, the shell is just another application reporting on the files it sees. As a result of this confusion and a desire to reduce disk footprint, many folks have endeavored to just delete this directory to save space.

There have been several blogs and even some “underground” tools that tell you it’s ok to delete the WinSxS directory, and it’s certainly true that after installation, you can remove it from the system and it will appear that the system boots and runs fine. But as described above, this is a very bad practice, as you’re removing the ability to reliably service, all operating system components and the ability to update or configure optional components on your system. Windows Vista only supports the WinSxS directory on the physical drive in its originally installed location. The risks far outweigh the gains removing it or relocating it from the system, given the data described above.

Where does the disk space go?

As we all know adding new functionality consumes additional disk space--in Windows or any software. In reality, “code” takes up a relatively small percentage of the overall Windows footprint.  The actual code required for a Windows Vista Ultimate install is just over 2GB, with the rest of the footprint going to “data” broadly defined. Let’s dig deeper into the use of storage in a Windows Vista installation and what we mean by "data".

Reliability and security were core considerations during the engineering process that built Windows Vista. Much of the growth in footprint comes from a number of core reliability features that users depend on for system recovery, performance, data protection, and troubleshooting. Some of these include system restore, hibernation, page file, registry back up, and logging. Each of these represent “backup state” that is available to the system to recover from any number of situations, some planned and others not. Because we know that different customers will want to make different tradeoffs of disk space relative to recovery (especially on small footprint devices) with Windows 7 we want to make sure you have more control than you currently do to decide ahead of time how much disk space to use for these mechanisms, and we will also tune our defaults to be more sensitive to overall consumption due to the changing nature of storage.

System restore and hibernation are features that help users to confidently recover their system and prevent data loss, in a number of situations such as low battery (hibernation), bad application installation or other machine corruption (system restore). Combined, these features consume a large percentage of the footprint. Because of the amount of space these use, they are easy to identify and make decisions regarding.

System restore protects users by taking snapshots of the system prior to changes and on regular intervals. In Windows Vista, system restore, is configured to consume 300mb minimally, and up to 15% of the physical disk. As the amount of space fills up with restore points, System Restore will delete older restore points to make room for new ones. The more space you have, the greater the number of restore points you have available to “roll back” to. We have definitely heard the feedback from Windows Vista customers around system restore and recognize that the it takes significant space and is not easy to tune. Some have already seen the pre-beta for Windows 7 provides an interface to manage the space better.

Hibernate is primarily used on mobile PCs and saves your work to the hard disk and puts the computer in an extremely low power state.  Hibernate is used on mobile PCs when the battery drains below a certain threshold or when turning the computer off without using Shut Down to extend battery life as much as possible.  On Windows Vista, Hibernate is also automatically used with Sleep on desktop PCs to keep a backup copy of open programs and work. This feature is called Hybrid Sleep and is used to save state to the hard disk in case power fails while the computer is sleeping.  Hibernate writes all of the content in memory (RAM) to a file on the hard drive named Hiberfil.sys.  Therefore, the size of the reserved Hiberfil.sys is equal to the amount of RAM in the machine.  In the Windows Vista timeframe, the amount of RAM being built into computers has increased significantly, thus the disk footprint of Hibernate is more noticeable than before. This space must be reserved up front to guarantee that in a critical low battery situation, the system can easily write memory contents to the disk.  Any mobile PC user that has experienced their computer automatically entering Hibernate when the battery is critically low can appreciate the peace of mind this footprint growth provides. While we're talking about RAM and disk footprint in the same paragraph, Mark Russinovich has a post this week on virtual memory and how big the swapfile could/should/can be that you might find interesting.

Now it’s clear that in the description above, I don’t account for the entire footprint required by Windows Vista. For instance, we also include many sample files, videos, high resolution backgrounds that allow users to easily customize their experience, and try out new features, but we’ve covered a couple of the more common questions out there.

It’s important that we consider more than just the size of the system once deployed, but we must also look at how the system grows over time as services write logs, updates, and service packs are installed, system snapshots are taken etc. For many, the “growth” over time of the installation proves to be the most perplexing—and we hear that and need to do better to (a) make smarter choices and (b) make it clearer what space is being consumed and can be reclaimed.

The following table provides one view of the installation footprint of a Windows Vista Premium/Ultimate installation. This includes the full installation, but to make it digestible this has been broken down into some logical categories and also highlights some specific features. Part of the reason to highlight specific feature is to illustrate the “costs” for items that have been raised as questions (or questionable).

Table of disk space utilization of Windows Vists SP1.

Here are some items worth calling out:

  • ~1GB driver support. Windows Vista works with thousands and thousands of different devices. The ability to plug in almost any device, even your old printer and have it get recognized and install automatically is something customers have come to expect from Windows. We receive lots of feedback wanting to remove some or all of these and each release we carefully scrub the “in-box” device support relative to what we see from telemetry in terms of used devices. The ability to install a printer or USB device while offline is a key value, especially with laptops representing over half of all PCs being sold. In the future we can possibly assume “always go to Windows Update” but we’re not there yet in most places in the world.
  • ~1GB of system growth in serviced and superseded components to support robust rollback and recovery, after installing critical security and functionality updates. We receive a lot of positive feedback about the robustness of servicing but at the same time, the desire to rollback a specific fix for any variety of reasons remains an important robustness and reliability measure. We also understand the feedback we have received regarding the disk requirements to install SP1 on top of RTM. We hope folks are aware of the vsp1cln.exe utility in the system32 directory, for those that are in need of disk space.
  • ~1GB hibernation support is necessary in order to prevent data loss when a machine has been in standby for many hours. This can be removed via the Disk Cleanup wizard or via an elevated command prompt (powercfg /h off).
  • ~315mb of Fonts. Windows users speak many different languages, often on the same PC, and wish Windows to “speak” to them. Windows Vista contains native font support to allow users with systems defaulted to one language to be able to read documents, or websites in another. As we know, however, fonts are east to delete should you desire.
  • ~52MB of log files. Whether it is the event log, servicing logs, or device installation logs (or more) this space is consumed and becomes critical when trying to diagnose a failure. These logs are often used by our support personnel or corporate helpdesks to diagnose a specific failure.

Engineering Windows 7

Windows disk space consumption has trended larger over time. While not desirable, the degree to which it’s been allowed is due in large part to ever-increasing hard drive capacity, combined with a customer need and engineering focus that focused heavily on recoverability, data protection, increasing breadth of device support, and demand for innovative new features. However, the proliferation of Solid State Drives (SSDs) has challenged this trend, and is pushing us to consider disk footprint in a much more thoughtful way and take that into account for Windows 7.

This doesn’t mean that we’re going to stop adding great features or make Windows less reliable or recoverable. As we look to the future, it’s critical that as we innovate, we do so treating the disk space consumed by our work as a valuable resource, and have a clearer design for how Windows uses it. We want to make sure that we are making smart choices for the vast majority of customers and for those desiring more control providing places to fine tune these choices as appropriate. This design goal isn’t about a type of machine, or specific design, all Windows editions benefit from efforts that focus on a reduction of the overall footprint.

For example, as we consider the driver support discussed above, Windows Vista with SP1 installs almost 1GB of drivers on the system to support plug and play of devices. This local cache can get out of date as IHVs release updates to their drivers, and as a result, users are pushed to Windows update to get the latest version once the device is installed.

Why not extend the PnP user experience to include (or only use) the Windows Update cache of drivers and save some disk space? This has several benefits:

  1. Because MobilePCs rarely lack a network connection, they can simply get the new driver from the web.
  2. People don’t have to install the driver twice on updated devices because they do the round trip to the web anyways.

With this example it’s easy to see how engineering for a minimal footprint might actually deliver a better experience for people when attaching new devices to their systems. At the same time, we want to be careful about going too far too soon. We get a tremendous amount of feedback regarding the “plug and play” experience or feedback about costly download times (if download is at all possible). For Windows 7 we are going to continue to be deliberate in what we include based on the telemetry of real world devices and reducing the inbox set to cover the most popular devices around the world. At the same time we will continue a very significant effort around having the best available Windows Update site for all devices we can possibly support.

Windows features installed by default make sense in most cases to support many scenarios. We should consider how we make some features/components (such as Media Center) optional when they are not required rather than installing them by default on every system. We’re committed to make more features of Windows optionally installed. As you might notice today in Windows, when you choose to add a feature that was not installed Windows does not require a source (a DVD or network location). This is because the feature is stashed away as part of a complete Windows install—this is itself a feature. We will always keep features available and will always service them even when components are not installed—that way if you add a component later you do not risk adding a piece of code that might have been exploited earlier. This is another important way we keep Windows up to date and secure, even for optional features.

System growth over time is an area where we need to provide more “transparency”. For instance, Windows will archive previous versions of updated system components to allow robust rollback. A new system will install patches as Windows Update makes them available, just as expected by design. As a Service Pack or other large update is installed that contains or supersedes any of the previous patches; we can simply recover the space used by the old updates sometime after the update is successfully installed.

Windows writes logs in many places to aid in troubleshooting and these logs can grow very large. For instance, when an application crashes, Windows will archive a very large dump file to support analysis of the failure. There are many good reasons for this behavior, but as we change our mindset towards footprint, we need to extend our scenarios to include discussions of how to manage the growth, and recover the disk space consumed whenever possible. Other areas where we are considering the default disk space reserved include System restore and hibernation. On a disk constrained system, the 1GB or more reserved to support hibernation is costly and there may be ways to shrink the size of hiberfil.sys. System restore should be configurable, and default in all cases to the minimally useful number of snapshots vs. a blanket 15% of the system disk.

At WinHEC we had several machines on display with 16GB drives/partitions and on those you could see there was plenty of free disk space. Like all the benchmarks, measuring disk space on the pre-beta is not something we’re encouraging at this time.

In conclusion, as we develop Windows 7 it’s likely that the system footprint will be smaller than Windows Vista with the engineering efforts across the team which should allow for greater flexibility in system designs by PC manufacturers. We will do so with more attention to defaults, more control available to OEMs, end-users and IT pros, and will do so without compromising the reliability and robustness of Windows overall.

-Michael Beck

Leave a Comment
  • Please add 4 and 3 and type the answer here:
  • Post
  • What can I say ?

    I have published here comment hour ago. It wasn't aggressive, inpolite and didn't break any other rule on this forum and had some questions to e7blog and some ideas about device drivers tool.

    Why was it deleted ? (maybe sending short email with explanation would be good idea ?)

    Shortly:

    drivers - why can't we have tool, which will have only one window: on the left tree with devices names and on the right their versions. It will be possible to delete or add driver to system using it.

    why system "is slowing down" after months of usage - it's going, because we have more and more garbage inside. It happens, because we have shared Registry, opened system directory for applications and other design decisions...

    WinSxS - it criticized many times. Why Microsoft can't give to customers what they want and prepare something else instead of it ? and why Explorer can't display additionally real size of these links on hdd ? (in other words: when directory X contains link and file, there will be displayed size of file + number of bytes used by link on hdd)

  • @marcinw -- there is no human involved in comment moderation.  The filtering that takes placed is algorithmic and determined by the third party code that runs this site.  From what I understand many elements including length, quoting, language, # comments by any one person, frequency, etc. all are weighted in terms of "spam detection".  I am subject to the same rules.

    However, since you asked I might suggest that you avoid repeating yourself and let you know that your point of view is well known and so sticking to new topics is probably polite for all of us.

    --Steven

  • I am adding my voice to those who you like more options on content when installing W7. I appreciate having the full installation option on some machines, but when I want a small footprint and have the exact drivers then I want to be able to have the option to not install extra drivers, fonts, ...

  • I absolutely much echo the statement, "I don't care about the size, but as a designer, I'd like control of what fonts show up on my system, because I don't want to wade through fonts I'll never use." The font list has gotten out of hand with all the international, system (e.g. widget-drawing), and math fonts that most people never use (at least directly)!

    Also, if you are going to cut down on drivers, please don't give network drivers the ax. There's nothing worse than installing Windows on a computer whose network card is not supported out of the box! (What am I supposed to do? Download the driver?!) Perhaps more standardization in network drivers would help--think of basic VGA-style functionality, only for network cards.

  • Steven, or anyone else on the windows team:

    1. I do agree that you own explorer MUST be able to identify hardlinks/junction points and exclude them from any diskspace calculations, i know Windows has been able to do this for a long time, you can get the attributes from cmd prompt and it does list the types correctly. If anything, this deficiency in Explorer is in fact a bug, not a feature.

    2. The biggest turnoff for me is Microsoft's continued refusal to externalize settings that interest much of the IT community and professional users. Registry tweaks are a godsend, but a terrible way to give pro users access to detailed settings. PLEASE make some kind of tool that externalizes a lot of these registry tweaks and gives users more control over features, rather than less - which seems to be a pattern in Windows development.

    3. I am further confused by decisions made by the UI team for Vista. Why would you cripple control (what little there was) that has exisited in previous versions of window, like toolbar positioning, button selection, panel locations, panel visibility. Why not create an XML based theming system that would allow you to control every aspect of the UI - if desired. Why do i have to put up with gigantic margins on everything in my UI? Why can i not turn off visibility of things i never use? And why does it suddenly take 3x the number of pixels to display the same amount of info that XP used to display. A lot of UI space is wated on eye candy and there is little control over this. Please rethink the CONTROL you allow to advanced users. I am fine with terrible defaults, and i do find them terrible, but give the ability to control them.

    thanks,

    Leon

  • Oh, those hard links in Windows Vista... I doubleclick on my C:\Documents And Settings, and why do I get that nasty "Access Denied", why can't I simply be brought to C:\Users, or to C:\Vartotojai instead, as that's what C:\Users is named in my language.

    Just show a different icon, that it's Hardlink, and that's it.

  • @steven_sinofsky,

    Thank you for explanation about removing posts. I will suggest something - this filtering should check everything during adding post, not after longer time (currently it's done this way and there is feeling, that this is work from human)

    I'm sorry, that few times (two or three) some posts were sent double.

    I'm not sure, if you wanted to write "it is polite" or "it isn't polite". If second, I'm sorry for that.

  • I'd never understood why it's needed to save all the memory for hibernation. I think that segments of memory with code only should save a reference to fragments of files where that code its saved.

  • Leeoniya,

    I see myself in you, but you don't mention vistas breadcrumb system. are you not annoyed by the way it prevents you from "breadcrumbing" all the way to the desktop, and long foldernames forces you to click that small arrow to access previous crumbs?

    If I had easy access to the desktop I wouldn't need to waste space for a left column with links to my computer and network and recycle bin. Vista really is not very effective for me, or am i doing something wrong?

  • afrazier@victorptg.com:

    When do I know a patch is working properly? For almost all patches and almost all users, almost all of the time, a patch can be considered to be working if I haven't uninstalled it within a day or two. If I leave it beyond that time, it must be because it works well enough, and so it can stay on my system until another patch fixes whatever subtle bugs existed in it. Yes, I'm aware that in rare cases, you may want to remove KB85373 two months after it was installed, and after you've installed two dozen other KB fixes on top of it. But that is a pretty rare case, and I think the *default* behavior in Windows should target the most common use scenario. Which is that once a hotfix has been installed, it stays there. (And further, why does Windows need to save the entire original installer, just to be able to uninstall later?) Obviously, the alternative should be supported, because *occasionally* that is what you want. Sysadmins who want absolute control and the ability to reinstall or roll back anything any time will want to store all possible installers and uninstall/rollback data, I'm just saying that for the common user, it just serves to bloat up \Windows.

    As for keeping MS Update up to date with drivers, I honestly have no clue. I assume vendors submit drivers to MS Update, which are then certified or go through some internal MS process before they're added. In any case, I wasn't really blaming anyone, just pointing out that for this to be useful, MS Update *must* have up-to-date drivers, regardless of whose responsibility this is.

  • Come on! 1 Gig for printer drivers? Remove them all! Drivers can stay on the system DVD and at the same time on a MS managed online driver storage. Why do you have Windows Updates at all if you cannot fetch the required driver on demand? Any given user will only use 1-2 printer in the lifetime of the OS. Also Language packs? How many language will be used an OS? Max 3 at the same time one would think. Why can't you prompt the user if he wants to keep the rest of the languages. And make available online for further changes. You need a nice font manager (Win 7 is nice) to disable and remove fonts. Also just make them available online.

    Handwriting and ink, Speech recognition, wallpapers etc these are all modules that should be easily deactivated/removed maybe, during install or after.

    See you are already 2 Gig leaner.

  • The first commenter made a really good point. OEM Crapware!

    It doesn't matter how fast a car can go, and how lightweight and stylish it is, if the garage you buy it from insists on attaching a caravan to it before you drive it away.

    McAfee and Norton slow PCs down. They show lots of annoying alerts and advertisements, and make the desktop look horrible. I know it brings it a lot of money to OEMs and retailers (who try and sell you up from the 60 day trials) but users should come first. I am glad Microsoft are releasing their own AntiVirus software for free. It's a shame they won't be bundling it with Windows 7. I just don't want to see all this thought and effort put in by the Windows 7 team wasted because some badly coded piece of bloatware gets bundled on every new PC.

    We need an option to choose where and how long we keep Windows installer packages for. On a solid state drive with 16GB of disk space, I may decide my 250GB pocket hard drive is a better place for this sort of thing. Yes it could be annoying if Word pops up saying it needs to install something while I am out, but since I always do a complete install anyway it wouldn't.

  • I still ask where is the new file system that you guys were working on for Vista. I think it was called WinFS. We need a better file system to reduce fragmentation (maybe get rid of it) and speed up file access.

  • WinFS had a lot of good things in it - tha main one being able to index and query your filesystem through some standard SQL like you would any database...which would have been sweet, because your filesystem performance can be tweaked immensely through indexes alone based on the typical type of work a user does.

    Looking forward, however, fragmentation will be much less of an issue as everything will start to move towards SSD-based storage, which is 100% random access and does not suffer from fragmentation whatsoever, there should be more pressure on motherboard and SSD manufacturers to move away from ATA controllers in favor of NVMHCI controllers that are optimized for random access storage devices, and windows 7 appears to have support for this: http://windows7center.com/windows-7-feature/windows-7-to-improve-on-solid-state-drive-ssd-performance/

    Leon

  • ..i should correct myself, the article says Windows 7 won't support NVMHCI :(. regardless, they're making optimizations to how existing controllers access SSD devices, so the point about fragmentation insignificance is still valid.

    Personally, in XP Pro SP3 - the performance difference is HUGE on my Lenovo T400 with a Super Talent Masterdrive OX 64GB vs the stock 7200 Hitachi. I dont know what people are talking about when they say there is little improvement - i do know, however, that some cheaper SSD drives shipped with crappy JMICRON controllers, so that might have been the cause.

    Leon

Page 3 of 7 (103 items) 12345»