Many folks have commented and written email about the topic of performance of Windows. The dialog has been wide ranging—folks consistently want performance to improve (of course). As with many topics we will discuss, performance, as absolute and measurable as it might seem, also has a lot of subtlety. There are many elements and many tradeoffs involved in achieving performance that meets everyone’s expectations. We know that even meeting expectations, folks will want even more out of their Windows PCs (and that’s expected). We’ve re-dedicated ourselves to work in this area in Windows 7 (and IE 8). This is a major initiative across each of our feature teams as well as the primary mission of one of our feature teams (Fundamentals). For this post, I just wanted to frame the discussion as we dig into the topic of performance in subsequent posts.  Folks might find this post on IE8 performance relevant along with the beta 2 release of IE 8. 

Performance is made up of many different elements. We could be talking about response time to a specific request. It might mean how much RAM is “typical” or what CPU customers need. We could be talking about the clock time to launch a program. It could mean boot or standby/resume. It could mean watching CPU activity or disk I/O activity (or lack disk activity). It could mean battery life. It might even mean something as mundane as typical disk footprint after installation. All of these are measures of performance. All of these are systematically tracked during the course of development. We track performance by running a known set of scenarios (there are thousands of these) and developers can run specific scenarios based on exercising more depth or breadth. The following represent some (this is just a partial list) of the metrics we are tracking and while developing Windows 7:

• Memory usage – How much memory a given scenario allocates during a run. As you know, there is a classic tradeoff in time v. space in computer science and we’re not exempt. We see this tradeoff quite a bit in caches where you can use more memory (or disk space) in order to improve performance or to avoid re-computing something.
• CPU utilization – Clearly, modern microprocessors offer enormous processing power and with the advent of multiple cores we see the opportunity for more parallelism than ever before. Of course these resources are not free so we measure the CPU utilization across benchmark runs as well. In general, the goal should be to keep the CPU utilization low as that improves multi-user scenarios as well as reduces power consumption.
• Disk I/O – While hard drives have improved substantially in performance we still must do everything we can do minimize the amount that Windows itself does in terms of reading and writing to disk (including paging of course). This is an area receiving special attention for Windows 7 with the advent of solid state storage devices that have dramatically different “characteristics”.
• Boot, Shutdown, Standby/Resume – All of these are the source of a great deal of focus for Windows 7. We recognize these can never be fast enough. For these topics the collaboration with the PC manufacturers and hardware makers plays a vital role in making sure that the times we see in a lab (or the performance you might see in a “clean install”) are reflected when you buy a new PC.
• Base system – We do a great deal to measure and tune the base system. By this we mean the resource utilization of the base system before additional software is loaded. This system forms the “platform” that defines what all developers can count on and defines the system requirements for a reasonable experience. A common request here is to kick something out of the base system and then use it “on demand”. This tradeoff is one we work on quite a bit, but we want to be careful to avoid the situation where the vast majority of customers face the “on demand” loading of something which might reduce perceived performance of common scenarios.
• Disk footprint – While not directly related to runtime performance, many folks see the footprint of the OS as indicative of the perceived performance. We have some specific goals around this metric and will dive into the details soon as well. We’ll also take some time to explain \Windows\WinSxS as it is often the subject of much discussion on technet and msdn! Here rather than runtime tradeoffs we see convenience tradeoffs for things like on disk device drivers, assistance content, optional Windows components, as well as diagnostics and logging information.

We have criteria that we apply at the end of our milestones and before we go to beta and we won’t ship without broadly meeting these criteria. Sometimes these criteria are micro-benchmarks (page faults, processor utilization, working set, gamer frame rates) and other times they are more scenario based and measure time to complete a task (clock time, mouse clicks). We do these measurements on a variety of hardware platforms (32-bit or 64-bit; 1, 2, 4GB of RAM; 5400 to 7200 RPM or solid-state disks; a variety of processors, etc.) Because of the inherent tradeoffs in some architectural approaches, we often introduce conditional code that depends on the type of hardware on which Windows is running.

On the one hand, performance should be straight forward—use less, do less, have less. As long as you have less of everything performance should improve. At the extreme that is certainly the case. But as we have seen from the comments, one person’s must-have is another person’s must-not-have. We see this a lot with what some on have called “eye candy”—we get many requests to make the base user interface “more fun” with animations and graphics (“like those found on competing products”) while at the same time some say “get rid of graphics and go back to Windows 2000”. Windows is enormously flexible and provides many ways to tune the experience. We heard lots on this forum about providing specific versions of Windows customized for different audiences, while we also heard quite a bit about the need to reduce the number of versions of Windows. However, there are limits to what we can provide and at the same time provide a reliable “platform” that customers and developers can count on and is robust and manageable for a broad set of customers. But of course within a known context (within your home or within a business running a known set of software) it will always be possible to take advantage of the customization and management tools Windows has to offer to tune the experience. The ability to have choice and control what goes on in your PC is of paramount importance to us and you will see us continue to focus on these attributes with Windows 7.

By far the biggest challenge in delivering a great PC experience relative to performance is that customers keep using their PCs to do more and more things and rightfully expect to do these things on the PC they own by just adding more and more software. While it is definitely the case that Windows itself adds functionality, we work hard to pick features that we believe benefit the broadest set of customers. At the same time, a big part of Windows 7 will be to continue to support choice and control over what takes place in Windows with respect to the software that is provided, what the default handlers are for file types and protocols, and providing a platform that makes it easy for end-users to personalize their computing experience.

Finally, it is worth considering real world versus idealized settings. In order to develop Windows we run our benchmarks in a lab setting that allows us to track specifically the code we add and the impact that has. We also work closely with the PC Manufacturers and assist them in benchmarking their systems as they leave the factory. And for true real-world performance, the Microsoft Customer Experience Improvement Program provides us (anonymous, private, opt-in) data on how machines are really doing. We will refer to this data quite a bit over the next months as it forms a basis for us to talk about how things are really working, rather than using anecdotes or less reliable forms of information.

In our next post we will look at startup and boot performance, and given the interest we will certainly have more to say about the topic of performance.

--Steven