Notes on comments.
Welcome to our blog dedicated to the engineering of Microsoft Windows 7
Many folks have commented and written email about the topic of performance of Windows. The dialog has been wide ranging—folks consistently want performance to improve (of course). As with many topics we will discuss, performance, as absolute and measurable as it might seem, also has a lot of subtlety. There are many elements and many tradeoffs involved in achieving performance that meets everyone’s expectations. We know that even meeting expectations, folks will want even more out of their Windows PCs (and that’s expected). We’ve re-dedicated ourselves to work in this area in Windows 7 (and IE 8). This is a major initiative across each of our feature teams as well as the primary mission of one of our feature teams (Fundamentals). For this post, I just wanted to frame the discussion as we dig into the topic of performance in subsequent posts. Folks might find this post on IE8 performance relevant along with the beta 2 release of IE 8.
Performance is made up of many different elements. We could be talking about response time to a specific request. It might mean how much RAM is “typical” or what CPU customers need. We could be talking about the clock time to launch a program. It could mean boot or standby/resume. It could mean watching CPU activity or disk I/O activity (or lack disk activity). It could mean battery life. It might even mean something as mundane as typical disk footprint after installation. All of these are measures of performance. All of these are systematically tracked during the course of development. We track performance by running a known set of scenarios (there are thousands of these) and developers can run specific scenarios based on exercising more depth or breadth. The following represent some (this is just a partial list) of the metrics we are tracking and while developing Windows 7:
We have criteria that we apply at the end of our milestones and before we go to beta and we won’t ship without broadly meeting these criteria. Sometimes these criteria are micro-benchmarks (page faults, processor utilization, working set, gamer frame rates) and other times they are more scenario based and measure time to complete a task (clock time, mouse clicks). We do these measurements on a variety of hardware platforms (32-bit or 64-bit; 1, 2, 4GB of RAM; 5400 to 7200 RPM or solid-state disks; a variety of processors, etc.) Because of the inherent tradeoffs in some architectural approaches, we often introduce conditional code that depends on the type of hardware on which Windows is running.
On the one hand, performance should be straight forward—use less, do less, have less. As long as you have less of everything performance should improve. At the extreme that is certainly the case. But as we have seen from the comments, one person’s must-have is another person’s must-not-have. We see this a lot with what some on have called “eye candy”—we get many requests to make the base user interface “more fun” with animations and graphics (“like those found on competing products”) while at the same time some say “get rid of graphics and go back to Windows 2000”. Windows is enormously flexible and provides many ways to tune the experience. We heard lots on this forum about providing specific versions of Windows customized for different audiences, while we also heard quite a bit about the need to reduce the number of versions of Windows. However, there are limits to what we can provide and at the same time provide a reliable “platform” that customers and developers can count on and is robust and manageable for a broad set of customers. But of course within a known context (within your home or within a business running a known set of software) it will always be possible to take advantage of the customization and management tools Windows has to offer to tune the experience. The ability to have choice and control what goes on in your PC is of paramount importance to us and you will see us continue to focus on these attributes with Windows 7.
By far the biggest challenge in delivering a great PC experience relative to performance is that customers keep using their PCs to do more and more things and rightfully expect to do these things on the PC they own by just adding more and more software. While it is definitely the case that Windows itself adds functionality, we work hard to pick features that we believe benefit the broadest set of customers. At the same time, a big part of Windows 7 will be to continue to support choice and control over what takes place in Windows with respect to the software that is provided, what the default handlers are for file types and protocols, and providing a platform that makes it easy for end-users to personalize their computing experience.
Finally, it is worth considering real world versus idealized settings. In order to develop Windows we run our benchmarks in a lab setting that allows us to track specifically the code we add and the impact that has. We also work closely with the PC Manufacturers and assist them in benchmarking their systems as they leave the factory. And for true real-world performance, the Microsoft Customer Experience Improvement Program provides us (anonymous, private, opt-in) data on how machines are really doing. We will refer to this data quite a bit over the next months as it forms a basis for us to talk about how things are really working, rather than using anecdotes or less reliable forms of information.
In our next post we will look at startup and boot performance, and given the interest we will certainly have more to say about the topic of performance.
Couple things to add to my prior post:
First, I do love Perfmon in its present form, so in no way am I suggesting it should be replaced with simpler tools. It could be extended, or an additional tool created, but hopefully Perfmon's existing features wouldn't go away.
Also, I wanted to take note of a couple of missing perfmon counters I have always wanted to see. All in the Process object:
Process/disk IO: per process counters for disk IO in megabytes/sec and/or as averages over last sample period. If there were another counter which expressed this as 'percentage of current disk queue length', that would rock!
Oops, sorry, that last post got submitted too soon. Here's the rest.
Process/paging counters: Right now you can't see page-ins or page-outs, broken down by which process is causing them. This data would be invaluable!
Process/network counters: and continuing on a theme, network IO broken down on a per-process basis.
Process/CPU queue. We do have the %CPU counter on a per-process bases. Would be nice to also be able to see which processes have the longest CPU queue lengths.
The general thrust here is, essentially the computer has four main resources we want to use effectively: CPU, memory, disk, and network. If we could clearly see how each process is using these resources, we'd have a lot better understanding of how to intelligently shed the load when the system isn't as performant as we'd like.
I second the fact that 3rd party programs are usually the culprit.
The Mojave experiment proved it clearly. Vista is up to the users expectancy.
So Microsoft has to find a way to struggle with these 3rd party applications which put the Windows brand reputation at risk.
A desirable idea is to extend the signed drivers concept to 3rd party applications.
You should put in place a certificate program enforcing strict energy and efficiency rules to deliver a "WHQL" like signature in 3rd party applications.
So ideally, if a user wants to launch an unsigned application, Windows 7 would warn him in a UAC way, warning him about power consumption and explaining him the impediment towards the efficiency of the OS.
I think the most common issue here is. Why does the new OS require latest hardware to run at about the same speed my old pc and previous os. Vista has lots of features (that most might be aware) that are good. But it does require more power.
1) Gamers. I haven't done this personally, but many have complained that the games run slower on vista vs xp. There is benchmarks on the web showing this. Are they valid? If anything, I would address this. I mention games, but the fact is its an overall problem on most softwares. Why is this happening? I'd love to tell you, look at line 512334 in your code, there's your problem. But really I can't and this is the biggest issue that just about everyone complains on. Why does X run slower in Vista than XP? Is there more overhead added? Why the extra overhead? Is it defender, search, shadow copy, etc.. causing this slow down? Can those be improved?
2) Background tasks. Are all the tasks running in background needed? Wireless, I don't have it, but its on by default in XP/Vista. Might not do any impact, but its still there. Taking memory, pooling what ever. How many tasks are really needed for users. There's a couple of sites showing which service to disable to improve speed. Many don't really affect the system, but never the less are all those services needed. In the end, more stuff running the more things can go wrong.
3) Simple tasks. File Copy, Move, Scanning network etc. There is no doubt that these tasks are slowing in vista. Many have been fixed by SP. But why did this happen!! Simple tasks should NOT BREAK! And even now, they are slower than 3rd party ones. Example I use teracopy to do my file copying. Its faster than default windows. Command line typing is faster, but I'm assuming this is due to no graphic over head. BUT does a progress bar slow the copy process that MUCH!?
In vista a big improvement appeared in a few places. When display some items list like (Add/Remove programs) it doesn't wait for the entire list to be created like in XP. You see the items appear on the fly. I think this feature would be nice in more places. It gives the user more feeling that the system is actually doing something. Some tasks just can't be made faster, but can they give you the illusion its faster? I find that when I go back to XP and do remove app, I walk away waiting for the list to populate, unlike vista I see it very fast. This is a big +.
I'm sure some will say, make the os boot faster. But I think that the boot up speed seems to be fast enough. Especially the improvements that as been done to vista from power management. Any time on making the OS boot 5-10 sec faster personally would be a waste of time. How often do you cold boot your OS.
The idea of having a task running in the background to optimize the system, sounds good, until that task runs while play a game or doing a video conference. Please make sure that any feature like that is "Optional" I like the fact that I can right click and perform defrag when I want and not when the OS "Thinks" it should. I could bring up a icon on my status saying hey its been awhile you haven't done X. Maybe have one window with all the different "Tasks" that can optimize.
Philcm - the background defragger is optional already. Just open the defrag program and turn off the automatic scheduling if you're going to be doing a video conference.
It should never run while you're playing a game, since it only runs when the system is idle.
Sorry I wasn't clear. I just picked defrag as an example. I've had other things run in background that slows my system. Specifically the index of search. The harddrive just spins like crazy and all slows down. And my demoing of a product looks bad.
That's why I mentioned background tasks in general is good, until they hurt you.
Defrag wasn't a good one to pick, since that one is actually nice and lets you stop. The search index is the one I have a real issue with. I understand that it needs to do it, otherwise things are out of sync. but does it require to slow down my system while its doing that task? Lowing process doesn't help since it really is kill the hard drive not the cpu.
I would really really love it if Microsoft actually gave it's consumers the option to do a customer install alongside the express install option. What I mean by this is things like Windows Media Player, Live Messenger, Windows Mail, Internet Explorer, SmartCard, etc. should be optional to the consumer. Why must I have every single program and/or service and/or process installed on my system by default? Sometimes turning it off isn't enough. I realize you have the ability to uninstall the above items after install. But why does the consumer have to uninstall it? If I fork out my hard earned cash for a retail copy of your OS, why can't I say what things I want to never be put on my hard drive? Please at least consider my plea. I really do beg of you to seriously consider this install option.
This computer needs 1:10 to show the logon screen and 2:58 to show the desktop.
It takes over 5 minutes to get a usable system on my other computer (disk trashing). It has got 4 GB RAM...
Both systems are fresh installs of Vista Ultimate with SP1 applied, no antivirus software, Windows Defender disabled and no logon scripts.
BeOS R5 takes 4 seconds to boot in Virtual PC on the very same machines. I'm aware of the fact that BeOS does a lot less than Windows Vista. But that's exactly the point I'm trying to make. Maybe Windows 7 should do a bit less. Not have 700 threads running after starting up. Use the KISS approach?
And most of the time it's just percieved performance - why does Vista wait until the preview is ready before showing the delete confirmation dialog? Why did it just take 10 seconds to open the "Computer Management" MMC? MMC could have shown the main window and populate the tree while loading all snap-ins...
Cutting 2 seconds off the boot time isn't going to help. People will notice halved boot times. Not if Windows 7 boots in 1:08 instead of 1:10.
But I'm happy to see that Microsoft cares about boot performance. This is the right direction. Now you just have to replace all computers inside MS with 2+ years old machines instead of measuring boot performance with state-of-the-art computers. Because most people use old computers.
As everyone has said, one SKU. Also, in order to appeal to the general market, maybe there needs to be a "Mac Mode" that simplifies the user experience without sacrificing features. Easier to use remote access would be very nice, as well as giving gamers a better reason to switch. Maybe make Valve's Steam Digital Distribution service integrated with it instead of trying to compete with them. If you can't beat 'em, join 'em!
Here's a quote from Jon Erickson's Hacking: The Art of Exploitation (http://www.amazon.com/Hacking-Art-Exploitation-Jon-Erickson/dp/1593271441 ; don't worry, its about system performance)
Please bear with reading the full thing, there's an important point in there.
""The hacks found in program exploits usually use the rules of the computer to bypass security in ways never intended. Programming hacks are similar in that they also use the rules of the computer in new and and inventive ways, but the final goal is efficiency or smaller source code, not necessarily a security compromise. There are actually an infinite number of programs that can be written to accomplish any given task, but most of these solutions are unnecessarily large, complex, and sloppy. The few solutions that remain are small, efficient, and neat. Programs that have these qualities are said to have elegance, and the clever and inventive solutions that tend to lead to this efficiency are called hacks. Hackers on both sides of programming appreciate both the beauty of elegant code and the ingenuity of clever hacks.
In the business world, more importance is placed on churning out functional code than on achieving clever hacks and elegance. Because of the tremendous exponential growth of computational power and memory, spending an extra five hours to create a slightly faster and more memory efficient piece of code just doesn't male business sense when dealing with modern computers that have gigahertz of processing cycles and gigabytes of memory. While time and memory optimizations go without notice by all but the most sophisticated users, a new feature is marketable. When the bottom line is money, spending time on clever hacks for optimization just doesn't make sense.""
I forgot to add to my above post::
But it does matter. To the end user, the developer, the power/ tech user, and even businesses.
Functional code, something that works.
*Elegant* code, something that works with greater efficiency of hardware resources and software. Something that's just better.
Please, optimize the code. Reorganize it. Make ever part of the Windows OS optimized and tweaked.
After seeing Vista, we will wait. Take as much time as you need. Take an extra year. As long as Microsoft gets this release correct.
I'm a system developer and regard myself as a power user of computers. I've got a quad core CPU, 4 GB RAM and lots of disk space.
I don't care about power consumption and keeping the CPU load low. I have a quad core CPU so please use it. You talk about optimizing for multi user scenarios, that's fine for a server OS (at least it might be). But for a standard file server/web server or desktop PC with a single purpose/user unused hardware is wast imho.
If you can use 3 of the 4 cores at 100% and do something useful that will make my work experience faster and more performant then please do so. The conclusion must be to do more stuff and try to anticipate the users actions when ever resources are idle. Use the CPU, disk and memory for what it's worth.
A lot of time my Vista using HDD for no apparent reason. I consider my PC install is fairly clean, no crapware, otherwise PC recover.
Most of time Vista waste time on HDD activities and Network Device Detection. Like when I tried to print, it will stop for awhile to check my network printer's status.
"We see this a lot with what some on have called “eye candy”—we get many requests to make the base user interface “more fun” with animations and graphics (“like those found on competing products”) while at the same time some say “get rid of graphics and go back to Windows 2000”"
I don't really think this is a contradiction though. It's just people being lousy at explaining what they want. What most people mean when they say "go back to Win2k" is not "I want it to be all grey and square and boxy with no animations". It's "I want it to be as fast to work with as 2k was".
I don't mind flashy graphics, but Vista annoys me because 1) the animations create artificial latency. If I do something, I want it to happen now, not 200 milliseconds from now, when some animation or fadein/out has completed. And 2) the Aero interface in Vista just is not as efficient as it could be. It places some heavier demands on CPU/GPU, which also slows things down a bit.
Now, everyone can agree that 2) should be optimized as much as possible. 1) is the tricky one. I don't mind colors or a nice-looking theme. I've always used the default XP theme, even when all the cool people stuck with the 2k one because it was "simpler" and XP looked too "childish". I even think Aero looks nice in still pictures for the most part. But all the extra flashy bits just get in my way when I try to use the OS.
Keep those concerns separated. Yes, a lot of people are saying "Just make it like 2k", but people are lousy at separating different concerns. What (many of them) mean is just that it should be as fast as 2k, not that it has to look identical.
Similarly, the groop asking for more eyecandy are not also asking for the interface to be slower. They're not asking for *longer* animations when they click a button. They just want it to look nice.