Notes on comments.
Welcome to our blog dedicated to the engineering of Microsoft Windows 7
Just about every email we receive and every comment we get comes with feedback—something to change, something to do more of, something to do less of, and so on. As we’ve talked about in this blog, acting on each one in an affirmative manner is easier said than done. What we can say for certain, is that we are listening to each and every comment, blog post, news story, MS Connect report, Send Feedback item, and of course all the data and telemetry. This post kicks off the discussion of changes made to the product with an overview of the feedback process. We'll get into specific changes shortly and we'll continue to return to the theme of changes in the Release Candidate (RC) over the next weeks. Yesterday on the IE Blog, you saw that we'll be updating IE 8 on Windows 7, and there we also talked about the feedback process in general.
Feedback about Windows 7 of course starts before we've written any code, and by the time we've got running code thousands of people outside of Microsoft have provided input and influenced the feature set and design of Windows 7. As we've seen, the input from even a small set of customers can often represent a wide variety of choices--often in alignment, but just as often in opposition. As we're developing the features for Windows 7 we work closely with PC makers, enterprise customers, and all types of customers across small business, education, enthusiasts, product reviewers and industry "thought leaders", and so on. We shape the overall "blueprint" of the release based on this wide variety of input. As we have design prototypes or code running, we have much more targeted and specific feedback by using tools such as usability tests, concept tests, benchmark studies, and other techniques to validate the implementation of this blueprint. Our goal with this level of feedback is for it to be representative of the broad set of Windows customers, even if we don't have a 1:1 interaction with each and every customer. Hopefully this post will offer some insights into this process overall--the tools and techniques, and the scope of feedback.
In the first few weeks of the Windows 7 beta we had over one million people install and use Windows 7. That's an astounding number for any beta test and while we know it has been fun for many folks, it has been a lot of work for us--but work that helps to raise the quality of Windows 7. When you use the beta you are automatically enrolled in our Customer Experience Improvement Program (anonymous feedback and telemetry, which is voluntary and opt-in in the RTM release). Just by using Windows 7 as a beta tester you are helping to improve the product--you are providing feedback that we are acting on in a systematic manner. Here is a sense of the scale of feedback we are talking about:
We have a variety of tools we draw on to help inform the decision making process. A key element that we have focused on quite a bit in Windows 7 is the role of data in making decisions. Everything we do is a judgment call as ultimately product development is about deciding what to get done from an infinite set of possibilities, but the role of data is essential and is something that has become far more routine and critical. It is important to be super clear—data is not a substitute for good judgment or an excuse to make a decision one way or another, but it most definitely informs the decision. This is especially true in an era where the data is not only a survey or focus group, but often includes a “sampling” of millions of people using Windows over the course of an extended time period.
A quick story from years ago working on Office, many years ago before the development of telemetry and the internet deciding what features to put in a release of Office could really be best described as a battle. The battle took place in conference rooms where people would basically debate until one or more parties gave up from fatigue (mental or otherwise)—essentially adrenaline-based product development. The last person standing, the one with the most endurance, or the one who pulled an all-nighter to write the code pretty much determined how features ended up or what features ended up in a product. Sort of like turning feature design over to a Survivor-like process. I’m sure many of you are familiar with this sort of process. The challenges with this approach are numerous, but inevitably features do not hold together well (in terms of scenarios or architecture), the product lacks coherency, and most importantly unless you happen to have a good match between the “winner” and the target customers, features will often miss the mark.
In the early 1990’s we started instrumenting Word and learning about how people actually used the software (this was before the internet so this was a special version of the product we solicited volunteers to run and then we would collect the data via lots of floppies). We would compile data and learn about which features people used and how much people used them. We learned things such as how much more people used tables than we thought, but for things very different than tables. We learned that a very significant amount of time the first suggestion in the spelling dictionary was the right correction (hence autocorrect). We learned that no one ever read the tip of the day (“Don’t run with scissors”). This data enabled us to make real decisions about what to fix, the impact of changes, and then when looked at the goals (the resulting documents) what direction to take word processing.
Fast forward to the development of Windows 7 and we’re focused on using data to help inform decisions we make. This data takes many forms and helps in many ways. I know a lot of folks have questions about the data – is it representative, how does it help fix things people should be using but don’t, what about doing new things, and so on. Data is an important element of making decisions, but not a substitute for clear product goals, meaningful customer engagement, and working across the ecosystem to bring Windows 7 to customers.
Let’s talk a bit about “bugs”. Up front it is worth making sure we’re on the same page when we use the much overloaded term bug. For us a bug is any time the software does something that someone one wasn’t expecting it to do. A bug can be a cosmetic issue, a consistency issue, a crash, a hang, a failure to succeed, a confusing user experience, a compatibility issue, a missing feature, or any one of dozens of different ways that the software can behave in a way that isn’t expected. A bug for us is not an emotional term, but just shorthand for an entry in our database representing feedback on the product. Bugs can be reported by a human or by the various forms of telemetry built into Windows 7. This broad definition allows us to track and catalog everything experienced in the product and do so in a uniform manner.
Briefly, it is worth considering a few types of data that help to inform decisions as some examples.
This type of feedback all represents structured feedback in that the data is collected based on a systematic study and usually has a hypothesis associated with it. We also have the unstructured feedback which represents the vast array of bug reports, comments, questions, and points of view expressed in blogs, newsgroups, and the Send Feedback button—these are unstructured because these are not collected in a systematic manner, but aggressively collected by any and all means. A special form of this input is the bug reporting done through the Connect program—the technical beta—which represents bug reports, feature suggestions, and comments from this set of participants.
The Windows 7 beta represents a new level of feedback in this regard in terms of the overall volume as we talked about above. If you go back and consider the size of the development team and the time it would take to just read the reports you can imagine just digesting (categorizing, understanding, flagging) issues let alone responding to them is a massive undertaking (about 40 Send Feedback reports per developer during that one week, though as you can imagine they are not evenly distributed across teams).
The challenge of how to incorporate all the feedback at this stage in the cycle is significant. It is emotional for us at Microsoft and the source of both considerable pride and also some consternation. We often say “no matter what happens, someone always said it would.” By that we mean, on any given issue you can be assured that all sides will be represented by passionate and informed views of how to resolve it, often in direct opposition to each other plus every view in the middle. That means for the vast majority of issues there is no right or wrong in an absolute sense, only a good decision within the context of a given situation. We see this quite a bit in the debates about how features should work—multiple solutions proposed and debate takes place in comments on a blog (people even do whole blogs about how things should work). But ultimately on the Windows development team we have to make a call as we’re seeing a lot of people are looking forward to us finishing Windows 7, which means we need to stop changing the product and ship it. We might not always make the right call and we’ll admit if we don’t make the right call, even if we find changing the behavior is not possible.
Making these decisions is the job of program management (PM). PMs don’t lock themselves in their offices and issue opinions, but more realistically they gather all the facts, data, points of view, and work to synthesize the best approach for a given situation. Program management’s role is making sure all the voices are heard, including beta testers, development, testing, sales, marketing, design, customer support, other teams, ISVs, IHVs, and on and on. Their role is to synthesize and represent these points of view systematically.
There are many factors that go into understanding a given choice:
These are just a few of the factors that go into considering a product change. As you can see, this is not something that we take lightly and a lot goes into each and every change. We consider all the inputs we have and consider all the data we can gather. In some ways it is easy to freeze thinking about the decisions we must make to release Windows 7—if you think too hard about a decision because you might start to worry about a billion people relying on something and it gets very tricky. So we use data to keep ourselves objective and to keep the decision process informed and repeatable. We are always humbled by the responsibility we have.
While writing this post, I received a “bug report” email with the explicit statement “is Microsoft going to side step this issue despite the magnitude of the problem” along with the inevitable “Microsoft never listens to feedback”. Receiving mail like this is tough—we’re in the doghouse before we even start. The sender has decided that this report is symbolic of Microsoft’s inability or lack of desire to incorporate critical feedback and to fix must fix bugs during development. Microsoft is too focused on shipping to do the right thing. I feel like I’m stuck because the only answer being looked for is the fix and anything less is a problem or further proof of our failure. And in the back of my mind is the reality that this is just one person with one issue I just happen to be talking to in email. There over a couple of million people using the beta and if each one, or for that matter just one out of 10, have some unique change, bug fix, or must do work item we would have literally years of work just to make our way through that list. And if you think about the numbers and consider that we might easily get 1,000,000 submitted new “work items” for a product cycle, even if we do 100,000 of them it means we have 900,000 folks who feel we don’t listen compared to the 100,000 folks who feel listened to. Perhaps that puts the challenge in context.
With this post we tried to look at some of the ways we think about the feedback we’re getting and how we evaluate feedback in the course of developing Windows 7. No area is more complex than balancing the needs (and desires) of such a large and diverse population—end-users, developers, IT professionals, hardware makers, PC manufacturers, silicon partners, software vendors, PC enthusiasts, sysadmins, and so on. A key reason we augment our approach with data and studies that deliberately select for representative groups of “users” is that it is important to avoid “tyranny of the majority” or “rule by the crowd”. In a sense, the lesson we learned from adrenaline -based development was that being systematic, representative, and as scientific as possible in the use of data.
The work of acting on feedback responsibly and managing the development of Windows through all phases of the process is something we are very sincere about. Internally we’ve talked a lot about being a learning organization and how we’re always learning how to do a better job, improve the work we do, and in the process work to make Windows even better. We take this approach as individuals and how we view building Windows. We know we will continue to have tough choices to make as everyone who builds products understands and what you have is our commitment to continue to use all the tools available to make sure we are building the best Windows 7 we can build.
Thank you for your kind words overall. I can offer some assistance to you on your wishlist.
Run As is discouraged for security purposes, but it is still available using shift+right-click.
You can create and customize a Quick Launch tool bar: http://www.howtogeek.com/howto/windows-7/add-the-quick-launch-bar-to-the-taskbar-in-windows-7/ is one location that has the full instructions.
On the “Manage Network Adapters” I recommend adding ncpa.cpl to the quick launch bar. I apologize this isn’t what you are asking for exactly.
On GINA, the add-in mechanism for login was redesigned for Vista, which Windows 7 inherits. Any ISV or IHV that used GINA is encouraged to use the new interface: http://msdn.microsoft.com/en-us/magazine/cc163489.aspx
Cisco for example has done this in their Vista compatible VPN client.
After reading this article, it would seem that there would be a great benefit to having a beta 2 before locking into the release candidate. One of the comments here was about how there seems to be no response to the Windows 7 taskforce suggestions, but I would guess that some of these suggestions have been looked at, and we just need a chance to use the latest build to see this. It's unfortunate that the public beta is the only opportunity to submit feedback with the hope that a large change will take place. Please don't rush 7.
"Our telemetry measures sequences of feature usage as well and we often spend considerable time understanding the starting point/end point of sequences. So not just quantity."
In the example given of Media Player's library I wonder if that telemetry measures the disjunction between the reduced feature set of v12 and the far-from bridging-the-gap additions to Explorer for managing media files. I think there would also be a lot of unquantifiable sequences as users try to figure out how to do something like rip a CD now that the Rip tab has gone. After using many versions of the player and writing articles on its use, I feel like a newbie trying random left and right clicks over the Media Player interface trying to figure out where they've hidden stuff.
"For us a bug is any time the software does something that someone one wasn’t expecting it to do."
One of the most passionately argued items in the Vista and W7 betas has been the reduction in UI contrast for so much of Windows (and in moreso for Media Player which likes to use grey text over various shades of blue). Those of us who are contemporaries of you Steve but may not be able to afford bleeding edge video equipment, find it increasing difficult to easily read so much on the screen because of color choices and font sizes that can ONLY be adjusted by going to one of the extreme High Contrast schemes. Even with so many people voting on these issues, the beta team does us the disservice of saying "NOT REPRO". Do we need to contribute telemetry data from our visual cortex before this is taken seriously?
"The work of acting on feedback responsibly and managing the development of Windows through all phases of the process is something we are very sincere about."
For two Windows betas now we've had problems with the Feedback tools that seem to require entirely new builds of Windows to address. In Build 7000 the flag to set feedback as Public almost never works, which means that almost every beta tester has to edit each bug after submission to reset that flag. Why not issue a patch? It's very likely that we're never going to see a pre-RC build so why not make beta-testers' work easier?
We're also unable to see attached screenshots on bugs, something simple promised during the Vista beta but still eluding Microsoft 3 years later in the W7 beta.
Could Microsoft put some effort in the XPDM graphics driver layer? For laptops upgrading the graphics card is not an option. And from Vista -> 7, the compatibility with these older drivers has deteriorated. I hope the engineers get enough telemetry data from the broken installs on such systems. Or get around talking the few graphics chip builders to update their drivers to the new stricter rules.
I can attest that to the folks I've encountered that have used Windows 7, they are very pleasantly surprised at the quality of the Beta build and the potential for the final product. All the users I've spoken to are itching to get the RC and it does span the userbase from the basic skilled users to the more advanced users. I'd also like to thank Microsoft for respond to issues very quickly.
The IE 7 update has improved some issues. However, I'm now getting the same amount of crashes as in IE 7 on Windows Vista. I hope the RC version of IE 8 improves upon this. However, I do love the recovery feature that allows me for the most part to resume whatever I happen to be working on. This is very useful and helps you cut down on wasted time.
I would have to agree that Microsoft does need a few rabbit's in the hat, a few aces, and a joker's in the deck of a "Wow" must have features that neither Linux or Apple has. A couple of these game changers or the killer app to movtivate folks who need something extra. In Windows 95, it was the clean UI that was the must have. In Windows 98, it was the tight integration of IE plus WMP that was the game changer. In Windows 2000, the NTFS switch and new driver model that was the must have. For Windows XP, besides streamlining 2000, it was the emergence of new technologies and stability. For Vista users, the security was supposedly the big sell, but it wasn't communicated and there was no "Wow" app considered a game changer. It was percieved as slow and it was correct.
Here we are slowly approaching Windows 7 RC, and there is a lot of promise with Windows 7. However, there isn't that killer apps, game changers, aces, jokers, etc. I really do hope between now and the RC, Microsoft has something hidden to spring onto users. Some set of killer apps that make the case. Those of us using the beta have been pretty much sold. I'm really speaking for the average joe, the not so technical user, the Mac skeptic, and other users who might not get it. Steven and Jon, just something for you guys to think about and cook up before Windows goes to RTM.
@ "needy state changes"
Please, i BEG you! give us a choice for the needy state taskbar blinking.
when there is a taskbar button flashing of a window that i cannot immediately divert my attention to, it drives me nuts. it's like my taskbar is visually shouting at me like a kid with ADHD demanding attention.
I always try to be kind regardless of the situation. Sugar vs Spice. From a security perspective, having administrators logon to their workstations with elevated domain credentials is directly against best practice. Just add a nice custom logon script on a DA workstation and welcome to hell. Maybe just a difference in opinion on this one. In response to your response to my wishlist.
1.) Even the new Cisco VPN 5.0.05.0.280-Beta for Vista DOES NOT support START BEFORE LOGON. This has not changed since the release of Vista and is a really big deal. Let me know if I am missing something on this one.
2.) Yes the hidden SHIFT-RIGHT-CLICK works for certain file types. However, most administrators will want to run Snap-In Consoles when running under alternate credentials. Unless I am missing something the RunAs different user does not work under these file types. Of course we can open a cmd under the context of an elevated account and then execute the msc but that is just another set of steps to do. I could also execute MMC.EXE and then add snap-in consoles. Just more steps once again.
3.) In Windows 7, if I right-click on "Network" the menu is practically empty. You could put it right below "Disconnect Network Drive" Why is it a major issue to add the Manage Network Adapters back when the menu itself is already small? I understand a quicklaunch is another workaround but I'm not going to deploy that to every single workstation as I want to make it transparent to our normal end users. In addition, do you have a reason why this option was removed from Vista in the first place?
I am very happy where you are going with Windows 7. As an enterprise customer, these are some of the smaller issues I see that can easily be fixed. Maybe you can work with Cisco to bring back "Start Before Logon" features that existed back in Windows XP. I appreciate the links outlining the GINA as they were very informative.
Thanks for this great post! This will surely encourage beta testers to submit more and higher quality feedback. You can at least count me in for that!
2 things BOTHER me about Windows VISTA, Windows Server 2008, & doubtless their offspring in Windows 7 (unless you can tell me otherwise on the latter):
1.) The removal of being able to use 0 as a blocking IP address in a HOSTS file (vs. 0.0.0.0 or 127.0.0.1, which are bigger, slower on load into the local DNS Cache (as well as slower flushes via ipconfig /flushdns) & also occupy more RAM once loaded, for NO GOOD REASON - 0 blocks as well as the other 2 do, & is smaller + faster!)
In this case, this happened on 12/09/2008 Microsoft "Patch Tuesday" updates, it wasn't LIKE that before then!
E.G.-> Here, using 0 as my blocking IP address in a FULLY normalized (meaning no repeated entries) HOSTS file with nearly 650,000 bad sites blocked in it, I get a 14++mb sized HOSTS file... using 0.0.0.0 it shoots up to 18++mb in size (& even worse using 127.0.0.1, to around the tune of 24++mb in size)... semseless & bloat creation is the result!
2.) The removal of IP Port Filtering GUI controls for it via Local Network Connections properties "ADVANCED" section (this is up there w/ when MS removed the GUI checkbox after NT 4.0 for IP Forwarding, only, this time, the difference is (and, it's a PAIN) is that it is NOT a single 1 line entry to hack via regedit.exe, but FAR MORE COMPLEX to do by hand)... port filtering is a USEFUL & POWERFUL security (& to a degree, speed also) enhancing feature!
Afaik, on THIS case (vs. #1 above)? It has always been that way in VISTA &/or Windows Server 2008... & not just the result of a Patch Tuesday modification.
Ordinarily, I wouldn't post anything that "puts down Windows" here, ESPECIALLY THIS SITE (since it's KNOWN to widely be a more-or-less largely "Anti-Microsoft" type of news website, lol, & facts like these give the 'antimicrosoft' faction here ammo to use), but...
Facts, are facts.
P.S.=> MAN, all that said & aside? I had to post those 2 objections I have to newer MS OS' - I mean, hey:
Doing both of those alterations (crippling ones imo) on MS' part? Dumb...
So, unless someone can show me a GOOD solid technical reason (because I have YET to find any reasons WHY both of those things were done) on why these cripplings were implemented in VISTA/Server2008, vs. Windows 2000/XP/Server 2003??
I will stick by that statement! apk
I hope you've done something about WMP 12's UI since build 7000. Features aside, WMP 11 had a more sensible design and looked much better.
I've been a beta tester since Windows NT 4.0, and I completely sympathize with the Windows dev team, however I decided to resign my beta tester credentials after receiving "will not fix" responses to the vast majority of my Vista bug reports and all of my Windows 7 bug reports.
I guess the inability to successfully run vanilla .vbs logon scripts without first turning off UAC isn't critical to Microsoft. And that's fine, I understand that you have to have your priorities. Some of my bug reports are stupid cosmetic things (those usually get fixed), and some are not. If MS is going to consistently flag the stuff that is actually >important< as "will not fix", then I'm not going to waste my time or yours continuing to file bug reports when the outcome is already known.
Great post. No one serious about release a product can just add in every suggestion. Months of delays (because extra features/changes always add more unexpected bugs!) vs. an early and controlled polished release. I know what I'd choose!
Marcinw - you're talking nonsense. The DRM in Vista doesn't effect me because none of my devices or windows media files are DRMed. In fact iTunes probably has actively running DRM code since I do actually buy songs/apps from the store.
You must understand that code doesn't slow down your system unless it's being constantly executed or it has large amounts of data in memory.
Vista may have built in DRM, just like the iPod and even the Mac OS X contain code to control what hardware the OS will run on (a form of DRM if you ask me).
If you won't a totally "open" non DRM system, then use one you can view the source code of, like Fedora.
However say goodbye to interoperability with all your devices, graphics cards, sound cards and online stores. You either live within the ecosystem and accept that your operating system will contain some DRM, or move to a different platform (or, as you've suggested, stay with an antiquated, and soon to be not-supported OS for ever and ever).
DRM = Minor issue. Move on.
1.) The removal of being able to use 0 as a blocking IP address in a HOSTS file
(vs. 0.0.0.0 or 127.0.0.1, which are bigger, slower on load into the local DNS Cache (as well as slower flushes via ipconfig /flushdns) & also occupy more RAM once loaded, for NO GOOD REASON - 0 blocks as well as the other 2 do, & is smaller + faster!)
E.G.-> Here, using 0 as my blocking IP address in a FULLY normalized (meaning no repeated entries) HOSTS file with nearly 650,000 bad sites blocked in it, I get a 14++mb sized HOSTS file... using 0.0.0.0 it shoots up to 18++mb in size (& even worse using 127.0.0.1, to around the tune of 24++mb in size)... senseless & bloat creation is the result!
2.) The removal of IP Port Filtering GUI controls for it via Local Network Connections properties "ADVANCED" section
(This is up there w/ when MS removed the GUI checkbox after NT 4.0 for IP Forwarding, only, this time, the difference is (and, it's a PAIN) is that it is NOT a single 1 line entry to hack via regedit.exe, but FAR MORE COMPLEX to do by hand)... port filtering is a USEFUL & POWERFUL security (& to a degree, speed also) enhancing feature!
P.S.=> WHY HAS THIS BEEN DONE? Makes NO sense people... none for efficiency & security @ least that I can see & thus? I'd like to know WHY these crippling things were done to otherwise possibly FINE OS (these things DO affect my decisions to upgrade & possibly those of others as well, something to consider)... apk
About this question...
The removal of being able to use 0 as a blocking IP address in a HOSTS file
I've got $20 bucks that says it was removed for one of two reason.because it was the
Let me try again.
The removal of being able to use 0 as a block ing IP address in a HOSTS file
I've got $20 bucks that says it was removed for one of two reason.
Because they also removed 1234567 as a valid IP to make it more obvious that you were being hacked.
I don't know if the 1234567 form is compatible with IPv6.