Engineering Windows 7

Welcome to our blog dedicated to the engineering of Microsoft Windows 7

The Windows Feedback Program

The Windows Feedback Program

  • Comments 72

Introducing Christina Storm who is a program manager on the Windows Customer Engineering feature team working on telemetry. 

In a previous article Steven has introduced the Windows 7 Feature Teams. I am a program manager working on telemetry on the Windows Customer Engineering team. Our team delivers the Windows Feedback Program, one of several feedback programs in place today that allow us to work directly with customers and make them part of our engineering process.

The Windows Feedback Program (WFP) has been active for several years during the Windows XP and Windows Vista product cycles, and we are currently ramping up to get all aspects of this program ready for Windows 7. At the core of this program is a large research panel of customers who sign up via our website http://wfp.microsoft.com during open enrollment. Customers choose to be part of a survey program, an automated feedback program or both. They then complete a 20-minute profiling survey, which later allows us to look at their feedback based on their profile. We have customers spanning a wide spectrum of computer knowledge in our program, and we are constantly working on balancing the panel to staff up underrepresented groups. The majority of customers who are spontaneously willing to participate in a feedback program like ours are generally enthusiastic about technology. They are early adopters of consumer electronics, digital devices and new versions of software. In contrast, customers who see the PC as a tool to get a job done tend to be a bit more reluctant to join. And we also need more female participants!

Customers who participate in the automated feedback program install a data collection tool developed by the Windows Telemetry Team. The privacy agreement of this program describes the data collections our tool performs. Here are a few examples:

  • Windows usage behavior including installed and used applications.
  • File and folder structures on your computer, including number of types of files in folders, such as number of jpg files in the Pictures folder.
  • System-specific information, such as hardware, devices, drivers, and settings installed on your computer.
  • Windows Customer Experience Improvement Program (CEIP) data.

From the collected data we create reports that are consumed by a large number of Windows feature teams as well as planners and user researchers. This chart below shows the answer to the following question: What is the most common file type that customers who participate in our program store on their PCs and what are the most popular storage locations?

Graph showing common file types and locations.  The most common file type are .jpg across all typical locations.

The results help us both with planning for handling the volumes of data customers store on their PCs as well as mimicking real-life scenarios in our test labs by setting up PCs with similar file numbers and file sizes and distribution of files on the PCs.

These data collections furthermore allow us to create reports based on profiled panelists. The above chart may look different if we created it based on data delivered only by developers and then compare it to data delivered only by gamers, just to name a couple of different profiles that participate in our program. The Windows knowledge level sometimes makes a difference, too. Therefore it is very important to us that customers participate in this program who consider themselves Windows experts as well as customers who don’t enjoy spending too much time with the PC, who just see the PC as a tool to get things done. Based on the data, we may decide to optimize certain functionality for a particular profile.

In general, we utilize this data to better understand what to improve in the next version of Windows.  Let’s take a look at how the representative sample has their monitors configured.  First what resolutions do customers run with on their PCs?  The following table lists typical resolutions and the distribution based on the Windows Customer Experience Improvement Program, which samples all opt-in PCs (Note the numbers do not add to 100% because not every single resolution is included):

Distribution of common screen resolutions.  Approximately 46% of customers run with 1600x1200 and 1280x1024.  Approximately 10% of customers run with HD resolution.

One thing you might notice is that about 10% of consumers are running with HD or greater resolution.  In some of the comments, people were asking if our data represented the “top” or “power users”.  Given this sample size and the number of folks with industry leading resolution I think it is reasonable to conclude that we adequately represent this and all segments.  This sample is a large sample (those that opt-in) of an extremely large dataset (all Windows customers) so is statistically relevant for segmented studies,

We have found that a large percentage of our program participants lower their display resolution from the highest usable for their display. Looking at the data coming from the Windows Customer Experience Improvement Program to compare to, and noticed a similar trend: over 50% of customers with 1600x1200 screen resolution displays are adjusting their resolution down to 1024x768, likely because they find it uncomfortable to read the tiny text on high resolution displays. The negative effect of this resolution change is the loss of fidelity to the point where reading text in editors and web browsers is difficult. High definition video content also won’t be able to render properly.

Here is the data just for customers with displays capable of 1600x1200:

Actual running resolution for customers with 1600x1200 capable displays shows that 68% of customers reduce their actual screen resolution.

In a future blog post, one of the program managers from the Windows Desktop Graphics team is going to describe what we have done with that information to improve display quality and reading comfort in Windows 7.

We also frequently use our data to select appropriate participants for a survey. A researcher may be interested in sending out an online survey only to active users of virtual machine applications. We would first determine that group of users by looking at our “application usage” data and then create the list of participants for the researcher. Sometimes we combine automatically collected data with survey responses to analyze the relationship between a customer’s overall satisfaction and their PC configuration.

At the current point in time, 50% of our panel participants are using Windows XP and 50% are using Windows Vista. We are currently not offering open enrollment. If you are interested in being invited to this program, please send an email to winpanel@microsoft.com indicating “Notify me for enrollment” in the subject line. If you’d like to add a bit of information about yourself, including your Windows knowledge level, that would be much appreciated! We will add you to our request queue and make our best effort to invite you when we have capacity.

When we release the Windows 7 beta we will also be collecting feedback from this panel and asking for participation from a set of Windows 7 beta users. Our current plans call for signing up for the beta to happen in the standard Microsoft manner on http://connect.microsoft.com. Stay tuned!

-- Christina Storm

Leave a Comment
  • Please add 4 and 1 and type the answer here:
  • Post
  • I have often wondered why the term "resolution" is incorrectly synonymous with "scale" in the computer world. I understand the concept of making an object that is 10 pixels wide, and therefore at different resolutions the object would resize, however I think this is the wrong approach.

    I believe that we have the power now to *separate* resolution and scale so that resolution truly means how fine the drawing of objects is, and scale would determine the actual physical size of the object.

    This concept presents some problems in determining exactly how large the physical size of a user's monitor is, but I would think that making the scale a relative variable would suffice for the average user. Windows would present a slider bar on the bottom of the screen which the user would drag left and right until the screen appeared to display things in a comfortable size.

    The resolution would truely be how fine an object is drawn, not just the relative size of objects to your monitor size, and would be a good start to solving the issue presented in the article.

  • What exactly does 'default resolution' mean when talking about CRTs? (What percentage of your sample still use CRTs?)

    I don't think I've ever seen an LCD with a native res of 1600x1200 and yet 20% of your sample reports that as being the default.

    A 'default resolution' seems like a strange concept when a display doesn't have a fixed number of pixels. I still use a 19" CRT at 1600x1200, but that's not to say it can't go higher (I just wouldn't be able to read anything). Likewise, I'm sure my old 17" CRT could display 1600x1200, but anything higher than 1280x960 would be difficult to use.

  • Thank you for the informative post. It answered all (!) questions I had regarding beta testing.

  • Thank you Christina for all the info you gave us about your department and the way you work to improve Seven.

    Your job is very important. You need knowledgeable and smart ppl to design, analyze and interpret the outcome of statistics.

    Vista, gave us access to lots of statistical data about performance, problem reports and solutions, and crash situations. Are these part of your job? I think some crash reports are inaccurate and are not actually caused from bad drivers/programs, but from bad hardware or Vista coding (ie D3D10 support).

  • @mariosalice

    you're Italian?

  • Does your feedback include statistics on multiple monitors?

    Most of the employees at my company are moving to dual monitors, and I wonder if Microsoft is working on any tools to help those of us with two or more monitors.

  • @Domenico

    Those are just the ones that had enough participants to warrant making their own map. Check this out: http://connect.microsoft.com/SiteImages/8ae1c7af-a580-4421-9cf3-43ad3e8b044b.gif

    120 countries :)

  • Super Nice CullenD

    :D :D

    will begin to have friends and relatives to make sentinels, when the download will be available

    ;)

  • A couple of points:

    1. Please don't bring back "My Documents."  You took a step in the right directions when you dropped to just "Users/$user_name".

    2. Have you considered that the profile of these documents seemingly matches the profile of temporary internet files?  Something tells me that users are not, by and large, saving that many .htm files.  It might be useful to separate out the total file counts from the "user browsable" file counts.  

    I'm shocked at how few .mp3 and .m4a files there are . . . it's not even on the list!

    3. My parents don't realize that you can adjust the DPI settings with the resolution.  IIRC on Windows XP this does not happen automatically, so text shrinks when resolution grows.  

    Could you make this happen automatically?  The X Window system sucks in a lot of ways, but almost every Linux window manager automatically sets the DPI based on the screen's reported width and height.  Just something to think about.

  • okay this may just be me...but:

    1. as stated by others, this proves that "my documents" doesn't work well, and MS should sort out with developers how the data is stored. Best e.g. of this is most people I know use itunes right? maybe abit of an assumption but itunes stores all the music in one place (unless you choose not to) but your files are assessable if you want to copy it over to another folder or something. why are not all apps like this?

    2. screen res... the higher the res the better. Higher DPI is also welcome although not everyones eyesight maybe good. I believe screens should also be set to their native resolution and this should NEVER be changed. Either using bmp or vector based windows, they should both allow the size or font or icons to be changed according to the users taste, like my old man can't see anything that's not blown up in his face.

    3. multiple screens... I use or have used multiple screens on both mac and windows. Definitely a useful feature. I have to say that the mac "spaces" is an amazing feature also. Yes I have heard that you can download it for windows, but why isn't it there already? On top of that, they have expose. For small screens and general computing, these features make a big difference in multi-tasking and productivity.

    I hope my comments help in providing opinions to improve Win 7.

  • Telemetry statistics are all about the past.

    How about the future?

    I mean something like SSD chips on board for the operating system and a second regular hard disk for documents, videos, photos, music etc.

    We should be able to choose or change the location of our documents, the pagefile, the hiberfil.sys etc.

  • Just to throw this in there... I never use Documents or any of the other built in folders for user files. This is because most applications have a nasty habit of dumping their junk in these folders. Random config files appear... useless program folders appear. After a while it starts to feel more like the graveyard of lost program parts more than MY documents.

  • 1. My documents aren't popular

    2. people don't like screens with smaller pixels. if you know it, maybe you can propose to companies producing notebooks, that they should produce models with smaller resolutions (bigger pixels - 0.29) like some time ago. Everybody will be happy: people will buy more devices, you will sell more licenses.

    3. I was trying Windows Server 2008 on VmWare. Although I was setting options for increasing interface size, they didn't work... Windows XP is maybe older, but here everything is working as expected.

  • Thanks for the very much appreciated information and I also recognie that this job (of collecting such datas) is very important.

    But frankly, I don't think that you need huge datas collections programs to discover that Vista's performance are disastrous, that those who wrote the code for Aero are Apple's infiltrator sabotaging Microsoft Windows (trying to keep it fun), that it takes too much space on the HDD, that poeple don't like pre-created folders, that the one-mile long path to "My Documents" in XP was ridiculous, etc.

    Not saying that datas collections are not useful, I applaud this job and the way you do it, but what I mean is that anyone -I say ANYONE- who try Vista would notice these shortcoming within the first minutes.

    Now it's time to fix the problems everybody is talking about. And I see that this blog disgress from this goal and bcome more and more a show off platform for various poeple and teams working at Microsoft with little info on Windows 7.

  • @Cuppa, I have a 20" LCD that is 1600x1200, I agree there pritty rare as they were expensive and there rarer now as 20" is now a widescreen format 1680x1050?

    At my company we have mostly 17" LCD's with 1280x1024 res, the number of times I have adjusted peoples settings is in the hundreds, most people can't stand the default font size and have no idea how to change it, they know the sider though.

    is DPI really the best way to fix things? If found that in XP many programs were messed up changing that, I adjusted the font and icon size in the classic NT4 dialouge with more sucess.

Page 2 of 5 (72 items) 12345