Engineering Windows 7

Engineering Changes to ClearType in Windows 7

One of the many passions held by Bill Gates is a passion for reading and so his desire to make reading on PCs a fantastic experience has been an effort ongoing for many years. In the 1998 COMDEX show, Bill Gates unveiled ClearType – hard to believe it was that long ago. Back when it was announced, very few of us had LCD monitors and those that did invested several thousand dollars in one that was 15” and 1024x768 (today one like that costs less than $100). The notions of smoothing and anti-aliasing have been around for many years and are common in the world of typography, animation, and games. ClearType took this to new levels by building on the properties of LCD panels. ClearType was subsequently included in Windows XP and continues in Vista and Windows 7—each release saw changes in the underlying technology, the fonts that support the technology, and the APIs available to developers. It is fair to say that over the years we have learned that there are a set of customers who simply find ClearType rendered screens less than appealing and wish to turn it off. We recognize this and want to make sure we provide the appropriate controls. ClearType is also part of the Windows platform and provides APIs callable and controllable by developers of applications. There is a conventional view that ClearType is a "visual preference" and through this post we want to show how there are elements that are such a preference but there are also elements that are APIs used by applications, just like applications can choose fonts, colors, and other attributes as required.  This post goes into the details of Windows 7’s implementation along with some history and background. Greg Hitchcock is the development lead on ClearType and has worked on it since the start. He’s also one of the most tenured members of the Windows 7 engineering team with only 6 folks having been at Microsoft longer -- Larry is one of them :-)!

--Steven

Based on feedback, we want to clarify how font rendering works in Windows 7 and give some background on how we chose ClearType font rendering to be the default in Windows. For those that dislike ClearType and want to change the system default setting to bi-level rendering, as were defaults in Windows Millennium, the quick answer is:

• Enter Appearance into the start menu search
• Select Adjust the appearance and performance of Windows from the Control Panel
• The setting that should be changed under the custom option is: Smooth edges of screen fonts, which should be turned off

The longer answer, as we will describe in this post, shows that changing the default setting is not as “black and white” as it may seem. As you have noticed, Windows 7 also includes a new ClearType tuner in the control panel which affords fine-grained control over rendering—we’ll talk about that some below as well.

ClearType

ClearType is a technology developed to improve both the appearance of font rendering and reading performance on computer displays. As most people spend over 80% of their time on computers reading on the screen, improvements in this area greatly improve the overall experience of Windows. The ClearType technology has continued to evolve and the latest improvements have been made in Windows 7, as discussed in this earlier E7 post.

In simple terms, ClearType works by using the underlying geometry of colored sub-pixels in the display as if they were full pixels—gaining extra resolution while at the same time using principles of human vision to remove the perception of color artifacts. Further details on the technology and how it uses human visual perception are described here. More specifically, the ClearType technology is optimized for LCD panels with red, green, and blue (RGB) striped sub-pixels that are oriented vertically, although it performs reasonably well on CRT displays (especially those that are aperture grille based) and even LCD panels with horizontally oriented RGB stripes. Although this might seem counterintuitive, through informal studies, we’ve found that about 70% of users prefer ClearType even on these non-optimal displays. Of the 30% who preferred other rendering techniques, their biggest concern with ClearType in these non-optimal cases was the loss of text contrast.

Other Types of Font Rendering in Windows

Given the complex world of many display types and a wide variety of users and their visual systems, how did we go about implementing ClearType into Microsoft Windows? Microsoft did not rush headlong into making ClearType the default rendering. The technology was first released in 2000 with the Windows CE product. The Windows CE environment is usually quite controlled in terms of the hardware used, so it was quite easy to verify that ClearType worked properly on each device, and either tune ClearType or adjust the hardware to optimize the onscreen reading experience. The first release of ClearType on the Windows platform was with Windows XP in 2001.

Bi-Level Rendering

Example of Bi-Level rendering.  Note if your browser scales this image the text will not be correctly represented.

Prior to Windows XP, two types of font rendering were supported in Windows. The first type of font rendering supported was bi-level rendering, more commonly referred to as “black and white” rendering, but sometimes also called aliased text. With bi-level rendering, two colors represent the font, the foreground color and the background color. This was the first type of rendering supported by TrueType when Windows 3.1 was released, and also the essential method of displaying fonts in bitmap form from Windows 1.0. Bi-level rendering, especially when generated from outline technologies like TrueType, is very difficult to optimize for low screen resolutions. Significant effort needs to be put into the font hinting for TrueType in order to get the best bi-level quality. It can reasonably take a skilled person 6 months to a year per font of hinting time to get this level of rendering detail. That would be multiplied by four for a four-member family. If the character sets are larger, as in some system fonts, it can take even longer.

Font Smoothing / Grayscale

 

The second form of rendering, known as font smoothing, became the default rendering in Windows 2000 and was first released as an option for Windows 95 in the Plus! Pack. Font smoothing is a hybrid grayscale anti-aliasing technique designed to improve the contrast of fonts over traditional anti-aliasing techniques. There are two factors that differentiate font smoothing from more traditional text anti-aliasing.

First, traditional anti-aliasing works by overscaling the font outline data and then downsampling. Font smoothing uses the same technique, except that it applies font hints prior to overscaling the outline data. Although we don’t have enough space here to fully describe font hinting, I can simplify it enough to say that it often uses a method called “grid fitting” to snap the vertical and horizontal edges of the font outlines so that they are aligned with the pixel grid. In this situation, most horizontal and vertical stems of a font, when overscaled, cover 100% of the underlying pixels, and when downsampled return the text foreground color, which is usually black. Diagonal and round features of the font will not have full coverage of the pixel and thus will return some shade of gray, reflecting the coverage of the underlying pixel. It should be noted that when text displays the “jaggies” (or more formally aliasing) this usually occurs on round or diagonal parts of the glyphs—exactly the areas receiving gray coverage with this method. This way of anti-aliasing is beneficial due to the higher contrast of the stems in the font at a slight cost of some spatial accuracy.

The second differentiating factor is that the fonts determine the exact size that the font smoothing turns on or off. Most fonts that provide this level of information turn on grayscale anti-aliasing below 9 pixels per em (PPEM.) That is the equivalent of 7 points on a 96 PPI screen. Above 9 PPEM, anti-aliasing is turned off until the main stems of the font are around two pixels wide, which is around 13 to 20 points, depending on the typeface. Once the main stems are two pixels wide, anti-aliasing remains on as the sizes increase. Two pixel wide stems are usually chosen because there is usually enough “backbone” of foreground colored pixels to keep the stem contrast high.  If the font does not have specific sizes for font smoothing, system defaults are used. There are independent defaults for both regular and bold typefaces. So although font smoothing was the default, most fonts, when displaying text at typical reading sizes, would render them bi-level.

Defaults for Font Rendering

With the addition of ClearType to Windows XP, there were now three types of font rendering available (Bi-level, Font Smoothing, ClearType). During the time period that Windows XP was being developed there was a clear transition underway from desktop systems with CRT monitors to laptops and desktops with LCD displays. Since this transition was far from complete, we felt that the default value for font rendering in Windows XP should be grayscale font smoothing, the same as Windows 2000. OEM’s who provide Windows XP on their systems could change this default, and in fact, by the time Windows XP SP 2 shipped, many of them had set the default to ClearType. It should be noted that OEMs can always change these settings as part of configuring a PC.

In Windows Vista, the system’s default font rendering was changed to ClearType. It is important to clearly understand what is meant by default font rendering. In Windows, the default font rendering is the rendering used when the application does not choose an explicit type of rendering. Some have confused this default value to mean that all applications must use this value. This view is inconsistent with the way text APIs worked when introduced in Windows 95’s font smoothing. In GDI, the API for choosing the current font has the rendering type explicitly as input. It is expected that there are situations where the application knows best what type of rendering should be used. For example, in displaying a print preview page with small text, traditional font smoothing might be the best choice for rendering. Likewise, if an application was targeting on-screen reading, it might be best to use ClearType as the rendering for that application. In some situations, like remote terminal services, the application might choose to use bi-level rendering to reduce the bandwidth of text information that needs to be sent to a remote client.

There are many examples where applications decide one way or another to use rendering other than the system default—just as applications that choose to use different fonts, colors, sizes, or other text attributes.  The most typical example is in applications that wish to have reproducible layout and flow of documents.  By being specific about which way to render text, the applications can be certain of how text will flow across different PCs.  Another common example, as mentioned above, is Print Preview where the ability to properly render representations of higher resolution output, particularly for small text sizes, is much improved.  We recognize that for some it is counter-intuitive that an aspect viewed as a “display” property is something that applications can choose to “over-ride”.  We’ve designed rendering so that the default case is to respect the setting, but applications, including Windows itself, might have elements that require explicit rendering techniques.

Although each application can make the choice on a per-font basis of which rendering to use, the majority of applications choose the default rendering. Therefore, making the decision to change the default for Windows Vista was not taken lightly. The trends in the hardware displays were strongly showing a rapid movement from CRTs to LCD-based displays as we have shown in earlier blog posts based on the Windows XP and Windows Vista real-world telemetry. Even though there were still CRTs in use, feedback from Windows XP customers was positive on the quality of ClearType rendering on CRTs. After we made the choice, the feedback on the decision to enable ClearType as the default for Windows Vista was overwhelmingly positive.

Even with the default rendering set to ClearType, there are some scenarios that can change the default. If an OEM is providing Windows on their hardware, they can change the default. In some situations, and this was more common with font smoothing in Windows 95, the hardware may not meet the minimum requirements for the rendering technology. In the case of both font smoothing and ClearType, a minimum of sixteen bits per pixel display resolution is required. (When rendering to an off-screen bitmap in GDI, it is important that it not be the default color depth of 1 bit per pixel if you desire to capture ClearType text.) In some cases, when optimizing for system performance, font smoothing (both ClearType and grayscale) can be disabled. In a similar fashion, using Remote Desktop to connect to another computer or session usually disables ClearType by default.

Changing the Default Rendering in Windows 7

Windows 7 maintains the same defaults as Windows Vista. There are several ways for the user to change the default values for font rendering in Windows 7. For those that want the default rendering to be bi-level, the user interface for this selection is in the performance section of the Control Panel. From the root of the control panel you can access it by selecting System and Security –> System –> Advanced System Settings –> Performance (Settings…). An easier way is to enter “Appearance” into the start menu search, and then select “Adjust the appearance and performance of Windows.” The setting that should be changed under the custom option is: Smooth edges of screen fonts, as shown in the figure.

The option of no font smoothing as the default value is considered to be an uncommon setting, so it is a little more difficult to find than other settings. If the user prefers to change the default font rendering selection to the Windows grayscaling anti-aliasing technique described earlier, in Windows 7 that is now done through the ClearType Tuner.

ClearType Tuner

The quality of the ClearType text can be optimized for you and your monitor. The ClearType Tuner is a new control panel component for Windows 7. Because there are differences in monitor characteristics and differences between readers’ eyes, there are font rendering options that can only be optimized by a reader looking at text on their monitor. The ClearType Tuner uses various samples of ClearType, presented in the form of an eye-test, to make fine grained adjustments to the ClearType algorithms. Each wizard page tunes a parameter such as monitor gamma (relationship between voltage and brightness), your sensitivity to color artifacts, and your preference for letter heaviness.

In order to switch between ClearType and grayscale, the setting “Turn on ClearType” on the opening page of the ClearType Tuner can be toggled.

Either way, the user is taken through the rest of the ClearType Tuning wizard for two reasons; if an application explicitly enables ClearType rendering, it is useful for that experience to be tuned, and some graphics platforms have more fine tuning of the rendering for both gray rendering and ClearType.

 

Font Design and Font Rendering

The availability of higher resolution font rendering techniques like ClearType has had a significant impact on the design of fonts for onscreen reading. From the early days of the printing press, as new technologies and printing styles were developed, typefaces were redesigned to take advantage of those technologies. For example, many typefaces still in use today incorporate “ink traps” into the design so that ink would not clog up key features of a glyph. This is an aspect of making specific design choices in the font in order to work the best with the technology. In traditional typeface design, the term font refers to a typeface at a given size. So a 10 point Times New Roman would be a different font from a 24 point Times New Roman. In the days of metal cast typography, each of these sizes were designed by a punch cutter to be optimized for the medium for which they were to be used, often with changes in stem contrast, x-height, or character spacing for a given size. The advent of photo typesetting in the mid-twentieth century was a step backwards in this regard, as it used one size as a type master, and then used optics to scale that master size to any other presentation size.

Microsoft Windows has taken the more traditional approach to digital outline fonts, and through a combination of font hinting and new typeface design we attempt to optimize each size for the medium for which they were intended. With Microsoft’s initial release of TrueType for Windows 3.1, the traditional typefaces Times New Roman, Arial, and Courier New were used as core fonts. In the creation of these fonts, one master size was chosen for the outline data, usually something around 10 or 12 point, and, similar to the technique used in photo-typesetting, the outlines could be scaled to any requested size for a given display resolution. But, going back to the more traditional ways, each size was carefully examined and changes were made to the basic design through font hinting—including changes to critical features like stem contrast, x-height, or glyph spacing. As earlier mentioned, hinting fonts to be tuned for a low-resolution medium like full pixels on a 96 PPI screen was very time consuming. To help in this process for Microsoft Windows, we commissioned or designed in-house new outline typefaces designs that were optimized for the world of 96 PPI bi-level rendering. These custom fonts include Tahoma, Verdana, Georgia, Trebuchet MS, and even Comic Sans MS. These fonts still needed to be hinted to tune the individual sizes, but because the typeface was designed with the medium in mind, it was a more straightforward process and less time consuming.

Even with typefaces tuned to the display medium, 96 PPI pixels on a screen are still larger than many of the features we’d like to show in a typeface—and that is where ClearType helps us. Therefore, with ClearType, it made sense to commission a new set of fonts that were optimized for this new medium. Now the existing fonts for Windows still work well with the technology, but this project was an attempt to get the very best design for onscreen reading using ClearType. This led to a new set of fonts that shipped and were tuned for Windows Vista. The ClearType Collection fonts of Calibri, Cambria, Consolas, Corbel, Candara, Constantia, the new user interface font Segoe UI, and the Japanese font Meiryo were designed for this medium. As part of the engineering work on these font projects along with the default setting of ClearType, we decided in the hinting process to do the fine, size-specific hinting only for ClearType, and not for bi-level rendering. This allowed us to focus our efforts on the fine levels of detail and quality for the vast majority of customers.

ClearType Fonts in Windows 7

A reasonable question for us to ask ourselves is what is the experience like in Windows 7 when bi-level or hybrid font smoothing is chosen as the default?

As mentioned earlier, not all applications will choose to render with the default settings. Microsoft Office and Internet Explorer will default in some cases to using ClearType rendering. Some applications that use fonts tuned for ClearType and not bi-level rendering may choose ClearType rendering to maintain the benefits of the font designs. Some applications need higher precision glyph widths like sub-pixel positioning or “natural width ClearType,” and would reflow if they were changed to bi-level or grayscale rendering. Other applications like Adobe Reader have their own built-in text rendering engine that is independent of the Windows graphics platforms. Likewise, platforms like Java on Windows also use their own rendering techniques.

In some situations with the Windows 7 Explorer, ClearType rendering will remain on so that Segoe UI will keep its optimal design. Changing the system font from Segoe UI to some other font could be problematic, leading to issues like reflowing dialog box entries, missing text due to wrapping, unlabeled buttons, etc. We know many would value global changes to the fonts used by Windows, but to maintain to reliably across resolutions, DPI, and languages to name a few issues means we cannot have total flexibility on the system font settings at this time.

Given the challenges of turning off ClearType, there are a few mitigations in the fonts to handle some scenarios where ClearType is not available. In the ClearType font Calibri, since it is the default font for Microsoft Office, an unusual technique was used to attempt to improve the quality of the font rendering when font smoothing grayscale was selected. In this case, as opposed to the normal situation where font smoothing was disabled at lower text sizes to remove the blur, at these lower sizes the font enabled grayscale in order to improve the character shape. Also, at a few key sizes, the Calibri font had some bitmap fonts embedded in the outline file. These bitmaps kick in when bi-level rendering is requested. These bitmaps were intended to handle the case where Calibri was being used in a Remote Terminal situation and the default for Remote Terminal was not set to ClearType for performance reasons.

ClearType Research on Performance

As mentioned earlier, one of the goals behind ClearType is to improve the performance of reading text on computer screens. We have supported several areas of research looking into measuring this work. The research is done at universities and published in peer-reviewed journals. We have another Microsoft blog, that among other things related to fonts, also describes some of the research work on reading performance. Since those blog entries give more detail and background, we’ll just describe some of the performance highlights.

• We’ve measured an improvement in word recognition accuracy of 17% using ClearType over bi-level rendering. 
• We’ve found a 5% speed improvement in reading speed and a 2% improvement in comprehension (this is remarkable) using ClearType over bi-level rendering. A 5% reading speed improvement may sound small, but the cumulative effects can be huge given the amount of time people spend reading.
• We’ve found the reading speed improvements of 5% continue over longer spans of text, and we’ve found that non-traditional reading tasks like document scanning are about 8% faster with ClearType over bi-level rendering.
• We’ve found that reading sub-optimal text causes eye fatigue by increasing squinting and decreasing the blink rate. (This may seem obvious, but prior to this work there was no understanding of the physiological mechanisms of eye fatigue.)

ClearType Research on Rendering Preferences

Another research question we’ve asked ourselves is why do some people prefer bi-level rendering over ClearType? Is it due to hardware issues or is there some other attribute that we don’t understand about visual systems that is playing a role. This is an issue that has piqued our curiosity for some time. Our first attempt at looking further into this involved doing an informal and small-scale preference study in a community center near Microsoft. This was done with two identical laptops, one with ClearType and one with bi-level rendering. They were placed side by side and participants were asked which version they preferred. This was done with three different samples. Here were the results:

Comments:

1. 35 participants.
2. Comments for bi-level rendering:
   Washed out; jiggly; sketchy; if this were a printer, I’d say it needed a new cartridge; fading out – esp. the numbers, I have to squint to read this, is it my glasses or it is me?; I can’t focus on this; broken up; have to strain to read; jointed.
3. Comments for ClearType:
   More defined, Looks bold (several times), looks darker, clearer (4 times), looks like it’s a better computer screen (user suggested he’d pay $500 more for the better screen on a $2000 laptop), sort of more blue, solid, much easier to read (3 times), clean, crisp, I like it, shows up better, and my favorite: from an elderly woman who was rather put out that the question wasn’t harder: this seems so obvious (said with a sneer.)

Two other additional preference tests were performed with 28 of 30 participants preferring ClearType to bi-level rendering in one study and another with 52 of 55 participants preferring ClearType. Combining these three tests, we get 113 of 120 participants preferring ClearType rendering over bi-level rendering. It is important to note that in a forced preference test like this, just because someone preferred ClearType, it does not mean that they also don’t like bi-level rendering. It is just a preference towards ClearType.

Further examination of those who prefer bi-level rendering is of great interest to us and we continue to research this topic and to work with university researchers as well. We expect to see published papers on this topic in the future.

Future Research

Going forward, much of our research is in finding ways to make the highest quality text rendering more accessible to everyone. Each visual system has its own characteristics, and just as the ClearType tuner allows us to tune the algorithm for display characteristics, it would also be nice to tune for visual system characteristics. For example, in the United States 7% of the male population is color blind. We believe that we can improve the ClearType algorithm so that text for a colorblind reader is even better than for a reader without colorblindedness. Researching ways to improve text rendering for those with high color sensitivity and lower visual acuity would be just as important for us.

Conclusion

Making the screen the best possible place to read is an exciting opportunity for us.  It blends the engineering challenges of working with many display technologies and human visual systems with the artistic challenge of creating a beautiful set of glyphs, where every subtle typographic nuance is important.  In doing this, we need to keep in mind how the science of reading must guide us in making the experience optimal for us—humans. Each rendering technology has advantages and disadvantages for different people; depending on the application in use there are tradeoffs involved. Many of these issues go beyond the ability for people to easily discern choices. Our job is to work hard to provide a great platform for developers as well as tools that people can use to make choices and control how they use their technology. Our goal should be that the out-of-box experience just works. We think that, most of the time, we’ve accomplished this and we also recognize this area is complex and there is a wide spectrum of feedback.

The team at Microsoft working on these problems has been together since 1990, developing fonts and font-rendering solutions, and working to get a better understanding of the science of reading. The team is made up of engineers, type designers/artists, and psychologists and we work with many other experts throughout Microsoft in attempting to tackle this tough, yet vitally important task. You spend over 80% of the time at the computer reading, so it should be as pleasant an experience as possible. The following article from IEEE Spectrum describes some of the issues we deal with related to the technology, art, and science of text.

--Greg

Improving Audio Glitch Resilience in Windows 7

Delivering excellent audio playback on a PC is one of those “much harder than it looks” technical challenges.  Unlike dedicated audio / video devices, PCs have a lot going on during playback of audio and the playback happens on an incredible array of hardware and software.  Many of you might be familiar with “glitches” that occasionally happen.  In this post, Kristin Carr, a program manager on our Devices and Media team, describes some of the engineering in Windows 7 to improve this area representing the work of a number of folks across the team.  One lesson I learned early in the product cycle is that we don’t say “glitch-free” but rather “glitch-resilient” and hopefully that will make sense as you read this.  --Steven

Have you ever used your PC to play an MP3 or a DVD? If you answered yes, you’re among the overwhelming majority of PC customers who use their computer for audio and video applications, encompassing everything from watching a movie to playing a game to viewing a YouTube clip. But you may have also had an experience where your audio or video wasn’t quite perfect – perhaps the video was a bit choppy or the audio stuttered. We call this a ‘glitch’ – a perceived discontinuity in your audio or video that interrupts the playback experience. In this blog post, we’ll be focusing on audio glitching: we’ll examine the ecosystem challenges that can cause glitches, and we’ll discuss the work we’ve been doing to improve the Windows 7 experience.

 

What Causes Glitching?

In previous posts, we’ve touched on a variety of ecosystem initiatives and challenges that we’ve undertaken for Windows 7, including application compatibility, accessibility, and system performance, among others. Tracing the root cause of audio glitching leads us to a similar place: because Windows runs on a huge variety of hardware configurations and multitasks between dozens of applications, it is challenging to ensure that all of the programs and drivers running on your computer will work together in exactly the way you expect.

Audio is especially sensitive. In order for you to hear music from your speakers, data needs to be delivered to your audio hardware approximately every 10 milliseconds, or 30 times in the blink of an eye! The challenge is that your PC is usually doing a lot of other things at the same time you’re listening to music, such as streaming that YouTube video or downloading that new song, and many of these other tasks have complex timing requirements as well. As you can imagine, it doesn’t take much – a slow network driver or a graphics driver that requires plenty of CPU time – to prevent your audio from reaching your ears in a continuous fashion.

So what are we doing to address this challenge? The answer is ‘lots!’ – and the remainder of this blog post will be devoted to discussing these things:

1. Gathering data in order to characterize the problem
2. Developing a systematic method to detect and analyze glitches
3. Getting these tests and tools widely deployed, both at Microsoft and by our Windows partners
4. Engaging with partners to detect, diagnose and fix glitching issues

Who Experiences Glitching?

In studying this during the Windows 7 development cycle, our first order of business was to gather data. We‘d heard reports of audio glitching, but we didn’t know the exact scope of the problem. How often do users hear their audio glitching? Are there certain machines that were worse than others? With these questions in mind, we set out to understand our problem space a bit better.

We gathered data by using the telemetry infrastructure built into Windows, which allows our users to report back to Microsoft with performance data and other statistics that help us improve the OS. For each machine that opted to contribute data to Microsoft, we measured the number of times that the underlying audio hardware was being starved for data (i.e., when the user might hear a glitch). This data was grouped into “sessions,” each of which represents the data collected on a single machine for a single day or the data collected between machine reboots, whichever is shorter.

Let’s dive into some of the results. First, let’s look at the overall rate of audio glitching:

Figure 1: Distribution of Glitch Counts per Session

The chart above shows data from external (non-Microsoft) RC users. Approximately 80% of sessions showed no glitching at all, but 4.3% showed 10 or more glitches, which indicates that audio glitching affects a significant number of users.

Once we figured out how often glitching occurs, we started looking into why it occurs. First, we broke the data down by laptop/desktop form factor:

Figure 2: Glitching Likelihood by Form Factor

From this data, we noticed that laptops were almost twice as likely to experience audio glitching. As a result, we’ve made sure to address and target mobile PCs as well as mobile scenarios (for example, playing music while running on battery) for better coverage in our glitching tests and diagnostic tools.

Next, we looked at glitching likelihood by PC manufacturer:

Figure 3: Glitching Likelihood by PC Manufacturer (Mfr)

This data showed that certain manufacturers were more likely to be susceptible to audio glitching than others. As a result, we made sure to spread our testing efforts across a wide spectrum of machines and manufacturers. In addition, we are using this data to work with manufacturers to see if we can identify components or specific causes that would result in higher glitch incidents.

Finally, we looked at glitching on a wide variety of PC models:

Figure 4: Breakdown of All Glitch Sessions by PC Model

In the chart above, we examined all of the sessions that had at least one glitch, and we looked for any correlation with the PC make and model as shown in the table above (actual machine names have been anonymized). The first thing to notice is that Machine A is responsible for more than three times as much audio glitching as any of the other machines on the list. This data confirmed earlier reports of audio glitching on this particular machine, which we traced to a graphics card that shipped in a faulty configuration. As a result, we were able to work with the manufacturer to improve the configuration.

This chart also helps to show how widespread the issue is. There were hundreds of PC models that showed evidence of glitching – in fact, it seemed difficult to find a single PC model for which audio glitching did not ever occur. On the other hand, most individual machines didn’t show any problems at all. The conclusion that we drew was that audio glitching was not caused by any one hardware configuration, but was dependent on all the different hardware and driver permutations a user could possibly encounter on their machine. It was clear that no machine was immune, and in order to improve the experience, we were going to need a far-reaching, system-wide solution to this problem.

Developing Tools to Diagnose Glitching

Once we had data on when and why glitching occurs, the Windows Devices & Media Performance team developed a comprehensive suite of tests that were centered around media playback scenarios and were designed to assess how well a PC performed at that scenario. During media playback, these tests recorded thousands of statistics about the system’s performance, including CPU load, the activity of all components on the system and their corresponding interactions, and whether glitching occurred, among other things. We intentionally covered a huge range of scenarios and configurations, including laptops running on battery power, hardware under stress, hundreds of media content types, and many more. The goal was to exercise each PC in a wide variety of user scenarios in order to uncover and isolate audio glitches.

In addition, the Devices & Media Performance team created a graphical tool to highlight glitches as well as the CPU activities that occurred before and during an audio glitch, which allows us to quickly diagnose any glitching problems that we uncover. For example, in the figure shown below, we can see a visual representation of when glitches occurred, and we can display related measurements that occurred at the time of the glitching in order to easily pinpoint any suspicious behavior.

Figure 5: Example Graphical View of Audio Glitch Troubleshooting

In this case, you can see four audio glitches (shown by red vertical lines in the top panel). Two panels down, we have displayed calls to the CPU that took longer than 3ms (called long ISRs/DPCs). In this example, you can see a direct correlation between audio glitches and long ISRs and DPCs, which are procedure calls executed by the operating system that have the potential to hog the CPU and produce audio glitches. From here, we can track down the components responsible for these calls in order to reduce or eliminate the glitching. This figure shows additional information than what we used to diagnose the particular problem discussed above; however, this information and the many other measurements are available to diagnose other glitches and media performance issues from across a wide range of sources.

Putting the Tools to Work

Armed with these tests and tools, our next step was to deploy them on as many systems as possible. As part of this effort, we are participating in a Windows-wide initiative to help OEMs test their PCs at or before ship time. Hundreds of OEM machines get shipped to Microsoft for use in our Windows lab where we run thousands of tests in order to validate and ensure the best user experience. What this means is that if we notice that a particular machine or configuration might be susceptible to glitching, we can work with the OEM to try to fix the problem before the consumer ever sees their new PC.

By running these tests and analyzing the results with our new tools, we’ve been able to find hundreds of potential issues that would result in audio glitches. In some cases, this analysis resulted in changes to the Windows code. In other cases, we have identified components developed by our partners that can lead to audio glitching.

Engaging with Windows Partners

Since the issues we identify with these tools often involve components from many different partners, an important aspect of this work is engaging with these partners. Until now, it has been almost impossible for manufacturers to know how their components will affect the system as a whole, but by making these tests and tools available, we are attempting enable these partners to see how their components interact and what the final impact on users will be.

As part of this effort, we have been working to ensure that our partners can take full advantage of these new tools and tests. We’ve talked with OEMs, ODMs (original design manufacturers, who traditionally assemble the PC for the OEM), hardware manufacturers, and software vendors. We’ve given presentations and tutorials, written whitepapers, and held video conference workshops. Our goal has been to make it as easy as possible to create glitch-resilient software and hardware.

In summary, this effort includes:

1. Sharing audio glitching telemetry data with our partners. Our partners have had very little concrete data on the prevalence of audio glitching. With the data we are now collecting, we can help them to diagnose problems and improve their products.
2. Running our suite of audio and video performance tests on the hundreds of machines that OEMs send us and communicating the results to our partners. By assessing as many systems as possible and providing these results, we begin to tackle the causes of audio glitching.
3. Providing the tools and support that enable our partners to understand how their components are interacting with everything else on a PC and enable them to more easily address the subtle issues that can result in audio glitching.

What’s Next

Ultimately, we and all of our Windows partners share a common customer (you!); by working with our partners, calling attention to these issues, and providing more insight into the root causes of audio glitching, we are continue to improve the audio experience for everyone.

Creating, Saving, Sharing Themes in Windows 7

When we posted the new "inbox" desktop backgrouns, the reactions showed just how personal, personalization can be.  Building on that theme of personalization (pun intended), we wanted to share some of the work we did on themes in Windows 7.  We’ve shared data about customization in previous releases of Windows and this post builds on that.  This is also an area where we know there is very broad spectrum of desires (needs) for personalization and we definitely had to balance the engineering and design efforts.  I’ve received mail from many folks wanting to personalize (tweak) nearly every pixel on the screen—from border width, to title bar transparency percentage, to height of taskbar, to color/size/location of the close button (I’ve received each of these in email more than once).  At the other end are customers who are enormously happy when they can easily change the background picture and color scheme, and many do.  With Windows 7 we picked a group of settings that we believe represent the most satisfying settings to broadly personalize, and would also provide the most robust platform that maintains application compatibility, and made those easy to change.  In addition we wanted to make it easy to package up those settings so you could save and share them.  We think of this as the start of bringing robust personalization (and customization) to a broader set of customers.  Katie Frigon, a program manager on the core user experience team, authored this post. 

--Steven

PS: Things are "slowing" down as we have talked about in how we will get to the RTM milestone.  You might have noticed the announcement we made today in Asia regarding Windows 7 release and availability. Thank you to everyone who has been using the RC and helping to reach the next milestone.

Creating and Sharing Windows 7 Themes

In early builds, you may have noticed that Windows 7 includes a variety of themes that change your desktop background, window color and sounds with a single click. These themes are located in the Personalization Control Panel which is easily accessed from the desktop context menu.

Personalization Control Panel

Desktop Context Menu

In the RC, you can see a number of new themes, for example the “Architecture” theme. This theme is comprised of six architectural photos which cycle on the desktop background, a complementary “Twilight” window color and the “Cityscape” sound scheme which was inspired by the sounds of an urban jazz club.

A theme is a coordinated set of Desktop Backgrounds, Window Colors and Sounds.

Windows provides a set of themes in box and if customers want more there is a prominent link in the Control Panel to get additional themes online. This link takes you to the Windows Online theme gallery where Microsoft provides additional content including a variety of international themes.

Personalization Control Panel: Get more theme online link

Creating a theme

While our customers enjoy the content we’ve provided both in the box and online we also know that they enjoy and desire the option to customize their PC’s even more than choosing a theme. Windows 7 continues to be about your PC reflecting you and what you do, as well as putting you in control of that experience. So, if you do want to go beyond the options in the box and on the web, it is easy to create and share your own themes. Creating your own theme can be as easy as just changing your desktop background image while keeping the rest of the settings the same or you can change all the settings one-by-one.

From our Beta Customer Experience Improvement Program data we see that customers are changing and creating themes. We also see many users changing the different settings, the most popular being desktop background:

Figure 1: Break out of theme type

Note: Only 15% of the beta users kept the default theme. 77% of the beta users created a custom theme by changing one or more elements of the inbox themes.

Figure 2: Percentage of Beta users selecting each theme component in a session

Note: 35% of beta users who opened the Personalization CPL clicked on “Desktop Background”.

Now let’s look at how you can change the different settings and save a custom theme. To start, you can change any of the theme settings by starting in the Personalization Control Panel.

Personalization Control Panel: Click on the items beneath the theme gallery to change your theme settings.

Let’s start with the desktop background control panel. This control panel has been enhanced for Windows 7 to support the pictures library and the new desktop background slideshow capabilities. If you choose the “Pictures Library”, we will show all of the pictures in that library including subfolders. All you need to do is select more than one photo to have them cycle as your desktop background slideshow. In this example, I have selected some of my favorite photos from a recent trip to Hawaii to use as my desktop background.

Desktop Background Control Panel: Windows 7 adds support for libraries and desktop
background slideshows. I’ve selected the pictures I want to use in my theme.

When personalizing your PC, you might want to go further than just changing your background. Changing your window color or sound scheme is simple, just click on the items beneath the themes gallery. We provide 16 window colors to choose from and the ability to pick a custom color as well. New to Windows 7, we include 14 sound schemes with the OS inspired by a variety of regional music traditions, so you have plenty to choose from. If that isn’t enough, you can include your own sounds if you want.

Window Color and Sound Control Panels: It is also easy to change your window color
or pick from 14 diverse sound schemes.

After you change the desktop background, window color or sound scheme, you will notice that we have created a new “unsaved theme” that contains your changes. Your unsaved settings will be preserved when trying other themes in the gallery so you can get back to your most recent customizations. If you are happy with your personalization settings, you can ensure that they are always available in the themes gallery by clicking “Save theme”.

Personalization Control Panel: I clicked "Save Theme" to ensure that my current
personalization settings will always be available in the themes gallery.

Sharing themes

After saving your personalization settings for your own use, you might want to share these settings with friends and family or bring the settings to another PC. Windows 7 allows you to share your themes by right-clicking on your current theme and selecting “Save theme for sharing”. After specifying a name and folder destination for your theme, Windows will collect all of your custom desktop background images, sounds, mouse pointers and icons into the new .themepack file format that can be applied on another computer running Windows 7.

Personalization Control Panel: When I’m ready to share my theme with Friends, Family and on the Web,
I right-click on my current theme and select “Save theme for sharing”.

Sometimes after I take a fun vacation I like to create a theme that reminds me of the trip. To do this I select the best photos from the trip to rotate as my desktop background and then pair those with a matching window color and Windows 7 sound scheme that best matches the mood of the trip. After I save as a new .themepack I can either share this file via Windows Live to friends and family or use it from another PC in my house via Homegroup.

Sharing with Windows Live

Since all of the personalization settings are now contained in a single file, it’s easy to upload the theme to Windows Live Skydrive and post a link to the theme on a Windows Live Spaces blog. Once my friends and family upgrade to Windows 7, they will be able to download themes from trips that we went on together so they can enjoy my photos on their desktop background.

Windows Live: I can also upload my theme to my Windows Live Skydrive
and add a link to the theme on my blog.

Sharing via Homegroup

In Explorer you can create a themes Library. Then from another computer in a Homegroup you just browse to the shared location and click on the desired theme to apply those settings with a single click.

Explorer: I created a themes library on one of my PC’s and shared it with my Homegroup.
From another PC in the home, I can click on any of these themes to apply them.

But wait…there’s more.

One additional way we’ve added value with Windows7 themes is by capitalizing on the growing popularity of RSS photo feeds to share photos. Enthusiasts can create a theme where the desktop background slide show points to an RSS photo feed. For example, my sister lives across the country and we only see each other about once a year. An easy way for me to keep her up to date on my family is to send her a Windows 7 theme which points to my RSS photo feed. When I upload new photos they will appear on her desktop automatically.

Because there are a few different ways to create an RSS photo feed, the process to include an RSS photo feed in a Windows 7 theme will only work if your RSS photo feed links to the high resolution photos using the “enclosures” method. The feed should only reference picture formats such as JPEG or PNG. Due to this limitation themes must be created manually when including an RSS photo feed.

So, to create one of these themes you can follow these steps:

1. Download the template from MSDN.
2. Open the template using Notepad.
3. Replace {themename} with the name you want to appear in the Personalization Control Panel themes gallery.
4. Replace {rssfeedurl} with the full path to your compatible RSS photo feed.
5. Save the changes as a file with the “.theme” extension.

It is ready for you to share! Send the file via email, etc. to your friends and family.

Photo sharing sites can also offer these Windows 7 RSS photo themes which provide more ways to connect their customers.

Looking ahead

Themes in Windows 7 make it possible for you to make the PC reflect you. Beyond my example of sharing personal photos as a theme, we hope that users will find new and creative ways to use themes in Windows 7. Wedding photographers can include Windows 7 themes in the packages they deliver to their clients, Artists can create themes that showcase their creative style and businesses can create themes that promote their brand. We look forward to seeing how you are using themes to Personalize these aspects Windows 7.

--Katie

PS: We've posted some additional themes you can download and use on http://windows.microsoft.com/en-US/Windows7/Personalize which is the US English link from the Themes control panel.

Safeguarding Windows 7 – Parental Controls

As you can imagine, our team is quite busy working through this next phase of Windows 7.  We definitely appreciate the millions of downloads and installs of the Windows 7 RC.  Things are going as we expect at this point.  On a personal note, I wanted to thank all the folks who have been sending me mail.  I’ve received a lot of kind words and support regarding the RC and quite a few people saying “hurry up and just release it”.  We outlined the steps we’re taking for this next milestone and aren’t going to rush things.  We’ve got a lot of work for sure!  Not that I’m counting, but I just crossed over 3,000 emails sent via the contact link in this blog.  While I haven’t answered all of them, I’ve done the best I can, and appreciate each and every exchange. 

Windows 7 includes a set of features for safeguarding your PC when used by children.  This post is by Vladimir Rovinsky, a program manager on our Safety Team, who details the features in Windows 7 specifically around Parental Controls.  This work is in addition to the safety of the OS itself and of course the features built into Internet Explorer to provide safety and security while browsing.  You might also want to check out Windows Live Family Safety which is part of Windows Live Essentials (http://download.live.com) which provides even more for safety and parental controls.  --Steven

Today, children are exposed to digital hazards more easily than any time in the past. Especially with the help of powerful search tools, convenient social networking applications, low cost tools and services for publishing videos and photographs, the web is awash with content that’s inappropriate for children, and full of people that parents want to bar from contacting their children.

These digital hazards are accessible to children through a variety of applications, including web browsers, instant messaging applications, media players, games, and email applications. Many of these applications have attempted to offer parental control features. However, they offer this functionality through variety of user interfaces, locations and include varied terminology. The duplication and inconsistency of parental control settings management can make it difficult for parents to maintain the correct settings across multiple applications.

Windows Vista Parental Controls provided a framework to solve these problems by offering:

• A single, central location in the Windows Control Panel to configure and manage parental control settings and activities;
• Built-in restrictions on web content and file downloads, time spent on the computer, application usage, game usage as well as the ability to log and view user activity.
• The Windows Parental Control platform public application programming interfaces (API) which expose in-box restriction settings and logging functionality to any application. For instance, Internet Explorer and Mozilla Firefox 3.0 are using these APIs to determine if file downloads should be blocked for a user.
• Integration with the User Account Control (UAC) to enforce standard user accounts for parentally controlled users; promotion of best practices for keeping kids safer on a Windows computer; for instance, encouraging the creation of separate standard accounts for managed children, password creation for parent accounts (administrators), etc.

To get a quick demo of Windows Vista Parental Controls in action, check out this video.

For more information about developing software for Windows Vista Parental Controls, see Using Parental Controls APIs.

Key Design Decisions for updates to Windows 7 Parental Controls

Responding to customer feedback and evolving nature of the web and challenges it poses to the parents, we strive to provide families with flexible and effective safety features. Our efforts for the Windows 7 release of Parental Controls were focused on the following objectives:

1. Further developing the extensibility of the Parental Controls platform to enable third-party developers to create richer Parental Control capabilities that integrate well with Windows 7 Parental Controls.

The Windows 7 Parental Controls platform was modified to allow multiple independent providers of Parental Controls functionality to be installed on the system and augment or fully replace the parental controls provided by Windows 7. Windows Vista allowed partial replacement of Windows Parental Controls; the web filter was replaceable. In Windows 7, in addition to the web filter components, the entire Windows 7 Parental Controls user interface can be replaced by third-party providers. The underlying enforcement of the offline restrictions will still be performed by Windows Parental Controls platform. Allowing a third party provider to replace the entire Windows Parental Controls user interface creates a consistent user experience that seamlessly combines existing Parental Controls functionality with the new ones introduced by the third-party provider.

The Windows Control Panel Parental Controls screen still remains the central location and launching point on Windows 7 for Parental Controls functionality regardless of whether it is provided by default (system) or by a third-party provider.

2. Removal of web content restrictions and activity viewing functionality from default (system) Parental controls provider and reliance on Windows Live or third-party providers for these capabilities.

The web is changing much faster than we can update the Windows operating system. For example, when Vista was released Social Networking was barely known. Now it has a thriving web presence. We need to keep web focused parental controls up with innovation. Because of this, we have moved them into Windows Live.

Web filtering and activity viewing capabilities can be more efficiently provided by Windows Live or a third-party solution that implement web based delivery of this functionality. For instance, Microsoft’s Windows Live Family Safety free application provides web content filtering, file downloads restrictions, and activity monitoring. It also provides online contact restrictions for children using Windows Live online applications (Windows Live Hotmail, Windows Live Messenger, etc).

You can learn more about Windows Live Family Safety solution here.

More information about Windows 7 changes to the Parental Controls platform can be found here.

Windows 7 Parental Controls User Interface Changes.

Elements new to Windows 7 Parental controls top-level screen can be seen on the following screen shot:

Figure 1 Windows 7 Parental Controls screen

1. The Additional controls section allows users to select a provider for additional controls such as web filtering, activity reporting, online contact management, etc. When a third-party controls provider’s installed on the computer, the screen displays the Select a provider drop down box that shows the currently selected (active) provider. A description of the provider’s functionality, as supplied by the provider, is shown below the drop down.
2. When the user account is selected by clicking user’s name or picture, the provider configuration for the user is launched. The provider can take over the default configuration UI for the in-box offline restrictions.  Optionally, provider generated status strings for user accounts are displayed under user account pictures.
3. An Icon supplied by provider is shown in the upper right corner of the screen.

Additional control providers can still rely on the default’s (system) provider UI for the configuration of in-box offline restrictions. If a provider chooses to do so, the User Controls screen can be presented to configure a user’s Parental Controls settings.

If an additional provider is selected and configured, the following new user interface elements are shown on the Windows 7 User Controls screen:

Figure 2 Windows 7 User Controls screen. Additional controls provider is installed and configured.

1. More Settings allows direct access to the currently selected provider’s functionality.
2. Web Restrictions allows access to the currently selected provider’s functionality.

Windows Parental Controls settings and Vista to Windows 7 upgrade

If a Windows Vista PC which has parentally managed user accounts with enabled web filtering restrictions is upgraded to Windows 7, parents (administrators) are warned during the upgrade as well as when opening the Windows 7 Parental Controls screen, that web filtering and activity reporting functionality is not part of Windows 7 Parental Controls.

 

Figure 3 Windows 7 Parental Controls screen. Some users have web filtering restrictions. No additional provider is installed.

Windows Vista Parental Controls settings (including web filtering and activity logs information) are preserved unchanged when upgrading from Windows Vista to Windows 7. Although web filtering settings and activity logs information are not used by Windows 7 Parental controls, their preservation allows third-party provider to honor these settings.

As you start using Windows 7, we hope these changes to Parental Controls capabilities will make you feel more confident and in control of how your family members are using computers and experiencing the web.

--Vladimir

Media Streaming with Windows 7

We’ve blogged about a number of features related to home networking and media in Windows 7.  A scenario which brings all these together in a pretty cool way is Media Streaming.  This scenario allows you to use a Windows 7 PC as a hub for media sharing—where you can share media with other PCs and devices on your home network via streaming, and even stream this information securely over the internet.  Scott Manchester on the Devices & Media program management team coordinated this post, but as you will see it represents work across the Core User Experience, Media Center, Networking, and even Windows Live chose to take advantage of the new APIs in this scenario.  This is a pretty detailed post and there’s a lot to try out.  Those of you using the RC to test things out, you can always install on another PC and use it for the 30-day period without requiring a new PID key.  Have fun!  --Steven

Windows 7 includes a number of exciting new media streaming features that enable you to enjoy your media collection on other PCs and devices in the home and while on the road from across the internet. We’ve created a networked media experience that is more friendly to use and simpler to set up. Now enjoying music, pictures, and video on your network connected PC or media device “just works” without concern for media formats, transports, or protocols.

There are a growing number of Network Media Devices (NMDs) certified to interoperate using an open and widely embraced industry standard called the Digital Living Network Alliance (DLNA). Windows 7 implements this open standard, which means that sharing media between NMDs, Windows PCs, Windows Home Server, and Extenders for Windows Media Center (including Xbox 360) is easier and more natural. Supporting this standard also means that the myriad of NMDs such as electronic picture frames, network radios, televisions, and others are companions to Windows 7 PCs and may seamlessly participate in the whole-home media experience.

Not Just for the Techie

We made it much simpler to configure media streaming. Before Windows 7, media streaming features were focused on media enthusiasts. To improve the setup experience, media streaming has been integrated with the new HomeGroup feature so in a typical home network configuration, media streaming is enabled and works by default. There is also a new “Stream” menu prominently displayed in the Window Media Player user interface (see figure below) that exposes simple scenario-based configuration options. These options allow you to:

1. Set up your home PC so you can access your media libraries while away from home
2. Allow other Windows 7 PCs and devices to push media to your Player and control it
3. Quickly authorize all home PCs and devices to access your media collection

Each of these scenarios will be discussed throughout this post.

HomeGroup introduces the concept of “shared libraries” for music, pictures, and video. As described in a previous blog post, these shared libraries are accessible from within the navigation pane of Windows Explorer and Windows Media Player, and from the “shared” view of each media category within Windows Media Center (see figures below). The scope of these libraries is the same from each of these views.

Windows Explorer will automatically discover and provide access to shared media libraries on other HomeGroup PCs. In addition, Windows Media Player and Windows Media Center will automatically discover shared libraries from:

1. Windows Media Player 11 and 12
2. Windows Home Server
3. All DLNA compliant media servers (e.g. network attached storage)

Who Can Access My Shared Media Libraries?

A HomeGroup is a secured set of Windows 7 PCs that can view and consume each other’s media seamlessly. Sharing is automatically set up among HomeGroup PCs and HomeGroup settings allow you to choose what types of media you would like to share; for example, you may choose to only share your music library and not your video or pictures.

In addition to all HomeGroup PCs being able to access your media, we made it easy to allow devices to access shared media libraries on Windows 7 PCs. This can be done conveniently from either HomeGroup settings or within Windows Media Player:

You can also choose to restrict which specific PCs or devices have access to your media by choosing “more streaming options…” from the Windows Media Player “Stream” menu.

Play To: Windows 7 as a Universal Remote Control for your Media Collection

In addition to playing media streamed from other shared media libraries within Windows Media Player, Windows 7 can now send media to be played on other Windows 7 PCs and DLNA-certified digital media renderers. We call this feature “Play To.” With “Play To,” you can browse or search from within Windows Media Player or Windows Explorer to find your desired media, and then choose where you want it to be played. A versatile remote control window is presented for each “Play To” session, providing you with the ability to control the entire experience.

It does not matter where media collections are stored. “Play To” is available for both local media libraries and for shared media libraries. If you would like to send media from one Windows 7 PC to another, choose “Allow remote control of my Player” from the Windows Media Player “Stream” menu on the receiving PC. This will cause Windows Media Player to be discovered in the “Play To” menu of other Windows 7 PCs on the same network.

When media streaming is enabled on your Windows 7 PC, “Play To” will be available in Windows Media Player and Windows Explorer via the right click menu for media items. If Windows 7 has not discovered a “Play To” capable PC or device on the network, this context menu will not be available. DLNA provides guidelines to certify different device categories and roles. Not every DLNA-certified device supports the “Play To” feature. Look for DLNA-certified Digital Media Renderers (DMR), and for the best performance, look for DMR devices that carry the “Compatible with Windows 7” logo.

Once you’ve selected media items to play on another PC or device, a “Play To” remote control window will launch providing standard controls like play, pause, stop, skip forward and backward, seek forward and backward, volume, and mute. Not every device will support all of the control features and some media types may not support seek. Once the “Play To” remote control window is launched, you can reorder or delete items, add to the queue, or toggle repeat. It’s even possible to add new media items from Windows Media Player or Windows Explorer by dragging them into this window.

There is no artificial limit to the number of “Play To” sessions you can launch. You may send pictures to a picture frame, video clips to a TV, and music to another Windows 7 laptop all at the same time. Furthermore, different types of media can be sent to a single destination, as shown in the example above.

What About the Xbox 360 and Extenders for Windows Media Center?

Xbox 360 has two ways to receive media streams from other Windows 7 PCs, which we refer to casually as “dashboard” mode and “extender” mode.

In dashboard mode, Xbox 360 functions in the role of a simple media player. While it’s not officially a DLNA-certified device, you can use Xbox 360 to browse the shared media libraries from Windows 7 PCs (there is also support for this in Windows Media Player 11) and pull content from those libraries for playback within the dashboard.

In extender mode, Xbox 360 (and other Extenders for Windows Media Center) is seen by Windows 7 PC’s on the network as both a Digital Media Player (DMP) and a Digital Media Renderer (DMR) device. Using the Extender for Windows Media Center on the Xbox 360, you can browse media libraries on other computers and pull that content for local playback, similar to the process of using Xbox 360 in dashboard mode. However, in extender mode Xbox 360 will also support “Play To” so that users of Windows 7 PC’s on the network can push content to it. All extenders, when associated with a Windows 7 PC, will be discovered in the “Play To” menu of other Windows 7 PCs.

Internet Access to Home Media

With Windows 7 we’ve also extended the media streaming experience outside the home and allow you to access your home media from anywhere in the world via the internet. We’ve made media streaming over the internet a natural extension of the experience within the home. For the experience to be seamless we needed to solve some significant technical challenges, such as:

1. Discovery – Resolving the computer name at home to a routable IP address
2. Privacy – Ensuring the home media is only accessible by authorized users
3. Security – Encrypting browsing and streaming of media to prevent eavesdropping
4. Reliability – Network connection speeds, media formats and bit rates, and router firewalls all create potential reliability issues for a seamless experience

To overcome these technical hurdles, we designed a model that uses an Online ID Provider to help facilitate discovery, privacy, and security. The new Online ID Provider infrastructure in Windows 7 allows you to link your Online ID (e.g. you@live.com) with your Windows user account. This enables an authentication/authorization server to provide the necessary privacy to establish a protected link between two Windows 7 PCs (e.g. your laptop on the road and your PC at home). Internet access to home media is enabled from the “Stream” menu in Windows Media Player.

The setup process walks you through linking an online ID with your Windows user account, which must be performed on both the home PC and remote PC. The same online ID must be used on both PCs in order to establish the connection between them. In order for remote PCs to access the home media collection, the PC at home (acting as a server) must be on a “Home” network location. Remote PCs (acting as clients) can browse and receive content streamed from the home PC from any network location (Public, Work, or Home). The network location is chosen when first connecting to any network and can be changed later from the Network and Sharing Center.

Reliability - Network Connection Requirements

Streaming media over the internet from home works best with an “always on” broadband connection. Broadband uplink speeds vary from a modest 200Kbps to 10Mbps or more. Downlink connection speeds will also vary from crowded hotspots, hotel rooms, and wireless network connections in friends’ homes. Regardless of the uplink or downlink speeds, we wanted to ensure that even high bit rate content (e.g. high definition recorded TV) could be streamed with a good experience. The internet media streaming feature uses advanced bandwidth detection algorithms and end-to-end network heuristics to determine how to stream content that is at a higher bit rate than the smallest link in the network path.

Another challenge with internet access to home media is creating a peer-to-peer connection between the remote client PC and the home PC serving the media. A typical home network will get a single unique IP address from an internet service provider, and this IP address is shared by all the devices and PCs in the home using Network Address Translation (NAT), a function of an Internet Gateway Device (IGD) or Wireless Router. This creates a challenge for a remote PC or device to make an unsolicited connection inside the home, both in terms of resolving the home’s unique IP address and traversing the NAT to communicate directly to a unique PC or device on the home network.

Windows 7 employs some advanced NAT traversal technologies to establish the peer-to-peer connection and, with most IGDs, will allow a reliable connection to the home PC from any remote PC. For best results you should use a wireless router or IGD that has been certified by the Windows Logo program.

Media Formats

In Windows 7 we let you enjoy the media you want and don’t trouble you with the need to know about file types or codecs in most cases. (For more details, see Table 1 below). In addition to supporting local playback of new formats, we can also ensure that the content will play on devices that may not support the codec, bit rate, container, or format of that content. We accomplish this by using the new transcoding support in Windows 7.

Let’s say for instance you have a DivX movie you want to watch on your new DLNA certified television which only supports WMV and MPEG2. Windows 7 will determine the capability of the TV (codec, bit rate, etc.) and dynamically convert the DivX video to a format the TV can play. The general rule of thumb is: if Windows Media Player can play the content on the PC then the content will almost always play back on the network connected device. Bandwidth estimation techniques are used for media streaming within the home and over the internet, which enables Windows 7 to transcode using the most optimal format and bit rate.

 Table 1: New Decoders in Windows 7

The format and bit rate chosen for transcoding, especially for video, is highly dependent on the CPU performance of the transcoding PC as identified by its Windows Experience Index:

We also created a flexible model for silicon partners to provide hardware accelerators that automatically work with media streaming and other Windows 7 features. This new acceleration model allows hardware developers to build media foundation proxies for media format encoders and decoders that are fully implemented in their hardware (perhaps in a GPU or additional hardware device). With hardware supported encoding and decoding, Windows 7 can offload the computationally demanding transcoding to dedicated hardware as a background task without affecting the CPU performance of the PC.

Digital Living Network Support in Windows 7

The Digital Living Network Alliance (DLNA) is a consortium of more than 200 companies interested in specifying technologies for exchanging media in home networks. The DLNA architecture is based on the UPnP specification, but in addition, DLNA specifies transport protocols (based on HTTP and RTP) and sets of media formats.

DLNA defines device roles (e.g. servers, players, renderers, etc.) and the protocols that these devices use to discover each other and communicate with each other (e.g. UPnP, HTTP, RTP, etc.). Windows 7 implements several of the DLNA device roles (see table 2 below) and it also implements the DLNA protocols required for communications and media exchange. With Windows 7, your PC will be able to interoperate with a broad variety of DLNA certified devices like TVs, stereo systems, cell phones, DVRs, game consoles, etc.

Table 2: DLNA Device Profiles Supported by Windows 7

Because Windows 7 implements several device roles, there are different ways in which you could choose to use a Windows 7 PC at home. The remainder of this section explains the different scenarios.

Scenario 1: You store your music, video, and pictures on a Windows 7 PC. You’ve recently acquired a TV with a DLNA logo. Using the TV, you can browse the media library available on the Windows 7 PC. You can use the TV to watch the video and pictures, and listen to music stored on the PC. Figure 1 illustrates this scenario. In this case, the Windows 7 PC behaves as a DMS. Notice that this scenario was already available in Windows Vista and in Windows XP using Windows Media Player 11.

Figure 1: The TV unit browses and plays content stored in a PC

Scenario 2: You have a Network Attached Storage (NAS) device where you store your music, video, and pictures. The NAS device implements a DMS. You open Windows Media Player on a Windows 7 PC. You can find the NAS device using Windows Media Player, and you can browse the media library available on the NAS device. You can watch the video or pictures, and listen to music stored on the NAS device. Figure 2 illustrates this scenario. In this case, the Windows 7 PC behaves as a DMP.

Figure 2: A Windows 7 PC browses and plays content stored on a NAS device

Scenario 3: You have a cell phone that not only takes pictures but can push the pictures to a Windows 7 PC. You can show the pictures to your friends using the large-screen display of the PC without the need to physically transfer the files to the PC with a USB thumb drive, for example. Figure 3 illustrates this scenario. In this case, the cell phone acts as a DMS and a DMC and the Windows 7 PC behaves as a DMR.

Figure 3: A cell phone pushes pictures for display on a Windows7 PC

Scenario 4: You’ve acquired a stereo system with the DLNA logo. On his Windows 7 PC, you’ve accumulated a vast collection of music with thousands of songs. Because your collection is large, you prefer to search, organize, and select songs using the rich capabilities of the Windows Media Player. Once you select the songs, you simply push the songs to your stereo system using “Play To.” You also have a NAS device containing an additional collection of music and video. You can use the Windows 7 PC to browse the content on the NAS device and push it to the stereo system. Figure 4 illustrates this scenario. In this case, the Windows 7 PC behaves as a DMS and a DMC.

Figure 4: A Windows 7 PC browses local content or shared content on the network. The PC then pushes the content for playback in a TV unit (DMR).

There's definitely a lot to enjoy here.  Have fun!! 

-- Scott, Tim and the Devices & Media team

Our Next Engineering Milestone

Back in January we released the Beta and updated you on our overall engineering process that will get us from Beta to the Release Candidate.  Today, downloading of the Release Candidate started and we’re already seeing a lot of installations and a lot of excitement.  On behalf of the team, I want to extend a thank you for all of the millions of people who have been running and testing the Beta who have helped to make the Release Candidate possible.  The feedback we have received, through all the mechanisms we have blogged about, has been an incredibly valuable part of Engineering Windows 7.  We continue to be humbled by the response to Windows 7.  Thank you!

This post is about the path from RC to what we call RTM, release to manufacturing.  RTM is not one point in time but a “process” as from RTM we enable the PC manufacturers to begin their processes of building Windows 7 images for new PCs, readying downloads for existing machines, and preparing the full supply chain to deliver Windows 7 to customers.  Thus RTM is the final stage in our engineering of Windows 7, but the engineering continues from RTM until you can purchase Windows 7 and Windows 7 PCs in stores at General Availability, or GA.

The path to RTM starts with downloads of the RC.  The RC is “done” and what we are doing is validating this against the breadth of the ecosystem and with partners.  It means, from our perspective, we have run many tests many times and are working to understand the quality of the release in a breadth sense.  We’re all familiar with this as we have done this same thing as we went from pre-Beta to Beta and from Beta to RC.  The primary difference with the RC is that we will not be changing the functionality or features of the product at this point—that’s the sort of thing we’ll save for a future release.   We’ve gotten tons of feedback on design and features and shown how we have digested and acted on this feedback throughout many posts on this blog.   We know we did not do everything that was asked, and we have also seen that we’ve been asked to do things that are tricky to reconcile.  We hoped through the dialog on this blog that we’ve shown our commitment to listening and balancing a wide variety of inputs, and how we have thought about the evolution of Windows.  

What sort of feedback are we looking for in the RC?  We are primarily focused on monitoring the behavior of the product through the telemetry, and of course making sure we did not introduce any regressions in any dimension from Beta quality.  One of the things we have done since Beta has continued to beef up telemetry—we’ve put in additional monitoring points in many systems.  We’re particularly interested in seeing what devices are installed, drivers that are required, and overall system performance.  We have telemetry points that monitor the UI responsiveness of the Start Menu, Internet Explorer (recently posted), Boot, Shutdown, Resume, and across all subsystems.  Of course in the final product, this telemetry is optional and opt-in, and it is always private. 

There are a series of specific types of reports that we are keeping an eye out for that would constitute changes we would make to the code between now and RTM.  Some of these might include:

• Installation – We have significant telemetry in the setup process and also significant logging.  Of course if you can’t set up at all that is something we are interested in and the same holds for upgrades from Windows Vista.  For the “enrolled” beta programs we have a mechanism to enlist a connection to Microsoft for these issues and for the broad community the public support groups are monitored. 
• Security issues – Obviously any vulnerability is a potential for something we would fix.  We will use the same criteria to address these issues as we would for any in-market product. 
• Crashes and Hangs – We are monitoring the “crash” reports for issues that arise that impact broad sets of people.  These could be Windows code, drivers, or third party software.  This information streams “real time” to Microsoft and we watch it very carefully.
• Device installation and compatibility – When you download a driver from Windows Update or install a driver via a manufacturer’s setup program this is a data point we collect.  We’ve had millions of unique PnP IDs through the Beta.  We also receive the IDs for devices that failed to locate drivers.  We are constantly updating this web service with pointers to information about the device (driver availability, instructions, etc.)
• Software installation – Similar to devices, we are also monitoring the installation process of software and noting programs that do not complete successfully.  Again we have the mechanism to help move that foreword and/or introduce compatibility work in the RTM milestone.
• Servicing – We will continue to test the servicing of Windows 7 so everyone should expect updates to be made available via Windows Update.  This includes new drivers and will also include patches to Windows 7.  Test Updates will be labeled as such.  We might also fix any significant issue with new code as well.  All of this in an effort to validate the servicing pipeline and to maintain the quality of the RC.
• New Hardware – Perhaps the most important category is making sure that we work with all the new hardware being made as we all use 7100.  Our PC Manufacturing partners and Hardware partners are engineering new PCs and these are combinations new to the market and new to the OS.  We’re working together to make sure Windows 7 has great support for these PCs and hardware.

All of the feedback will be evaluated and whether the issue is with Windows itself or with hardware, software, or OEM partner code we will work closely across the entire ecosystem to do what is necessary to deliver excellent fully integrated PCs.  This goal is more important than anything else at this point.  The depth of this work is new for the team in terms of spending engineer to engineer time across a broad range of partners to make sure everyone is ready together to deliver a great PC experience.

Overall, while many have said that the quality of the Beta was on par with past RCs (remember how some even suggested we release it as final!), we are working to do an even better job with Windows 7.  We think we have the tools in place to do that. 

While the RC itself was compiled about 2 weeks ago, it takes a bit of time to go through the mechanics of validating all the ISOs and images that are released.  In the meantime we continue doing daily builds of the product.  The daily builds are incorporating code changes to address the above types of issues that impact enough customers that on balance the code change is more valuable than the potential of a regression.  Throughout this process, every change to the code is looked at by many people across development and test, and across many different teams.  We have a lot of engineers changing a very little bit of code.  We often say that shipping a major product means “slowing everything down”.  Right now we’re being very deliberate with every change we make.

The RTM milestone is not a date, but a process.  As that process concludes, we are done changing the code and are officially “servicing” Windows 7.  That means any subsequent changes are delivered as fixes (KB articles) or banked for the first service pack.  Obviously our ability to deliver fixes via Windows Update has substantially changed the way we RTM and so it is not unreasonable to expect updates soon after the product is complete as we have done for both Windows XP and Windows Vista. 

Between now and the RTM milestone we will make changes to the code in response the above inputs.  We are decelerating and will do so “gracefully” and not abruptly.  We do not have a “deadline” we are aiming to meet and the quality (in all dimensions) of the product and a smooth finish are the most important criteria for Windows 7.  In addition, we have a lot of work going on behind the scenes to build Windows 7 in nearly 100 languages around the world and to make sure all the supporting materials such as our Windows web site, SDK, resource kits, and so on are ready and available in a timely manner.

Once we have entered the RTM phase, our partners will begin to make their final images and manufacture PCs, and hardware and software vendors will ready their Windows 7 support and new products.  We will also begin to manufacture retail boxes for shipment around the world.  We will continue to work with our enterprise customers as well and based on the RTM process the volume license products will be available as well.

Delivering the highest quality Windows 7 is the most important criteria for us at this point—quality in every dimension.  The RTM process is designed to be deliberate and maintain the overall engineering integrity of the system.  Many are pushing us to release the product sooner rather than later, but our focus remains on a high quality release.

Ultimately our partners will determine when their PCs are available in market.  If the feedback and telemetry on Windows 7 match our expectations then we will enter the final phases of the RTM process in about 3 months.  If we are successful in that, then we tracking to our shared goal of having PCs with Windows 7 available this Holiday season. 

--Steven and Jon

Support and Q&A for Solid-State Drives

There’s a lot of excitement around the potential for the widespread adoption of solid-state drives (SSD) for primary storage, particularly on laptops and also among many folks in the server world.  As with any new technology, as it is introduced we often need to revisit the assumptions baked into the overall system (OS, device support, applications) as a result of the performance characteristics of the technologies in use.  This post looks at the way we have tuned Windows 7 to the current generation of SSDs.  This is a rapidly moving area and we expect that there will continue to be ways we will tune Windows and we also expect the technology to continue to evolve, perhaps introducing new tradeoffs or challenging other underlying assumptions.  Michael Fortin authored this post with help from many folks across the storage and fundamentals teams.  --Steven

Many of today’s Solid State Drives (SSDs) offer the promise of improved performance, more consistent responsiveness, increased battery life, superior ruggedness, quicker startup times, and noise and vibration reductions. With prices dropping precipitously, most analysts expect more and more PCs to be sold with SSDs in place of traditional rotating hard disk drives (HDDs).

In Windows 7, we’ve focused a number of our engineering efforts with SSD operating characteristics in mind. As a result, Windows 7’s default behavior is to operate efficiently on SSDs without requiring any customer intervention. Before delving into how Windows 7’s behavior is automatically tuned to work efficiently on SSDs, a brief overview of SSD operating characteristics is warranted.

Random Reads: A very good story for SSDs

SSDs tend to be very fast for random reads. Most SSDs thoroughly trounce traditionally HDDs because the mechanical work required to position a rotating disk head isn’t required. As a result, the better SSDs can perform 4 KB random reads almost 100 times faster than the typical HDD (about 1/10^th of a millisecond per read vs. roughly 10 milliseconds).

Sequential Reads and Writes: Also Good

Sequential read and write operations range between quite good to superb. Because flash chips can be configured in parallel and data spread across the chips, today’s better SSDs can read sequentially at rates greater than 200 MB/s, which is close to double the rate many 7200 RPM drives can deliver. For sequential writes, we see some devices greatly exceeding the rates of typical HDDs, and most SSDs doing fairly well in comparison. In today’s market, there are still considerable differences in sequential write rates between SSDs. Some greatly outperform the typical HDD, others lag by a bit, and a few are poor in comparison.

Random Writes & Flushes: Your mileage will vary greatly

The differences in sequential write rates are interesting to note, but for most users they won’t make for as notable a difference in overall performance as random writes.

What’s a long time for a random write? Well, an average HDD can typically move 4 KB random writes to its spinning media in 7 to 15 milliseconds, which has proven to be largely unacceptable. As a result, most HDDs come with 4, 8 or more megabytes of internal memory and attempt to cache small random writes rather than wait the full 7 to 15 milliseconds. When they do cache a write, they return success to the OS even though the bytes haven’t been moved to the spinning media. We typically see these cached writes completing in a few hundred microseconds (so 10X, 20X or faster than actually writing to spinning media). In looking at millions of disk writes from thousands of telemetry traces, we observe 92% of 4 KB or smaller IOs taking less than 1 millisecond, 80% taking less than 600 microseconds, and an impressive 48% taking less than 200 microseconds. Caching works!

On occasion, we’ll see HDDs struggle with bursts of random writes and flushes. Drives that cache too much for too long and then get caught with too much of a backlog of work to complete when a flush comes along, have proven to be problematic. These flushes and surrounding IOs can have considerably lengthened response times. We’ve seen some devices take a half second to a full second to complete individual IOs and take 10’s of seconds to return to a more consistently responsive state. For the user, this can be awful to endure as responsiveness drops to painful levels. Think of it, the response time for a single I/O can range from 200 microseconds up to a whopping 1,000,000 microseconds (1 second).

When presented with realistic workloads, we see the worst of the SSDs producing very long IO times as well, as much as one half to one full second to complete individual random write and flush requests. This is abysmal for many workloads and can make the entire system feel choppy, unresponsive and sluggish.

Random Writes & Flushes: Why is this so hard?

For many, the notion that a purely electronic SSD can have more trouble with random writes than a traditional HDD seems hard to comprehend at first. After all, SSDs don’t need to seek and position a disk head above a track on a rotating disk, so why would random writes present such a daunting a challenge?

The answer to this takes quite a bit of explaining, Anand’s article admirably covers many of the details. We highly encourage motivated folks to take the time to read it as well as this fine USENIX paper. In an attempt to avoid covering too much of the same material, we’ll just make a handful of points.

• Most SSDs are comprised of flash cells (either SLC or MLC). It is possible to build SSDs out of DRAM. These can be extremely fast, but also very costly and power hungry. Since these are relatively rare, we’ll focus our discussion on the much more popular NAND flash based SSDs. Future SSDs may take advantage of other nonvolatile memory technologies than flash.
• A flash cell is really a trap, a trap for electrons and electrons don’t like to be trapped. Consider this, if placing 100 electrons in a flash cell constitutes a bit value of 0, and fewer means the value is 1, then the controller logic may have to consider 80 to 120 as the acceptable range for a bit value of 0. A range is necessary because some electrons may escape the trap, others may fall into the trap when attempting to fill nearby cells, etc… As a result, some very sophisticated error correction logic is needed to insure data integrity.
• Flash chips tend to be organized in complex arrangements, such as blocks, dies, planes and packages. The size, arrangement, parallelism, wear, interconnects and transfer speed characteristics of which can and do vary greatly.
• Flash cells need to be erased before they can be written. You simply can’t trust that a flash cell has no residual electrons in it before use, so cells need to be erased before filling with electrons. Erasing is done on a large scale. You don’t erase a cell; rather you erase a large block of cells (like 128 KB worth). Erase times are typically long -- a millisecond or more.
• Flash wears out. At some point, a flash cell simply stops working as a trap for electrons. If frequently updated data (e.g., a file system log file) was always stored in the same cells, those cells would wear out more quickly than cells containing read-mostly data. Wear leveling logic is employed by flash controller firmware to spread out writes across a device’s full set of cells. If done properly, most devices will last years under normal desktop/laptop workloads.
• It takes some pretty clever device physicists and some solid engineering to trap electrons at high speed, to do so without errors, and to keep the devices from wearing out unevenly. Not all SSD manufacturers are as far along as others in figuring out how to do this well.

Performance Degradation Over Time, Wear, and Trim

As mentioned above, flash blocks and cells need to be erased before new bytes can be written to them. As a result, newly purchased devices (with all flash blocks pre-erased) can perform notably better at purchase time than after considerable use. While we’ve observed this performance degradation ourselves, we do not consider this to be a show stopper. In fact, except via benchmarking measurements, we don’t expect users to notice the drop during normal use.

Of course, device manufactures and Microsoft want to maintain superior performance characteristics as best we can. One can easily imagine the better SSD manufacturers attempting to overcome the aging issues by pre-erasing blocks so the performance penalty is largely unrealized during normal use, or by maintaining a large enough spare area to store short bursts of writes. SSD drives designed for the enterprise may have as high as 50% of their space reserved in order to provide lengthy periods of high sustained write performance.

In addition to the above, Microsoft and SSD manufacturers are adopting the Trim operation. In Windows 7, if an SSD reports it supports the Trim attribute of the ATA protocol’s Data Set Management command, the NTFS file system will request the ATA driver to issue the new operation to the device when files are deleted and it is safe to erase the SSD pages backing the files. With this information, an SSD can plan to erase the relevant blocks opportunistically (and lazily) in the hope that subsequent writes will not require a blocking erase operation since erased pages are available for reuse.

As an added benefit, the Trim operation can help SSDs reduce wear by eliminating the need for many merge operations to occur. As an example, consider a single 128 KB SSD block that contained a 128 KB file. If the file is deleted and a Trim operation is requested, then the SSD can avoid having to mix bytes from the SSD block with any other bytes that are subsequently written to that block. This reduces wear.

Windows 7 requests the Trim operation for more than just file delete operations. The Trim operation is fully integrated with partition- and volume-level commands like Format and Delete, with file system commands relating to truncate and compression, and with the System Restore (aka Volume Snapshot) feature.

Windows 7 Optimizations and Default Behavior Summary

As noted above, all of today’s SSDs have considerable work to do when presented with disk writes and disk flushes. Windows 7 tends to perform well on today’s SSDs, in part, because we made many engineering changes to reduce the frequency of writes and flushes. This benefits traditional HDDs as well, but is particularly helpful on today’s SSDs.

Windows 7 will disable disk defragmentation on SSD system drives. Because SSDs perform extremely well on random read operations, defragmenting files isn’t helpful enough to warrant the added disk writing defragmentation produces. The FAQ section below has some additional details.

Be default, Windows 7 will disable Superfetch, ReadyBoost, as well as boot and application launch prefetching on SSDs with good random read, random write and flush performance. These technologies were all designed to improve performance on traditional HDDs, where random read performance could easily be a major bottleneck. See the FAQ section for more details.

Since SSDs tend to perform at their best when the operating system’s partitions are created with the SSD’s alignment needs in mind, all of the partition-creating tools in Windows 7 place newly created partitions with the appropriate alignment.

Frequently Asked Questions

Before addressing some frequently asked questions, we’d like to remind everyone that we believe the future of SSDs in mobile and desktop PCs (as well as enterprise servers) looks very bright to us. SSDs can deliver on the promise of improved performance, more consistent responsiveness, increased battery life, superior ruggedness, quicker startup times, and noise and vibration reductions. With prices steadily dropping and quality on the rise, we expect more and more PCs to be sold with SSDs in place of traditional rotating HDDs. With that in mind, we focused an appropriate amount of our engineering efforts towards insuring Windows 7 users have great experiences on SSDs.

Will Windows 7 support Trim?

Yes. See the above section for details.

Will disk defragmentation be disabled by default on SSDs?

Yes. The automatic scheduling of defragmentation will exclude partitions on devices that declare themselves as SSDs. Additionally, if the system disk has random read performance characteristics above the threshold of 8 MB/sec, then it too will be excluded. The threshold was determined by internal analysis.

The random read threshold test was added to the final product to address the fact that few SSDs on the market today properly identify themselves as SSDs. 8 MB/sec is a relatively conservative rate. While none of our tested HDDs could approach 8 MB/sec, all of our tested SSDs exceeded that threshold. SSD performance ranged between 11 MB/sec and 130 MB/sec. Of the 182 HDDs tested, only 6 configurations managed to exceed 2 MB/sec on our random read test. The other 176 ranged between 0.8 MB/sec and 1.6 MB/sec.

Will Superfetch be disabled on SSDs?

Yes, for most systems with SSDs.

If the system disk is an SSD, and the SSD performs adequately on random reads and doesn’t have glaring performance issues with random writes or flushes, then Superfetch, boot prefetching, application launch prefetching, ReadyBoost and ReadDrive will all be disabled.

Initially, we had configured all of these features to be off on all SSDs, but we encountered sizable performance regressions on some systems. In root causing those regressions, we found that some first generation SSDs had severe enough random write and flush problems that ultimately lead to disk reads being blocked for long periods of time. With Superfetch and other prefetching re-enabled, performance on key scenarios was markedly improved.

Is NTFS Compression of Files and Directories recommended on SSDs?

Compressing files help save space, but the effort of compressing and decompressing requires extra CPU cycles and therefore power on mobile systems. That said, for infrequently modified directories and files, compression is a fine way to conserve valuable SSD space and can be a good tradeoff if space is truly a premium.

We do not, however, recommend compressing files or directories that will be written to with great frequency. Your Documents directory and files are likely to be fine, but temporary internet directories or mail folder directories aren’t such a good idea because they get large number of file writes in bursts.

Does the Windows Search Indexer operate differently on SSDs?

No.

Is Bitlocker’s encryption process optimized to work on SSDs?

Yes, on NTFS. When Bitlocker is first configured on a partition, the entire partition is read, encrypted and written back out. As this is done, the NTFS file system will issue Trim commands to help the SSD optimize its behavior.

We do encourage users concerned about their data privacy and protection to enable Bitlocker on their drives, including SSDs.

Does Media Center do anything special when configured on SSDs?

No. While SSDs do have advantages over traditional HDDs, SSDs are more costly per GB than their HDD counterparts. For most users, a HDD optimized for media recording is a better choice, as media recording and playback workloads are largely sequential in nature.

Does Write Caching make sense on SSDs and does Windows 7 do anything special if an SSD supports write caching?

Some SSD manufacturers including RAM in their devices for more than just their control logic; they are mimicking the behavior of traditional disks by caching writes, and possibly reads. For devices that do cache writes in volatile memory, Windows 7 expects flush commands and write-ordering to be preserved to at least the same degree as traditional rotating disks. Additionally, Windows 7 expects user settings that disable write caching to be honored by write caching SSDs just as they are on traditional disks.

Do RAID configurations make sense with SSDs?

Yes. The reliability and performance benefits one can obtain via HDD RAID configurations can be had with SSD RAID configurations.

Should the pagefile be placed on SSDs?

Yes. Most pagefile operations are small random reads or larger sequential writes, both of which are types of operations that SSDs handle well.

In looking at telemetry data from thousands of traces and focusing on pagefile reads and writes, we find that

• Pagefile.sys reads outnumber pagefile.sys writes by about 40 to 1,
• Pagefile.sys read sizes are typically quite small, with 67% less than or equal to 4 KB, and 88% less than 16 KB.
• Pagefile.sys writes are relatively large, with 62% greater than or equal to 128 KB and 45% being exactly 1 MB in size.

In fact, given typical pagefile reference patterns and the favorable performance characteristics SSDs have on those patterns, there are few files better than the pagefile to place on an SSD.

Are there any concerns regarding the Hibernate file and SSDs?

No, hiberfile.sys is written to and read from sequentially and in large chunks, and thus can be placed on either HDDs or SSDs.

What Windows Experience Index changes were made to address SSD performance characteristics?

In Windows 7, there are new random read, random write and flush assessments. Better SSDs can score above 6.5 all the way to 7.9. To be included in that range, an SSD has to have outstanding random read rates and be resilient to flush and random write workloads.

In the Beta timeframe of Windows 7, there was a capping of scores at 1.9, 2.9 or the like if a disk (SSD or HDD) didn’t perform adequately when confronted with our random write and flush assessments. Feedback on this was pretty consistent, with most feeling the level of capping to be excessive. As a result, we now simply restrict SSDs with performance issues from joining the newly added 6.0+ and 7.0+ ranges. SSDs that are not solid performers across all assessments effectively get scored in a manner similar to what they would have been in Windows Vista, gaining no Win7 boost for great random read performance.

A Little Bit of Personality

Greetings!  Based on the data we’re seeing we know a lot of folks on MSDN/TechNet/Connect are probably busy using the RC (Release Candidate) for Windows 7.  Thank you!!!  And of course many folks are looking forward to downloading the RC and using it as we expand the downloads—we’re looking forward to the participation and seeing the data that will help us validate the RC.  We’ve talked about making sure that you are “in control” of Windows 7 and one of the ways that people are in control of their PC is to personalize the experience.  With the RC you’re going to see some of the new personalization “elements” in Windows 7.  In this post, Denise Trabona and Samuel Moreau of our product design team provide a behind the scenes look at some of the work.  Be sure to check out the links below the images as you can see a lot more work by these talented artists.  Note, these are just thumbnails for this post so be sure to enjoy the full screen images in the RC.  --Steven

PS: Just a reminder, that just as with the pre-beta and beta we’ll be testing out Windows Update and the system for doing patches and updates.  So along with new drivers you might also see some other updates flowing through the system. 

One of the most exciting parts of engineering Windows 7 has been the wide variety of work that gets done over the course of a full product cycle. As evidenced by the variety of topics just in this blog, one can see that we are hard at work at all levels of the product. For fun, we thought folks might enjoy hearing some of the story behind the new personalization work in Windows 7.

As some folks have noticed, we are unveiling some new personalization content (wallpapers, glass colors and sounds schemes) in the RC build which allows people greater flexibility to personalize their experience. One thing we know is that Windows users love to express themselves by changing the desktop background and like many past releases, Windows 7 includes content in the box that allows you to begin customizing your experience immediately.

A picture speaks a thousand words

In developing the personalization features, we knew that we wanted great content for people to express their personal style. Because the desktop background is such a vibrant surface, we wanted to focus on providing quality content that demonstrated how creative people could be with this feature. When folks send us screenshots using the feedback button, we are regularly inspired by the rich diversity and personality of the wallpapers that people choose.

As we thought about how we wanted to approach personalization in Windows 7, we knew one way was to honor our lineage. In the past photography has been featured heavily Windows. Some of that photography has been quite beautiful and has become a proud tradition we wanted to maintain. In addition, we also wanted to explore new territory and expand our visual palette. In the realm of photography, we kept a theme focused on landscape photography which is our tradition, but added new themes for architecture and nature. Much of the this imagery is from our stock imagery partners, but we also had the good fortune to work with a talented local photographer named Will Austin, who has photographed all over the world on many subjects with an emphasis on architecture. Will’s photos provide a little bit of the local flavor of the Seattle area that we are proud to call home.

Raising the bar of inspiration and delight

With the photography covered, we tried to broaden our coverage to include additional images that would inspire, delight and invigorate people’s imaginations. We wanted to stretch into some new content that felt unique, timely, and with a distinct point of view. Our goal was content that balanced the timelessness of great photography with graphical illustrations that are energetic, modern, and fresh. On top of it all it was also important to achieve a rich variety in the illustrations to appeal to different tastes, genders and ages, color ranges from quiet to loud, and from large compositions to small and detailed.

Inspired by our neighbors in Zune, we worked with an agency called 72 and Sunny to search for illustrators around the world to create one-of-a-kind art work for you to have in Windows 7. In the process of looking through tons of samples, we sought a group of artists whose styles seemed both incredibly varied, to cover the broad diversity we were after, and maintained a common thread that we felt was applicable to the overall tone we were striving to achieve. Then began the fun part, with little more than some simple guiding words (light, energetic, inspiring, optimistic, etc.), the artists went off with a blank canvas to create concept sketches of their original pieces.

Iterate and refine

We still remember the first chance we got to review the artist’s initial sketches and concept work, and right from that moment, we knew that these images were going to be a lot of fun. The next step was to iterate back and forth a few times to make sure certain goals were achieved and get little details just right. For example, a couple of things that were important to us were how the image flowed under the new task bar and striking the right balance between visually compelling, and not too distracting when it came to finding that important file on your desktop. It’s tricky to find the right balance and we were fortunate to have an amazingly talented set of artists and our friends at 72 and Sunny to work with on this project.

Windows is for the whole world

Finally, we wanted to recognize the global audience of Windows by seeking out illustrators with varied backgrounds and styles with the intention of representing and appealing to people all around the world.

With that, we are honored to introduce the amazingly talented artists and the work that they contributed to Windows 7 personalization.

 

 

Yuko Kondo
From Japan, now resides in London, England

 

Katharina Leuzinger
Born to Swiss and Japanese parents in Zurich, Switzerland, Katharina Leuzinger now resides in London, England

 

Osmand Nosse
Wicklow, Ireland

Klaus Haapaniemi
From Finland, now based in London, England

Chris Sickles of Red Nose Studios
Indiana, United States

Punga
Buenos Aires, Argentina

Pomme Chan
Born and educated in Bangkok, Pomme Chan now resides in London, England.

Kustaa Saksi
Amsterdam, Netherlands

 

 

 

 

 

Paul Hwang and Benjamin Lee of Nanosphere
Los Angeles, California

Adhemas Batista
From Sao Paulo Brazil, now resides in Los Angeles, California

Kai and Sunny
London, England

Nan Na Hvass
Born to a Danish father and Chinese mother in Swaziland, Africa, Nan Na Hvass now resides in Copenhagen, Denmark.

We hope this post has given you some insight into the Windows 7 content. We also hope that we achieved the goals we set out for ourselves with this element of Windows 7.

-Denise Trabona and Samuel Moreau

Improvements to AutoPlay

As mentioned before on this blog (regarding our UAC changes) and on the IE blog (regarding the SmartScreen® filter for malware), we have an increased focus to enable customers to be in control and feel confident about the software that they choose to run on their computers. Folks on this blog have also commented about the concerns they have specifically in the AutoPlay area. This blog entry addresses some of the changes that we have made to increase customer confidence when using their media and devices with Windows.  It is authored by Arik Cohen, a program manager on the Core User Experience team. –Steven  [Note: There was a technical problem so this post was reposted in its entirety.]

Certain malware, including the Conficker worm, have started making use of the capabilities of AutoRun to provide a seemingly benign task to people – which masquerades as a Trojan Horse to get malware onto the computer. The malware then infects future devices plugged into that computer with the same Trojan Horse. For further information about Conficker please visit http://www.microsoft.com/protect/computer/viruses/worms/conficker.mspx

In the following example for a USB flash drive that has photos, malware registers as the benign task of “Open folders to view files.” If you select the first “Open folders to view files” (circled in red), you would be running malware. However, if you select the second task (circled in green), you would be safe running the Windows task.

Infected USB AutoPlay

People are confused why they have two tasks that appear to do the same thing – and even a knowledgeable person who is careful not to run software from an untrusted source can easily make the mistake of selecting the first task. As a result, people lose confidence and don’t feel in control.

A growing attack

While presenting an AutoRun task in AutoPlay has been available since Windows XP, we have seen a marked increase in the amount of malware that is using AutoRun as a potential method of propagation. According to the Security Intelligence Report, an enterprise study by Forefront Client Security found that the category of malware that can propagate via AutoRun accounted for 17.7% of infections in the second half of 2008 – the largest single category of malware infections.

The chart below shows the increasing amount of detection reports by Microsoft anti-virus software of the class of infections that spread via AutoRun. (Note: The actual method of infection cannot be determined.)

Infection Detections of Malware that Spread via AutoRun

Currently, disabling AutoPlay completely is the only solution for consumers and enterprises to gain confidence with the use of USB flash devices on their computer. Guidance on disabling AutoPlay is available here.

Increasing customer confidence

Windows 7 introduces key changes to AutoPlay that keep you from being exposed inadvertently to malware like Conficker when doing your common scenarios with devices (e.g., get to the files on your USB flash drive, download pictures from an SD card, etc.).

In particular, Windows will no longer display the AutoRun task in the AutoPlay dialog for devices that are not removable optical media (CD/DVD.) because there is no way to identify the origin of these entries. Was it put there by the IHV, a person, or a piece of malware? Removing this AutoRun task will block the current propagation method abused by malware and help customers stay protected. People will still be able to access all of the other AutoPlay tasks that are installed on their computer.

With these changes, if you insert a USB flash drive that has photos and has been infected by malware, you can be confident that the tasks displayed are all from software already on your computer:

Infected USB AutoPlay after AutoPlay changes

On the other hand, if you insert a CD that offers software to install, Windows will still display the AutoRun task provided by the ISV during their media creation process. For example:

AutoPlay for a CD that offers an AutoRun Task

You will first see this updated AutoRun experience in the Windows 7 RC build, and we will be bringing this change to Vista and XP in the future.

Ecosystem Impact

We are working with our ecosystem partners to help mitigate situations where this AutoRun change will have an impact on them.

CDs and DVDs (including CD emulation), where the IHV specified AutoRun task authored during manufacturing, will continue to provide the AutoRun choice allowing customers to run the specified software. IHVs of generic mass storage devices should expect that people will browse the contents of the device to launch any software. The new behavior will allow customers to continue to use AutoPlay (including all Windows and ISV installed tasks) to access their media and devices while not being presented with tasks from malware. Additionally, device classes, such as portable media players and cell phones, now support Device Stage™ on Windows 7. Device Stage offers the IHV a multifunction alternative to AutoPlay where they can present links to software and common tasks, and provides additional features as you use the device.

As you try out the Windows 7 RC, we hope these changes will make you feel more confident and in control when using your media and devices.

-Arik Cohen

Engineering Windows 7 Graphics Performance

One of the areas of any release of Windows that receives a significant amount of testing and scrutiny is the performance of graphics—desktop graphics all the way to the most extreme CAD and game graphics.  The amazing breadth of hardware supported for Windows and the broad spectrum of usage scenarios contributes to a vibrant ecosystem with many different goals—from just the basics to the highest frame rates on multiple monitors possible.  In engineering Windows 7 we set out to improve the “real world” performance of graphics as well as continue to improve the most extreme elements of graphics. This is work we do in Windows 7 and work our partners do as they work to improve the underlying hardware/software combination through drivers (note: Windows Vista drivers continue to work as they did in Windows Vista, but we've also been working with partners on updated drivers for Windows 7 which many of you have been testing through Windows Update downloads). This post looks at this spectrum of engineering as well as the different ways performance is measured.  Ultimately we want to inform you about what we have done in engineering Windows 7, while we leave room for the many forums that will compare and contrast Windows 7 on different hardware and in different scenarios.  This post is written by Ameet Chitre, a program manager on our Desktop Graphics feature team.  --Steven

If you have gone online to check out or purchase a new PC, you must have noticed that “faster graphics” and “great performance” are always some of the key selling points. People have come to expect faster systems which enable them to edit photos, do email, watch high-definition videos and play the latest 3D games all with greater ease, often shuffling between these tasks seamlessly. Quite a few of these users refer to the enthusiast community blogs and various review sites which run graphics benchmarks and report results evaluating how fast the graphics of new hardware or software performs. Traditionally graphics performance has been measured and analyzed through 3D games but it also impacts what we call “desktop scenarios” - such as when you are using the Windows UI, moving or maximizing windows, or scrolling within Word or IE etc. The performance requirements for these desktop scenarios are quite different from3D games. In fact, this is the reason in Windows Vista Experience Index (WinEI) we give you rating for these two scenarios separately, highlighted in the image below:

Figure 1. WEI sample with Graphics capabilities highlighted.

Graphics performance is usually assessed through many benchmarks. These can be classified into 2 broad categories:

• Scenario benchmarks: These report performance of particular scenarios, e.g. the frame rate when flying over a terrain in a flight simulator. Many of the popular games have in-built benchmark modes that demonstrate how fast the graphics perform in scenes that are typical for that game.
• Synthetic benchmarks: These highlight the performance of a particular aspect of graphics such as the ability to draw a number of lines as fast as possible.

However, there are plenty of things that we all do on our PCs that don’t have benchmarks tracking them that are still quite critical to make fast. In these cases we use the instrumentation within Windows to obtain timing information and then analyze the performance.

This blog entry discusses various aspects of graphics performance - both gaming and desktop graphics performance. It calls out the changes we made in Windows 7 to address user feedback as well as to take advantage of modern hardware to improve graphics performance.

Desktop Responsiveness

Many have experienced scenarios where an application, or Windows itself, stops responding momentarily. This is type of a performance issue that can be impacted significantly by the performance of graphics in the PC. We categorize these as desktop responsiveness issues.  Improving responsiveness, both in real terms and by avoiding non-responsive moments, is one of the key ways that performance is improved in the system.  It is also hard to measure.

Measuring desktop responsiveness is a hard problem since a number of issues which affect responsiveness aren’t easily reproducible and there is a great variety of them. They are rarely caught by either kind of benchmark as these issues are dependent on real-world combinations of factors. For Windows 7 we spent a great deal of time looking at these performance glitches using a mechanism in test versions of Windows 7 which has the ability to record key OS events and when they occurred. During real-world testing when we encounter a responsiveness problem, the tester can hit a record key and enter a small description of the issue encountered. The event history with diagnostic information called a “performance trace” is written out to a file and uploaded to a server where a team of performance analysts parse the data to figure out the cause of the responsiveness issue. This process has been successful to the extent that today most responsiveness issues can be quickly tracked down and root-caused.

Using this methodology, we analyzed thousands of desktop responsiveness traces where the tester experienced a frozen desktop anywhere from 100msec to several seconds. The type of issues ranged from an antivirus blocking disk access for all applications while updating itself on the vendor’s website to an application doing network access from a UI thread. In a significant portion of these traces, we found that a GDI application is waiting on another GDI application which is experiencing slowdowns due to excessive paging activity. This was the single-most frequently occurring cause of all desktop responsiveness issues, which without this data we probably would not have assumed. Based on these investigations, we worked to improve the architecture in these two key areas:

1. GDI Concurrency: Improve responsiveness when multiple applications are running. This required a re-architecture of the code around GDI (Graphics Device Interface) synchronization objects or “locks”.
2. Reduction of overall memory footprint of Windows: Reduce the memory overhead of composition in the Desktop Window Manager (DWM), which is the component responsible for rendering the desktop. This helps reduce overall paging activity and thus is especially important for low-memory PCs, especially using shared graphics memory.

These are described in more detail below.

GDI Concurrency

A number of performance traces we investigated in the context of desktop responsiveness pointed us to the design of a key synchronization mechanism in GDI. The performance challenge happens because the design of GDI in Windows Vista allows only a single application to hold a system-wide exclusive global lock.  While this seems obvious in hindsight, when this decision was originally made the performance characteristics of different parts of the system made this optimistic implementation perfectly reasonable.

Figure 2. Existing architecture of GDI concurrency.

GDI applications running simultaneously vie for this global lock in order to render on the screen. The application that accesses the global lock prevents other applications from rendering till it releases the global lock. The situation often gets exacerbated when the application that is holding the lock needs to page in a large amount of memory from the disk since moving the memory from the disk to RAM takes a relatively long time. The above picture shows two GDI applications running simultaneously, contending for the global lock. If App X gets hold of the lock, it can render to the screen while App Y is unable to do so and waits for App X to finish.

Figure 3. Windows 7 architecture of GDI concurrency.

The solution to the problem was therefore to reduce the lock contention and improve concurrency by re-architecting the internal synchronization mechanism through which multiple applications can reliably render at the same time. Contention due to the global exclusive lock is avoided by implementing a number of fine-grained locks which are not exclusive but aid parallelism. The increased number of fine-grained locks adds a small overhead for scenarios where only a single application is rendering at a time.

Special attention was paid to GDI application compatibility as changing internal synchronization mechanism in the most widely used API stack could potentially give rise to timing issues such as deadlocks and rendering corruption.

This work also resulted in better rendering performance of concurrent GDI applications on multi-core CPUs. Multi-core Windows PCs benefit from these changes as more than one application can now be rendering at the same time.

After the GDI concurrency work was implemented in the Windows 7 builds leading to the Beta, we saw a large reduction in the number of desktop responsiveness issues reported by our testers which were caused by one application blocking another one due to GDI. To further validate the scalability of our new implementation, we wrote tests that draw 2D GDI primitives and measured the rendering throughput by launching simultaneously multiple such applications. The throughput is measured by adding together the frame rate (FPS) of each application window. Below is a sample of these results on a quad-core CPU system.

Figure 4. GDI Concurrency and Scalability.

Without the Windows 7 GDI concurrency, the rendering throughput of these applications is effectively limited to the performance of a single CPU core. Since only a single application can acquire the global exclusive lock while the others are waiting, this scenario doesn’t benefit from multiple CPU cores. This demonstrates that GDI applications in Windows 7 are now much less dependent on one another. This benefit will not need any new display drivers; it will work on any Vista (WDDM 1.0) and newer display drivers.

Desktop Graphics - Reduced Memory Footprint

Another area which affects system responsiveness is memory usage. Simply put, increased system memory (RAM) usage leads to an increased paging activity which directly leads to reduced system responsiveness. Thus, for the best responsiveness, all applications, processes and OS components need to use as little system memory as possible.

In Windows Vista, the amount of memory required to run multiple windows scales linearly with the number of windows opened on the system. This results in more memory pressure when there are more windows or if the monitors have higher resolution. It gets worse if you have more than one monitor. As part of investigating various means to improve system responsiveness, we saw a great opportunity in reducing the usage of system memory by DWM. In Windows Vista, every GDI application window accounts for two memory allocations which hold identical content – one in video memory and one in system memory. DWM is responsible for composition of the desktop through the graphics hardware. Hence, it requires a copy of the same allocation in video memory, which is easily accessible by the graphics hardware. The duplicate copy present in system memory is required because GDI is being rendered utilizing the CPU completely in the operating system without any assistance or “acceleration” by the graphics hardware. As the CPU performs all the tasks for rendering GDI applications, it requires an easily accessible cacheable copy of memory.

 

Figure 5. Existing memory allocations.

Windows 7 saves one copy of the memory allocation per application window by getting rid of the system memory copy entirely. Thus, for a GDI application window visible on the desktop, the memory consumed is cut in half.

Figure 6. Windows 7 memory allocations.

We achieved the reduction in system memory by accelerating the common GDI operations through the graphics hardware - the WDDM drivers accelerate these to minimize the performance impact of the CPU read-back of video memory. This was necessary as performing these operations otherwise on the CPU would incur a heavy performance penalty. In order to decide which GDI operations to accelerate, it was important to understand the usage pattern of various GDI applications. We profiled the top 100 GDI applications to learn more about their calling patterns and frequency and nature of the GDI operations.

 

Figure 7. Calling patterns and frequency of GDI operations for 100 GDI-based applications.

Based on real-world application statistics, a tiny snapshot of which is seen above, we worked with our graphics IHV partners to provide support in their drivers to accelerate the most commonly used GDI operations. Windows 7 systems with these updated drivers, called “WDDM v1.1” will thus benefit from this memory savings work. Please note that WDDM 1.0 drivers continue to function and are fully supported on Windows 7.  You might have seen Windows Update providing these 1.1 drivers during the Beta—these drivers are themselves in Beta.

Figure 8. Desktop Window manager memory consumption comparison using WDDM 1.1 v. WDDM 1.0.

The above data shows that the memory savings become more and more pronounced when you have multiple application windows visible on the desktop. Since you save a lot of system memory, the paging activity gets reduced – as a result, your system responsiveness improves for the same workload.

Certain trade-offs had to be made for the desktop responsiveness improvements which benefit a wide range of systems. For example – the elimination of the duplicate system memory copies which “speed up” certain operations introduced slightly reduced performance as the CPU now has to read data back from the video memory. An analysis of real-world application statistics showed that these operations were rare. However, certain GDI micro-benchmarks which issue these operations show some performance degradation. This is something to note if you are running existing benchmarks that stress specific GDI operations repeatedly, which isn’t necessarily a reflection of real-world performance.  Our observation has been that these slow-downs do not impact the end-user functionality directly and that the memory savings directly result in Windows 7 being much responsive overall.  The improvements overall are definitely noticeable on memory constrained PCs with shared memory graphics.

Gaming Performance

No article on graphics performance is complete without talking about gaming, which is still the most widely analyzed and discussed aspect of graphics performance. There are a number of popular benchmarks such as 3D Mark as well as in-game benchmarks which are really a mode in which you can run your game where it renders the game scenes and animations without any user interaction. This area has thus been well tracked by the gaming industry through various industry benchmarks, which are pretty realistic and representative of actual games. The different benchmarks and tests are widely discussed and gamers all well-versed in the subtleties of these measurements and translating them into recommendations depending on their hardware, drivers, and gaming expectations.

For Windows 7, we have worked closely with our Graphics IHV partners, helping them improve the WDDM drivers’ gaming performance with specific changes to how Windows 7 works under the hood, while maintaining the same driver model and compatibility. Our continued investments in performance tools has helped us and our IHV partners track down and analyze various gaming performance bottlenecks and fix them in subsequent driver releases. The fundamentals of the Windows Display Driver Model remain unchanged in Windows 7. Some policies around GPU scheduling and memory management were changed to enable better performance in certain scenarios.

Because these benchmarks are very sensitive to the specific hardware, firmware, drivers, and overall system and because these are so widely measured and discussed elsewhere we are going to leave these comparisons to third parties.  Like many areas in Windows 7, our commitment is to engineer even better performance across many dimensions.  We believe it is better for you to experience these efforts directly.  In comparing Windows 7, we would encourage the comparison using Windows Vista SP1 and keep in mind the difference you might see in WDDM 1.0 v. 1.1 and that the 1.1 drivers are still under development.

In Closing…

As you can see, in engineering Windows 7 we have worked hard to improve the architecture for graphics for real-world performance. It benefits both ends of the hardware spectrum – by enabling low physical memory systems to run a leaner and faster Windows and at the same time enabling multi-core PCs render multiple graphics applications much more efficiently.

-Ameet

Ink Input and Tablet PC

There’s a strong community of developers who take advantage of the ink input/TabletPC functionality to develop unique solutions for specific markets (medicine, education, line of business) and create software in Windows that builds on this experience to streamline how these end-users interact with information on their PC (usually with unique form-factors such as slates or wall mounted PCs).  Earlier this week I received a great email asking “what’s new for us” from the head of development for one such ISV (medical software) and so we put together an overview of the new functionality.    Several Program Managers on the team authored this post. 

Also, as you have noticed, the site has had some uptime troubles over the past 10 days or so and I think we’re all back to normal.  That’s ok since we’ve also been pretty busy in the Windows 7 hallway :-)   --Steven

Tablet PC Input Panel

Hi, my name is Jan-Kristian and I’m a Program Manager on the Core User Experience team for Windows 7. One of my focus areas is pen and touch text input, and I’d like to share some of the exciting things we have been working on.

The Tablet PC Input Panel, what we often called the TIP for short, is the tool to insert text using handwriting into any Windows application. It also has a soft-keyboard you can use for text entry. The Input Panel has been around since the first version of Windows XP Tablet PC Edition, and we’ve made steady improvements to the user experience in each version.

The new Writing Pad

Our goal with the TIP is to make it as light-weight as possible so you can think about what you are writing and not how you are doing it. We received a lot of positive feedback on the improvements we did to the Input Panel in Windows Vista, but there were still areas that caused confusion or took more steps than necessary.

Windows Vista Input Panel – The handwriting recognition results are shown as small text bubbles under the writing surface. To verify recognition you need to look down at the bubbles, if you see an error you then tap on a bubble to bring up a secondary window for correction.

Based on analysis of our telemetry from Vista and usability tests we focused on two significant areas of improvement for Windows 7:

• Simplify the experience – Handwriting should be an easy, natural, flowing experience. What we found though, was that using the TIP caused a “high cognitive load”, which means you have to think too much about what you are doing. Your eyes needed to dart back and forth between what you’ve written and the little bubbles down below and corrections meant entering another mode and even then often meant rewriting the whole word. Our goal was to simplify this and make it less taxing.
• Add flexibility – We’re all accustomed to the flexibility of using a mouse and keyboard for input. Handwriting with the Windows Vista Input Panel had minimal flexibility because the ink-based model made it hard to edit a sentence once it was written – there was no way to insert more text between words, or to easily replace words. Our goal here was to bring the editing experience more in line with what you are used to with the mouse and keyboard while exploiting the power of the pen.

Creating a new model

To achieve these goals we needed to make fundamental changes to the writing pad. As we explored different ideas we decided on a model where ink was converted in-place to text as the user was writing. Although this sounds like a straightforward UI model, there were a lot of open questions on what the right behavior should be: when do we convert, how big should the text be, what font should we use… The only way to make sure we created a natural and efficient handwriting experience was to get real user feedback. We utilized the RITE (Rapid Iterative Testing and Evaluation) method. RITE testing is a cycle-based usability method that was developed at Microsoft as part of usability testing of the Age of Empires II game. For each cycle you try to make small improvement to the user experience and then you re-test to see how well it worked. We went through roughly 20 cycles before we had a design that we felt was ready to be documented.

One of the most important things we adjusted during RITE testing was the timing for the automatic ink to text conversion. Converting too early or too late would break the user experience; to get this right we had to do a lot of behind the scenes work. Our final solution is a combination of a distance trigger (automatically adapting to the user’s average word spacing), recognizer-result-based trigger, and a time-based trigger. Another factor was the text size, in the end we use dynamic sizing to closely match the size of the handwriting.

The new text-based UI in the writing surface allows you to get to the text they wanted faster. Having a single representation of the text makes the experience less complex and reduces the height of the Input Panel. Using text instead of ink makes the writing surface much more flexible as we can move the text around as much as we want – inserting a word between two words is now as simple as just starting to write in the space and we will auto-grow the space as much as needed.

With the ink to text conversion working we needed a correspondingly natural way of editing recognized text. Gestures seemed to be the perfect solution for this - we were creating a pen-based UI, so we should use the pen. We limited ourselves to a small number of gestures: delete, split (add space), and join. We collected samples of how people would perform these three actions on paper. Based on our collected data we then created our gestures. To make the gestures discoverable, we added the “gesture panel”, which is an interactive “cheat sheet” in the title bar of the Input Panel.

Let’s take a look at how this all comes together in the new Windows 7 writing surface [Ed. Note, used YouTube with Windows Live Photo Gallery on Windows 7]:

Writing Pad: The new writing pad in action, animation is used to provide meaningful transitions so that the user can easily see the result of their actions.

Smart Corrections

Our telemetry showed us that corrections were one of the more painful parts of using the TIP in Vista, to correct a word you often had to rewrite all of the characters. In Windows 7, we leveraged work from Microsoft Research to design the Smart Correction feature to make word corrections much faster. Now you just start correcting a word left-to-right and Windows performs a new recognition every time you enter a character. This constrained recognition will almost always give you the desired result within a few character corrections.

Smart Corrections: “worked” is auto-corrected to “wonderful” with just a single character change. All you have to do is start correcting the word from the left and it will keep updating until you get the word you want.

Entering URLs

One extra writing pad feature worth mentioning: our instrumentation data showed that the most used applications with the Input Panel are web browsers, and when you are browsing one of the main scenarios is to enter URLs.

Entering URLs: The flexibility of the new writing pad makes entering URLs easy by pre-populating parts of the URL

Notice how the different URL segments are separated and all have alternates that make sense. The alternates are based on what you use most frequently, so if you choose “.net” a lot then that will become the top alternate and set by default in the URL template. The “Insert” button also changed to “Go to” to let the user insert the URL and navigate to it with a single click.

Touch Keyboard

The Input Panel also has a soft-keyboard available which is great for the Pen or Touch. Some of the updates we made might seem like only visual changes, but they were very deliberate and have a big impact on the usability of the touch keyboard. For example, touch screen PCs are often used in mobile situations, we had to be very careful with the color and contrast of the key labels to make sure they are visible on a dimmed screen or in less than optimal light conditions.

The Windows 7 Touch Keyboard

One of the challenges with using a touch based keyboard is the lack of tactile feedback. Coupled with this is the fact that user’s fingers cover keys as they are being tapped. How does the user know that they hit the right key (or even hit a key at all) when they are covering the key with their finger? If a user has to switch focus between the text field and the touch keyboard for every key press they will quickly tire of typing using touch. We wanted to give the user a simple little nudge; “we heard you”, and “yes, you just hit this key”. Our solution was to let the released key have a short glow fade-out effect. This glow feedback gives the user a subtle confirmation that they hit the key they wanted (or not).

The keyboard now supports multitouch so you can press “ctrl+c” or “shift+a” etc. We also enabled key rollover, meaning you can press another key before you finger has lifted off the previous one – this enables a much more natural typing experience. Don’t worry, if you prefer the sticky modifier-key model, where you press “Ctrl” then press “c”, it is still supported.

Predictive Text and Handwriting Personalization

Hi, my name is Jen and I’m a Program Manager on the Tablet PC Handwriting Recognition team. In our previous post you heard from my co-worker Yvonne about new handwriting recognizers. I’m going to talk about some of the new features that leverage or augment our recognizers including Predictive Text and Personalization as well as our new East Asian Recognizers.

Text Prediction

One of our goals on the Tablet PC team is to make it as efficient as possible to enter text when using your keyboard is inconvenient or impractical. To achieve this we’ve made investments in the TIP, we’ve improved overall handwriting recognition accuracy, and we’re leveraging some of these same technologies to deliver Text Prediction on the soft keyboard. Text Prediction offers possible completions for the word being entered, and may offer suggestions for next words or phrases.

Text Prediction – Here I am trying to input the word “Microsoft” using the US English soft keyboard. After entering the first two letters “Mi”, the word “Microsoft” is proposed as the first option. I can then select this option and have the word “Microsoft” inserted into a document.

In Windows 7, we support Text Prediction for English, French, Italian, German, and Spanish using the soft keyboard as well as for Traditional Chinese and Simplified Chinese using handwriting in character by character mode. This section will focus on the Latin based languages; examples for Chinese can be found in the following section.

As we developed Text Prediction, our primary goal was to speed up user input. To do so, we had to make sure predictions were relevant. Our recognizers use a lexicon to improve recognition accuracy. The system lexicon ships with the recognizer and is a fancy name for a word list of the most commonly used words in a given language. Using this lexicon, the out-of-the-box predictions are good, but by including additional user-specific words (your words), we can improve accuracy significantly. This is where Text Harvesting comes in.

In Windows Vista, we shipped Text Harvesting (or Automatic Learning) for US English and UK English to improve handwriting recognition. In Windows 7, this feature will be available for all languages. It allows us to automatically extend the system lexicon based on the words you type when writing email. Text Harvesting is done on a per user basis, so your data is specific to you. From the results of Text Harvesting, we build a new lexicon containing your specific vocabulary and also increase the probability of words already you commonly use, this is use for both handwriting recognition and text prediction. The results are impressive, after augmenting the lexicon with your words and usage patterns, prediction can seem almost magical in its ability to predict which words you are going to enter next!

Windows 7 also includes support for Custom Dictionaries, these are specialized word lists that can be added to the system. Companies can develop domain specific dictionaries, such as for medicine, chemistry etc. and add them to the system – predicting acetaminophen is a lot faster than typing it!

Improvements in East Asian Handwriting Recognition

Significant improvements were made to handwriting recognition on the four East Asian languages we support: Traditional Chinese, Simplified Chinese, Korean and Japanese. For many people, handwriting is an efficient input method for these languages due to the large character set.

There are two input modes for East Asian handwriting: character by character mode (or box mode) and freestyle mode (or line mode). In box mode, you input a single character at a time into a fixed width box. In freestyle mode, you write the characters cursively on a line and do not have to concentrate on spacing. Which mode you chose depends on your preference; box mode is slightly more constrained but has text prediction and is more accurate, whereas line mode is closer to natural handwriting.

Traditional Chinese in Line Mode – The top pane contains the user’s writing and the bottom pane contains the recognized text.

In addition to core accuracy across all these languages improvements, we also use personalization to improve the user experience. One method of personalization is to adapt to how you write using the Shape Collector. The Shape Collector is a Wizard that allows you to train handwriting recognition on your individual handwriting style. For the four East Asian Languages, you can use the Shape Collector in a “troubleshooting” mode to improve recognition of a specific character or word, or to add a character or word that is unsupported.

We also learn as you write and correct mistakes. If you write a character and it is initially misrecognized, you can view the alternates list and select the character you intended. We will learn based on this action, and be more likely to provide that as the first choice the next time you write the character.

In Windows 7, Simplified Chinese and Traditional Chinese also support Text Prediction in box mode. For these languages, Text Prediction is especially valuable as writing individual characters can be time consuming. The user writes in character by character mode and is provided with options to complete their word or phrase without having to write the whole thing. In the case below, the user wants to input 中华人民共和国 and only has to input the first two characters (中华) to get the desired text as a prediction. The gray words represent what has been entered and the black shows the predictions.

Notice in this example that Text Prediction is working on both characters together (中华) as well as just the second character (华). As with the other languages, Text Prediction also works with user-specific vocabulary. If a user inputs the same words multiple times, the recognizer will learn this behavior.

We have made significant improvements to Traditional Chinese, Simplified Chinese, Japanese, and Korean handwriting recognition since Windows Vista. These were based on improving our overall customer experience by improving accuracy and throughput. Our goal is to give customers a more natural way to input in these languages and a viable alternative to keyboard use.

Math Handwriting Recognition

Have you ever written a math paper in Word or performed calculations in Mathematica, and spent hours creating equations using a multitude of buttons or a complex linear format, thinking: “Oh, what I wouldn’t give for an easy-to-use input method?” Well, your wishes have just come true, in addition to improving handwriting in Windows 7, we have also invested in recognizing ink drawn math equations.

Hi, my name is Marko and I am the Program Manager for math recognition in Windows. Let me show you the Math Input Panel that provides you with the most natural and efficient way of math input: handwriting recognition. We have taken a completely new approach to this problem and raised math handwriting recognition to a whole new level in terms of functionality, performance and area coverage.

The Math Input Panel (or MIP) is designed to be used with a tablet pen on a Tablet PC, but you can use it with any input device such as a touchscreen, external digitizer or even a mouse. MIP outputs the recognition result via the clipboard in MathML format, a standardized mathematical markup language. Any equation you write and recognize in MIP reaches your destination application in a completely editable form – you can insert and edit the output as you would edit any text.

We spent a lot of time researching and identifying as many areas of math as possible and endless different math notations. The final result is a great coverage of high school and college level math, and of even more advanced areas.

Math Input Panel – Have you heard of the formula recognized in the example above? You haven’t J, well, it’s the Schwarz formula used in complex analysis!

Using MIP is really simple and straightforward. You write the well-formed math expression (this means that will not get recognized, whereas will) just as you do with pencil and paper and the recognizer takes over. The recognized math is shown in the preview area. As no recognizer is perfect, the great power of MIP lies in its ability to provide a fantastic correction experience (let’s be honest, sometimes even a human is not sure what has been written – you should see my handwriting!).

In case your handwritten math is misrecognized, you can select any part of it (symbols or whole sub-structures) and correct it either by selecting an alternate from a drop-down list or by rewriting part of the expression. Usually fixing one part of the equation automatically fixes the rest, in just one step.

Math Input Panel Correction

All you have to do now is tap Insert and you have just painlessly created an equation in your word-processing or computational program.

There are many other cool features like History, moving ink around, and dragging and dropping ink into MIP from other inking applications such as OneNote, all of which you can explore on your own. For software developers, the MIP can be embedded into your applications – check out the documentation on MSDN.

Delivering a quality upgrade experience

This is a little bit of a tricky post to write because we’re going to be asking everyone using our Windows 7 Beta to help us out, but doing so is going to take a little time and require a bit of a commitment to helping test the next milestone. This has been a remarkably valuable and beneficial testing cycle for Windows as we have had a tremendous amount of very rigorous testing and usage. We’ve had millions of people install and use the Beta since January and as we’ve talked about, the feedback and telemetry have been of tremendous value as we finalize the product. The effort of Beta testers has contributed immensely to our ability to deliver a high-quality product to hundreds of millions of customers. We continue to follow the plan we have previously outlined and this post is no announcement of any news or change in plans. Since we know many people are running the Beta we want to provide a heads up regarding the behavior of the Release Candidate (RC) as it pertains to upgrades. Of course we are working hard on the RC and following the schedule we have set out for ourselves.

A big part of the beta process is making sure we get as much “real world” coverage of scenarios and experiences as possible and monitor the telemetry of those experience overall. One of the most challenging areas to engineer is the process of upgrading one release of Windows to another. When you think about it, it is the one place where at one time we need to run a ton of code to basically “know” everything about a system before performing the upgrade. During the development of Windows 7 we routinely test hundreds of original OEM images from Windows Vista and upgrade them and then run automated tests validating the upgrade’s success. We also test thousands of applications and many thousands of devices as they too move through the upgrade process.

Many of you installed the Windows 7 beta on a PC running Vista. We received that telemetry and acted on it accordingly. We believe we’ve continued to improve the upgrade experience throughout the release. Similarly, based on our telemetry most of you did clean installations onto new drive partitions. Through this telemetry we learned about the device ecosystem and what drivers were available or missing. We also learned about PC-specific functions that required installing a driver / application (from XP or Vista) to enable support for buttons, connectors, or other hardware components. Together we get great coverage of the setup experience.

We’ve also learned that many of you (millions) are running Windows 7 Beta full time. You’re anxious for a refresh. You’ve installed all your applications. You’ve configured and customized the system. You would love to get the RC and quickly upgrade to it from Beta. The RC, however, is about getting breadth coverage to validate the product in real-world scenarios. As a result, we want to encourage you to revert to a Vista image and upgrade or to do a clean install, rather than upgrade the existing Beta.  We know that means reinstalling, recustomizing, reconfiguring, and so on.  That is a real pain.  The reality is that upgrading from one pre-release build to another is not a scenario we want to focus on because it is not something real-world customers will experience. During development we introduce changes in the product (under the hood) that aren’t always compatible with what we call “build-to-build” upgrade.  The supported upgrade scenario is from Windows Vista to Windows 7. Before you go jump to the comment section, we want to say we are going to provide a mechanism for you to use if you absolutely require this upgrade.  As an extended member of the development team and a participant in the Beta program that has helped us so much, we want to ask that you experience real-world setup and provide us real-world telemetry.

If you do follow the steps below, you might run across some oddities after upgrade. We experience these internally at Microsoft occasionally but we don’t always track them down and fix them because they take time away from bugs that would not only manifest themselves during this one-time pre-release operation. From time to time we’ve noticed on a few blogs that people are using builds that we have not officially released and complained of “instabilities” after upgrade. Nearly all of these have been these build-to-build issues. We’ve seen people talk about how a messenger client stopped working, a printer or device “disappears”, or start menu shortcuts are duplicated. These are often harmless and worst case often involves reinstalling the software or device.

We’re just trying to be deterministic and engineer the product for the real world. Speaking of the real world, many have asked about upgrading from Windows XP. There's no change here to the plan as has been discussed on many forums.  We realized at the start of this project that the “upgrade” from XP would not be an experience we think would yield the best results. There are simply too many changes in how PCs have been configured (applets, hardware support, driver model, etc.) that having all of that support carry forth to Windows 7 would not be nearly as high quality as a clean install. This is something many of you know and already practice. We do provide support for moving files and settings and will prompt at setup time, but applications will need to be reinstalled. We know that for a set of customers this tradeoff seems less than perfect, but we think the upfront time is well worth it.

So when you try to upgrade a pre-RC build you will find that you’re not able to and setup will tell you and you can then exit gracefully. You can install as a clean installation and use the Windows Easy Transfer feature as well (run this from your current installation of course) if you wish to move your accounts, settings, files, and more. To bypass the version check, the instructions below will use a mechanism that is available for enterprise customers (so we are also testing this as well). It is not a simple command line switch. We didn’t make it multi-step on purpose but wanted to stick to using proven, documented and tested mechanisms.

These instructions will be brief. Since everyone reading is a well-versed and experienced beta tester you know ALWAYS BACK UP YOUR MACHINE before running any OS installation and NEVER TEST AN OS ON YOUR ONLY COPY OF ANY DATA. Testing a pre-release product means just that—it is testing and it is pre-release. Even though this is a Release Candidate, we are still testing the product. We have very high confidence but even if an error happens once in 1,000,000 we want to make sure everyone is taking the precautions normal for a pre-release product.

One other related caution is INSTALL ONLY OFFICIALLY RELEASED BUILDS FROM MICROSOFT. It will always be tempting to get the build with the “mod” already done but you really never know what else has been done to the build. There’s a thrill in getting the latest, we know, but that also comes with risks that can’t even be quantified. For the RC we will work to release a hash or some other way to validate the build, but the best way is to always download directly from Microsoft.

Here’s what you can do to bypass the check for pre-release upgrade IF YOU REALLY REALLY NEED TO:

1. Download the ISO as you did previously and burn the ISO to a DVD.
2. Copy the whole image to a storage location you wish to run the upgrade from (a bootable flash drive or a directory on any partition on the machine running the pre-release build).
3. Browse to the sources directory.
4. Open the file cversion.ini in a text editor like Notepad.
5. Modify the MinClient build number to a value lower than the down-level build. For example, change 7100 to 7000 (pictured below).
6. Save the file in place with the same name.
7. Run setup like you would normally from this modified copy of the image and the version check will be bypassed.

These same steps will be required as we transition from the RC milestone to the RTM milestone.

Again, we know many people (including tens of thousands at Microsoft) are relying on the pre-release builds of Windows 7 for mission critical and daily work, making this step less than convenient. We’re working hard to provide the highest quality release we can and so we’d like to make sure for this final phase of testing we’re supporting the most real world scenarios possible, which incremental build to build upgrades are not. At the same time everyone on the beta has been so great we wanted to make sure we at least offered an opportunity to make your own expert and informed choice about how to handle the upgrade.

We’re always humbled by the excitement around the releases and by the support and enthusiasm from those that choose to run our pre-releases. We’re incredibly appreciative of the time and effort you put into doing so. In return we hope we are providing you with a great release to work with at each stage of the evolution of the product. Our next stop is the RC…see you there!

THANK YOU!

--Windows 7 Team

PS: At Step 1 above many of you are probably thinking, “hey why don’t you just let me mount the ISO and skip the plastic disc”. We’ve heard this feedback and we deserve the feedback. We don’t have this feature in Windows 7 and we should have. So please don’t fill the comments with this request. There are several third party tools for mounting and if you’ve got a Vista image there’s a good chance your PC came with those tools on it.

Touching Windows 7

We’ve come a long way in engineering Windows 7 since we first provided an engineering preview of Windows 7 and the work we are doing to support the touch interface paradigm back at the D: All Things Digital conference.  We chose to kick-off the discussion about engineering Windows 7 with touch scenarios because we know this is a long-lead effort that requires work across the full ecosystem to fully realize the benefit.  For Windows 7, touch support is engineered by building on our advances in input technology we began with the TabletPC work on Windows XP.  Touch in Windows 7 requires improvements in hardware, driver software, core Windows user experience, and of course application support.  By having this support in an open platform, consumers and developers will benefit from a wide variety of choices in hardware, software, and different PC form factors.  Quite a few folks have been a little skeptical of touch, often commenting about having fingerprints on their monitor or something along those lines.  We think touch will become broadly available as the hardware evolves and while it might be the primary input for some form factors (such as a wall mounted display in a hospital, kiosk, or point of sale) it will also prove to richly augment many scenarios such as reading on a convertible laptop or a “kitchen PC”.  One of my favorite experiences recently was watching folks at a computer retailer experience one of the currently available all-in-one touch desktops and then moving to another all-in-one and continuing to interact with the screen—except the PC was not interacting back.  The notion that you can touch a screen seems to be becoming second nature rather quickly.  This post is our first dedicated blog on the subject. This is a joint effort by several people from the touch team, mostly Reed Townsend, Dave Matthews, and Ian LeGrow. -Steven

Windows Touch is designed to enhance how you interact with a PC. For those of us that have been living and breathing touch for the last two years we’re excited to be able to deliver the capability to people using Windows 7. In this blog we’re going to talk about what we’ve done to make Windows touchable. We approached this from a number of different directions: key improvements to the core Windows UI, optimizing for touch in key experiences, working with hardware partners to provide robust and reliable touch PCs, and providing a multitouch platform for applications.

Making Windows Touchable

With Windows 7 we have enriched the Windows experience with touch, making touch a first-class way to interact with your PC alongside the mouse and keyboard. We focused on common activities and refined them thoughtfully with touch in mind. You will have the freedom of direct interaction, like being able to reach out and slowly scroll a web page then flick quickly to move through it. With new touch optimized applications from creative software developers you will be able to immerse yourself as you explore you photos, browse the globe, or go after bad guys in your favorite games.

While providing this touchable experience, we made sure you are getting the full Windows 7 experience and not a sub-set just for touch. We’ve been asked if we are creating a new Touch UI, or “Touch Shell” for Windows – something like Media Center that completely replaces the UI of Windows with a version that is optimized for touch. As you can see from the beta, we are focused on bringing touch through the Windows experience and delivering optimized touch interface where appropriate.  A touch shell for launching only touch-specific applications would not meet customers’ needs – there would be too much switching between “touch” mode and Windows applications. Instead, we focused our efforts on augmenting the overall experience so that Windows works great with touch.

We took a variety of approaches – some broad, and some very targeted to support this goal:

• Touch gestures: Windows 7 has a simple set of touch gestures that work in many existing applications. These include the basics of tap and drag, as well as scroll, right-click, back, forward, zoom, and rotate. More details on how gestures work is described later.
• Improved high DPI support: Windows 7 has improved high dpi support (see High DPI blog post). The broad benefit to touch is that UI elements are rendered closer to their intended size – usually larger – which makes small buttons, links, and other targets easier to access with touch.
• Improved window management: The updated taskbar and windows arranging features go a long way towards making Windows easier to use with touch. There have been several subtle but critical touch optimizations:
  • The taskbar buttons and thumbnails are ideally sized for pressing with touch, and specific behaviors are tuned for touch input. For example, the Jump Lists can be accessed with a simple drag up from the taskbar, and when opened with touch, the shortcuts in the Jump Lists are drawn with extra vertical spacing to make them easier to select.
  • Aero Peek has been tuned to work with touch – the show desktop button is twice as wide (the only visual sign you are on a Windows Touch PC) and instead of hovering (which you can’t do with touch), a press-and-hold on the button activates Aero Peek.
  • Sizing and positioning windows is easy with Aero Snap – just drag a window to a screen edge. Furthermore, this was tuned with special touch thresholds so that you don’t have to drag to the absolute edge of the screen – a better balance for touch usage.
• Refinements to key experiences: The top browsing and media activities were refined to provide an optimized touch experience. IE8 includes support for the core touch gestures (scrolling, back, forward, zoom) as well as an optimized address bar that opens by dragging down, and extra spacing in favorites and history lists when opened with touch for easy selection. In Windows Media Player, the transport controls (play, pause, etc) have larger clickable areas even though they still look the same size – so that they are easier to touch.
• Touch keyboard: The on-screen keyboard has been optimized for touch with glow key feedback that’s visible when your finger is covering the letter and multitouch support for natural typing behavior and key combinations. It’s designed for quick usage, like entering a URL.

Overall, the Windows Touch features are designed to work together to deliver a great end-to-end touch experience. For example, the goal with IE8 was to deliver a seamless touch browsing experience, this includes the panning, zooming, URL entry, and several interface enhancements. For this reason, all the new touch features require the presence of a multi-touch digitizer – more on that further down.

Gestures

The Windows Touch gestures are the basic actions you use to interact with Windows or an application using touch. As we noted above, because the gestures are built into the core of Windows, they are designed to work with all applications, even ones that were never designed with touch in mind.

Our mantra with gestures has been “Predictable + Reliable = Habits”. To be predictable the action should relate to the result – if you drag content down, the content should move down. To be reliable, the gesture should do roughly the same action everywhere, and the gesture needs to be responsive and robust to reasonable variations. If these conditions are met then people are far more likely to develop habits and use gestures without consciously thinking about it.

We’ve intentionally focused on this small set of system-wide gestures in Win7. By keeping the set small we reduce misrecognition errors – making them more reliable. We reduce latencies since we need less data to indentify gestures. It’s also easier for all of us to remember a small set! The core gestures are:

• Tap and Double-tap – Touch and release to click. This is the most basic touch action. Can also double-tap to open files and folders. Tolerances are tuned to be larger than with a mouse. This works everywhere.
• Drag – Touch and slide your finger on screen. Like a dragging with a mouse, this moves icons around the desktop, moves windows, selects text (by dragging left or right), etc. This works everywhere.
• Scroll – Drag up or down on the content (not the scrollbar!) of scrollable window to scroll. This may sound basic, but it is the most used (and most useful – it’s a lot easier than targeting the scrollbar!) gesture in the beta according to our telemetry. You’ll notice details that make this a more natural interaction: the inertia if you toss the page and the little bounce when the end of the page is reached. Scrolling is one of the most common activities on the web and in email, and the ability to drag and toss the page is a perfect match for the strengths of touch (simple quick drags on screen). Scrolling is available with one or more fingers. This works in most applications that use standard scrollbars.
• Zoom – Pinch two fingers together or apart to zoom in or out on a document. This comes in handy when looking at photos or reading documents on a small laptop. This works in applications that support mouse wheel zooming.
• Two-Finger Tap – tapping with two fingers simultaneously zooms in about the center of the gesture or restores to the default zoom – great for zooming in on hyperlinks. Applications need to add code to support this.
• Rotate – Touch two spots on a digital photo and twist to rotate it just like a real photo. Applications need to add code to support this.
• Flicks – Flick left or right to navigate back and forward in a browser and other apps. This works in most applications that support back and forward.
• Press-and-hold – Hold your finger on screen for a moment and release after the animation to get a right-click. This works everywhere.
• Or, press-and-tap with a second finger – to get right-click, just like you would click the right button on a mouse or trackpad. This works everywhere.

For touch gestures, seeing them in action is important so here is a brief video showing the gestures in action:

 

In order to make the gestures reliable, we tuned the gesture detection engine with sample gesture input provided by real people using touch in pre-release builds; these tuned gestures are what you will see in the RC build. We have a rigorous process for tuning. Similar to our handwriting recognition data collection, we have tools to record the raw touch data from volunteers while they perform a set of scripted tasks. We collected thousands of samples from hundreds of people. These data were then mined looking for problems and optimization opportunities. The beauty of the system is that we can replay the test data after making any changes to the gesture engine, verifying improvements and guarding against regression in other areas.

This has led to several important optimizations. For example, we found that zooms and rotates were sometimes confused. Detecting zoom gestures only in applications that don’t use rotation has resulted in a 15% improvement in zoom detection.

Further analysis showed that many short gestures were going unrecognized. The gesture recognition heuristics needed to see 100ms or 5mm worth of data before making a decision about what gesture the user was performing. The concern that originally led to these limits was that making a decision about which gesture was being performed too early would lead to misrecognition. In fact, when we looked at the collected user data, we found we could remove those limits entirely – the gesture recognition heuristics performed very well in ambiguous situations. After applying the change and replaying the collected gesture sample data, we found zoom and rotate detection improved by about 6% each, and short scrolling improved by almost 20%!

Supporting Gestures

Gestures are built into the system in such a way that many applications that have no awareness of touch respond appropriately, we have done this by creating default handlers that simulate the mouse or mouse wheel. Generally this gives a very good experience, but there are applications where some gestures don’t work smoothly or at all. In these cases the application needs to respond to the gesture message directly.

In Windows, several experiences have been gesture enabled. We’ve spent a considerable amount of effort on IE8 – ensuring scrolling and zooming are smooth and that back and forward are at your fingertips. Media Center, which is a completely custom interface ideally suited to touch, added smooth touch scrolling in galleries and the home screen. The XPS Viewer has gesture support that will could become a model for many document viewing apps. Scrolling and zoom work as you would expect. When zooming out beyond a single page, pages start to tile so you can view many at a time. When zoomed out in that fashion, double tapping on any page jumps back to the default view of that page. A two-finger tap restores the view to 100% magnification. These predictable behaviors become habit forming quickly.

Working with the Hardware Ecosystem

A major benefit of the Windows ecosystem is diversity – PCs come in all shapes and sizes. To help ensure that there is a great Windows Touch experience across the many different types of PCs we have defined a set of measurements and tests for Windows Touch that are part of the Windows Logo. We’ve been working with touch hardware partners since the beginning of Windows 7 to define the requirements and ensure they are ready for launch.

Our approach has been to provide an abstraction of the underlying hardware technology. We’ve specified a requirements for the quantitative aspects of the device, such as accuracy, sample rate, and resolution, based on the requirements to successfully enable touch features. For example, we have determined the necessary accuracy values for a device so people can successfully target common UI elements like close boxes, or what sample rate and resolution are required to ensure quality gesture recognition.

The requirements form the basis for the Windows Touch logo program. For consumers, the logo tells you that the PC and all of its components are optimized for Windows. Component level logo, which is what we grant to Touch digitizers helps the OEMs choose a device that will deliver a great touch experience.

Based on the quantitative requirements, we built an interactive test suite that includes 43 separate tests, all validating the core requirements under different conditions. There are single point accuracy tests at various locations on the screen, including the corners which are often harder for accuracy but critical to Windows. There are also several dynamic tests where accuracy is measured while drawing lines on the screen – see the screenshot below of Test 7. In this test, two lines are simultaneously drawn using touch along the black line from the start to the end. The touch tracings must remain within 2.5 mm of the black line between the start and end points. The first image below shows a passing test where the entire tracing is green (apologies for the fuzziness – these are foot long tracings from a large screen that have been scaled down).

 

  Figure 1: A passing line accuracy test from the Windows 7 Touch logo test tool

Not all devices pass the tests. Below is a screenshot of a device that is failing. This one has some noise – notice the deviation from the line in red. These errors need to be resolved before it would receive the logo. Errors like this can result in misrecognized gestures.

 Figure 2: A failing line accuracy test from the Windows 7 Touch logo test tool

To ensure repeatability of the tests, we’ve built a set of plastic jigs with tracing cut-outs, see photo below. This particular jig is used for 5 of the tests and measures accuracy while tracing an arc.

Figure 3. Plastic jigs with tracing cut-outs for testing.

The testing tool is available to our partners now, we’re working closely with several of them to help tune the performance of their devices to meet the requirements and deliver a great touch experience. We have set-up an in-house testing facility that will be testing every device submitted for Logo.

With the Release Candidate, OEMs and IHVs will be able to finalize the logo process for systems designed for Windows 7. Today we already have several hardware partners that have provided us with devices and drivers for testing.

Windows Touch for Software Developers

We also want to talk a little about the touch platform for software developers.  Windows 7 provides a rich touch platform for applications. We’ve already mentioned gestures, there’s also a lower level platform that gives developers complete control over the touch experience. We think about it in a Good-Better-Best software stack.

Good

The “good” bucket is what touch-unaware applications get for free from Windows 7. Windows provides default behaviors for many gestures, and will trigger those behaviors in your application in response to user input. For example, if someone tries touch scrolling over a window that is touch-unaware, we can detect the presence of various types of scrollbars and will scroll them. Similarly, when the user zooms, we inject messages that provide an approximation of the zoom gesture in many apps. As a developer you can ensure that the default gestures work just by using standard scrollbars and responding to ctrl-mouse wheel messages.

Better

The “better” bucket is focused on adding direct gesture support and other small behavior and UI changes to make apps more touch-friendly. For instance, there is a new Win32 window message, WM_GESTURE (preliminary MSDN docs), that informs the application a gesture was performed over its window. Each message contains information about the gesture, such as how far the user is scrolling or zooming and where the center of the gesture is.

Applications that respond to gestures directly have full control over how they behave. For example, the default touch scrolling is designed to work in text centric windows that scroll primarily vertically (like web pages or documents), dragging horizontally does selection rather than scrolling. In most applications this works well, but if an app has primarily horizontal scrolling then the defaults would have to be overridden. Also, for some applications the default scroll can appear chunky. This is fine with a mouse wheel, but it feels unnatural with touch. Apps may also want to tune scrolling to end on boundaries, such as cells in a spreadsheet, or photos in a list. IE8 has a custom behavior where it opens a link in a new tab if you drag over it rather than click it.

In addition to gestures, there are subtle optimizations applications can make for touch if they check to see if touch is in use. Many of the subtle touch behavior optimizations in Windows were enabled in this manner. Larger Jump List item spacing for touch, larger hot spots for triggering window arranging, and the press and hold behavior on the desktop Aero Peek button with touch are all features written with the mouse in mind, but when activated via touch use slightly different parameters.

Best

Applications or features that fall into the “best” bucket are designed from the ground up to be great touch experiences. Apps in this bucket would build on top of WM_TOUCH – the window message that provides raw touch data to the application. Developers can use this to go beyond the core system gestures and build custom gesture support for their applications. They can also provide visualizations of the touch input (e.g. a raster editing application), build custom controls, and other things we haven’t thought of yet!

We also provide a COM version of the Manipulations and Inertia APIs from Surface. The Manipulations API simplifies interactions where an arbitrary number of fingers are on an object into simple 2D affine transforms and also allows for multiple interactions to be occurring simultaneously. For instance, if you were writing a photo editing application, you could grab two photos at the same time using however many fingers you wanted and rotate, resize, and translate the photos within the app. Inertia provides a very basic physics model for applications and, in the example above, would allow you to “toss” the photos and have them decelerate and come to a stop naturally.

Showcasing Touch

We’ve previously demonstrated, Microsoft Surface Globe, an interactive globe done in partnership with the Surface effort. Spinning the globe works as you would expect from a real-world globe, but with a touchable globe you can grab and stretch the view to zoom in, rotate, and move the view around. Interacting with the globe and exploring the world is the majority of the UI, and it is exceedingly easy to use with touch. Other features like search and adding markers to the map have also been designed with touch in mind.

Here’s another video to get an idea of what we’re talking about:

 

We’re eagerly looking forward to seeing new touch-optimized user interfaces and interactions. If you’re thinking about writing touch applications or adding touch support to your existing app, you should start with the MSDN documentation and samples.

What Next?

We’ve noted several touch updates in the RC. If you have the Windows 7 Beta you can experiment with touch using a PC that supports multiple touch points. Please note that the multitouch PCs available today were developed while the Windows 7 requirements were also defined, so while we believe they can support Windows 7’s requirements, only the maker of the PC can provide the logoed drivers for Windows 7 and support the PC on Windows 7. Keeping that caveat in mind, today there are a few multitouch PCs on the market:

• HP TouchSmart All-in-One PCs (IQ500 series & IQ800 series)
• HP TouchSmart tx2 Tablet PC
• Dell Latitude XT or XT2 Tablet PC

To enable multitouch capabilities on these PCs running the Windows 7 Beta you will need to make sure you have the latest multitouch beta drivers. Remember these are pre-release drivers and are not supported by Microsoft, Dell or HP. And again, they still need to pass through the Windows Logo process we described above before they are final.

• For HP TouchSmart All-in-One PCs: The pre-release driver is available from Windows Update. After you have installed the Window 7 Beta, open Windows Update from the Start menu. You might have to click the “Check for Updates” link on your left so it will find the driver, it is Optional right now so you’ll have to select it before it will install. Alternatively you can download it from the NextWindow website.
• For the Dell Latitude XT, XT2 and HP TouchSmart tx2 Tablet PCs: the drivers are available on N-Trig’s website. N-Trig is the company that makes the digitizer in these PCs (you should read the release notes, there are some limitations, like no pen support – that will be fixed by RC – you should be aware of and how to switch between Windows Vista and Windows 7).

We often get asked about single-touch PCs. Will they work with Windows 7? There are many types of hardware available for touch and many screens and PCs can provide single touch (usually based on resistive touch technology).  A single-touch PC will have the same functionality on Windows 7 as it does on Vista, but this functionality will not be extended to the Windows 7 capabilities. As we noted earlier, Windows Touch in Windows 7 is comprised of a collection of touch enhancements, several of which require multitouch, that work together to deliver a great end-to-end touch experience.

As form factors change and the demands of our user interfaces change, input methods change and grow as well. We’re excited about the unique benefits touch offers the user, and the new places and new ways it enables PCs to be used.  We expect PCs of all form factors and price points to provide touch support and so it makes sense that these PCs will be able to take advantage of the full range of Windows 7 capabilities.

Windows 7 is designed to provide efficient ways to use multitouch for the most common and important scenarios, while being a natural and intuitive complement to the mouse and keyboard people use today.

Keep in Touch!

- Windows Touch Team

Federating Windows Search with Enterprise Data Sources

The Windows Explorer has evolved by enabling you to find all sorts of content by searching for it. Many of you have used the search features in Windows Vista (based on our instrumented data) from the start menu or from the search box in Explorer. It has been a long time since most of us could remember where everything is by carefully managing our folder hierarchy and finding things based on file name alone.  We often rely on domain specific search (in music players, mail clients, photo clients) but with Windows Vista and Windows 7 we make it possible to search within a namespace and across namespaces.  This post is about a new feature based on Search that allows searching across PCs and even servers in an Enterprise setting.  Alwin and Scott, program managers, and Brandon, a developer, on the “Find and Organize” feature team authored this post. --Steven

Finding your stuff

Whether you’re searching or browsing, Windows Explorer is really about finding your stuff, and once you’ve found it, doing something with it (such as copying, opening, deleting, etc). For data that lives on your PC or home network, Windows 7 has invested in HomeGroup and Libraries (subjects for a future posting from our team) to provide an easier and richer experience than ever before. However, we didn’t stop there. Over the last few years, we’ve seen enterprise customers’ important content migrate towards (or aggregated in) centralized content stores, such as SharePoint. These products typically provide great features for team collaboration, document versioning and workflow management, archiving, retention policy enforcement, and other centrally-managed functionality that IT managers appreciate.

Important enterprise data is found on local machines, in a variety of centralized content stores and also beyond the firewall

Unfortunately, this has placed an extra burden on customers to learn each new content store’s user interface, often asking them to give up familiar desktop features like drag-and-drop. Given their collaborative focus, these sites grow organically and it can become hard to remember where a particular document was stored and then wade through long lists of them every time you want to get back to it. Enterprise customers have asked us for a solution that simplifies finding important content in these various data stores but without leaving their normal Windows work flows.

As we looked at this trend and the lack of integration with content management and content indexing web services, we used these guiding principles in developing a solution:

• Natural for people to use. Customers want a more consistent experience for finding and working with data in these disparate content stores, and would like us to bridge the local and remote content experiences by helping them “roll over” from one to the other.
• Easy for IT admins to deploy. IT admins don’t like to deploy code, and want low-maintenance solutions that are easy to manage. Meanwhile customers want to connect up these sources without going through long and tedious installation processes or having to get help every time they want to set up a new search location.
• Easy for developers to adopt. Developers want to enable this functionality in their offerings quickly and easily. There are a lot of data sources which need to be supported because IT folks don’t want to be locked in to a specific server technology.

Choosing to build Federated Search

Federated Search wasn’t the only way to address these challenges. The brute force approach would have been to take our existing Windows Search indexing technology and just use it on these content stores—that index the remote content on a local PC. This isn’t a very realistic solution since it’s inefficient to have all content indexed over the network by each person’s machine, especially when the content is changing at a rapid pace and represents a large corpus. Corporate retention policies may also prevent keeping even a local index of certain sensitive data.

Fortunately, there’s a better option – Federated Search. Federated Search enables you to search a remote web service from Windows explorer and get results back that you can act on like any normal file. The largest barrier to doing Federated Search has already been taken care of too. That is, most of these content stores are already indexed on the server, or at least on some server. There are several great offerings that will accomplish this, such as Microsoft Search Server. Not only do these servers index this content, but many of them already expose search results via a standard web protocol. This is largely thanks to the prevalence of OpenSearch and RSS enabled clients (including Internet Explorer and Microsoft Search Server, among many others).

For Windows 7, we’ve added support for Federated Search using OpenSearch v1.1 and worked to make the experience a seamless one. We found this solution strikes a good balance by leveraging the strengths of content services and the strengths of local file interactions within Windows.

Natural to use

Using Windows Explorer, people are familiar with several important user interface and interaction elements. They know how to use the navigation pane to change what they’re looking at. They know how to scroll around, how to select an item (or several), and they know how to double-click to open them. Most people know how to right-click for context-sensitive options related to their selection, or how to find those options presented in the command bar. They know they can drag and drop items to move them around. They know how to change view modes. We hope that they know how to search their current location using the search box, and in Windows 7 we think we’ve made it much easier to discover and use the Preview Pane to make sure they’ve got the right result.

Searching a SharePoint site using the new Federated Search support in Windows Explorer

Much of the usefulness of building Federated Search into Explorer is our ability to take advantage of this knowledge and familiarity. This may seem obvious once you see it in action, but behind the scenes there’s quite a lot going on to make all of this happen. For example, some applications such as Microsoft Word already know how to work with web URLs. So opening a Word document from a web server is fairly straightforward. But the majority of applications you’ll encounter really only understand how to open files on the local machine or via standard network file sharing protocols. This includes everything from the built-in software like Notepad and Paint, to third-party software like Photoshop or iTunes.

To handle this case, we implemented a “just in time” download solution, which will download the file to the internet cache before opening an application or taking actions (like using the SendTo menu) which require local files. This lets us offer searches that are very “lightweight” from a server load perspective, where we display metadata and icons or thumbnails without ever requesting the actual file. Then if you take an action like previewing or opening an item, we will do some behind-the-scenes work to make a local copy of the file only if necessary.

That enables us to work with the existing application ecosystem without asking anything of developers. However, applications can also take steps to offer even better functionality in many cases. For example, Windows Photo Viewer has added support for non-file items. So if you open a picture result in the built-in photo viewer, it’s the photo viewer that downloads the item, not Explorer. This may not seem like a big deal, but it lets the photo viewer enable the forward and back buttons to jump to the next or previous result – and it will download that image on-demand. Starting at the PDC we began reaching out to third-party ISVs to encourage them to implement similar enhancements for Federated Search scenarios, and we will continue to offer guidance on how to best integrate with all of the newest Explorer features.

Finally, we support all the standard clipboard and drag-and-drop operations. So if you drag a Word document from a Federated Search query onto your desktop, it will be copied there. You’ll even see the familiar Windows Explorer copy dialog, with progress indication, cancel ability, conflict resolution, and so on.

But wait, there’s more! Windows Explorer is a great tool that many customers know and love. But some people use it without even knowing it. Countless Windows applications make use of what we call the Common File Dialog. This is a special Explorer window that lets you find and choose items to be opened or inserted into your current application, without ever leaving it. If you’ve ever clicked File and then Open or Save in an application menu, you’ve probably seen some version of this dialog. PowerPoint, for example, uses the common file dialog to insert pictures. That means from inside PowerPoint you can click Insert Picture, select the Federated Search link for your image repository, search for the picture you want, and then insert it directly into PowerPoint. This works for any existing application that supports the Common File Dialog, and there are a whole lot of them!

Inserting a picture into PowerPoint’s using Federated Search

Our Federated Search solution is all about simple lightweight access with a common, familiar user interface. This has a lot of benefits as we described above, but there are also cases where a server’s web interface will offer its own benefits. This might involve advanced query building, browsing, or server work-flow tasks, for example. So Windows 7 builds a bridge to these content repositories. After doing a search against a supported location, you will see a “Search on Website” button in the command bar which allows you to seamlessly send the query up to the service’s web interface in the default web browser. You’ll also see the “Open File Location” menu item when you right-click on a search result. Selecting that option will launch the web browser to the specific location in the document repository where the file is stored.

This seamless integration of Federated Search within Windows allows customers to greatly simplify their workflow for getting at remote files while still being able to easily take advantages of the advanced functionality of content repositories.

Simple to deploy

Our next challenge was to make it easy for customers to get these new connections onto their machines. It wouldn’t be practical to ship Windows with a connection to every solution in the world, so we shifted to a way that would make it very easy for any web service to deploy a connection to their specific service.

The model we came up with is similar to the way you add favorites from the web today. A web service can place a link to an .osdx file somewhere on their web page (see Channel 9’s search page for an example). The .osdx file is a simple XML file that uses the OpenSearch description document format to describe how to connect to the web service, and gives the web service some control of how the data is presented in Windows Explorer. When a person clicks on the link, Windows performs an ultra-lightweight install process that adds a search connector to that web service and places a link to that it in the Windows Explorer favorites.

If you are an administrator in an enterprise environment, you will likely want to provide some pre-installed search connectors for your users to search the company intranet or a popular internal SharePoint site for example. You can do this by deploying the search connector (.searchconnector-ms) files to your users’ machines via typical deployment techniques such as imaging, group policy preferences or startup scripts. The beauty is that it’s just a simple XML configuration file and there’s no code that needs to get installed on their machines. It’s also possible to pin one of these as a link from the Start menu through group policy. In the group policy editor look for the policy in this area: User Configuration> Administrative Templates > Windows Components > Windows Explorer. The policy name is “Pin Libraries or Search connectors to Search again links and start menu”.

Launching a Federated Search of an enterprise Intranet from the Start Menu

Easy to adopt

Of course this technology depends on having services that support it. Although there are only a few services that provide a .osdx for you today, there are many existing services that already support the basic OpenSearch requirements.

We’re already seeing positive initial reactions from enthusiasts and ISVs alike echoing that it is indeed easy to enable your service to work with our Federated Search platform. If you’re a developer and want to enable an existing web based service to support Windows 7 Federated Search, you’ll need to provide a web service that accepts an http GET request with the search terms embedded somewhere in the URL and be able to return the results as an RSS or Atom feed. These requirements are typically very easy to meet for most applications that already provide search services via a web browser.

Your web service results should include the basic RSS tags like <link>, <title>, <description>, <pubDate> to get started but there’s much more that you can include in the RSS output and customization you can do within the .osdx file to enhance the experience for the end user.

For more information, we’ve published the Windows 7 Federated Search implementer’s guide with detailed information on how to enable your data source to work with Windows Federated Search. There’s also a recorded PDC session that demonstrates how to build a Windows Federated Search compatible web service for an existing SQL database.

- Brandon Paddock, Scott Dart & Alwin Vyhmeister, Find and Organize

Designing Aero Snap

Here’s a behind the scenes look at the design of the Aero Snap feature in Windows 7.  We thought it would be fun to take a look at the overall design process of the feature and the tools and techniques used.  This feature poses a unique design challenge in that you just use the feature without any user-interface specifically to invoke it.  As with all features this is a collaboration across all of our engineering disciplines.  For this post, Stephan Hoefnagels, a Senior UX designer, presents the design perspective.  --Steven (P.S., keep an eye out on the Microsoft MIX conference this week!)

In Managing Windows windows and Follow-up: Managing Windows windows we talked about, and you shared, some interesting window management scenarios that we might address in Windows 7. We also touched on some data around typical configurations, as well as goals that guide our thinking in this area.

In this post we’d like to have a closer look at the Aero Snap feature that many of you have already been able to experience in our PDC builds, and of course the Beta. We’ll briefly describe the feature itself, but mostly we’d like to invite you to take a behind-the-scenes peek at our design process so far, and share our iterations, challenges and considerations.

Goals and scenarios

As we explained more in depth in our previous post, our top goal for the Aero Snap feature is to provide you with an effortless way to position your windows the way you want them. We want to reduce the number of clicks and precise movements needed to perform common activities. In a general sense, we want you to be able to manage your windows with confidence and create a feeling of power and control. This is something we touched on in our post on the Windows 7 taskbar as well, and really a theme that weaves through much of our new desktop experience.

Before we look at how we address our goals in the design, a quick note. In the scenarios below you’ll notice sequences of interactions that are fully written out. Sequences like: select the window, click the caption button, and then resize the window. Looking at interactions at this level of detail makes for somewhat awkward reading. These individual steps are so small, and so frequent, that for most of us they normally barely register. Why not simply gloss over some of those details? Well, we spell out sequences of events this way to force us to be consciously aware of the amount of “overhead” that is sometimes involved to get to the task at hand. It forces us to realize what we normally might not. Also, besides providing insight in the problems, it provides us with the right level of detail to consider our solutions.

Now, let’s look at the design!

Side-by-side windows

As many of you mentioned, doing a drag drop operation from one window to another is sometimes a pain. Windows tend to overlap and to get them positioned right can require a lot of fine mouse movement. Oftentimes the steps are as follows: select a window, resize it appropriately, and position it on the screen so that enough of it shows for it to be a meaningful drag or drop target. Then repeat the same actions with the other window. Similarly, comparing content in two windows requires a lot of mouse clicks, resize actions, careful arrangement, window switches, and a fair amount of mouse mileage.

With Aero Snap you can grab a window and move your mouse to the edge of the screen and the window will resize to fill half the screen. Repeat with the other window. Now with two easy motions you have a setup that makes both of these scenarios much easier to accomplish!

Put windows side-by-side with Aero Snap by moving the cursor to the edge of the screen (left to right, top to bottom)

Vertical maximized windows

We know our users love the maximized window state. Many love it so much that they maximize all of their windows and never even run in any other state! However, with screens increasing in resolution and widescreen layout becoming more prevalent, the maximized window state can lose some of its appeal in certain cases. E-mail is an example. Reading long lines of text across the screen is not ideal. Your eye simply cannot track a line all the way across. Web browsing is another example. Content will sometimes not fill the entire width of the screen, leaving a lot of unused white space on the side.

Now, with Aero Snap you can you can maximize a window in the vertical direction only. When you resize a window to the top of the screen, it will also resize all the way to the bottom. Great for reading long blog posts!

Vertically maximize a window with Aero Snap by resizing the window to the edge of the screen

Moving maximized windows from screen to screen

We realize there are a few multi-mon users out there, especially amongst the readers of this blog ;-). Ever wanted an easier way to move a maximized window to another screen? A way that’s quicker than clicking the restore caption button, moving the mouse to the title bar, dragging the window over, and finally clicking the maximized caption button again? With Aero Snap you can simply drag a maximized window down, move it over and snap it to the top, all in one gesture. Finally!

Arranging windows on laptops

Arranging windows on a desktop PC can sometimes involve excessive fiddling, fine mouse movements and lots of mouse mileage, as touched on above. On laptops this situation is further exacerbated by the lack of a mouse, making some of these movements even more cumbersome. For these scenarios, and our power users, we’ve introduced some convenient key combinations. Hold down the Windows key and one of your arrow keys to give it a try. You may want to try holding down Shift as well, especially if you’re in a multi-mon setup.

Design process

OK - So those are the scenarios, and the way we chose to address them. Seems straightforward enough, right? Especially for those of you that have been using Aero Snap in out Beta build. It all behaves as you’d expect. But how did we get here? Well, in this next bit we’d like to something that we don’t do that often and really give you a peek into our design process so far and some of the snags we ran into along the way. Keep in mind that this is a brief overview and by no means exhaustive. While we allude to usability testing for instance, this post is by no means meant to give insight into the scope of those efforts. There will certainly be opportunities to talk about some of those topics in the future.

Let’s have a look!

Sketching

We’re back in early 2007, and after we established window management as a potential area of interest, and identified several appealing scenarios (again, see our previous post for more detail on that stage of the process), we started with brainstorms to generate ideas. “How can we make window arranging more efficient?” “More direct?” “More fun?” are some of the questions we asked ourselves. This was a multidisciplinary process, with disciplines like design, user research, program management and development involved. All in all there were around a handful of us, certainly less than a dozen, thinking about this space at the time.

We imposed a significant constraint on ourselves: we wanted to achieve our goals without introducing any extra widgets in our UI. Imagine an extra caption button on the window title bar for instance: this may not seem too bad for one window, but when we’re talking about 10 windows or more, we’ve all of a sudden introduced a significant amount of clutter on your screen. And that’s something that we simply didn’t feel good about. After all, as we shared in our UX Principles talk at the PDC, we believe in “Solving Distractions, not Discoverability”.

We captured our, many, ideas in very quick sketches that we shared via an internal website. Transferred from the whiteboards in our offices and hallways, these took less than 5 minutes to sketch each. The sketches below are some actual examples in which you can start some of the Aero Snap ideas forming.

Early ideas are captured in quick and disposable sketches

Interactive prototyping in early code

With so many ideas on paper we were eager to try out the best ones. Now was the time to prototype. Note that we are still very early in the process. We wanted our prototypes to be interactive, and we wanted to be able to live with them in our day-to-day work. So we chose to implement the ideas using early code that we could run on our work machines. For example, the image below shows a “smart resizer” prototype running on Windows Vista. Of course these prototypes are not “done” features that we could actually ship: they merely get the basic ideas working, and they definitely have more than a few “quirks” (bugs ;-)). What’s important however, is that they allow us to experience just enough of the interactions ourselves, as well as get feedback in usability studies.

Early “smart resizer” prototype running on Windows Vista, note the taskbar button created by the prototype (and the date in the calendar to get a sense of where we are in the process)

We use this firsthand experience and early usability feedback to iterate on the ideas as we hone in on the final design. We approach this part of the process in a very open minded way and allow ourselves to be surprised. Sometimes ideas that don’t look like much on paper prove to be startlingly powerful. On the other hand, we found out that others look much better on paper than they were in practice! Since we’re not invested in any one idea or implementation yet, we freely refine, and drop ideas.

Finally, the prototypes ease us into thinking about design details. And we might stumble on some insights too (of course we tell ourselves the real feat is to recognize the insight and hold on to it). Here’s an example from an e-mail at the time from one of the team members about a new version of the “smart resizer” prototype:

I noticed something changed. In the original version if I resized it to the max it “supersized” the window. Then if I resized the window smaller, it jumped back to the normal restored state. It was as if the supersize state was different than restored state. With this version when I supersize the window and then resize it again later, it doesn’t jump back to the previous restored size. There was something kind of nice about the supersized state being different than the restored state. We should think about it more and consider making that a part of the design.

After many a prototype we settled on the concept of side-by-side windows and vertical maximized windows. We’re getting clearer on what we want to build.

Detailed design: state transitions

OK - We’ve whittled down our ideas to a good, but not fully specced, set of interactions and behaviors. Time to start filling in the blanks by asking detailed questions. “What does it mean to have a side-by-side window in a world where there used to be only minimized, restored and maximized windows?” “How exactly would you get to and from this new window state?” The e-mail snippet above already pointed to some of these questions.

Currently the common window states are (left to right) minimized, restored, and maximized, how would side-by-side and vertical maximized windows fit in?

Let’s look at the state problem in detail. Below is an example of two proposals made during this time that show how you can move from one window state to another, for all the different states. Which model is better?

Two proposals detailing the various ways we can transition between states

To answer that question we considered more specific questions like “What states do we want to link directly and how do we move between them?” “Is it compelling to go from a vertical maximized state directly to a maximized state?” “Should the vertical maximized and side-by-side states behave similar, as they look similar?” Our answers of course guided us to the model that you are now familiar with, which is model B.

But that’s not the end of it. More details need to be worked through, and more questions come up. “What if I want to move a vertical maximized window sideways?” “Resize its width?” “Then pull it down?”

Soon you’ll find yourself in elaborate sequences of events, many possible actions, and even more possible outcomes. Which would be the most expected when actually using the entire system?

To help guide our decision making process we established some guiding statements. Assumptions that we hold to be true. Examples are: “The intuitive way to undo an effect triggered by a mouse movement is to make the opposite mouse movement.” And: “It should always be effortless to go back to the previous “restored” state so as to avoid excessive work to get the window back to a reasonable size.” Or: “If the user specifies a width for a window in a given state, that size should be preserved across state changes when it makes sense.”

Using these statements we were able to answer questions like the ones above in a predictable way and as a result craft a predictable experience. And while the underlying state transitions and rules are fairly complex when added all together, the resulting behavior is, we hope, intuitively understood. That’s definitely something we’re aiming for.

Conflicting rules Here’s an example of a sequence problem for the really attentive reader. When working through our many new state transitions, sometimes the rules that determine what should happen, conflict. Consider the following scenario. Two rules in our new window model are: Dragging a window to the top of the screen maximizes it The intuitive way to undo/cancel an effect triggered by a mouse movement is to make the opposite mouse movement OK – Let’s try the following scenario in your Windows 7 build. Start with a restored window. Vertically maximize it by resizing it to the top of the screen. Release the mouse. Drag (don’t resize) down the window and drag it to the top again, all in one motion. Release the mouse. What happened? Your window should be maximized. Which means in this case we chose to follow rule 1. We could have also followed rule 2, in which case your window would be vertically maximized. We figured rule 1 would more accurately reflect the user’s mental model for this scenario. This is just one example of the decisions we had to make for each transition to and from a window state.

Throughout this process we made sure to preserve the subtleties in the current model. One small example that some of you may be familiar with that we did tweak is the scenario of dragging a window title bar off screen. In previous Windows releases we would snap the window back halfway in an attempt to provide you with just enough space to still move the window around, while optimizing vertical space as much as possible. We chose to be a little more deliberate and straightforward here by snapping back so that the entire title bar is visible. This way you can start to rely on the fact that all of your windows are always very easy to grab, also with touch, and move around. If we really felt half of a title bar is enough, why don’t we always half it, right? For our vertical maximized window states we of course chose to keep the entire title bar visible as well, leading to a solid story all around.

Storyboards and other visualizations

State diagrams are of course only one way to look at the world. We used various ways to communicate different aspects of the feature, picking our medium based on familiarity, availability and intent. We didn’t even shy away from the occasional ASCII art as you can see below! We’ll simply use whatever tool gets the point across. Most of all, interaction storyboards were a very valuable technique to help us understand the user flows, and even though this is only a small sample, you can see we did quite a few of those.

Feature details are communicated throughout using appropriate means

Accidental triggers

Besides figuring out the right state transitions, one or our biggest discussions was around when a state transition should exactly occur, or in other words: when the feature is triggered. We talked a lot about “accidental triggers”, i.e. running into the feature without deliberately setting out to do so.

 

Aero Snap is triggered by touching the edge of the screen with the mouse

From the very beginning we always made sure that our feature did not get in the way of current scenarios such as tucking a window off-screen to the side. After all, we don’t want to have a detrimental effect on your current, expected, way of managing your windows. That’s why you have to literally touch the edge of the screen with your mouse, not the window edge, to trigger the transitions.

However, at this time, the feature as you know it now was different in one very important, fundamental way. In our early builds, Aero Snap followed an “instant commit” model. When you moved your mouse to the top of the screen, your window would literally be maximized while you’re still dragging. That is, before you even release the mouse. Moving back in one motion would literally restore the window. We liked this approach as the “preview” was very accurate, i.e. the preview was the window, and there is a certain directness in not having a commit model.

Because our UI is invisible by design, we expected some accidental triggering. In fact in some sense we were relying on accidental triggers to help with the discoverability of the feature. However, after living with the builds for a while we got a little worried because accidental triggering seemed higher than we expected. From our telemetry data we saw people running into the feature, and then cancelling it nearly twice as often as committing. In our usability studies we observed confusion as to what exactly triggered the behavior. Was it the window touching the edge that did this? A gesture? Or something else?

We’re now in early 2008. What should we do? Cut the entire feature? We actually did consider this as an option. Again, we really respect our current window management behaviors, and the last thing we wanted to do is degrade the experience. More constructively, we took on the challenge to come up with a design tweak that would address these issues. We explored several solutions. Some conservatively centered on smoothing out the resize transitions, so an accidental trigger would at least be smooth. A conservative approach for sure, but probably not satisfying enough as a real solution. Others focused on trying to detect user intent more accurately, based on the angle of motion into the screen edge, or the speed of the motion. This proved much too complex to be predictable. Maybe we could trigger the transition only when double-bumping the edge? We were worried that this would degrade the experience of fluidity and flow of the current model.

In the end we settled on the implementation you’re familiar with. We don’t trigger until you release the mouse, making it easy to back out of the effect before anything happens to your window. In addition we provide you with a smooth preview animation, and a cursor effect to help you understand what just happened. This way you can be more deliberate in the future, and use the feature to your full advantage.

Did we solve the issues? Feel free to let us know :-)

Look and feel

It’s interesting to think back and realize that up to this point we had essentially designed a feature without any visible UI whatsoever. Now all of a sudden we have window previews, and a cursor effect. What should those look like, and how should they behave?

Well, luckily we had some things to go on. At this point we had already established a clear picture of our taskbar look and feel (we call it “personality” and will talk about it more in a later post). We had settled on using glass sheets for Aero Peek. We saw an opportunity to use the same visual representation for our preview windows. But how should the glass sheets appear? After experimentation, we settled on a scale animation that originates from the cursor. This gives a subtle hint as to where this preview window came from. We also made sure to animate our transitions. Try this in your build for instance: move a window to the top, and then to the side, in one motion without releasing the mouse. Notice the smooth morph of the preview? Why did we spend time on this? We believe “Small Things Matter, Good and Bad” of course.

Light effects are used to indicate the snap trigger, and glass sheets for snap previews

Finally, we tied into some of our ideas around “light” for the trigger indication. We tuned this to be noticeable, but not too loud and made sure to synch with other light effects in the system such as our touch feedback.

We hope this post has given you some insight in the Aero Snap feature, including our design process. We would love to hear your thoughts!

Stephan Hoefnagels