These postings are provided "AS IS" with no warranties, and confer no rights. Use of included code samples are subject to the terms specified at Microsoft -
Notes on comments.
Welcome to our blog dedicated to the engineering of Microsoft Windows 7
We’ve been quite busy for the past two months or so working through all the feedback we’ve received on Windows 7. It should be no surprise but the Release Candidate for Windows 7 will have quite a few changes, many under the hood so to speak but also many visible. Some have asked if the featureset is "frozen" then what will we change--we change a lot of things in the beta based on feedback and we try to do so in a systematic manner with the focus on the goals for the release. The goal of having a fully functional Beta was to make sure we received reliable feedback and not a lot of "hey this doesn't work at all" sorts of reports. This has allowed us to really focus on delivering a refined RC where the changes we made are all the reflection of feedback we have received.
Building on the previous post that looked at the broad view of feedback, we want to start posting on the feedback and the engineering actions we’ve taken in responding to the feedback. We won’t be able to cover all the changes (as we’re still busy making them), but for today we wanted to start with a sampling of some of the more visible changes. We’re still on the same path working towards the release candidate and of course we know everyone is anxious for the next phase of our path to RTM. In the meantime, our full time machines are still running the Beta build.
Today’s post is from Chaitanya, who has previously posted on some of the core user interface work. --Steven
This blog post talks about a few of the improvements that will be in our Release Candidate (RC) based upon customer feedback. There are many under the hood changes (bug fixes, compatibility fixes, performance improvements, and improvements) across the entire dev team that we just don’t have room to discuss here, but we thought you’d enjoy a taste of some changes made by three of our feature teams: Core User Experience, Find & Organize and Devices & Media. The comments in this article come from a variety of verbatim sources, with identifying information withheld.
1. Windows Flip (ALT + TAB) with Aero Peek
We’ve received overwhelmingly positive feedback about Aero Peek and how it helps customers switch windows with increased confidence. Daniel wrote to tell us “I’m wondering why Peek was never implemented for the ALT + TAB window. The thumbnails look/behave the same way as the taskbar thumbnails when you hover the mouse over them. It seems logical that they would exhibit the peek behavior, too”. We decided to make this change since we heard many requests for it. One can still quickly flip between and cycle through running windows using the ALT+TAB keys, but when more window information is needed Aero Peek will appear. This is triggered by a time delay as you pause while keyboarding through running windows.
Aero Peek triggered from Windows Flip (ALT+TAB)
2. Windows Logo + <#> keyboard shortcut
Enthusiasts often ask us for more keyboard shortcuts to simplify their common tasks. Efficiency is key. We’ve answered with a very powerful new keyboard shortcut for the taskbar that may just alienate mice everywhere. Pressing Windows Logo + <#> (where <#> corresponds to an item’s order in Quick Launch) in Vista would simply launch the item. As part of our unification of Quick Launch with the taskband in Windows 7, we now beef up the shortcut so it can both launch and switch. For example, if IE wasn’t running in Fig 1 then Windows Logo + 2 will launch the program (as it did in Vista). If IE is running with a single window, the same shortcut will now switch to the program. The magic really begins when IE is running with several windows or tabs—holding down the Windows Logo and tapping the 2 key repeatedly will actually cycle through the open IE items off the taskbar (with Aero Peek, of course). Letting go simply switches to the corresponding window. Think of this as per-program ALT +TAB shortcut for the first 10 items on the taskbar. If you need a new instance for IE, simply use SHIFT + Windows Logo + <#>. A program’s Jump List may also be accessed via ALT+ Windows Logo + <#>. Finally, you can even flip back to the last active window of a program by using CTRL+ Windows Logo + <#> (this also works by holding CTRL with a mouse click on a taskbar button). Keyboard aficionados rejoice!
3. Needy State
“Needy window” is the internal term we use for a window that requires your attention. Since the ‘90s, the taskbar has always provided some type of visualization to alert the customer to this state such as by flashing the button. A careful balance must be struck between providing information and not irritating the customer. With the new taskbar, we received feedback that Outlook reminders or a Messenger chat sometimes went unnoticed because needy windows were too subtle. For example, Mudassir opened a bug to say “The flashing is not obvious enough to get user's attention. Sometime I don't even notice it. It flashes for a little bit and then stops. If I am away the icon flashes and stops before I come back. The icon is not noticeable.” We’ve made three changes that should address the issue. First, we changed the flashing animation curve to make it more noticeable (from a sine to a sawtooth wave). Second, we used a bolder orange color. Finally, we wanted to double the number of flashes which is currently set to three. As a nod to Windows 7, we decided to go with seven flashes instead.
4. Taskbar “Open With”
Quick Launch always supported the ability to drop a file onto a pinned program and have it open with that program. The new taskbar on the other hand, always treats a drop as a pin command. Drop a program and the program is pinned. Drop a file and the file will be pinned under its respective program’s Jump List and that program automatically gets pinned to the taskbar. It was important for us to keep drag/drop consistent. We believe that for most cases people will open files through the desktop by just double-clicking them or from the Jump List and the default program will open. However, there are some scenarios when a customer wants to open a certain file type with another program. We heard this feedback and decided to revive “Open With” drag/drop on the taskbar with a keyboard modifier. One can hold down SHIFT and drop the file on the desired program.
5. Taskbar scaling
We’ve reclaimed lots of space on the taskbar by unifying launching/switching, by collapsing open windows and by cleaning the notification area. Still, some have asked for even more room to pin the programs they use regularly. We’ve made a change to squeeze in 24-39% more icons before the taskbar scrolls; depending upon your resolution, icon size and assuming the default notification area. Table 1 illustrates the new button capacity before the taskbar begins to scroll as well as the capacity growth since Beta. We believe customers will find more than enough room to pin their common programs.
Maximum taskbar button capacity before scrolling
% Increase from Beta (large/small icons)
25% / 36%
25% / 38%
25% / 32%
24% / 39%
6. Anchoring taskbar thumbnails
Hovering or clicking on a taskbar button surfaces all the running windows for that program. Upon seeing a set of open thumbnails, Kozlow asked “How do I know which application has opened the thumbnails group?” In other words, the thumbnails didn’t appear visually connected to the taskbar. We made a visual update that now keeps the color hot-track effect on when the mouse is over a thumbnail. In fig 2 you can see that IE retains its blue Color Hot-track visual even though the mouse is over a thumbnail.
Color Hot-track stays active when the mouse hovers over taskbar thumbnails
7. Newly installed programs
“Customer in control” is so strong a mantra for Windows 7 we don’t even allow programs to pin themselves to the taskbar when they are installed. This is a task expressly reserved for the customer. We’ve gotten some requests to make this goal a bit easier so now when a program is installed, it is automatically and temporarily surfaced at the bottom of the Start Menu. The customer can easily discover this new addition, launch it directly and optionally drag it to the taskbar for convenient access in the future.
8. Jump List length
Jump Lists are proving to be a valuable tool to quickly jump to commonly access files, folders, links and tasks. Steve filed a bug in which he said “The whole point of the jump list is to make it easier to jump to your favorite locations. However, it doesn't save me time having to scan through a long list of frequent locations.” In other words, sometimes it’s hard to parse an item when the list gets too long. Our telemetry data informs us that in most cases customers are clicking on the first 10 items. Therefore, we’ve updated Jump Lists so that only a maximum of 10 items may be automatically suggested (this doesn’t apply to tasks or pinned items). Don’t worry—there’s even a setting for enthusiasts to customize the length of the list.
9. Increased pinning flexibility with Jump List
For organizational, scaling and identification purposes, the taskbar is designed to hold files, folders and links in a program’s Jump List. Items can only be pinned to the Jump List of programs registered to handle that file type. Based on feedback we’ve received we now allow one to pin items to a Jump List belonging to a program that isn’t registered to handle that file type. Better yet, pinning the item in most cases will create a new registration so that launching it from the Jump List will always open the file with that specific program. For example, one can pin an .HTML file to Notepad’s Jump List and when clicked on from the menu, the file will always open in Notepad even though IE by default handles the file type.
10. Desktop icon and gadget view options
Windows 7 makes gadgets far easier to manage, view and access by building them directly into the desktop. David’s feedback matches what others were telling us: “In Vista, I was able to hide desktop icons while my gadgets were still visible and available. I liked this feature in Vista, especially with all the icons that are constantly dropped on the desktop by app installers. I don't want to see the icons, but I still want to see my gadgets.” In Beta it was impossible to separate desktop icons from gadgets under the View setting available by right-clicking on the desktop. We made a change to afford independent control to each so that one can opt to hide just her gadgets or just her desktop icons.
11. Aero Peek for touch
We’re excited about Peek and we further refined its functionality. Our touch customers enjoy the benefits of direct manipulation, but inform us they feel left out of some of new functionality that’s available for the mouse and keyboard. We’ve made two improvements that spreads the love. First, the taskbar’s thumbnails now support a touch gesture so one can drag her finger across the UI and trigger Aero Peek. Also, the Show Desktop button is improved so a press-and-hold will allow the customer to peek at the desktop. A regular tap in both these scenarios still to commits the switch.
12. Multi-touch touch keyboard
A funny thing happens when one uses touch to interact with a software keyboard for the first time. The natural instinct is to press multiple buttons simultaneously like they do with a real keyboard. It’s quite reasonable to try to use SHIFT + <letter> to capitalize, for example. RC ushers in multi-touch support for the Touch Keyboard so that customers enjoy a more realistic experience.
13. Multi-touch right-click
People who are rely on touch give us mixed feelings towards tap and hold to bring up a context menu. This approach works, but it also involves a slight delay. We now have a fast new multi-touch gesture for right-click. Simply touch an item with one finger and use another finger to tap and summon a context menu.
14. Drag/Drop and selection
In Beta there was no discoverable way to select text in a website that scrolled both horizontally and vertically. Customers are now able to drag/drop and select items with touch, even inside scrolling pages. The new behavior is optimized for the two most common actions by touch customers—scrolling up and down and dragging left to right.
15. Internet access feedback
The new network experience from the taskbar’s notification area makes it much easier to find and connect to networks. People seem to also really like the wireless signal strength that is available at a glance. In our effort to simplify the experience we removed indications for some advanced scenarios. Based upon feedback, we’ve decided to introduce a new overlay icon which now reveals when there is a local connection without internet access.
16. User Account Control
If you’ve been following this blog, then you already know about a recent design change we’ve made that will prompt for any modification made to the UAC Control Panel. For more information, please refer to the earlier post on UAC Feedback and Follow-Up.
17. Locking a machine without a screensaver
It isn’t uncommon for IT administrators to want their corporate machines to auto-lock after a certain amount of time. In Beta, enabling this functionality required a screensaver to be set. We’ve since made a change to allow this functionality even when no screensaver is specified.
18. Faster access to High Performance power plan
Clicking on the battery flout from the taskbar notification area offers two different power plans: Balanced and Power saver. Windows 7 laptops are configured by default to use the Balanced plan since this setting best balances a good experience while promoting more environmentally friendly power use. However, some customers tell us they want to be able to quickly toggle between Balanced and High Performance (yet another power plan). We’ve taken a change to now show the latter in the flyout menu when it is enabled under the Power Options Control Panel.
19. Custom theme improvements
We’ve always known customers love personalizing their Windows experience. At the center of this expression of individuality are ingredients such as the desktop background, glass color, sounds and screensavers. In Windows 7 we’ve introduced themes that make it easy to enable a whole package of default combinations or for customers to save their own creations. However, during Beta we heard feedback along the lines of “I just changed my background or color and I see the change, but I thought it was saved when it really wasn’t”. We added text under each theme to not only aid in identification, but also to provide feedback on the state of a theme. The new “Unsaved Theme” text also ties better to the nearby “Save theme” command. These tweaks should make personalization a more predictable and enjoyable experience.
Windows Media Player
20. Improved Internet Radio playback
Internet radio playback continues to gain in popularity. We received feedback that sometimes playback of radio streams may be inconsistent depending on network conditions. It’s worth noting that our understanding of this issue was greatly helped by the broad scale of usage across so many customers and network topologies and our telemetry in the Beta. Windows Media Player has made changes to make streaming playback more reliable and resilient.
21. Improved playback support for video content from digital camcorders and cameras
Customers loved the increased range of formats natively supported by the Windows 7 Beta, but noticed areas where they wanted broader support. For example, one was unable to seek to a specific spot in the video in Windows Media Player or Windows Media Center for AVCHD content that was imported from a digital camcorder. We’ve addressed this. Also, while the support for video from some digital cameras worked great, we also got feedback about supporting a broader set of devices out of the box. We’ve since added support for Windows Media Player to natively support the .MOV files used to capture video for many common digital cameras.
22. Cleaner Now Playing view
Customers are sharing positive reviews of Media Player’s new light-weight Now Playing view. Still some have asked to make the experience even cleaner. We’ve responded with a visual update that is more lightweight and compact.
23. Filtering content that cannot be played
Media Player’s library view is designed to surface and showcase one’s content. However, in some cases items were displayed that couldn’t be played. For example, Apple’s lossless .M4A or .H263 MPEG-4 content would be shown in a library even though Media Player could not play them. In RC, this content will no longer appear in the library view so that there is better expectation of what is supported by the player.
24. Resume from sleep
Customers are used to resuming a CD or DVD after an interruption. With customers choosing new low-cost, smaller form-factor, machines without optical drives, an increasingly popular scenario is to have content played directly from the hard drive. In Beta, it was not possible to resume playback on such content after a laptop goes to sleep. Customers assume the experience should match that of physical media so we fixed the experience to meet this expectation.
25. Quieting Windows Media Player sync relationships
When Media Player is open and a portable media player or a USB drive is inserted, we trigger a dialog to determine whether a sync relationship should be created with the new device. Our original goal was to be proactive and help customers make a decision in context, but we received comments that this experience is jarring. As a result, we will no longer interrupt when the player is running. This is consistent with our “customer in control” goal of Windows 7 and we trust people can manually configure this should they wish to.
26. Easier access to advanced settings
What enthusiast doesn’t want to tweak her player settings? This was echoed by several comments so we’ve made it easier to access and adjust settings. The equalizer, play speed, SRS WOW and other options are now surfaced via the Now Playing context menu under Enhancements.
27. Jump List improvement
Media Player’s Jump List provides quick access to the content customers consume. The list becomes even more powerful and complete in the RC now that we also include items launched from Explorer.
28. Enriching the Device Stage ecosystem
Customers have been so positive about the new Device Stage experience, one of the biggest pieces of feedback we got was “Why aren’t even more of my devices supported?” We’ve taken that feedback to heart and then took the feedback to our IHV and OEM partners to get their support for more devices. Our hardware partners in turn asked us to make it easier to integrate with the Device Stage and we worked with them on improvements. Although Windows already supports tens of thousands of devices, customer feedback on the Beta introduces even more device support in RC via the new Device Stage experience.
29. Improving the headphone experience
Customers informed us that sometimes their audio streams did not properly move from the default speakers to their headphones. The fix required an update to the algorithm we use to detect new devices. In RC the transition works more reliably.
30. Increased audio reliability
In some cases people reported not having any audio device after installing Beta. The problem is that some audio hardware does not work out of the box with our inbox audio class driver. Amazingly there are over 26,000 custom audio drivers and while many are on Windows Update, many are still not. The Release Candidate tightens the Windows Logo test to better ensure clean install delivers baseline functionality for speakers and microphones. Furthermore, we will continue to populate Windows Update with frequently needed drivers.
Windows Explorer and Libraries
31. Improved header
It is great to see customers realize the convenience and power of libraries. Having files aggregated into one convenient view, without worrying where they are all physically located, simplifies many scenarios. The library header in Beta showed only a static string that reflected how many locations were represented as part of the library. We heard feedback that this wasn’t very clear and more importantly, customers preferred to have more information so that they could be better orientated themselves. The RC will introduce a new header that updates to reveal the subfolder as one browses a library. Furthermore, the “Arrange by” views are better expose in the upper right, in proximity of the other view and search controls.
32. Reduced confusion with drag/drop
The Release Candidate will remove the ability to drag/drop a folder into the Libraries node in the Explorer navigation pane. We know some liked this functionality to create a new library, but it also presented some serious design issues. For example, some were surprised to find a new library was created when their intent was to simply copy the folder. More seriously though, there were circumstances where people then deleted the original folder thinking it was already copied. Data loss is a grave concern of ours and we don’t want customers to suffer from such a mistake. Don’t worry though—one can still easily create a new library using the “New Library” or “Include in Library” commands in the Explorer command bar.
33. Reviving familiar entry points
Mando writes, “In Win7 the Win+E shortcut opens an explorer window but the path is “Libraries” instead (which isn’t where I want to go most of the time). Is there a way to configure the target folder of “Win+E” or is there an alternate shortcut that will get me to the “Computer” path like it did in Vista?” RC reverts the behavior and now the shortcut will launch the “Computer” Explorer. Also, we changed the link in Start Menu -> Username to match the Vista behavior.
34. FAT32 support
Local FAT32 hard disk drives were not support in libraries for Beta. RC libraries will now support non-removable FAT32 and NTFS hard disk drives thanks to the feedback we received.
35. Arrangement view enhancements
It’s been great to see people’s reaction to the arrangement views in libraries. Being able to browse using metadata certainly makes quick work of finding files. We’ve received many requests to further enhance the arrangement views in a variety of ways and we’ve made a number of changes in response to them. For starters, RC makes it easier to switch arrangement views—one can now do so directly from the view context menu, which is the familiar home of switching the view mode, sorting, and grouping. Second, the specific arrangement views themselves have been enhanced for RC. The “Month” and “Day” views in the Pictures library now group together both the pictures and videos taken on the same date, whereas previously the videos were split out into a separate group. The “Artist” and “Genre” views in the Music library now show the thumbnails for up to three unique albums per artist or genre instead of typically just one in Beta. The Videos library now features a Length view that lets customers split out the shorter clips from longer movies in their video collection. Finally, we’ve made it so that changing the grouping of the Folder view in a library is now remembered just like other arrangement view customization. People who prefer to see their files grouped a particular way no longer have to reset the grouping each time.
36. Improving performance through data
Feedback comes to us in many different forms. Typically it consists of comments customers share. However, some of the most valuable information actually comes to us automatically when people just use Windows. PerfTrack, for example, is a telemetry system that provides us with invaluable real-world performance data on over 500 different Windows scenarios. The exciting aspect of PerfTrack is that it represents what people are really experiencing “out in the wild”. Performance is a very important to both the engineering team as well as to our customers and we strive to continuously improve this area. The topic has been discussed in several posts on this blog.
Let’s look at just one example of a Windows scenario that was improved with the help of PerfTrack. The two graphs below show the performance of opening the Start Menu for both Beta and for a more recent version of Windows 7. Some caveats first—the sample sizes are different (after all Beta did go to a far wider audience) and these numbers shouldn’t be taken too literally since they really do just represent a snapshot. The different colors denote performance against the “interaction class”—the acceptable experience range defined by each feature team. In this case we want the Start Menu to appear within 50ms to 100ms. A trace capturing tool running on each machine lets us investigate and fix what may be impacting performance. The charts shows in Beta 85% of interactions were within the acceptable range (i.e. green or yellow, but not red). After examining the traces and making some optimizations, we find 92% of interactions are this range for a more recent build.
Start Menu Open Times for Windows 7 Build 7000 (Beta)
Start Menu Open Times for Windows 7 Build 7033
As is evident from this sample of changes, we’ve been very busy improving Windows 7 based upon what our customers are telling us in many forums.
- Chaitanya Sareen
As we connect through this blog and through all of those talking about Windows 7 it is clear that folks have a lot of passion around many topics. We learned early on about the passion around the boot/startup sequence and how important it was for that to go by quickly! At the same time, we know that it is really dull to watch a HDD light blink when resuming a machine from hibernate or powering up a machine. To improve this first connection with people, we set out to improve the boot sequence—jazz it up if you will. It sounds pretty easy, but as we looked into solving this we found it is a pretty significant engineering challenge. Our goal was to have fun while having no impact on the boot performance of the system. To explain this engineering and describe the boot sequence, Karen Wong, a program manager on our Core User Experience feature team, authored this post. --Steven
We use the word “personality” to refer to some of the characteristics of software that connect emotionally with people. ‘Light’ and ‘energy’ are some of the terms we use to describe the personality of Windows 7. As we designed Windows 7 it became clear that in order to showcase these elements of Win7’s personality, we needed go above and beyond what we did with Vista’s boot visuals.
From a design perspective, we know that the visual presentation of a feature plays a key role in the user’s perception of performance and quality. Our objective was to make Windows boot beautiful and was inspired by our Windows 7 personality of light and energy; and the way these forms reveal themselves in nature became our design palette. Words such as “bioluminescence”, “organic”, “humble beauty”, and “atmosphere” came up frequently in our brainstorming sessions. We know that in isolation these might sound a bit corny, but this is all part of the overall goals of Windows 7.
Over two dozen boot sequence designs were created, reviewed, and user tested to evaluate them against our goals. Designs varied in the saturation and/or brightness of color, the complexity of motion, and lighting effects. Here are some sketches from our design journey:
The final design in Windows 7 shows energy approaching from four directions, that join to form a light that projects through a window (of course it is no coincidence that the Windows logo resembles a window!). A subtle pulse indicates progress thereafter; another design detail that reinforces the liveliness of Windows 7’s personality.
From a design perspective, this new boot sequence met all of our design goals, and we were excited to send it out into the world. However, boot had to be more than just a pretty face. From an engineering perspective, we had some clear challenges to overcome, as we knew that “time to desktop” was still the most important thing to users. Visual delight could not trump getting to the desktop faster and many of you have been critical of features that are dubbed “eye candy” – the boot sequence is not going to be one of those features for sure.
If we had kept everything the same from Vista and simply updated the boot animation to the new Win7 look, we would not have achieved new levels performance and quality that we aspire to. In fact, significant code changes were required in order to make the new boot animation even possible in Win7.
In Vista, the boot loader is using a low resolution 640x480 screen, and file size required for the green animated progress bar is very small. Furthermore, the Vista boot screen had low color depth – 16 bits per pixel (bpp). We increased this in Win7 to 32 bpp, which enabled the color richness you see in the new boot animation. Updates to the Vista boot progress indicator were achieved via the CPU, which was susceptible to I/O time, therefore causing occasional glitches in the animation. With the low resolution screen, limited color depth, and susceptibility to glitches – we knew we had our work cut out for us if we wanted to build something fancier for Win7.
We started with the Win7 boot loader using a different mechanism to display the boot animation. It gets a pointer to the frame buffer from the firmware (either BIOS or UEFI firmware), and displays a higher resolution image (1024 x 768). It animates the image while the kernel and boot critical device drivers are loaded into memory. Since the native graphics driver for the display is not loaded into memory and initialized yet, the animation is run by using the CPU, and by updating the frame buffer for the graphics display. We made an additional optimization - to have the CPU use write-combined caching to accelerate performance.
Michael Fortin’s blog entry on boot performance describes how the early stage of boot is I/O bound, as it is loading the kernel, device driver files, and other system component files. We therefore limited the dimensions of the boot animation to a small region of the screen, to avoid introducing any delay during the early stage of boot. A larger animation area would require loading a larger set of animation images, which adds to the file I/O. The animation images are compressed by incorporating the bitmaps as resources, which are then compressed using WIM image compression. WIM image compression reduces the overall file size, thereby reducing the I/O required to read them in. It also reduces the on-disk footprint. Animating a smaller region of the screen, and using a slightly lower frame rate also keeps the CPU overhead of updating the frame buffer to a low enough level, that there is no added overhead to the boot time.
Another change we made that improved not only the performance of boot, but the quality, was the reduction of transitions in graphics mode. These transitions occur during initialization of the graphics subsystem and Windows shell. In Vista, these cause the boot experience to be less smooth, as the display changes (flashes black) a few times before presenting the user with a logon screen (or the user’s desktop if there is only one system user).
After looking deep into our boot architecture for performance and quality improvements to enable the new animation, we were pleasantly surprised that the act of beautifying the boot animation created a new opportunity to further decrease time to desktop. In Vista, when a customer powered on the machine, the boot sequence included an animation of the Windows flag, or ‘pearl’, before reaching the login screen (or the desktop if the user is set to auto-login). Due to the Vista boot architecture constraints, this pearl animation can only play after boot code has already completed.
Vista Boot Sequence, with Pearl Animation
Now that new boot visuals display a rich animation that reflects the Windows 7 personality, the pearl animation seemed out-of-date and redundant, and was removed. As a result, we saved the time it takes to play this animation after boot is complete.
Windows 7 Boot Sequence, Pearl Animation Removed
You may also be wondering what happened to the startup sound. In Vista, the sound had to be synchronized with the pearl animation to produce the highest quality experience. This has potential performance impact on some hardware, as we require the system’s sound stack to be loaded to complete the pearl sequence. In the cases when we are waiting for the system’s sound playback to be ready, a delay can occur in getting to the desktop. As such, we changed the sound to now play asynchronously, anytime after the logon screen loads. On most hardware that we tested, this is right when the logon screen displays. We heard customer feedback in Vista that the sound played and caught your attention, but boot was not yet complete. So in addition to performance benefits, this change also improves the user experience by letting users know when their machine is ready for use.
The sum of the boot code optimizations and removal of the pearl animation from Vista enabled us to add a rich, high-quality animation during boot, with no increase in the time it takes a user to reach the desktop.
The boot experience varies depending on the user’s hardware. We made some design decisions to ensure the best visual experience across a wide range of hardware, however the time it takes a system to get to the desktop is mainly hardware-dependent.
For example, you may notice that there is a delay before the animation starts during boot, and this delay time varies depending on system hardware. To optimize for showing immediate feedback, we actually display text on the boot screen before Windows has had a chance to start all the processors on the system. It is only when that is complete that the animation can run asynchronously to the rest of the I/O during boot (which as discussed earlier is necessary for optimal performance and quality).
You may also notice that the Windows flag’s dimensions during boot may change slightly on different screen sizes. Due to technical constraints in Win7, boot is always displayed in our recommended minimum resolution – 1024x768, regardless of the system’s native resolution. Today, most hardware is set to stretch the boot sequence to fill the screen, as opposed to centering it. Consequently, the boot animation is usually stretched on screens that are of different aspect ratio than 1024x768; however, we did test the sequence on common aspect ratios to ensure that visual quality was preserved.
With all this hard work to improve the boot experience, we couldn’t let it go to waste. As such, users will also have this experience when they resume from hibernate.
We know many of you might be asking if you could include your own animation or customize this sequence. This is not something we will support in Windows 7. We’ve talked about and shown a great many “personalization” elements of Windows 7 already, such as the new themepacks which you can try out in the beta. The reasons for this should be pretty clear, which is that we cannot guarantee the security of the system to allow for arbitrary elements to be loaded into memory at boot time. In the early stages of starting Windows, the system needs to be locked down and execute along a very carefully monitored and known state as tools such as firewalls and anti-virus checking are not yet available to secure the system. And of course, even though we’re sure everyone would follow the requirements around image size, content, etc. due to performance we would not want to build in all the code necessary to guarantee that all third parties would be doing so. One of our design goals of Windows 7 was around making sure there are ample opportunities to express yourself and to make sure your PC is really your PC and so we hope that you’ll understand why this element is one we need to maintain consistently.
This was a quick behind the scenes look at something that we hope you enjoy. With Windows 7 we set out to make the experience of starting a Windows PC a little more enjoyable, and from the feedback we’ve seen here and in other forums, we think we’re heading in the right direction. In addition to our efforts to make boot fast, we also have a goal to make the system robust enough, such that most of you will not see this new boot animation that often and when you do it will be both enjoyable and fast!
Just about every email we receive and every comment we get comes with feedback—something to change, something to do more of, something to do less of, and so on. As we’ve talked about in this blog, acting on each one in an affirmative manner is easier said than done. What we can say for certain, is that we are listening to each and every comment, blog post, news story, MS Connect report, Send Feedback item, and of course all the data and telemetry. This post kicks off the discussion of changes made to the product with an overview of the feedback process. We'll get into specific changes shortly and we'll continue to return to the theme of changes in the Release Candidate (RC) over the next weeks. Yesterday on the IE Blog, you saw that we'll be updating IE 8 on Windows 7, and there we also talked about the feedback process in general.
Feedback about Windows 7 of course starts before we've written any code, and by the time we've got running code thousands of people outside of Microsoft have provided input and influenced the feature set and design of Windows 7. As we've seen, the input from even a small set of customers can often represent a wide variety of choices--often in alignment, but just as often in opposition. As we're developing the features for Windows 7 we work closely with PC makers, enterprise customers, and all types of customers across small business, education, enthusiasts, product reviewers and industry "thought leaders", and so on. We shape the overall "blueprint" of the release based on this wide variety of input. As we have design prototypes or code running, we have much more targeted and specific feedback by using tools such as usability tests, concept tests, benchmark studies, and other techniques to validate the implementation of this blueprint. Our goal with this level of feedback is for it to be representative of the broad set of Windows customers, even if we don't have a 1:1 interaction with each and every customer. Hopefully this post will offer some insights into this process overall--the tools and techniques, and the scope of feedback.
In the first few weeks of the Windows 7 beta we had over one million people install and use Windows 7. That's an astounding number for any beta test and while we know it has been fun for many folks, it has been a lot of work for us--but work that helps to raise the quality of Windows 7. When you use the beta you are automatically enrolled in our Customer Experience Improvement Program (anonymous feedback and telemetry, which is voluntary and opt-in in the RTM release). Just by using Windows 7 as a beta tester you are helping to improve the product--you are providing feedback that we are acting on in a systematic manner. Here is a sense of the scale of feedback we are talking about:
We have a variety of tools we draw on to help inform the decision making process. A key element that we have focused on quite a bit in Windows 7 is the role of data in making decisions. Everything we do is a judgment call as ultimately product development is about deciding what to get done from an infinite set of possibilities, but the role of data is essential and is something that has become far more routine and critical. It is important to be super clear—data is not a substitute for good judgment or an excuse to make a decision one way or another, but it most definitely informs the decision. This is especially true in an era where the data is not only a survey or focus group, but often includes a “sampling” of millions of people using Windows over the course of an extended time period.
A quick story from years ago working on Office, many years ago before the development of telemetry and the internet deciding what features to put in a release of Office could really be best described as a battle. The battle took place in conference rooms where people would basically debate until one or more parties gave up from fatigue (mental or otherwise)—essentially adrenaline-based product development. The last person standing, the one with the most endurance, or the one who pulled an all-nighter to write the code pretty much determined how features ended up or what features ended up in a product. Sort of like turning feature design over to a Survivor-like process. I’m sure many of you are familiar with this sort of process. The challenges with this approach are numerous, but inevitably features do not hold together well (in terms of scenarios or architecture), the product lacks coherency, and most importantly unless you happen to have a good match between the “winner” and the target customers, features will often miss the mark.
In the early 1990’s we started instrumenting Word and learning about how people actually used the software (this was before the internet so this was a special version of the product we solicited volunteers to run and then we would collect the data via lots of floppies). We would compile data and learn about which features people used and how much people used them. We learned things such as how much more people used tables than we thought, but for things very different than tables. We learned that a very significant amount of time the first suggestion in the spelling dictionary was the right correction (hence autocorrect). We learned that no one ever read the tip of the day (“Don’t run with scissors”). This data enabled us to make real decisions about what to fix, the impact of changes, and then when looked at the goals (the resulting documents) what direction to take word processing.
Fast forward to the development of Windows 7 and we’re focused on using data to help inform decisions we make. This data takes many forms and helps in many ways. I know a lot of folks have questions about the data – is it representative, how does it help fix things people should be using but don’t, what about doing new things, and so on. Data is an important element of making decisions, but not a substitute for clear product goals, meaningful customer engagement, and working across the ecosystem to bring Windows 7 to customers.
Let’s talk a bit about “bugs”. Up front it is worth making sure we’re on the same page when we use the much overloaded term bug. For us a bug is any time the software does something that someone one wasn’t expecting it to do. A bug can be a cosmetic issue, a consistency issue, a crash, a hang, a failure to succeed, a confusing user experience, a compatibility issue, a missing feature, or any one of dozens of different ways that the software can behave in a way that isn’t expected. A bug for us is not an emotional term, but just shorthand for an entry in our database representing feedback on the product. Bugs can be reported by a human or by the various forms of telemetry built into Windows 7. This broad definition allows us to track and catalog everything experienced in the product and do so in a uniform manner.
Briefly, it is worth considering a few types of data that help to inform decisions as some examples.
This type of feedback all represents structured feedback in that the data is collected based on a systematic study and usually has a hypothesis associated with it. We also have the unstructured feedback which represents the vast array of bug reports, comments, questions, and points of view expressed in blogs, newsgroups, and the Send Feedback button—these are unstructured because these are not collected in a systematic manner, but aggressively collected by any and all means. A special form of this input is the bug reporting done through the Connect program—the technical beta—which represents bug reports, feature suggestions, and comments from this set of participants.
The Windows 7 beta represents a new level of feedback in this regard in terms of the overall volume as we talked about above. If you go back and consider the size of the development team and the time it would take to just read the reports you can imagine just digesting (categorizing, understanding, flagging) issues let alone responding to them is a massive undertaking (about 40 Send Feedback reports per developer during that one week, though as you can imagine they are not evenly distributed across teams).
The challenge of how to incorporate all the feedback at this stage in the cycle is significant. It is emotional for us at Microsoft and the source of both considerable pride and also some consternation. We often say “no matter what happens, someone always said it would.” By that we mean, on any given issue you can be assured that all sides will be represented by passionate and informed views of how to resolve it, often in direct opposition to each other plus every view in the middle. That means for the vast majority of issues there is no right or wrong in an absolute sense, only a good decision within the context of a given situation. We see this quite a bit in the debates about how features should work—multiple solutions proposed and debate takes place in comments on a blog (people even do whole blogs about how things should work). But ultimately on the Windows development team we have to make a call as we’re seeing a lot of people are looking forward to us finishing Windows 7, which means we need to stop changing the product and ship it. We might not always make the right call and we’ll admit if we don’t make the right call, even if we find changing the behavior is not possible.
Making these decisions is the job of program management (PM). PMs don’t lock themselves in their offices and issue opinions, but more realistically they gather all the facts, data, points of view, and work to synthesize the best approach for a given situation. Program management’s role is making sure all the voices are heard, including beta testers, development, testing, sales, marketing, design, customer support, other teams, ISVs, IHVs, and on and on. Their role is to synthesize and represent these points of view systematically.
There are many factors that go into understanding a given choice:
These are just a few of the factors that go into considering a product change. As you can see, this is not something that we take lightly and a lot goes into each and every change. We consider all the inputs we have and consider all the data we can gather. In some ways it is easy to freeze thinking about the decisions we must make to release Windows 7—if you think too hard about a decision because you might start to worry about a billion people relying on something and it gets very tricky. So we use data to keep ourselves objective and to keep the decision process informed and repeatable. We are always humbled by the responsibility we have.
While writing this post, I received a “bug report” email with the explicit statement “is Microsoft going to side step this issue despite the magnitude of the problem” along with the inevitable “Microsoft never listens to feedback”. Receiving mail like this is tough—we’re in the doghouse before we even start. The sender has decided that this report is symbolic of Microsoft’s inability or lack of desire to incorporate critical feedback and to fix must fix bugs during development. Microsoft is too focused on shipping to do the right thing. I feel like I’m stuck because the only answer being looked for is the fix and anything less is a problem or further proof of our failure. And in the back of my mind is the reality that this is just one person with one issue I just happen to be talking to in email. There over a couple of million people using the beta and if each one, or for that matter just one out of 10, have some unique change, bug fix, or must do work item we would have literally years of work just to make our way through that list. And if you think about the numbers and consider that we might easily get 1,000,000 submitted new “work items” for a product cycle, even if we do 100,000 of them it means we have 900,000 folks who feel we don’t listen compared to the 100,000 folks who feel listened to. Perhaps that puts the challenge in context.
With this post we tried to look at some of the ways we think about the feedback we’re getting and how we evaluate feedback in the course of developing Windows 7. No area is more complex than balancing the needs (and desires) of such a large and diverse population—end-users, developers, IT professionals, hardware makers, PC manufacturers, silicon partners, software vendors, PC enthusiasts, sysadmins, and so on. A key reason we augment our approach with data and studies that deliberately select for representative groups of “users” is that it is important to avoid “tyranny of the majority” or “rule by the crowd”. In a sense, the lesson we learned from adrenaline -based development was that being systematic, representative, and as scientific as possible in the use of data.
The work of acting on feedback responsibly and managing the development of Windows through all phases of the process is something we are very sincere about. Internally we’ve talked a lot about being a learning organization and how we’re always learning how to do a better job, improve the work we do, and in the process work to make Windows even better. We take this approach as individuals and how we view building Windows. We know we will continue to have tough choices to make as everyone who builds products understands and what you have is our commitment to continue to use all the tools available to make sure we are building the best Windows 7 we can build.
When we started the “E7” blog we were both excited and also a bit uneasy. The excitement is obvious. The unease is because at some point we knew we would mess up. We weren’t sure if we would mess up because we were blogging about a poorly designed feature or mess up because we were blogging poorly about a well-designed feature. To some it appears as though with the topic of UAC we’ve managed to do both. Our dialog is at that point where many do not feel listened to and also many feel various viewpoints are not well-informed. That’s not the dialog we set out to have and we’re going to do our best to improve.
This post is an attempt to get both the blog right and the feature right. We don’t like where we are in terms of how folks are feeling and we don’t feel good – Windows 7 is too much fun and folks are having too much fun for us to be having the dialog we’re having. We hope this post allows us to get back to having fun!
To start we’ll just show representative comments from the spectrum of feedback. We’ll then talk about the changes we’re making and also make sure we’re all on the same page regarding how we move forward. In terms of comments we’ve heard the following:
You have 95% of the people out there think you got it wrong, even if they are the ones that got it wrong. The problem is that they are the one's that buy and recommend your product. So do you give them a false sense of increased security by implementing the change (not unlike security by obscurity) and making them happy, or do you just fortify the real security boundaries?
You have 95% of the people out there think you got it wrong, even if they are the ones that got it wrong. The problem is that they are the one's that buy and recommend your product. So do you give them a false sense of increased security by implementing the change (not unlike security by obscurity) and making them happy, or do you just fortify the real security boundaries?
And @Thack says:
Jon, Thanks for sharing your thoughts. I understand your points. Now, I want add my voice to the call for one very simple change: Treat the UAC prompting level as a special case, such that ANY change to it, whether from the user or a program, generates a UAC prompt, regardless of the type of account the user has, and regardless of the current prompting level. That is all we are asking. No other changes. Leave the default level as it is, and keep UAC as it is. We're just talking about the very specific case of CHANGES to the UAC prompting level. It will NOT be a big nuisance - most people only ever change the UAC level once (if at all). Despite your assurances, I REALLY WANT TO KNOW if anything tries to alter the UAC prompting level. The fact that nobody has yet demonstrated how the putative malware can get into your machine is NO argument. Somebody WILL get past those other boundaries eventually. Even if you aren't convinced by my argument, then the PR argument must be a no-brainer for Microsoft. PLEASE, Jon, it's just a small change that will gain a LOT of user confidence and a LOT of good PR. Thack
Thanks for sharing your thoughts. I understand your points.
Now, I want add my voice to the call for one very simple change:
Treat the UAC prompting level as a special case, such that ANY change to it, whether from the user or a program, generates a UAC prompt, regardless of the type of account the user has, and regardless of the current prompting level.
That is all we are asking. No other changes. Leave the default level as it is, and keep UAC as it is. We're just talking about the very specific case of CHANGES to the UAC prompting level.
It will NOT be a big nuisance - most people only ever change the UAC level once (if at all).
Despite your assurances, I REALLY WANT TO KNOW if anything tries to alter the UAC prompting level.
The fact that nobody has yet demonstrated how the putative malware can get into your machine is NO argument. Somebody WILL get past those other boundaries eventually.
Even if you aren't convinced by my argument, then the PR argument must be a no-brainer for Microsoft.
PLEASE, Jon, it's just a small change that will gain a LOT of user confidence and a LOT of good PR.
With this feedback and a lot more we are going to deliver two changes to the Release Candidate that we’ll all see. First, the UAC control panel will run in a high integrity process, which requires elevation. That was already in the works before this discussion and doing this prevents all the mechanics around SendKeys and the like from working. Second, changing the level of the UAC will also prompt for confirmation.
Sometimes, inconsistency with your own ideals is a good thing. Make an exception, if only to put people's fears to rest.
Sometimes, inconsistency with your own ideals is a good thing. Make an exception, if only to put people's fears to rest.
That sums up where we are heading. The first change was a bug fix and we actually have a couple of others similar to that—this is a beta still, even if many of us are running it full time. The second change is due directly to the feedback we’re seeing. This “inconsistency” in the model is exactly the path we’re taking. The way we‘re going to think about this that the UAC setting is something like a password, and to change your password you need to enter your old password.
The feedback is that UAC is special, because it can be used to disable silently future warnings if that change is not elevated and so to change the UAC setting an elevation will be required. To the points in the comments, we also don’t want to create a sense or expectation of security that is not there—you should still not download code and run it unless you trust the source. HTML, EXE, VBS, BAT, CMD and more are all code and all have the potential to alter the environment (user settings, user files) running as a standard user or an administrator. We’re focused on helping people make sure that code doesn’t get on the machine without consent and many third party tools can help more as well. We want people to be comfortable with the new UAC control and the new default setting, so we’ll make the changes outlined above as the feedback has been clear.
While we’re discussing this we want to make sure we’re all on the same page going forward in terms of how we will evaluate the security of Windows 7. Aside from the UAC setting, the discussion of the vulnerability aspects of the Windows 7 Beta have each started with getting code on the machine, which the mechanisms of Windows have prevented in the cases shown. We have also heard of security concerns that involve multiple steps to demonstrate a potential exploit. It is important to look at the first step—if the first step is “first get code running on the machine” then nothing after that is material, whether it is changing settings or anything else. We will treat very seriously the ability to get code on a machine and run without consent. As Jon’s post highlighted briefly, the work in Windows 7 is about the increased protections in place to secure your PC from acquiring and running code without your consent, and of course we continue to make sure Windows code is secure from both tampering or circumventing the protections in the system.
We want to reiterate the security of the system overall. Windows 7 is SD3+C and is designed to be more secure that Vista—that’s our priority. None of us want to have Windows 7 be perceived as being less secure than Vista in any way, because our design point is to make sure it is more secure that Windows Vista, by default.
We said we thought we were bound to make a mistake in the process of designing and blogging about Windows 7. We want to continue the dialog and hopefully everyone recognizes that engineering, perhaps especially engineering Windows 7, is sometimes going to be a lively discussion with a broad spectrum of viewpoints expressed. We don’t want the discussion to stop being so lively or the viewpoints to stop being expressed, but we do want the chance to learn and to be honest about what we learned and hope for the same in return. This blog has almost been like building an extra product for us, and we’re having a fantastic experience. Let’s all get back to work and to the dialog about Engineering Windows 7. And of course most importantly, we will continue to hear all points of view and share our point of view and work together to deliver a Windows 7 product that we can all feel good about.
--Jon and Steven
Hi, Jon DeVaan here to talk to you about the recent UAC feedback we’ve been receiving.
Most of our work finishing Windows 7 is focused on responding to feedback. The UAC feedback is interesting on a few dimensions of engineering decision making process. I thought that exploring those dimensions would make for an interesting e7 blog entry. This is our third discussion about UAC and for those interested in the evolution of the feature in Windows it is worth seeing the two previous posts (post #1 and post #2) and also reading the comments from many of you.
We are flattered by the response to the Windows 7 beta so far and working hard at further refining the product based on feedback and telemetry as we work towards the Release Candidate. For all of us working on Windows it is humbling to know that our work affects so many people around the world. The recent feedback is showing us just how much passion people have for Windows! Again we are humbled and excited to be a part of an amazing community of people working to bring the value of computing to a billion people around the world. Thank you very much for all of the thoughts and comments you have contributed so far.
UAC is one of those features that has a broad spectrum of viewpoints with advocates staking out both “ends” of the spectrum as well as all points in between, and often doing so rather stridently. In this case we might represent the ends of the spectrum as “security” on one end and “usability” on the other. Of course, this is not in reality a bi-polar issue. There is a spectrum of perfectly viable design points in between. Security experts around the world have lived with this basic tension forever, and there have certainly been systems designed to be so secure that they are secure from the people who are supposed to benefit from them. A personal example I have, is that my bank recently changed the security regimen on its online banking site. It is so convoluted I am switching banks. Seriously!
As people have commented on our current UAC design (and people have commented on those comments) it is clear that there is conflation of a few things, and a set of misperceptions that need to be cleared up before we talk about the engineering decisions made on UAC. These engineering decisions have been made while we carry forth our secure development lifecycle principles pioneered in Windows XP SP2, and most importantly the principle of “secure by default” as part of SD3+C. Windows 7 upholds those principles and does so with a renewed focus on making sure everyone feels they are in control of their PC experience as we have talked about in many posts.
The first issue to untangle is about the difference between malware making it onto a PC and being run, versus what it can do once it is running. There has been no report of a way for malware to make it onto a PC without consent. All of the feedback so far concerns the behavior of UAC once malware has found its way onto the PC and is running. Microsoft’s position that the reports about UAC do not constitute a vulnerability is because the reports have not shown a way for malware to get onto the machine in the first place without express consent. Some people have taken the, “it’s not a vulnerability” position to mean we aren’t taking the other parts of the issue seriously. Please know we take all of the feedback we receive seriously.
The word “vulnerability” has a very specific meaning in the security area. Microsoft has one of the leading security agencies in the world in the Microsoft Security Response Center (firstname.lastname@example.org) which monitors the greater ecosystem for security threats and manages the response to any threat or vulnerability related to Microsoft products. By any definition that is generally accepted across the world wide security community, the recent feedback does not represent a vulnerability since it does not allow the malicious software to reach the computer in the first place.
It is worth pointing out the defenses that exist in Windows Vista that keep malware from getting on the PC in the first place. In using Internet Explorer (other browsers have similar security steps as well) when attempting to browse to a .vbs file or .exe file, for example, the person will see the prompts below:
Internet Explorer 8 has also introduced many new features to thwart malware distribution (see http://blogs.msdn.com/ie/archive/2008/08/29/trustworthy-browsing-with-ie8-summary.aspx ). One of my favorites is the SmartScreen® Filter which helps people understand when they are about to visit a malicious site. There are other features visible and hidden that make getting malware onto a PC much more difficult.
A SmartScreen® display from IE 8
Additionally, if one attempts to open an attachment in a modern email program (such as Windows Live Mail) the malware file is blocked:
Much of the recent feedback has failed to take into account the ways that Windows 7 is better than Windows Vista at preventing malware from reaching the PC in the first place. In Windows 7 we have continued to focus on improving the ability to stop malware before it is installed or running on a PC.
The second issue to untangle is about the difference in behavior between different UAC settings. In Windows 7, we have four settings for the UAC feature: “Never Notify,” “Notify me only when programs try to make changes to my computer (without desktop dimming),” “Notify me only when programs try to make changes to my computer (with desktop dimming),” and “Always Notify.” In Windows Vista there were only two choices, the equivalent of “Never Notify” and “Always Notify.” The Vista UI made it difficult for people to choose “Never Notify” and thus choosing between extremes in the implementation. Windows 7 offers you more choice and control over this feature, which is particularly interesting to many of you based on the feedback we have received.
The recent feedback on UAC is about the behavior of the “Notify me only when programs try to make changes to my computer” settings. The feedback has been clear it is not related to UAC set to “Always Notify.” So if anyone says something like, “UAC is broken,” it is easy to see they are mischaracterizing the feedback.
The Purpose of UAC
We are listening to the feedback on how “Notify me only when…” works in Windows 7. It is important to bring in some additional context when explaining our design choice. We choose our default settings to serve a broad range of customers, based on the feedback we have received about improving UAC as a whole. We have learned from our customers participating in the Customer Experience Improvement Program, Windows Feedback Panel, user surveys, user in field testing, and in house usability testing that the benefit of the information provided by the UAC consent dialog decreases substantially as the number of notifications increases. So for the general population, we know we have to present only key information to avoid the reflex to “answer yes”.
One important thing to know is that UAC is not a security boundary. UAC helps people be more secure, but it is not a cure all. UAC helps most by being the prompt before software is installed. This part of UAC is in full force when the “Notify me only when…” setting is used. UAC also prompts for other system wide changes that require administrator privileges which, considered in the abstract, would seem to be an effective counter-measure to malware after it is running, but the practical experience is that its effect is limited. For example, clever malware will avoid operations that require elevation. There are other human behavior factors which were discussed in our earlier blog posts (post #1 and post #2).
UAC also helps software developers improve their programs to run without requiring administrator privileges. The most effective way to secure a system against malware is to run with standard user privileges. As more software works well without administrator privileges, more people will run as standard user. We expect that anyone responsible for a set of Windows 7 machines (such as IT Administrators or the family helpdesk worker (like me!)) will administer them to use standard user accounts. The recent feedback has noted explicitly that running as standard user works well. Administrators also have Group Policy at their disposal to enforce the UAC setting to “Always Notify” if they choose to manage their machines with administrator accounts instead of standard user accounts.
Recapping the discussion so far, we know that the recent feedback does not represent a security vulnerability because malicious software would already need to be running on the system. We know that Windows 7 and IE8 together provide improved protection for users to prevent malware from making it onto their machines. We know that the feedback does not apply to the “Always Notify” setting of UAC; and we know that UAC is not 100% effective at stopping malware once it is running. One might ask, why does the “Notify me only when…” setting exist, and why is it the default?
The creation of the “Notify me only when…” setting and our choice of it as the default is a design choice along the spectrum inherent in security design as mentioned above. Before we started Windows 7 we certainly had a lot of feedback about how the Vista UAC feature displayed too many prompts. The new UAC setting is designed to be responsive to this feedback. A lot of the recent feedback has been of the form of, “I’ll set it to ‘Always Notify,’ but ‘regular people’ also need to be more secure.” I am sure security conscious people feel that way, and I am glad that Windows 7 has the setting that works great for their needs. But what do these so called “regular people” want? How to choose the default, while honoring our secure design principles, for these people is a very interesting question.
In making our choice for the default setting for the Windows 7 beta we monitored the behavior of two groups of regular people running the M3 build. Half were set to “Notify me only when…” and half to “Always Notify.” We analyzed the results and attitudes of these people to inform our choice. This study, along with our data from the Customer Experience Improvement Program, Windows Feedback Panel, user surveys, and in house usability testing, informed our choice for the beta, and informed the way we want to use telemetry from the beta to validate our final choice for the setting.
A key metric that came out of the study was the threshold of two prompts during a session. (A session is the time from power up to power down, or a day, whichever is shorter.) If people see more than two prompts in a session they feel that the prompts are irritating and interfering with their use of the computer. In comparing the two groups we found that the group with the “Always Notify” setting was nearly four times as likely to have sessions with more than two prompts (a 1 in 6.7 chance vs a 1 in 24 chance). We gathered the statistic for how many people in the sample had malware make it onto their machine (as measured by defender cleaning) and found there was no meaningful difference in malware infestation rates between the two groups. We will continue to collect data during the beta to see if these results hold true in a much broader study.
We are very happy with the positive feedback we have received about UAC from beta testers and individual users overall. This helps us validate our “regular people” focus in terms of the trade-offs we continue to consider in this design choice. We will continue to monitor the feedback and our telemetry data to continue to improve our design choices on UAC.
So as you can see there is a lot of depth to the discussion of UAC and the improvements made in Windows 7 in UAC itself and in improving ways to prevent malware from ever reaching a PC. We are working hard to be responsive to the feedback we received from Vista to provide the right usability and security for people of all types. We believe we’ve made good progress and are listening carefully to the feedback on our UAC changes. Again please accept our most sincere thanks for the passion and feedback on Windows 7. While we cannot implement features the way each and every one of you might wish, we are listening and making a sincere effort to properly weigh all points of view. Our goal is to create a useful, useable, and secure Windows for all types of people.
Even with the pictures and videos so commonplace on PCs, many of us spend most of our time looking at and interacting with text. Yet few of us stop to think about the depth of technology required to render text well and that this is an area that continues to benefit from improved technology in displays, graphics cards, as well as the APIs available to developers. In Windows 7, The support for text and fonts in GDI continues to provide the foundation for compatibility and application support. Building on the foundation of the modern DirectX graphics infrastructure, Windows 7 enhances the text output available to developers with DirectWrite. This is a new API subsystem and one that over time you will see adopted more broadly by applications from Microsoft, independent software developers, and within Windows itself. This post will also talk about improvements to ClearType and the Fonts, both available as part of the improvements to the GDI-based text APIs. This work was introduced at the PDC (pointers towards the end of the post). This post is by Worachai Chaoweeraprasit, a development lead on our Graphics feature team. --Steven
One of the high-level goals of Windows 7 is to have even better graphics – graphics with higher fidelity. To that end, my team is looking into how to improve one of the most basic graphic elements in Windows, and that is text – the thing that’s always right in your face, but we hope you’ll never actually see it.
About 80% of the time people spend with their PC is to either read or write. This should come as no surprise when you realize that text is essentially how the machine talks back to you, and until we have a technology that would allow it to interject thought directly into our brains, text would probably continue to be the way we receive information from the computer screen.
Studies have shown that good text leads to better productivity. Essentially we are wired as human to be incredibly good at capturing words and making a smooth, rapid transition between them – the basis of reading. We’re so good at it that we can do it unconsciously with incredible speed given that the text is optimized for that process. This might explain why many can sink in to a good book for hours, but some quickly become tired after staring at the computer screen for a while. Any visual-related factor that could disrupt the reading process effectively slows us down. Good text, therefore, is text that is tuned to support the human reading process with minimal distraction possible.
The evenness of the white surrounding each letter, word, line, and paragraph plays a huge role in keeping the pace of reading while the black elements holds our attention together. A line too long, a word too tight, a paragraph too uneven, any of these conditions take us farther and farther away from the message being delivered but closer and closer to the mere medium delivering it. The art of text is essentially to make the actual text itself disappears before your eyes, so that the ideas it delivers reappear in your head. The study of how to prepare proper text is known as typography. And, as a typographer would say: good typography is not to be seen; only the bad ones are. As a platform, the role of Windows is to deliver great presentation of text and offering software developers great tools for creating the best presentation possible in the context of the software they develop.
People tend to develop habits and often over time these become the preferred way of getting things done. The more mundane the activity is, the easier we become attach to it, and the harder we’re willing to change. When it comes to text on your screen, the same screen you look at days in and days out. It could quickly become awkward if that completely changes overnight – even for the better. So, how do we go about improving on what we all become so used to? We want to make sure to support what is there and improve it, while supporting existing methods. But, before we get to understand the improvement, let’s first take a closer look into the current implementation really is and what challenges it presents over the years.
The current implementation is the product of text rendering design based on device pixel. The dimension of text at a certain size eventually translates into a fixed number of pixels in horizontal and vertical direction on the device surface. A 10-point text would translate to roughly 80 pixels height on a typical printer device of 600 dpi, while the same text would merely acquire 13 pixels on a 96 dpi monitor. This physical screen condition was hardly adequate for the quality we’re seeking for good text on screen.
Fortunately, the advent of ClearType during the past decade has largely improved the clarity aspect of quality. ClearType leverages the anatomy of the LCD pixel structure and takes advantage of the human visual system to distribute the energy typically emit to a whole display pixel, across the neighboring sub-pixels in the LCD’s typical 3-color channels making up each individual pixel, to create the visual illusion of higher resolution raster quality on a lower resolution device. As the result, ClearType text looks significantly sharper than the typical text on an LCD display, mitigating a large portion of the quality problem on a display technology that would become hugely popular a few years later.
Another pleasant design of the original ClearType in Windows was that it has improved the clarity of text without breaking application compatibility – that is, it doesn’t change the actual size of each individual glyph in either direction, nor did it change the distance between the two adjacent ones. This is the reason one could turn it on or off at will without having to “store” the selected option in the document or application. It is entirely per-user rendering preference. In Windows 7 we also improved the ClearType Text Tuner in keeping with our theme of being in control of your PC experience, by providing even more granular choices when tuning ClearType (and of course you can still turn it off).
But like many other things in the world, the coin comes in two faces. While it is able to preserve backward compatibility, it is limited by its own leverage unable to advance the state of the art. The width and height of the individual glyph and the nominal distances between the adjacent two remain fixed to the rounded number of screen pixels at a given size.
One of the graphics improvements we made in Windows 7, therefore, is to move from the physical pixel model of the past, and instead creating a new design around what we call the “device independent pixel” unit (or “DIP”), a “virtual pixel” that is one-ninety-sixth an inch in floating-point data type. In this model, a glyph (or any other geometric primitive for that matter) can size to fractional pixels, and be positioned anywhere in between the two pixels. The new ClearType improvement allows sizing and placement of glyph to the screen’s sub-pixel nearest to its ideal condition, creating a more natural looking word shape and making text on screen looks a lot closer to print quality.
The following figure shows the side-by-side comparison of the same word between the original or today’s ClearType (above) and the Windows 7 improvement – Natural ClearType (below), which does require calling the new APIs to render. Notice the width of the letters in the word and the spacing between them, as well as how the more consistent width and spacing improves the overall appearance of the entire word. Note that all the letters are placed with its nominal spacing and there is no kerning adjustment being applied here. A great article by Kevin Larson – a researcher in the Advanced Reading Technology team, discusses in details the scientific aspect of word recognition.
The ability to be more precise in approximating the screen placement of natural text also lends itself to a very nice side-effect, and that is the fact that text can now be placed on the line with no regards to the actual display device’s resolution. It means a UI designer can design an application UI knowing it’ll look the same on all other screens as it appears on his or her screen regardless of what type of display device the users might have. This fact is also particularly handy for software localization where the translated text produces the same layout everywhere.
This improvement could also offer a more realistic view of a print document on screen, or make the screen document looks closer to its print counterpart. It could also improve the quality of document zooming. Imagine document zoom that could go in and out in the same manner as what you would see when pulling the actual print page closer and farther away from your sight. It could mean a more joyful experience for online reading.
The Font is the heart and soul to typography, much like photo is to photography. A lot more fonts are shipped with Windows these days while even more are developed around the world. Windows Vista shipped with 40% more fonts comparing to Windows XP. Windows 7 is expected to ship with 40+ new fonts, just to underscore this trend. We’ve also added some additional viewing/categorization capabilities using the Windows 7 Explorer to improve working with a large library of related fonts.
The default common controls’ font dialog and the font chunk in Windows 7 Ribbon are also updated to be more intelligently selective of what fonts to be present to the user of the current user’s profile. Depending on a number of settings including the current UI language, the user locale, and the current set of keyboard input locales, the font list hides fonts of languages not typically used by the user of different culture and locale. For example, all the international fonts are automatically hidden away from a typical English user to reduce clutter and promote better productivity in common system applications such as NotePad, WordPad and Paint. Third-party application utilizing the Ribbon or the common controls’ font dialog could also have the same benefit. The user still retains the option of selecting any desired font back to the view by explicitly marking it in the Windows 7 Control Panel’s Font applet.
This growth, however, introduces some new opportunities for improvement. We’ve long treated fonts as system-wide resources. It gets “installed” on the machine and kept in a single flat namespace managed by the core part of the operating system. It may be interesting to some that the font named “Arial Black” isn’t really in the same grouping as “Arial Narrow” or “Arial”. This is because as far as the operating system is concerned, they are just different fonts with different names. And because font is uniquely identified by its name, you can’t have multiple versions of the same font at the same time.
Because font is system resource, non-traditional usage of font such as font embedded within the document, and font used exclusively in an application is done through the mechanism known as private installation, which involves making sure the font name is unique before installing it programmatically but doing so by hiding it from others to see. Private font is just like font installed publicly as far as the operating system internal is concerned.
An important improvement in Windows 7’s new font system is the notion of “font collection” which allows partitioning of fonts sharing the same usage into a separate namespace. The system collection is similar to what exists today and is created and managed by the system whereas custom collection can be created and managed, as many as needed, entirely by the application program. This allows document to have its own set of fonts local to it, and third-party application or plug-in to ship with its own font used exclusively within the program. This partitioning not only reduces unnecessary system-wide font update and allows update to happen only locally as needed, it also allows access to multiple versions of the same font in different collections.
The new font system also improves the way fonts are organized within the collection. It supports the notion of weight-width-slope variation where fonts with the same stylistic root but vary in its weight (thin, light, bold, black, etc.), width (wide, narrow, etc.), or slope (italic, oblique) are grouped together in the same font family. For instance, “Arial Narrow” becomes a variation or face in the “Arial” family. This grouping model is advocated by the CSS recommendation.
Fonts also represent art and artistic expression. The technology helping create font is therefore the artist’s tool of expression. An important technology called OpenType emerged during the past decade. It enables new ways type design can be realized. OpenType allows designer to define how glyphs interact and transform in stages. The designer then exposes this function as an executable unit known as the “font feature” for application programmable access.
OpenType was an offshoot of the TrueType Open technology Microsoft developed in 1994-95. The TrueType Open technology added the GSUB, GPOS, BASE, JSTF, and GDEF tables to the TrueType format. The primary usage at the time was to help with the creation of Arabic font due to the inherent complexity of the task. Microsoft chose to rename the technology to OpenType in 1996 and Adobe added their CFF glyph outline format to the technology in the same year. Today OpenType is used to improve readability of text as well as to express new and exciting type design in various languages.
However, despite its long-time presence and availability, the usage of OpenType in the Windows world remains largely in specialized programs. The Windows native graphics system has not fully embraced OpenType for its mainstream usage of text. This absence discourages many designers as there is no standard way in Windows to test the feature they produce. Likewise, its limited exposure doesn’t encourage discoverability for mainstream application developers. Improving this and transitioning to this improved rendering technology is a multi-step and multi-release investment done so as to maximize the benefit while minimizing the disruption that might be introduced as incompatibilities. Windows 7 takes another step on this path. We know for many that care deeply about this area there is a strong desire to move faster. We are doing our best to balance the speed of transition with the desire to maintain compatibility.
Windows 7 new text system not only uses available OpenType features internally but also allows access to any feature made available in the font in the high level programming interface, making it easier for application developer to discover and exercise the font feature in mainstream scenario. Windows 7 also ships with a brand new OpenType font “Gabriola” developed by a well-respected typographer John Hudson. Gabriola makes heavy use of contextual letterforms and offers an unprecedented number of stylistic sets for different usages of the font in different occasions. The figure below enumerates all stylistic sets available in this font; notice the subtleties and not-so-subtle way to distinguish each stylistic set.
The figure below also demonstrates the power of contextual letterforms in the eighth rendition of Gabriola’s stylistic set (“ss07”) to produce different ways the same word is rendered depending on where it’s at in the line.
Rendering text is complex and involved, even though it seems like something that should be straight forward. There are probably hundreds of ways to format text in a document and often many paths that ultimately yield the same results. HTML/CSS is a complex standard and is a great example of the richness of how text may be formatted and typeset. Underneath the formatting logic lies the language requirement – the rule of writing for the language. Windows has long been supporting Unicode – another complex standard for global data interchange. Windows supports an increasing number of Unicode script in every single release. The mapping from the input text to the final glyphs in the font requires intricate transformation, which involves parsing of font data and analyzing the language writing pattern. Once the glyph is finalized, it is then rasterized, merged and filtered into the final visual on the display device.
Due to this staging nature, different types of applications require different support from the text system. While a typical application such as the legendary “Hello world” type application may be satisfied with only the ability to get some text out showing to the user. The same level of support is hardly adequate for document preparation system such as Microsoft Word and Adobe InDesign. Some of the more mature application code bases may also have to deal with different graphics systems. This makes it harder in practice for a text system that tie to a particular graphics model to really be widely useful across the wide variety of applications in the Windows ecosystem.
It became obvious to us early on during the planning stage of Windows 7 that text processing is not homogeneous, and different types of applications have different needs and requires different levels of support. The appropriate level of programming access to the text functionality is as important as the functionality itself. The new text system in Windows 7 is assembled into a self-sufficient system called DirectWrite. The API is provided in four layers – the interfaces for font data, rendering support, language processing, and typesetting, each built upon the others with the lower layer makes no requirement to the upper one, and none depends on a specific graphics model. To illustrate the latter point, the figure below shows a sample application that uses the new typesetting interface and language processor while the final rendering happens as an extruded filled 3D geometry from the 2D graphics environment also new to Windows 7 called Direct2D. Both systems were introduced in PDC 2008 as the new graphic foundation in Windows 7.
DirectWrite preserves developer’s investment in existing technologies such as GDI and GDI+ in three important aspects. First, the previously described layering design of DirectWrite allows for the clean separation between the two fundamental processes of placing and rendering of text. It enables applications to use DirectWrite to place text while having it rendered onto traditional graphic surfaces such as GDI and GDI+. The reverse scenario in which the application may use GDI to place text while having it rendered through DirectWrite is also naturally supported. The second aspect of compatibility comes from the fact that DirectWrite also supports all existing methods for placing and rendering text found in GDI. A DirectWrite application can use DirectWrite to place and render text in the same manner as GDI does without actually using GDI. Text placed and rendered under this compatibility mode is indistinguishable from GDI text from the user’s point of view, and as such preserving existing layout of application UI and text document. Lastly, DirectWrite exposes a set of APIs that interoperate with GDI. An application selecting a GDI font object can turn it into a DirectWrite’s font object and vice versa. Since the font system is at the low end of the DirectWrite API layer, it provides a natural interoperability point that is fundamental enough to ensure high degree of data preservation and correctness. Once the application is able to acquire a DirectWrite’s font object, it can in turn use it in any other DirectWrite API requiring a DirectWrite font from that point onward. The conversion from a DirectWrite’s font object back to a GDI font object allows the rest of the GDI-based application to function with no change while still being able to reap the benefit of using DirectWrite’s new and improved font model. As in some real world examples, the XPS print rasterizer in Windows 7 is implemented on top of DirectWrite and utilizes DirectWrite’s interoperability API to convert back to a GDI font as part of the conversion of an XPS-based print job for a non-XPS printer driver. The Windows 7 XPS Viewer also uses DirectWrite alongside the GDI+ graphic rendering for its onscreen display.
There’s a lot more to the details of the API. In the PDC session linked to above, Leonardo Blanco and Kam VedBrat go into the details of DirectWrite and Direct2D and how to develop applications such as this.
The world has changed a lot since the first text APIs of Windows GDI, such as TextOut or ExtTextOut in Windows NT 3.1 (or the subsequent API additions). The evolution of support for text is a critical part of the underpinnings of Windows 7. We continue to improve this most “basic” element of a graphical operating system so that regardless of the language, script, or device used to render text, Windows will offers a great set of tools and APIs for developers and a great experience for end-users.
Microsoft has been working on handwriting recognition for over 15 years going back to the Pen extensions for Windows 3.0. With the increased integration and broad availability of the handwriting components present in Windows Vista we continue to see increased use of handwriting with Windows PCs. We see many customers using handwriting across a wide variety of applications including schools, hospitals, banking, insurance, government, and more. It is exciting to see this natural form of interaction used in new scenarios. Of course one thing we need to continue to do is improve the quality of recognition as well as the availability of recognizers in more languages around the world. In this post, Yvonne, a Program Manager on our User Interface Platform team, provides a perspective on engineering new recognizers and recognition improvements in Windows 7. --Steven
Hi, my name is Yvonne and I’m a Program Manager on the Tablet PC and Handwriting Recognition team. This post is about the work we’ve done to improve recognition in handwriting for Windows 7.
Microsoft has invested in pen based computing since the early 1990s and with the release of Windows Vista handwriting recognizers are available for 12 languages, including USA, UK, German, French, Spanish, Italian, Dutch, Brazilian Portuguese, and Chinese (Simplified and Traditional), Japanese and Korean. Customers frequently ask us when we plan to ship more languages and why a specific language is not yet supported. We are planning to ship new and improved languages for Windows 7, including Norwegian, Swedish, Finnish, Danish, Russian, and Polish, and the list continues to grow. Let’s explore what it takes to develop new handwriting recognizers.
Windows has true cursive handwriting recognition, you don’t need to learn to write in a special way – in-fact, we’ve taught (or “trained” as we say) Windows the handwriting styles of thousands of people and Windows learns more about your style as you use it. Over the last 16 years we’ve developed powerful engines for recognizing handwriting, we continue to tune these to make them more accurate, faster and to add new capabilities, such as the ability to learn from you in Vista. Supporting a new language is much more than adding new dictionaries – each new language is a major investment. It starts with collecting native handwriting, next we analyze the data and go through iterations of training and tuning, and finally the system gets to you and continues to improve as you use it.
The development of a new handwriting recognizer starts with a huge data collection effort. We collect millions of words and characters of written text from tens of thousands of writers from all around the world.
Before I describe our collection efforts, I would like to answer a question we are frequently asked: “Why can’t you just use an existing recognizer with a new dictionary?” One reason is that some languages have special characters or accents. But the overriding reason is because people in different regions of the world learn to write in different ways, even between countries with the same language like the UK and US. Characters that may look visually very similar to you can actually be quite different to the computer. This is why we need to collect real world data that captures exactly how characters, punctuation marks and other shapes are written.
Setting up a data collection effort is challenging and time consuming because we want to ensure that we collect the “right kind of data”. We carefully choose our collection labs in the respective countries for which we develop recognizers.
Before we start our data collection in the labs, we configure our collection tools, prepare documentation, and compile language scripts that will guide our volunteers through the collection process. Our scripts are carefully prepared by native speakers in the respective language to ensure that we collect only orthographically correct data, data from different writing styles, and data that covers all characters, numbers, symbols and signs that are relevant to a specific language. All of our scripts are proofread and edited before they are blessed to be used at the collection labs.
Once our tools and scripts are ready, we open our labs and start to recruit volunteers to donate their handwriting samples. Our recruitment efforts ensure that we have balanced demographics such as gender, age, left handiness, and educational background that represent the majority of the population for that country.
A supervisor at the lab instructs the volunteers to copy the text as it is displayed in the collection tool in their own writing style. What is important to note is that we want to collect writing samples that accurately represent the person’s natural way of writing. We therefore encourage volunteers to treat “pen and tablet” like “pen and paper”. If one of the volunteers tends to writes in big, curvy strokes, then we want to collect his/her big, curvy strokes during the collection session. High quality data in this context refers to data that was naturally written.
Here is a snapshot of what our collection tool looks like:
Figure 1: Collection Tool
A collection session lasts between 60-90 minutes at which point our volunteer has donated a significant amount of handwritten data without feeling fatigued. The donated data is then uploaded and stored in our database at Microsoft ready for future use. The written samples contain important information like stroke orders, start- and end points, spacing, and other characteristics that are essential to train our new recognizer.
Let’s take a look at some of our samples in our database to illustrate the great variation among ink samples:
Figure 2: Ink samples illustrating different stroke orders.
The screenshot shows how three different volunteers inked the word “black”. The different colors are used to illustrate the exact stroke orders in which the word was written. Our first two volunteers used five strokes to write the word “black”; our third volunteer used four strokes. Please also note how our third volunteer used one stroke only to ink the letters “ck”, while our first volunteer used three strokes for the same combination of letters. All of this information is used to train our recognizers.
Once we have collected a sufficient amount of inked data, we split our data into a training set, used by our development team, and a “blind” set, used by our test team. The training set is then employed to train the Neural Network, which is largely responsible for the magic that is taking place during the recognition process. Good, naturally written data is essential in developing a high quality recognizer; the recognizer can’t be any better than its training set. The more high quality data we feed into our Neural Network, the more equipped we are to handle sloppy cursive handwriting.
Our Neural Network is a Time-Delay Neural Network (TDNN) that can handle connected letters of cursive scripts. A TDNN takes ink segments of preceding and following stroke segments into consideration when computing the probabilities of letters, digits and characters for each segment of ink. The output of the TDNN is powerful but not good enough when handwriting is sloppy. In order to come within reach of human recognition accuracy, we have to employ information that goes beyond the shape of the letter: we call this the Language Model context. The majority of this Language Model context comes in form of the lexicon, which is a wordlist of valid spellings for a given language. For many languages, this is the same lexicon that the spellchecker uses. The TDNN and the lexicon work closely together to compute word probabilities and output the top suggestions for the given input.
Training the Neural Network is an involved process that takes time. We often experiment with borrowing data from other languages to increase the size of the training data with the ultimate goal to boost recognition accuracy. Borrowing characters from other languages does not always lead to success. As I mentioned above, stroke order, letter shape, writing styles and letter size can differ significantly from country to country and can have a negative impact on the performance of the TDNN. It often takes us several rounds of training, re-training and tuning before we find “the right formula” that will lead to high recognition accuracy.
How do we know if we are headed in the right direction when we build a new recognizer? This is an important question that the test team and native speakers answer for us. The test team is responsible for generating our recognition accuracy metrics that reflect how good our recognizer is. These accuracy metrics are based on our blind test set which is the collected data that development could not use for training. In addition to our accuracy metrics, we work with native speakers in house and at our world-wide subsidiaries to get feedback and further input.
In the previous paragraphs I have outlined how we develop high quality recognizers that can handle a wide variety of different writing styles. But there is more as each person can also train the recognizer his/her unique writing style. The training that is done to teach the recognizer a personal writing style is the same training that happens before Microsoft ships the product. The only difference is that we are now collecting unique training data from a specific person (and not that of thousands of people). We call this process “Personalization”.
Figure 3: Personalization Wizard (Sentence module).
As the screenshots of our Personalization wizard illustrates, a person is asked to write the requested sentence to provide his/her ink samples. The more data a person donates during the personalization process, the better the recognizer will become. In addition to providing writing samples based on specified sentences, a person can target specific recognition errors, shapes, and characters that will all be used for training. Our Personalization feature is complex and offers a variety of different modules that enable a person to optimally tune the recognizer. We are proud to announce that Personalization will be available for all Vista languages and all new Windows 7 languages. We encourage you to use this feature to improve your recognition accuracy.
We continue to work on improving our recognizers which also means that we are incorporating our customers feedback through online telemetry (anonymously, privately, voluntary, and opt-in). In Windows Vista we released a new feature called “Report Handwriting Recognition Errors”, which gives people the opportunity to submit those ink samples that the recognizer did not recognize correctly. After the person has corrected a word in the Tablet Input Panel (TIP), we enable a menu that allows a person to send the misrecognized ink together with its corrected version to our team.
Here is a screenshot of what our error reporting tool looks like:
Figure 4: With “Report Handwriting Recognition Errors” people can choose which of the misrecognized ink samples they want to submit.
We receive approximately 2000 error reports per week. Each error report is stored in our database before we analyze it and use it to improve our next generation of recognizers. As you can imagine, real world data is extremely helpful because it is only this type of data that can reveal shortcomings of our recognizers.
We value and appreciate every single error report. Keep sending us your feedback, so that we can use it to improve the magic of our present and future recognizers.
– Yvonne representing the handwriting recognition efforts