These postings are provided "AS IS" with no warranties, and confer no rights. Use of included code samples are subject to the terms specified at Microsoft -
Notes on comments.
Welcome to our blog dedicated to the engineering of Microsoft Windows 7
This post is from Michael Bernstein, a development lead on the User Interface Platform team where he focuses on accessibility. Accessibility is the term we apply to the APIs and features that enable Windows to be used, to be accessible, by as many people as possible so that, regardless of physical or cognitive abilities, everyone has the ability to access the functions of Windows. To enable this, Windows includes both built-in accessibility utilities as well as APIs used by third party assistive technology aids and by application developers to make sure their software is also accessible. This is a topic that is extremely important to Microsoft and one that is a key tenet in the engineering of Windows 7. Microsoft also has a corporate-wide group dedicated to making sure that PCs are easier to see, hear, and use. You can read more about Microsoft’s accessibility initiatives on http://www.microsoft.com/enable/. --Steven
Hi, I’m the development lead for Accessibility and Speech Recognition experiences in Windows 7, and I wanted to write about how we thought about Accessibility in Windows 7.
We wanted to make Windows 7 the most accessible operating system that Microsoft has ever produced. It became clear as we planned this release, however, that the notion of Accessibility is not as simple as it may appear. It is tempting to think about Accessibility like Security: either you have a known failure, or your system is believed to be secure/accessible. This definition turns out to be limited, though. How do you deal with the fact that the needs of customers who are blind are very different from the needs of customers who are deaf? The needs of customers who are blind are even different from those of customers with reduced vision: a magnification tool is useless for one group and crucial for the other. And what do we make of cases where something is technically accessible but practically frustrating, like a common user scenario that takes 36 keystrokes to execute? Clearly, Accessibility wasn’t going to boil down to a simple yes/no question. It is really more like a particular kind of usability, but usability for a specific set of audiences with individual needs.
Since the questions we were asking were complex, the answers ended up being complex, too. We chose a four-part strategy to improve Accessibility in Windows 7.
In Windows Vista, Microsoft delivered a new core component for Accessibility called UI Automation. UI Automation enables a user’s assistive technology (AT) to programmatically drive the UI of an application, and allows applications to expose their accessible functionality in a richer way than was possible in previous versions of Windows. More questions can be asked about a piece of UI, and that UI can be manipulated in richer ways. UI Automation also introduced the idea of Control Patterns: any given piece of UI can decide how it should be controlled. Buttons expose the Invoke pattern, indicating that they can be pushed; Combo Boxes expose ExpandCollapse, indicating that they can be opened and closed. We let different controls be different, instead of trying to force them all into the same mold. All this was introduced in Windows Vista and adoption is still ongoing.
In Windows 7, we invested in improving the performance of the UI Automation system and created a new, native-code API for UI Automation to make sure that it can be used effectively by a wide range of assistive technology software. Now applications written in C++, as well as those written using the .NET Framework, can take advantage of UI Automation.
We also did a bunch of work to make sure that the UI Automation system was integrated even more closely with the legacy Microsoft Active Accessibility (MSAA) system and developed new bridging techniques between the best of the new and the old technologies. UI Automation Clients can read Accessibility information from MSAA applications, and vice versa, to ensure maximum Accessibility regardless of which accessibility API an application used originally. Since the UI Automation and MSAA systems cooperate closely in many scenarios, we decided to name the combination of the two, calling it the Windows Automation API. This architecture forms the foundation for the rest of our Accessibility effort, and we’re pleased to have this Accessibility foundation Windows 7.
We also improved the Accessibility utilities that we include in the box with Windows. Microsoft works closely with many different AT software companies who deliver software to make Windows more accessible to customers with disabilities, but we also include a set of utilities to make sure that our customers’ early experiences are accessible, even before installing any other software. We decided to enhance two of those utilities in Windows 7: the On-Screen Keyboard and the Magnifier.
The most noticeable change to the On-Screen Keyboard is the improved look and feel, but there are also more subtle enhancements. The appearance of this utility had not changed since Windows XP; our customers were also asking for it to be resizable. We addressed both of these by working closely with Tablet developers to share a common code base between the Tablet Soft Keyboard and the On-Screen Keyboard. Both keyboards now have an attractive appearance that is in tune with Windows 7 and both are now resizable. The keyboards still are distinct, though, because customers use them differently: Tablet users may want to switch dynamically between handwriting and typing, whereas On-Screen Keyboard users may need modes where they can hover or scan to keys, if they have disabilities that prevent them from clicking. Along these lines, we also added basic text prediction to help customers with disabilities enter text more quickly. If you have ever tried typing with an on-screen keyboard, you can appreciate how significantly text prediction can improve text entry speed.
The Magnifier came in for a deeper overhaul. The Magnifier in Windows Vista and Windows XP was not an intuitive experience: when you pointed at part of the screen, the magnified content appeared in a separate window, usually docked at top of the screen. You had to point at one place and look at another. We considered two basic solutions to this problem: you could zoom into the entire screen or you could make the magnified area follow the pointer while leaving the rest of the screen the same. These became our two primary modes for the Windows 7 Magnifier: Full-screen mode and Lens mode.
Full-screen mode is great when you want to increase the size of everything on the screen at once. As you move the mouse or keyboard focus around the middle of the screen, the view stays still; if you move towards the edge, the Magnifier scrolls the view to keep up. One downside of this mode is that you can lose track of your context. To address that usability issue, we added a context animation that zooms out to show you where your work area is relative to the whole screen, and then zooms back in.
Lens mode, on the other hand, is nice when you just want to zoom in on one particular thing. In this mode, the lens centers on the mouse pointer, which feels much like using a magnifying glass. You can re-size the lens to be very wide and short, which can be nice if you are reading a document and want to magnify it line by line. We based our design on the popular Microsoft IntelliPoint magnifier, a design you can now enjoy with any mouse.
We also addressed customer feedback about the Magnifier window taking up too much space on the screen. We moved the most commonly used controls like zoom in/out to a small toolbar, which fades out to a semi-transparent watermark when you aren’t using it. The remaining options are available in an Options dialog when you need them. Last, we gave almost everything a keyboard shortcut, so if you really don’t want to see the UI, you don’t have to use it. Win-+ will zoom you in any time you are using Windows 7.
These tools directly improve Accessibility for customers with low vision and dexterity disabilities. It should be obvious, but making the PC easier to see or interact with benefits everyone and so these two examples also show the broad appeal of AT tools – at the PDC we showed both the On-Screen Keyboard and the Magnifier and I think it is fair to see everyone saw the benefit of using these tools themselves, regardless of abilities.
Windows APIs cannot provide Accessibility all by themselves; it is vital for Windows-based applications to do their part in providing Accessibility data for AT programs to use. For example, a screen reader may sound excellent, but if it can’t read your favorite web browser, what good is that? Assistive tools like screen readers and magnifiers are clients of the Accessibility system, while the applications that you want to use, like web browsers and word processors, are providers. It takes both to make the whole experience accessible--you need both a high-quality client and a well-written provider to have a good Accessible experience. There are more providers in the software ecosystem, so it is hard for us to work one-on-one with every provider to make sure they are well-written.
To address this challenge, our team developed the UI Accessibility Checker (AccChecker for short) and UI Automation Verify (UIA Verify) utilities, which can scan an application (a provider, really) and report on common Accessibility problems. Software developers can use AccChecker and UIA Verify to detect problems in their provider code before a customer ever uses it. Quality assurance engineers can use them to verify the quality of their firm’s work. We believe this is so important that we released AccChecker and UIA Verify as open-source software to make it available to the widest possible audience. If you are not a programmer, you may never use these utilities directly, but you may well benefit from the bugs they helped to eliminate before they ever reached you.
To make sure that Windows features themselves were good providers, we borrowed an idea from the Software Development Lifecycle, risk assessment. Before a line of code was written, each planned Windows 7 feature was rated on its Accessibility risk. Features that use more basic, off-the-shelf common controls are usually more accessible because Windows provides built-in providers for off-the-shelf components; features that do fancy, custom drawing have more work to do. This planning process made each team aware of how much accessibility risk it was taking on, so that they could plan appropriately. Once the features were all rated, the list was sorted by risk so that our team could reach out to teams with high-risk features and make sure that they had the resources and tools they needed to make their feature properly accessible. We also ensured that they received more hands-on testing and validation. As a result, most Windows features are more accessible than they have been in previous releases, making for a better overall customer experience.
To wrap up, we've emphasized Accessibility in engineering Windows 7. We’ve made good progress on improving the core architecture for Accessibility and enhancing the included tools like On-Screen Keyboard and Magnifier. The AccChecker and UIA Verify tools have made it much easier to validate applications to ensure that they will be compatible with current assistive tools as well as future tools based on the Windows Automation API. Our approach to Accessibility for the features and providers in Windows itself has become more thorough, consistent and integrated, thanks to the hard work of hundreds of engineers across the company. We’re proud of what we have accomplished in Windows 7 and hope that it will help customers with disabilities to realize their full potential and have a more enjoyable experience with Windows.
Aka: A developers view of the Windows 7 Engineering process
This post is by Larry Osterman. Larry is one of the most “experienced” developers on the Windows team and has been at Microsoft since the mid 1980’s. There are only three other folks who have worked at Microsoft longer on the entire Windows team! Personally, I remember knowing about Larry when I started at Microsoft back in 1989—I remember he worked on “multimedia” (back when we used to host the Microsoft CD-ROM Conference) and he was one of those people that stood up and received a “5 Year” award from Bill Gates at the first company meeting I went to—that seemed amazing back then! For Windows 7, Larry is a developer on the Devices and Media team which is where we work on audio, video, bluetooth, and all sorts of cool features for connecting up devices to Windows.
Larry wrote this post without any prodding and given his experience on so many Windows releases these thoughts seemed really worthwhile in terms of sharing with folks. This post goes into “how” we work as a team, which for anyone part of a software team might prove pretty interesting. While this is compared and contrasted with Vista, everyone knows that there is no perfect way to do things and this is just a little well-informed perspective.
So thank you Larry! --Steven
Thanks to Steven and Jon for letting me borrow their soapbox :-).
I wanted to discuss my experiences working on building Windows 7 (as opposed to the other technical stuff that you’ve read on this blog so far), and to contrast that with my experiences building Windows Vista. Please note that these are MY experiences. Others will have had different experiences; hopefully they will also share their stories here.
The experience of building Windows 7 is dramatically different from the experience of building Vista. The rough outlines of the product development process haven’t changed, but organizationally, the Windows 7 process is dramatically better.
For Windows Vista, I was a part of the WAVE (Windows Audio Video Excellence) group. The group was led by a general manager who was ultimately responsible for the deliverables. There was a test lead, a development lead and a program management lead who reported to the general manager. The process of building a feature roughly worked like this: the lead program managers decided (based on criteria which aren’t relevant to the post) which features would be built for Windows and which program managers would be responsible for which feature. The development leads decided which developers on the team would be responsible for the feature. The program manager for the feature wrote a functional specification (which described the feature and how it should work) in conjunction with development. Note that the testers weren’t necessarily involved in this part of the process. The developer(s) responsible for the feature wrote the design specification (which described how the feature was going to be implemented). The testers associated with the feature then wrote a test plan which described how to test the feature. The program manager or the developer also wrote the threat model for the feature.
The developer then went off to code the feature, the PM spent their time making sure that the feature was on track, and when the developer was done, the tester started writing test cases.
Once the feature was coded and checked into the source tree, it moved its way up to the “winmain” branch. Aside: The Windows source code has been arranged into “branches” – the root is “winmain”, which is the code base that would ultimately become Windows Vista. Each developer works in what are called “feature branches”, which merge changes into “aggregation branches”, the aggregation branches move into winmain.
After the feature was coded, the testers tested, the developers fixed bugs and the program managers managed the program :-). As the product moved further along, it got harder and harder to get bug fixes checked into winmain (every bug fix carries with it a chance that the fix will introduce a regression, so the risk associated with each bug fix needs to be measured and the tolerance for risk decreases incrementally). The team responsible for managing this process met in the “ship room” where they made decisions every single day about which changes went into the product and which ones were left out. There could be a huge amount of acrimony associated with that – often times there were debates that lasted for hours as the various teams responsible for quality discussed the merits associated with a particular fix.
All-in-all, this wasn’t too different from the way that features have been developed at Microsoft for decades (and is basically consistent with what I was taught back in my software engineering class back in college).
For Windows 7, management decided to alter the engineering structure of the Windows organization, especially in the WEX [Windows Experience] division where I work. Instead of being fairly hierarchical, Steven has 3 direct reports, each representing a particular discipline: Development, Test and Program Management. Under each of the discipline leads, there are 6 development/test/program management managers, one for each of the major groups in WEX. Those 2nd level managers in turn have a half a dozen or so leads, each one with between 5 and 15 direct reports. This reporting structure has been somewhat controversial, but so far IMHO it’s been remarkably successful.
The other major change is the introduction of the concept of a “triad”. A “triad” is a collection of representatives from each of the disciplines – Dev, Test and PM. Essentially all work is now organized by triads. If there’s ever a need for a group to concentrate on a particular area, a triad is spun off to manage that process. That means that all three disciplines provide input into the process. Every level of management is represented by a triad – there’s a triad at the top of each of the major groups in WEX, each of the second level leads forms a triad, etc. So in my group (Devices and Media) there’s a triad at the top (known as DKCW for the initials of the various managers). Within the sound team (where I work), there’s another triad (known as SNN for the initials of the various leads). There are also triads for security, performance, appcompat, etc.
Similar to Windows Vista, the leads of all three disciplines get together and decide a set of features that go in each release. They then created “feature crews” to implement each of the features. Typically a feature crew consists of one or two developers, a program manager and one or two testers.
This is where one of the big differences between Vista and Windows 7 occurs: In Windows 7, the feature crew is responsible for the entire feature. The crew together works on the design, the program manager(s) then writes down the functional specification, the developer(s) write the design specification and the tester(s) write the test specification. The feature crew collaborates together on the threat model and other random documents. Unlike Windows Vista where senior management continually gave “input” to the feature crew, for Windows 7, management has pretty much kept their hands off of the development process. When the feature crew decided that it was ready to start coding (and had signed off on the 3 main documents), the feature crew met with the second level triad (in my case with DKCW) to sanity check the feature – this part of the process is critical because the second level triad gets an opportunity to provide detailed feedback to the feature crew about the viability of their plans.
And then the crew finally gets to start coding. Sort-of. There are still additional reviews that need to be done before the crew can be considered “ready”. For instance, the feature’s threat model needs to be reviewed by one of the members of the security triad. There are other parts of the document that need to be reviewed by other triads as well.
A feature is not permitted to be checked into the winmain branch until it is complete. And I do mean complete: the feature has to be capable of being shipped before it hits winmain – the UI has to be finished, the feature has to be fully functional, etc. In addition, when a feature team takes a dependency on another Windows 7 feature, the feature teams for the two features MUST sign a service level agreement to ensure that each team knows about the inter-dependencies. This SLA is especially critical because it ensures that teams know about their dependants – that way when they change the design or have to cut parts of the feature, the dependent teams aren’t surprised (they may be disappointed but they’re not surprised). It also helps to ensure tighter integration between the components – because one team knows the other team, they can ensure that both teams are more closely in alignment.
Back in the Vista day, it was not uncommon for feature development to be spread over multiple milestones – stuff was checked into the tree that really didn’t work completely. During Win7, the feature crews were forced to produce coherent features that were functionally complete – we were told to operate under the assumption that each milestone was the last milestone in the product and not schedule work to be done later on. That meant that teams had to focus on ensuring that their features could actually be implemented within the milestone as opposed to pushing them out.
For the nuts and bolts, The Windows 7 development process is scheduled over several 3-month long milestones. Each milestone allowed for 6 weeks of development and 6 weeks of integration – essentially time to fine-tune the feature and ensure that most of the interoperability problems were shaken out.
Ok, that’s enough background (it’s bad when over half a post on Windows 7 is actually about Windows Vista, but a baseline needed to be established). As I said at the beginning, this post is intended to describe my experiences as a developer on Windows 7. During Windows 7, I worked on three separate feature crews. The first crew delivered two features, the second crew delivered about 8 different features all relatively minor and the third crew delivered three major features and a couple of minor features. I also worked as the development part of the WEX Devices and Media security team (which is where my series of post on Threat Modeling came from – I wrote them while I was working with the members of D&M on threat modeling). And I worked as the development part of an end-to-end scenario triad that was charged with ensuring that scenarios that the Sound team defined at the start of the Windows 7 planning process were actually delivered in a coherent and discoverable way.
In addition, because the test team was brought into the planning process very early on, the test team provided valuable input and we were able to ensure that we built features that were not only code complete but also test complete by the end of the milestone (something that didn’t always happen in Vista). And it ensured that the features we built were actually testable (it sounds stupid I know, but you’d be surprised at how hard it can be to test some features). As a concrete example, we realized during the planning process that some aspect of one of the features we were working on in M2 couldn’t be completed during the milestone. So before the milestone was completed, we ripped the feature out (to be more accurate, we changed the system so that the new code was no longer being built as a part of the product). During the next milestone, after the test team had finished writing their tests, we re-enabled the feature. But we remained true to the design philosophy – at the end of the milestone everything that was checked into the “main” branch was complete – it was code AND test complete, so that even if we had to ship Windows 7 without M3 there was no test work that was not complete. This is a massive change from Vista – in Vista, since the code was complete we’d have simply checked in the code and let the test team deal with the fallout. By integrating the test teams into the planning process at the beginning we were able to ensure that we never put the test organization into that bind. This in turn helped to ensure that the development process never spiraled out of control. Please note that features can and do stretch across multiple milestones. In fact one of the features on the Sound team is scheduled to be delivered across three milestones – the feature crews involved in that feature carefully scheduled the work to ensure that they would have something worth delivering whenever Windows 7 development was complete.
Each of the feature crews I’ve worked on so far has had dramatically different focuses – some of the features I worked on were focused on core audio infrastructure, some were focused almost entirely on UX (user experience) changes, and some features involved much higher level components. Because each of the milestones was separate, I was able to work on a series of dramatically different pieces of the system, something I’ve really never had a chance to do before.
In Windows 7, senior management has been extremely supportive of the various development teams that have had to make the hard decisions to scale back features that were not going to be able to make the quality bar associated with a Windows release – and there absolutely are major features that have gone all the way through planning only to discover that there was too much work associated with the feature to complete it in the time available. In Vista it would have been much harder to convince senior management to abandon features. In Win7 senior management has stood behind the feature teams when they’ve had to make the tough decisions. One of the messages that management has consistently driven home to the teams is “cutting is shipping”, and they’re right. If a feature isn’t coming together, it’s usually far better to decide NOT to deliver a particular feature then to have that feature jeopardize the ability to ship the whole system. In a typical Windows release there are thousands of features and it would be a real shame if one or two of those features ended up delaying the entire system because they really weren’t ready.
The process of building 7 has also been dramatically more transparent – even sitting at the bottom of the stack, I feel that I’ve got a good idea about how decisions are being made. And that increased transparency in turn means that as an individual contributor I’m able to make better decisions about scheduling. This transparency is actually a direct fallout of management’s decision to let the various feature teams make their own decisions – by letting the feature teams deeper inside the planning process, the teams naturally make better decisions.
Of course that transparency works both ways. Not only were teams allowed to see more about what was happening in the planning process, but because management introduced standardized reporting mechanisms across the product, the leads at every level of the hierarchy were able to track progress against plan at a level that we’ve never had before. From an individual developer’s standpoint, the overhead wasn’t too onerous – basically once a week, you were asked to update your progress against plan on each of your work items. That status was then rolled up into a series of spreadsheets and web pages that allowed each manager to track all the teams’ progress against plan. This allowed management to easily and quickly identify which teams were having issues and take appropriate action to ensure that the schedules were met (either by simplifying designs, assigning more developers, or whatever).
In general, it’s been a total blast building 7. We’ve built some truly awesome features into the operating system and we’ve managed to keep the system remarkably stable during that entire process.
As most folks (finally) get the beta and start to set aside some time to install and try out Windows 7, we thought it would be a good idea to start to talk about how we support devices through testing and work across the PC ecosystem. This is a big undertaking and one that we take very seriously. As we talked about at the PDC, this is also an area where we learned some things which we want to apply to Engineering Windows 7. While this is a massive effort across the entire Windows organization, Grant George, the VP of Test for the Windows Experience, is taking the lead in authoring this post. We think this is a deep topic and I know folks want to know more so consider this a kick-off for more to come down the road. –Steven
One of the most important responsibilities in a release of Windows is our support of, and compatibility with, all of the devices and their associated drivers that our users have. The abstraction layer in Windows to connect software and hardware is a crucial part of the operating system. That layer is surfaced through our driver model, which provides the interface for all of our partners in the multi-faceted hardware ecosystem. Windows supports a vast range of devices today – audio devices (speakers, headsets…), display devices (monitors…), print, fax and scan devices, connectivity to digital cameras, portable media devices of all shapes, sizes and functions, and more. Windows is an open platform for companies across the globe who develop and deliver these devices to the marketplace and our users – and our job is to make sure we understand that ecosystem and those choices and verify those devices and drivers work well for our customers – which includes partnering with those device providers throughout the engineering of Windows7.
Drivers provide the interface between a device and the Windows operating system – and are citizens of the WDM (Windows Driver Model). WDM was initially created as an intermediary layer of kernel mode drivers to ease the authoring of drivers for Windows. There are different types of drivers. Class drivers (which are hardware device drivers that supports an array of devices of a similar hardware class where hardware manufacturers make their products compatible with standard protocols for interaction with the operating system) and device-specific drivers (provided by the device manufacturer for a specific device and sometimes a specific version of that device) are the two most common.
Support for our hardware partners comes in the form of the Windows Driver Kit (WDK) and for certification, the Windows Logo Kit (WLK). The WDK enables the development of device drivers and as of Vista replaced the previous Windows Driver Development Kit (DDK). The WDK contains all of the DDK components plus Windows Driver Foundation (WDF) and the Installable File System kit (IFS). The Driver Test Manager (DTM) is another component here, but is separate from the WDK. The Windows Logo Kit (WLK) aids in certifying devices for Windows (it contains automated tests as well as a run-time framework for those tests). These tests are run and passed by our hardware vendor partners in order to use the Microsoft “Designed for Windows™” logo on devices. This certification process helps us and our hardware partners ensure a specific level of quality and compatibility for devices interacting with the Windows operating system. Hardware devices and drivers that pass the logo kits tests qualify for the Windows logo, driver distribution on Windows Update, and can be referenced in the online Windows Marketplace.
With Windows 7 we have modified driver model validation, new and legacy device testing, and driver testing. Compared to Vista, we now place much more emphasis on validating the driver platform and verifying legacy devices and their associated drivers throughout our product engineering cycle. Data based on installed base for each device represents an integral part of testing, and we gather this data from a variety of sources including the voluntary, opt-in, anonymous telemetry in addition to sources such as sales data and IHV roadmaps. We have centralized and standardized the testing mechanics of the lab approach to this area of the product in a way that yields much earlier issue/bug discovery than in past releases. We have also ramped up our efforts to communicate platform or interface changes earlier with our external hardware partners to help them ensure their test cycles align with our schedule. In addition, we draw a more robust correlation between the real-world usage data, including recent trends, and prominence of each device and the prioritization it is given in our test labs. This is especially important for new and emerging devices that will come to market right before and just after we release Windows 7 to our customers.
Another important element in bringing a high quality experience to our Windows 7 users in device and driver connectivity and capability is the staging of our overall engineering process in Windows 7. For this release all of our engineering teams have followed a well structured and staged development process. The development/coding of new features and capabilities in Windows 7 was broken out in to 3 distinct phases (milestones) with dedicated integration and stabilization time at the end of each of these three coding phases. This included ensuring our code base remained highly stable throughout the development of Windows 7 and that our device and driver test validation was a constant part of those milestones. Larry discussed this in his post as some might recall. Program Managers, Developers and Testers all worked in super close partnership throughout the coding phases. Our work with external partners – especially our device manufacturer partners – was also enhanced through early forums we provided for them to learn about the changes in Windows 7 and also work closely with us on validation. Much more focus has been put on planning and then executing - planning the work and then working the plan. Our belief is that this yields much more predictability to developing and delivering our new features in Windows 7 both from a feature content and overall schedule standpoint. We recognize that this raised the bar on how our external partners see us execute and deliver on that plan when we say we will, but we also hope it increases their confidence in how they engage with us in validating the device experience during our development and delivery of Windows 7.
Our program management team helps us drive device market share analysis. Most of their data comes from our Customer Experience Improvement Program. This gives us data on the actual hardware in use across our customer base. For example there are over 16,000 unique 4-part hardware IDs for display devices alone. Like many things, we understand that it only takes a single device not functioning well to degrade an overall Windows experience or upgrade—we definitely want to re-enforce this shared understanding.
New devices typically have a small initial user base, but the driver will often be mostly new code (or the first time a code-base has seen a new device). As the device enters the mainstream, market share grows and most manufacturers continue to develop and improve their drivers. This is where for our customers, and our own testing, it’s important to always have the latest drivers for a given device.
Over a device’s lifetime, we work closely with our external device partners and represent as faithfully as possible in our test labs, a prioritized way of ensuring old and new devices continue to work well with Windows. By paying very close attention to trends in the market place across our device classes, we can make guided decisions in the context of these areas:
Another benefit of close market tracking is creating an equivalence-based view of a device family.
We use the notion of equivalence classes to help us define and prioritize our hardware (device) test matrix. Creating equivalence classes involves grouping things into sets based on equivalent properties across related devices. For example, imagine if we worked for a chemical company and it was our job to test a car polish additive on actual automobiles. Given a fixed test budget, we would want to maximize the number of makes and models we test our product on. We begin by analyzing the current market space so we can make the best choices for our test matrix.
Let’s say the first test car we analyze is a blue 2003 Ford Mustang. We also know that the same blue paint is used on all of Ford’s 2003 and 2004 models and is also used on all of Mazda’s 2005 models. This means our first automobile represents several entries in our table based on equivalence:
Now let’s look at a silver 2001 Mercedes C240. We know that Mercedes and Chrysler have a relationship and upon further investigation we find Chrysler used the same silver paint on their 2006 through 2009 models. Now our equivalence class based test matrix looks like this:
By carefully analyzing each actual automobile, we have established an equivalence relationship that we can leverage to maximize implicit test coverage. Testing one make and model is theoretically equivalent to testing many. Of course we recognize in the real world different companies might use different techniques for applying paint, as one variable, so there are subtleties that require additional information to property class attributes for testing purposes.
Testing computer devices is very similar. Even though there are thousands of different devices on the market, many of them share major components, are die-shrinks of a previous revision, or differ only in terms of memory, clock-rate, pixel count, connector, or even the type of heat sink. Take for example display devices. There are over 16,000 display devices on the market. But the equivalence view reveals that 90% of the market is represented by about 60 different GPUs. By adding a few more to a carefully constructed test matrix based on equivalence it is possible to represent over 99% of all GPUs. Driver writers also leverage equivalence by targeting drivers at a range of hardware. Driver install packages indicate devices they support via hardware IDs.
All modern computer devices are assigned a unique hardware ID based on the device vendor, type, and class. Most IDs (PCI, PC Card, USB, and IEEE 1394 devices) are assigned by the industry standards body associated with that device type.
Let’s look at the device ID of my display adapter:
If I visit PCI-SIG (the standards body associated with all PCI device ID assignment) and do a search on 10DE, I’m told I this is an NVidia PCI ID. If I look further on my system in
I can find NVidia drivers (folders that start with nv_lh). If I open one of the driver .INF files on my machine I see this tell-tale line:
NVIDIA_G92.DEV_0611.1 = "NVIDIA GeForce 8800 GT”
NVIDIA_G92.DEV_0611.1 = "NVIDIA GeForce 8800 GT”
Further inspection of the driver .INF file tells me that the same G92 GPU is used for all of these devices:
A bit of online research reveals other interesting information: “The 8800 GT, codenamed G92, was released on October 29, 2007. The card is the first to transition to 65 nm process, and supports PCI-Express 2.0. It has a single-slot cooler as opposed to the double slot cooler on the 8800 GTS and GTX, and uses less power than GTS and GTX due to its 65 nm process.” -WikiPedia
So in theory, if I was to run a test on my display adapter, there’s a good chance I’d get the same results as I would on any of these other related devices.
One of our primary goals for Windows 7 is compatibility with all Vista certified drivers and to ensure that people have a seamless upgrade experience. This breaks down into several requirements that guide how we test:
One question we are asked about quite a bit is the availability of drivers. There are three primary reasons drivers end up looking for folks: clean installation of Windows, attaching device to a new computer, wanting the updated driver. We definitely recognize that for the readers of this blog, both as enthusiasts and often the support/IT infrastructure for corporations, friends, and families, that the ability to acquire drivers and reliably update machines is something of a “hobby” we all love to hate. We all want the latest and greatest—no more and no less.
A clean installation is one we are all definitely valuing during the beta phase of Windows 7. It should be clear that a clean install, as important as it is to many of us, is not a routine/mainstream experience. Nevertheless, the combination of in-box drivers and those available via Windows Update will serve a very broad set of PCs (for example, you should see most of the drivers installed for the new Atom-based machines if you do a clean install). On the other hand, some drivers for PCs are only available from the PC maker and for a variety of reasons are not available for download from Windows Update or even the device manufacturer’s site. For example, mobile graphics drivers are generally available only from the PC maker and not from the graphics component maker—this is a decision they make because of the way these chipsets are delivered for each PC maker.
Obviously attaching an existing device to a new PC is a common occurrence. In this case you may have long ago lost the CD/DVD that came with a device and you just plug it in (because you ignored the warning saying “please run the setup program first”). Again, our goal is to provide these via Windows Update. Often IHVs have updates or significantly large downloads that for a number of reasons are not appropriate to deliver via Windows Update. In that case we can also alert you, with a link many times, to seek the driver from the vendor of the device.
Updating drivers is something we are all familiar with as we often read “get the latest driver” to address issues. We all see this particularly in the enthusiast gamer space where newer drivers also improve performance or offer more features, in addition to improving overall. The primary way to get updated drivers is generally through optional updates in Windows Update, though again many times the latest and greatest must be downloaded directly from an IHV (independent hardware vendor) site.
Our goal is clearly to make sure that drivers for the broadest set of devices are available and high quality. There are many equal partners that contribute to delivering a PC and all the associated devices and we work hard to develop a systematic way to reach the broadest set of customers with high quality software and support.
The table below provides examples of some of the explicit devices we have directly tested thus far during the development of Windows 7. This is just a sampling of that direct testing - many more devices have been directly tested that are not shown here or are covered through equivalence classing.
This information is available in many sources, such as the WHQL web site that lists all qualified devices. For the purposes of this blog we thought it would be fun to provide a list here which we think will most certainly serve as the basis for discussion.
Radeon X300/X550/X1050 Series
Radeon 9800 Pro
Radeon Xpress Series
Radeon Xpress 1200
Radeon X700 PRO
Radeon X800 CrossFire Edition
Mobility Radeon X300
Radeon X850 CrossFire Edition
Radeon X1950 Series
Mobility Radeon X1300
Mobility Radeon X1400
Mobility Radeon HD3200
Radeon HD 2600 XT
Radeon HD 3850
Radeon HD 3870
Radeon HD 3200
Radeon HD 2400
Radeon HD 2900 XT
Radeon HD 2600
Radeon HD 4850
ATI Technologies, Inc. RAGE XL PCI
RADEON 7000 Series
Analog Devices Inc.
iSight 640x480 Firewire
X5 Stereo BT Headset
Print / Scan
Digital Rebel XT
i470D Photo Printer
PowerShot A720 IS
CASIO COMPUTER CO.,LTD.
Live! Cam Optia AF
WebCam Live! USB
Webcam NoteBook 640x480 USB
WebCam Instant 352x288 USB
WebCam NX Pro 640x480 USB
Live! Cam Notebook Pro 640K USB 2.0
Live! Cam Video IM Pro VGA USB 2.0
Webcam Live Ultra 640x480 USB 2.0 Manual Focus Ring
Creative Labs, Inc.
Creative Technology Ltd
NOMAD MuVo TX
Zen Vision M
DSM - 520
DSM - 510
Stylus Color C88+
Stylus Color C84/C85
Stylus Color C86/C87
Stylus Color C64
Stylus Photo R265
Stylus Photo R220
Stylus Photo R320
Stylus Photo 1270
Stylus Photo R200
Stylus Photo 1280/1290
Stylus Color 900/N
Stylus Color C62
Stylus Photo 820
Stylus Color 660
Stylus Color 640
EasyCam USB PC Camera 640x480
Deskjet D1400 series
Deskjet D2400 Series
Deskjet F2100 series
Color LaserJet 2600
Deskjet 3900 Series
Deskjet D4200 Series
Officejet 6200 Series
Officejet 6300 Series
Officejet Pro L7500
Officejet Pro L7600
Officejet 7400 Series
Officejet 5510 Series
Officejet 7300 Series
LaserJet 3030 MFP
Officejet 6100 Series
Officejet V40 Series
Photosmart D7400 Series
PSC 950 Series
Officejet G Series
Photosmart Pro B8350
LaserJet 4345 MFP
Color LaserJet 4700
Color LaserJet 5550
Color LaserJet 3800
Color LaserJet 3600
Color LaserJet 3000
Business Inkjet 1200D
Color LaserJet 4550
Color LaserJet 4600
Color LaserJet CP4005
Color LaserJet 3700
Color LaserJet 3500
LaserJet 9000 MFP
LaserJet 4 Plus
Color LaserJet 1500L
LABTEC WEBCAM PRO 961358
Web Cam Plus 352x288 USB 2.0 Manual Focus Motion Detection
Z42 Color JetPrinter
Z25 Color JetPrinter
Z45 Color JetPrinter
QuickCam Pro 9000
Quickcam Communicate STX VGA Fixed Focus USB 2.0
QuickCam Chat VGA w/Image Capture USB 2.0
961400-0403 QuickCam Notebook Deluxe 1.3MP MF USB 2.0
QuickCam Pro 4000 640x480 USB 2.0
QuickCam Pro 5000 640x480 USB 2.0
Quickcam Vision Pro1
Quickcam Vision Pro2
961403 QuickCam Fusion 1.3MP USB 2.0
QuickCam Messenger 640x480 USB
QuickCam Messenger Refresh 640x480 USB
QuickCam Notebooks Pro 1.3MP USB 2.0
QuickCam Zoom 640x480 USB
QuickCam Communicate 640x480 USB 2.0
QuickCam Orbit MP 1.3MP USB 2.0
QuickCam Orbit 640x480 USB 2.0
QuickCam for Notebooks Pro
LifeCam VX-1000 VGA USB 2.0
LifeCam VX-6000 1.3MP USB 2.0
LifeCam VX-3000 1.3MP USB 2.0
Xbox Live Vision (Xbox 360)
Wireless Picture Frame
Nero8 Home Media
GeForce 7400 Go
Geforce 7950 GX2
Geforce 8400 GS
GeForce 8400M GS
Geforce 8600 GT
Quador NVS 130m
GeForce 9600 GT
GeForce 8800 GT
Geforce 8400GS (G98)
Geforce 9800 X2
Geforce GTX 260
GeForce4 MX 420
GeForce FX 5200
Geforce FX 5900
GeForce Go 6150
Microline 184 Turbo
Discovery 655 or 665
Realtek 262 HD Audio codec
Realtek 268 HD Audio codec
Realtek 660 HD Audio codec
Realtek 862 HD Audio codec
Realtek 883 HD Audio codec
Realtek 888 HD Audio codec
Realtek 885 HD Audio codec
Realtek 882 HD Audio codec
Realtek 861 HD Audio codec
Realtek 662 HD Audio codec
Realtek Semiconductor Corp
S3 Graphics Chrome 440/430 Series
Sansa View Mp3 Player
Zone player ZP80
Gigabeat V2 PMC
Audio Advantage Micro
The Windows Explorer has evolved by enabling you to find all sorts of content by searching for it. Many of you have used the search features in Windows Vista (based on our instrumented data) from the start menu or from the search box in Explorer. It has been a long time since most of us could remember where everything is by carefully managing our folder hierarchy and finding things based on file name alone. We often rely on domain specific search (in music players, mail clients, photo clients) but with Windows Vista and Windows 7 we make it possible to search within a namespace and across namespaces. This post is about a new feature based on Search that allows searching across PCs and even servers in an Enterprise setting. Alwin and Scott, program managers, and Brandon, a developer, on the “Find and Organize” feature team authored this post. --Steven
Whether you’re searching or browsing, Windows Explorer is really about finding your stuff, and once you’ve found it, doing something with it (such as copying, opening, deleting, etc). For data that lives on your PC or home network, Windows 7 has invested in HomeGroup and Libraries (subjects for a future posting from our team) to provide an easier and richer experience than ever before. However, we didn’t stop there. Over the last few years, we’ve seen enterprise customers’ important content migrate towards (or aggregated in) centralized content stores, such as SharePoint. These products typically provide great features for team collaboration, document versioning and workflow management, archiving, retention policy enforcement, and other centrally-managed functionality that IT managers appreciate.
Important enterprise data is found on local machines, in a variety of centralized content stores and also beyond the firewall
Unfortunately, this has placed an extra burden on customers to learn each new content store’s user interface, often asking them to give up familiar desktop features like drag-and-drop. Given their collaborative focus, these sites grow organically and it can become hard to remember where a particular document was stored and then wade through long lists of them every time you want to get back to it. Enterprise customers have asked us for a solution that simplifies finding important content in these various data stores but without leaving their normal Windows work flows.
As we looked at this trend and the lack of integration with content management and content indexing web services, we used these guiding principles in developing a solution:
Federated Search wasn’t the only way to address these challenges. The brute force approach would have been to take our existing Windows Search indexing technology and just use it on these content stores—that index the remote content on a local PC. This isn’t a very realistic solution since it’s inefficient to have all content indexed over the network by each person’s machine, especially when the content is changing at a rapid pace and represents a large corpus. Corporate retention policies may also prevent keeping even a local index of certain sensitive data.
Fortunately, there’s a better option – Federated Search. Federated Search enables you to search a remote web service from Windows explorer and get results back that you can act on like any normal file. The largest barrier to doing Federated Search has already been taken care of too. That is, most of these content stores are already indexed on the server, or at least on some server. There are several great offerings that will accomplish this, such as Microsoft Search Server. Not only do these servers index this content, but many of them already expose search results via a standard web protocol. This is largely thanks to the prevalence of OpenSearch and RSS enabled clients (including Internet Explorer and Microsoft Search Server, among many others).
For Windows 7, we’ve added support for Federated Search using OpenSearch v1.1 and worked to make the experience a seamless one. We found this solution strikes a good balance by leveraging the strengths of content services and the strengths of local file interactions within Windows.
Using Windows Explorer, people are familiar with several important user interface and interaction elements. They know how to use the navigation pane to change what they’re looking at. They know how to scroll around, how to select an item (or several), and they know how to double-click to open them. Most people know how to right-click for context-sensitive options related to their selection, or how to find those options presented in the command bar. They know they can drag and drop items to move them around. They know how to change view modes. We hope that they know how to search their current location using the search box, and in Windows 7 we think we’ve made it much easier to discover and use the Preview Pane to make sure they’ve got the right result.
Searching a SharePoint site using the new Federated Search support in Windows Explorer
Much of the usefulness of building Federated Search into Explorer is our ability to take advantage of this knowledge and familiarity. This may seem obvious once you see it in action, but behind the scenes there’s quite a lot going on to make all of this happen. For example, some applications such as Microsoft Word already know how to work with web URLs. So opening a Word document from a web server is fairly straightforward. But the majority of applications you’ll encounter really only understand how to open files on the local machine or via standard network file sharing protocols. This includes everything from the built-in software like Notepad and Paint, to third-party software like Photoshop or iTunes.
To handle this case, we implemented a “just in time” download solution, which will download the file to the internet cache before opening an application or taking actions (like using the SendTo menu) which require local files. This lets us offer searches that are very “lightweight” from a server load perspective, where we display metadata and icons or thumbnails without ever requesting the actual file. Then if you take an action like previewing or opening an item, we will do some behind-the-scenes work to make a local copy of the file only if necessary.
That enables us to work with the existing application ecosystem without asking anything of developers. However, applications can also take steps to offer even better functionality in many cases. For example, Windows Photo Viewer has added support for non-file items. So if you open a picture result in the built-in photo viewer, it’s the photo viewer that downloads the item, not Explorer. This may not seem like a big deal, but it lets the photo viewer enable the forward and back buttons to jump to the next or previous result – and it will download that image on-demand. Starting at the PDC we began reaching out to third-party ISVs to encourage them to implement similar enhancements for Federated Search scenarios, and we will continue to offer guidance on how to best integrate with all of the newest Explorer features.
Finally, we support all the standard clipboard and drag-and-drop operations. So if you drag a Word document from a Federated Search query onto your desktop, it will be copied there. You’ll even see the familiar Windows Explorer copy dialog, with progress indication, cancel ability, conflict resolution, and so on.
But wait, there’s more! Windows Explorer is a great tool that many customers know and love. But some people use it without even knowing it. Countless Windows applications make use of what we call the Common File Dialog. This is a special Explorer window that lets you find and choose items to be opened or inserted into your current application, without ever leaving it. If you’ve ever clicked File and then Open or Save in an application menu, you’ve probably seen some version of this dialog. PowerPoint, for example, uses the common file dialog to insert pictures. That means from inside PowerPoint you can click Insert Picture, select the Federated Search link for your image repository, search for the picture you want, and then insert it directly into PowerPoint. This works for any existing application that supports the Common File Dialog, and there are a whole lot of them!
Inserting a picture into PowerPoint’s using Federated Search
Our Federated Search solution is all about simple lightweight access with a common, familiar user interface. This has a lot of benefits as we described above, but there are also cases where a server’s web interface will offer its own benefits. This might involve advanced query building, browsing, or server work-flow tasks, for example. So Windows 7 builds a bridge to these content repositories. After doing a search against a supported location, you will see a “Search on Website” button in the command bar which allows you to seamlessly send the query up to the service’s web interface in the default web browser. You’ll also see the “Open File Location” menu item when you right-click on a search result. Selecting that option will launch the web browser to the specific location in the document repository where the file is stored.
This seamless integration of Federated Search within Windows allows customers to greatly simplify their workflow for getting at remote files while still being able to easily take advantages of the advanced functionality of content repositories.
Our next challenge was to make it easy for customers to get these new connections onto their machines. It wouldn’t be practical to ship Windows with a connection to every solution in the world, so we shifted to a way that would make it very easy for any web service to deploy a connection to their specific service.
The model we came up with is similar to the way you add favorites from the web today. A web service can place a link to an .osdx file somewhere on their web page (see Channel 9’s search page for an example). The .osdx file is a simple XML file that uses the OpenSearch description document format to describe how to connect to the web service, and gives the web service some control of how the data is presented in Windows Explorer. When a person clicks on the link, Windows performs an ultra-lightweight install process that adds a search connector to that web service and places a link to that it in the Windows Explorer favorites.
If you are an administrator in an enterprise environment, you will likely want to provide some pre-installed search connectors for your users to search the company intranet or a popular internal SharePoint site for example. You can do this by deploying the search connector (.searchconnector-ms) files to your users’ machines via typical deployment techniques such as imaging, group policy preferences or startup scripts. The beauty is that it’s just a simple XML configuration file and there’s no code that needs to get installed on their machines. It’s also possible to pin one of these as a link from the Start menu through group policy. In the group policy editor look for the policy in this area: User Configuration> Administrative Templates > Windows Components > Windows Explorer. The policy name is “Pin Libraries or Search connectors to Search again links and start menu”.
Launching a Federated Search of an enterprise Intranet from the Start Menu
Of course this technology depends on having services that support it. Although there are only a few services that provide a .osdx for you today, there are many existing services that already support the basic OpenSearch requirements.
We’re already seeing positive initial reactions from enthusiasts and ISVs alike echoing that it is indeed easy to enable your service to work with our Federated Search platform. If you’re a developer and want to enable an existing web based service to support Windows 7 Federated Search, you’ll need to provide a web service that accepts an http GET request with the search terms embedded somewhere in the URL and be able to return the results as an RSS or Atom feed. These requirements are typically very easy to meet for most applications that already provide search services via a web browser.
Your web service results should include the basic RSS tags like <link>, <title>, <description>, <pubDate> to get started but there’s much more that you can include in the RSS output and customization you can do within the .osdx file to enhance the experience for the end user.
For more information, we’ve published the Windows 7 Federated Search implementer’s guide with detailed information on how to enable your data source to work with Windows Federated Search. There’s also a recorded PDC session that demonstrates how to build a Windows Federated Search compatible web service for an existing SQL database.
- Brandon Paddock, Scott Dart & Alwin Vyhmeister, Find and Organize
This post continues the discussion of Compatibility testing from our test team. --Steven
In the previous blog post "Application Compatibility Testing for Windows 7" we talked about the importance of Application Compatibility and work we are doing to engineer this in Windows 7. In this post we will examine the challenge that emerges as we consider the world wide audience that Windows serves.
This blog post will cover the following areas:
For Windows 7 we have made significant investment in application compatibility, ensuring applications that worked on Vista, continue to work on Windows 7 and we’ve also rescued some applications that were broken in Vista to work on Windows 7 (more on that later). As we’ve talked about, there are some applications that are OS version specific by design (utilities, firewalls, security, etc.) and those are not included in this discussion.
One of the biggest challenges in International Application Compatibility is what applications we test, the scale of testing, and what it means for us to say that an application “works”. For Windows 7 we are testing over 1200 applications across 25 specific markets. We have improved our coverage over Vista by adding over 300 more international applications.
We look at applications in 3 buckets.
Categories 1 & 2 are pretty straightforward. There are a known set of key applications and scenarios used around the world and we must ensure these applications function in Windows 7. Category #3 is where there is some complexity.
The applications list we build for 3rd Party Local Applications is built using a number of methods. First, we build on the list of applications we have used in previous versions Windows (XP/Vista, etc). If it worked on Vista, it must work on Windows 7.
Next we work with our teams in markets around the world to rank top applications in particular markets. It is amazing to see the diversity in application use around the world. The application testing list is based on a combination of market data where it is available, individual knowledge of markets, culture, revenue, usage and even sometimes just “word on the street”. The cultural knowledge in these markets is probably most critical to our success. For example, casual gaming in Korea is hugely popular and we need to ensure our Windows 7 testing accounts for this.
Our goal in selecting applications is to test as many applications as we can that will expose the most issues across different scenarios and markets.
These scenarios include:
Once we build the list of applications we need to test the next process is acquiring them. We acquire applications in a variety of ways but many times we have to buy an application from a retail store just as any end user would. Other methods we use to acquire applications include downloading full featured trial versions, purchasing software, and working with ISVs to acquire their applications to ensure compatibility.
Testing applications means more than just installing them and making sure they launch. Every application gets a unique test plan written for it to cover as much functionality as we can. We write test cases to cover primary and secondary application functions – for our word processing example this would include opening a file, typing a letter, adjusting formatting, save, and print, emailing a copy to someone, etc. These applications go through 6 or more test passes during the product cycle.
Now, we can’t test every piece of every application and we do run into some interesting challenges when we focus on a worldwide audience. Many applications depend on location specific information (meaning if you aren’t testing the application in that location – you aren’t likely to have the information needed). Examples include Brazilian citizen’s CPF ID, or Brazilian personal number of identification which would be required to test something like tax preparation software. We run into similar problems with SMS applications requiring active local mobile phone accounts.
Along with the core tenet of ensuring that any application that worked on Windows Vista also work on Windows 7 we have a stretch goal to “raise the bar” and make applications work on Windows 7 that never worked on Windows Vista. For Windows 7, we have some good news early in the development cycle. So far we have made over 30 applications that were “broken” on Vista work on Windows 7. This means that Windows 7 will have higher application compatibility than Windows Vista. We are continuing to push this number up. Below is a table of the # of applications by language that we have made to work on Windows 7 but didn’t’ work on Vista.
Asure Purchase/Sale/Stock Master 2008
Cyberlink DVD Suite v6
Asure Accounting Master 2008
Haufe Personal Office Professional - Haufe Formular-Manager
Compedia Timmy in English World
Compedia Moomins: The Search for the Ruby
Compedia The Puzzling Time Quest
Finson Costo del Lavoro Italian v2
Finson Falco 6
Finson Progetto Condominio
Finson Contintasca 7
Kenchako Adventure 9.0
WZ Editor 5.0
Overland LOKI: with Japanese Manual
WF-Fakturka dla Windows
Nahlik eTeacher 5
Mexico Federal Taxes Simplified SAT: Individual Taxes
IKEA Home Kitchen Planner
Along with ensuring these applications work on Windows 7 we have taken an extra step for our existing Vista customers. Of the applications outlined in the above table, 27 of the fixes we made have been back ported to Windows Vista for possible inclusion in future updates. We really wanted to raise the bar for application compatibility and go beyond just looking at Vista as the baseline.
There is a lot of information here and hopefully gives you some insight into what it means for us to make the application experience (application compatibility) on Windows 7 as high as possible for users around the world. We started out with a goal of making sure if an application worked on Windows Vista it should work on Windows 7. We have taken that further by bringing applications that never worked on Vista to work on Windows 7 and even future updates to Vista.
As mentioned before on this blog (regarding our UAC changes) and on the IE blog (regarding the SmartScreen® filter for malware), we have an increased focus to enable customers to be in control and feel confident about the software that they choose to run on their computers. Folks on this blog have also commented about the concerns they have specifically in the AutoPlay area. This blog entry addresses some of the changes that we have made to increase customer confidence when using their media and devices with Windows. It is authored by Arik Cohen, a program manager on the Core User Experience team. –Steven [Note: There was a technical problem so this post was reposted in its entirety.]
Certain malware, including the Conficker worm, have started making use of the capabilities of AutoRun to provide a seemingly benign task to people – which masquerades as a Trojan Horse to get malware onto the computer. The malware then infects future devices plugged into that computer with the same Trojan Horse. For further information about Conficker please visit http://www.microsoft.com/protect/computer/viruses/worms/conficker.mspx
In the following example for a USB flash drive that has photos, malware registers as the benign task of “Open folders to view files.” If you select the first “Open folders to view files” (circled in red), you would be running malware. However, if you select the second task (circled in green), you would be safe running the Windows task.
Infected USB AutoPlay
People are confused why they have two tasks that appear to do the same thing – and even a knowledgeable person who is careful not to run software from an untrusted source can easily make the mistake of selecting the first task. As a result, people lose confidence and don’t feel in control.
A growing attack
While presenting an AutoRun task in AutoPlay has been available since Windows XP, we have seen a marked increase in the amount of malware that is using AutoRun as a potential method of propagation. According to the Security Intelligence Report, an enterprise study by Forefront Client Security found that the category of malware that can propagate via AutoRun accounted for 17.7% of infections in the second half of 2008 – the largest single category of malware infections.
The chart below shows the increasing amount of detection reports by Microsoft anti-virus software of the class of infections that spread via AutoRun. (Note: The actual method of infection cannot be determined.)
Infection Detections of Malware that Spread via AutoRun
Infection Detections of Malware that Spread via AutoRun
Currently, disabling AutoPlay completely is the only solution for consumers and enterprises to gain confidence with the use of USB flash devices on their computer. Guidance on disabling AutoPlay is available here.
Increasing customer confidence
Windows 7 introduces key changes to AutoPlay that keep you from being exposed inadvertently to malware like Conficker when doing your common scenarios with devices (e.g., get to the files on your USB flash drive, download pictures from an SD card, etc.).
In particular, Windows will no longer display the AutoRun task in the AutoPlay dialog for devices that are not removable optical media (CD/DVD.) because there is no way to identify the origin of these entries. Was it put there by the IHV, a person, or a piece of malware? Removing this AutoRun task will block the current propagation method abused by malware and help customers stay protected. People will still be able to access all of the other AutoPlay tasks that are installed on their computer.
With these changes, if you insert a USB flash drive that has photos and has been infected by malware, you can be confident that the tasks displayed are all from software already on your computer:
Infected USB AutoPlay after AutoPlay changes
Infected USB AutoPlay after AutoPlay changes
On the other hand, if you insert a CD that offers software to install, Windows will still display the AutoRun task provided by the ISV during their media creation process. For example:
AutoPlay for a CD that offers an AutoRun Task
AutoPlay for a CD that offers an AutoRun Task
You will first see this updated AutoRun experience in the Windows 7 RC build, and we will be bringing this change to Vista and XP in the future.
We are working with our ecosystem partners to help mitigate situations where this AutoRun change will have an impact on them.
CDs and DVDs (including CD emulation), where the IHV specified AutoRun task authored during manufacturing, will continue to provide the AutoRun choice allowing customers to run the specified software. IHVs of generic mass storage devices should expect that people will browse the contents of the device to launch any software. The new behavior will allow customers to continue to use AutoPlay (including all Windows and ISV installed tasks) to access their media and devices while not being presented with tasks from malware. Additionally, device classes, such as portable media players and cell phones, now support Device Stage™ on Windows 7. Device Stage offers the IHV a multifunction alternative to AutoPlay where they can present links to software and common tasks, and provides additional features as you use the device.
As you try out the Windows 7 RC, we hope these changes will make you feel more confident and in control when using your media and devices.
There’s been a ton of interest in how we have improved user account control (UAC) and so we thought we’d offer a quick update for folks. We know most of you have discovered this and picked a setting that works for you, and we're happy with the feedback we've seen. This just goes into the details on the choice of defaults. --Steven
In an earlier blog post we discussed the why of UAC and its implications for Windows, the ecosystem, and our customers. We also talked about what we needed to do moving forward to address the data and feedback we’ve received. This blog post will provide additional detail on our response and what you can expect to see in the upcoming beta build in early 2009.
As mentioned in our previous post, and your comments supported this, the goals for UAC are good and important ones. User Account Control was created with the intention of putting you in control of your system, reducing cost of ownership over time, and improving the software ecosystem. It is important not to abandon these goals. Instead, we want to address feedback we’ve received and build on the telemetry we have using those to improve the overall experience without losing sight of the goals with which we agree.
For those of you using 6801 you have started to see the benefits of prompt reduction and our new and improved dialog designs. You also have seen our efforts to give the user greater control of their system – the new UAC Control Panel. The administrator now has more control over the level of notification received from UAC. Look for the UAC Control Panel to appear in Start Search, Action Center, Getting Started, and even directly from the UAC prompt itself. Of course, the familiar ways to access it from Vista are still present.
Figure 1: UAC Control Panel
The UAC Control Panel enables you to choose between four different settings:
We know from the feedback we’ve received that our customers are looking for a better balance of control versus the amount of notifications they see. As we mentioned in our last post we have a large number of admin (aka developer) customers looking for this balance, our data shows us that most machines (75%) run with a single account with full admin privileges.
Figure 2. Percentage of machines (server excluded) with one or more user accounts from January 2008 to June 2008.
For the in-box default, we are focusing on these customers, and we have chosen number 2, “Notify me only when programs try to make changes to my computer”. This setting does not prompt when you change Windows settings (control panels, etc.), but instead enables you to focus on administrative changes being requested by non-Windows applications (like installing new software). For people who want greater control in changing Windows settings frequently, without the additional notifications, this setting results in fewer overall prompts and enables customers to zero in on the key remaining notifications that they do see.
This default setting provides the right degree of change notification that a broad range of customers’ desire. At the same time we’ve made it easy and readily discoverable for the administrator to adjust the setting to provide more or fewer notifications via the new control panel (and policy). As with all of our default choices we will continue to closely monitor the feedback and data that come in through beta before finalizing for ship.
--UAC, Kernel, and Security program managers
Microsoft has been a global software company for a long time and has always put a lot of effort into engineering our products for a global customer base. It is also an area where the engineering is complex—probably a lot more complex than many might think—and one where we are always trying to learn and improve. Building global software is a responsibility for everyone on the team. We also have feature teams dedicated to developing both global and market specific features—whether it is font handling or doing East Asian language input as two examples. We of course have a significant engineering effort that goes into localizing (“translating” is not quite accurate) Windows into nearly 100 languages. Julie Bennett represents the global development and localization teams and she and John McConnell on her team collaborated across the team to author this post that provides an overview of engineering for a global market. --Steven
Many of the readers of the e7 blog are located outside of the United States or speak a language other than English, so we thought it would be useful to share the international and multi-lingual improvements in Windows 7. Our goal for Windows 7 is to deliver exciting features that benefit users worldwide as well as features that make Windows feel local to every user. Like Windows 7's focus to improve the fundamental scenarios of performance and reliability, we improved our processes to allow us to deliver a great customer experience in every language and every country we serve, including delivery of Windows 7 as close to simultaneously as possible worldwide. This blog entry discusses some of the new features and improved processes that we believe make Windows 7 a great worldwide release.
The international features of Windows 7 are pervasive across the system, from such low-level aspects as the supported characters in NTFS file names (now upgraded to match Unicode 5.1) to such high-level aspects as the selection of backgrounds and themes (now including locally-relevant photos). But there are certain features which are intrinsically critical for proper support of the world’s many languages and cultures, and we will describe some of those here.
Language and writing are at the heart of any culture and thus support for fonts is essential to supporting international users. Windows 7 significantly increases both the range and quality of fonts. We have added fifty new fonts:
As you might guess from the font names in the above table, many of the new fonts are for non-Latin scripts. In fact, Windows 7 will be the first version of Windows to ship with more fonts for non-Latin scripts than for Latin-based scripts. One major area of improvement is for the languages of India. To the nine (9) fonts for Indian languages that shipped in Vista, Windows 7 adds forty (40) more. Windows 7 will now include multiple fonts (often in multiple weights) for each of the official languages of India.
Aparajita: A New Devanagari Font in Regular, Bold, Italic and Bold-Italic
Besides new fonts, we have also improved many of the existing fonts. For example, we have added over two thousand (2,000) glyphs to Consolas, Calibri, Cambria Bold, and Cambria Math. But the most dramatic improvements have been to some of the non-Latin scripts. For example, Windows 7 does a much better job rendering the common Lam-Alef ligature in Arabic (see the illustration below) and in the placement of vowel marks.
Left: Lam-Alef Ligture in Vista Right: Lam-Alef Ligature in Windows 7
Changes to fonts (even clear improvements) are always tricky because of backwards compatibility issues. For example, if a character changes width or position, it may cause existing documents to reflow (repaginate), which is unacceptable. Therefore, whenever we change a font, we must run extensive verification tests against the changes to ensure the font metrics and other tables are unchanged. In the case of the Lam-Alef fix shown above, we discovered that there were existing applications that relied on the (undocumented) order of the glyphs within the old font. These applications would break if we simply replaced the glyphs. The font team worked closely with the international application compatibility team to ensure that changes we made did not affect the order of glyphs within the font, thus providing backward compatibility.
With so many new and expanded fonts for Windows 7, we also wanted to help users manage their fonts more easily. For the first time in years, we have done a complete overhaul of the font control panel.
The first picture below shows the font control panel with the large icon view. The most obvious change is that the font icons now convey much more information about the appearance of the font. The content of the icon gives a hint as to the glyph repertoire of the font. The style of the icon matches the style of the font. Non-Latin fonts show typical glyphs from the script for the font to see how it is designed. A more subtle change is that some font icons are faded to indicate fonts that are installed, but hidden. Hidden fonts will not show by default in the ribbon and font dialogs. Users can now use the font control panel to tune the fonts that they regularly use. By hiding fonts they never use, users can simplify choosing the correct font within applications. By default, only fonts supporting languages that can be written with the users installed input locales (keyboard layout plus language) will be shown. For example, users with English and French input locales will see only the Latin fonts, whereas users with the Japanese input method installed will see only the Japanese fonts. Users can override these defaults by right-clicking on any of the fonts in the control panel. Hidden fonts are still installed so an existing application that uses a hidden font will behave identically.
Font Control Panel with Large Icon View
The next picture below shows the font control panel with the detailed view. Now users can see much more information about the font. For example, the user can sort fonts by style, whether they are hidden, and information about the creator of the font. Font files generally contain information only in the design language of the font (e.g. a Japanese font might contain only information in Japanese). In Windows 7, we needed a solution that would work for all languages and for all fonts, so we created a hybrid approach that combines information from the font itself with metadata (an XML file that provides the information about the fonts on the system).
Font Control Panel with Detail View
Windows 7 has increased opportunities for personalization. New themes, backgrounds, and sounds make it easy to customize Windows 7 to match your personality. To the extent that our preferences are influenced by our language and location, Windows 7 reflects this with the introduction of Local Packs. Local Packs provide customized Windows 7 visual themes for a specific region. These visual themes contain locally relevant wallpaper images, custom aero glass colors, and regional sound schemes. Windows® Internet Explorer® Favorites and RSS feeds may also be updated when the Local Pack is activated on an end user’s computer. For example, adding and enabling the Local Pack for France will add a market-customized theme for France to the end user’s Personalization control panel and a number of links to useful French Public Sector websites and RSS Feeds to the user’s profile.
Customized Themes in the Personalization control panel
The Local Pack content provides users with seamless local experiences right out of the box. Users are never exposed to Local Packs per se, they just select their Location as normal during Windows Welcome, and appropriate local content is exposed to them based on that setting.
Users looking for visual themes for other countries, or indeed any other areas of interest, can find them on the Windows Online Gallery, which is accessible via the “Get more themes online” link in the Personalization control panel.
Other new features include five (5) new locales (bringing the total number of locales supported to two hundred and ten (210)), twelve (12) new input locales, and improvements to sorting for traditional Chinese characters. Also, we have generally updated our system databases to the latest version of the Unicode Standard (5.1). There are also interface improvements that should allow developers to create better globalized applications. Extended Linguistic Services (ELS) is a cool new feature we describe below in the International Timeliness and Quality section.
Perhaps one of the most important improvements outside the core international features has been in Search, which now recognizes more languages. For example, Windows 7 desktop search now recognizes Russian morphology (the rules for single and plural, tenses, and case). This means that searches for a particular word in Russian will now match not only that exact word, but also the common variations of the word, yielding significantly better results.
In previous versions of Windows, final delivery of every language to every market took several months. For Windows 7, we changed how we worked on international releases to significantly shorten this delta so that all users worldwide can enjoy Windows as simultaneously as possible. This goal had far reaching implications on how we perform our work as engineers and on how we interact with partners and customers during our public testing phases.
To understand our approach, we should first explain two important concepts: localization and globalization.
Localization is the process of adapting the user experience into another language. Beyond the translation of strings, it can also include activities such as resizing dialogs and mirroring icons for right-to-left languages, such as Hebrew and Arabic. Localization bugs, such as the mistranslation of a menu item, are defects introduced during this process.
Globalization, on the other hand, is the process of producing a product that works well in every country no matter the user interface language setting. A globalization bug may be as simple as showing a UI element in the wrong language and as complex as not properly handling right-to-left scripts. Globalization bugs are inherently more serious than localization bugs as they usually affect many or all languages and often require re-thinking the technical design. In past Windows releases, repairing globalization bugs contributed to the necessity of the long release deltas. For Windows 7 we worked to prevent, find, and fix globalization bugs as early in the development process as possible.
To prevent common globalization bugs, pseudo-localized builds were created. Pseudo-localization is a process that creates a localized product in an artificial language. That language is identical to English except that each character is written with a different character that visually resembles the English character. Except for being entirely machine generated, we create the pseudo-localized builds exactly the same way as we create the localized builds. Because even monolingual US software developers can read pseudo-localized text, it has proven to be an excellent way to find globalization problems early in the development cycle. In the Windows 7 beta, some UI elements were still in their pseudo-localized form, causing some interesting theories about what the meaning might be. We hope we have solved the mystery with this blog post. :-)
Control Panel Dialog in Pseudo-localized Windows 7
Beta is always an exciting time for us as it is our first real chance to hear from you about our efforts. We are thrilled that people from over one hundred and thirteen (113) countries downloaded the Windows 7 Beta. With such a large and diverse beta program, we must have highly scalable processes to gather and incorporate your feedback. In Windows 7, we are very excited about some new approaches we took here.
In the past, localization languages for Windows beta releases were selected for a mix of pragmatic reasons. While this ad hoc approach had benefits, too often we found that serious globalization defects were not reported because they did not manifest in the chosen languages. For the Windows 7 Beta, our priority was to find globalization bugs and therefore we have concentrated on four languages (plus English) that experience has shown are most likely to find specific types of defects:
By concentrating on these four languages during Beta, we maximized our chances to find and fix the globalization bugs that affect many languages. This in turn gave us more time to improve the localization of all languages before we release the actual product. The pictures below show two bugs found during Beta that illustrate the advantages of focusing on these pilot languages.
Globalization Defects Found During Windows 7 Beta
In addition to our goal of finding globalization bugs via these languages, we also asked some of our OEM customers to provide feedback on the language aspects within their manufacturing processes. Since many of the OEMs are located in East Asia, we also localized Windows 7 Beta for Simplified Chinese, Traditional Chinese, and Korean.
In part because of the engineering process improvements described above, we were able to deliver more language packs for Windows 7 RC than we have ever been able to do in the past for Windows. For those of you running the Ultimate version of Windows 7 RC, you will have noticed the following thirty-two (32) Language Packs available for download on Windows Update:
32 Windows 7 RC Language Packs on Windows Update
One thing we will do differently in the future is to ensure that all languages available at Beta are also available at RC (e.g. not including Hindi for Windows 7 RC). We will correct this for future versions.
With Windows 7 beta localized into five languages and globally enabled for hundreds more, we received beta bugs from customers all over the world. We rely on these bug reports to help us improve Windows 7, so we devote much time to reading customer bug reports to determine product issues. Because bugs come from worldwide customers in many languages, we look for ways not only to understand their feedback, but also to address it as quickly as possible. The faster we can understand the issue, the better chance we have of addressing the feedback. As we receive bug reports in all the many languages that our customers speak, this has sometimes posed quite a challenge.
In the past, we have handled multilingual bug reports using manual processes, where individual bugs were examined and then manually translated one-by-one for appropriate follow-up by the feature team that owned the affected component. This is a time-consuming and error-prone exercise that scales poorly to a program as large and diverse as the Windows 7 beta. In the worst case, valuable international feedback has missed the window to affect the final product, and thus slipped to a Service Pack or subsequent release.
In Windows 7, by using the language detection API in the new Extended Linguistic Services (ELS), we have been able to automatically detect the language of customer bugs as they are reported. ELS functionality is new for Windows 7 and available to any developer who wants to leverage advanced linguistic functionality in the operating system. Beginning in Windows 7, developers may use ELS to provide language and script detection of any Unicode text, as well as transliteration to map text between writing systems. To use these Windows 7 services and all further services that we will add in subsequent releases, developers need only to learn one simple and unified interface. The ability to detect over one hundred (100) languages is available for all Windows 7 application developers, and we are happy to be able to apply this functionality to triage and handle beta feedback you send us from around the world. We use our own international developer functionality to improve our ability to respond to customer issues globally.
Once we have detected the language, we take the resulting text and use the machine translation support that is available online from Live Translator. This allows us to translate the text to English to get a sense of your feedback. Our engineers can then search our feedback database for specific features or areas of functionality. This also helps us in our efforts to ensure international application compatibility, as we can learn about potentially problematic international application experiences as soon as customers report them. Machine translation does not provide a perfect translation, but it does allow us to determine which issues might require further investigation. This in turn allows us to hear and respond to customer issues with a much faster turnaround time than we have had in previous releases, which means better quality in Windows 7 when we release it to the world.
By the end of Windows 7 Beta, we had used this process to translate 35,408 issues and comments submitted using the Feedback tool.
The end result of the work to improve globalization and localization quality is reflected in the announcement that all fully localized releases of Windows 7 will be available within two weeks of the initial release wave with all languages available in October. We hope (and believe!) end users will find the overall quality of these releases to be the best ever.
36 Windows 7 language releases available in October 2009
In addition to the 36 languages that will be released in October, there will be additional languages available for download as Language Interface Packs (LIPs) onto any Windows 7 edition as part of the Local Language Program (LLP). The LLP is a partnership with governments, universities, and language experts from around the world. (You can find more information on the LLP at http://www.microsoft.com/unlimitedpotential/programs/llp.mspx.) Work on a LIP starts at RTM and continues for many months based on the schedules of our partners. Two (2) LIPs will be available for download when Windows 7 is available in October – Catalan and Hindi. Additional LIPs will become available for download over the following months based on the schedules of our partners. We are happy to have improved the delivery time of the first 38 languages (36 + 2 LIPs) and recognize that future releases are an opportunity to improve further. Creating a track record of dependable release schedules on our part will help everyone around the world plan better for a more unified release timeline.
More information about Extended Linguistic Services (ELS) and other cool new features of Windows 7 are available on-line on MSDN. In particular, you can download the Windows SDK for Windows 7 and read about what is new in the ‘International’ section. Also, the new Go Global Developer Center on MSDN has a wealth of information about international technologies.
If you want to send us feedback, please comment on this blog entry or use the Feedback button in Windows 7. We love to hear from you (in any language).
-- Windows International Team
Delivering a new release of Windows includes a major effort to insure that applications continue to function as well on the new release as they have on the previous release. At the PDC we talked about some of the new areas of Windows Vista that reduced this level of compatibility, such as changes we made around the OS security model. With Windows 7 we renewed our engineering efforts to maintain compatibility. As with device testing, compatibility testing is an effort that spans the entire engineering organization, though we also have a group that is dedicated to this effort. This post was authored by a set of folks and coordinated by Grant George, the corporate vice president for testing in the Windows Experience team. --Steven
We have taken a very proactive approach to Application Compatibility and our process starts from the moment we first plan our product schedule and design and check in code for Windows and runs through all of our engineering processes and disciplines leading up to our final release (and beyond).
Our main Application Compatibility goal for Windows 7 is to make sure that most all applications which work on Windows Vista will continue to work seamlessly on Windows 7. We do have to be careful about making this claim to be universal because there is a class of applications that are always updated in tandem with a new Windows release. These applications are primarily system utilities, diagnostics, and security software—the common thread is that they make assumptions about the underlying implementation of Windows internals and thus require updates. We carefully coordinate with a large set of ISVs who provide these applications. This was talked about earlier this month as we announced our Ecosystem Readiness Program, which we will discuss more below.
At the start of our product cycle we review our new features and changed designs to ensure that every element of Windows 7 has Application Compatibility in mind. Our engineering process includes automated quality checks to assure public APIs don’t change, and our test engineers have the right tools, engineering time and information that is used to find application issues as early as possible in our development cycle. Telemetry information is collected to assess and prioritize the breadth of applications our users depend on, paired with market data and install base information, across a wide variety of software categories to make sure they work as expected in our new OS version.
Below we expand on how we deliver on this goal.
Rich telemetry from external usage and the broad software marketplace helps us to provide a list of the most popular and critical applications, updated frequently throughout our development cycle. Our engineers then acquire these applications and build automated tests that verify each one works as expected on Windows 7.
Changes that could impact application compatibility are followed up closely, for example Legacy Code Removal which involves removing code that existed in the previous release of the product is strictly guarded and tightly managed and should not happen without proper documentation and engagement with the impacted parties. For example if we need to deprecate an API call, the documentation for this API will be updated and we will wait until the impact of the removal is minimal as indicated by telemetry data unless it is required sooner to fix a security issue.
Throughout the development process we are running tests in the background creating an ongoing validation of new code relative to application compatibility. As code is getting ready to be checked into the main build, if a compatibility failure is detected in an automated regression test the checkin is halted. At that point the code is scanned for known compatibility issues and if an issue is detected the developer is asked to fix the problem. Of course we also develop new tests throughout the course of developing Windows 7 in order to broaden our coverage of third party software.
We have several teams dedicated to application compatibility as part of the Windows development effort. These teams provides guidance on how to build in application compatibility, provides data on application dependencies, and information on what applications may be impacted for any particular change we make in the Windows platform. These teams also reviews new feature designs, as well as other planned changes, to ensure that the engineering team has fully taken application compatibility into account so that we achieve the tenet of "keeping applications working on Windows 7".
In addition to working with the internal Windows engineering teams, we also reach out to 3rd party developers writing Windows applications to ensure these partners have all the information they need to make their solutions fully compatible with Windows Vista and Windows 7. Furthermore if we do uncover any issues that may need to be resolved by the 3rd party developers, we collect all the information, resources and guidance and engage in a conversation with those external developers to help them understand and fix these issues.
We recently announced the Windows 7 Ecosystem Readiness Program. This program provides partners with access to Windows 7 builds and tools they need to test solutions for Windows 7, Windows Vista, Windows Server 2008 and Windows Server 2008 R2. The Ecosystem Readiness Program also facilitates testing multiple components of the ecosystem together to improve the overall user experience. Rather than just focusing on getting a specific OEM product, software application, or hardware device certified, we will be bringing multiple components together to verify a rich user experience that delivers quality, reliability, and performance as well as innovation through new feature adoption.
As mentioned earlier we use telemetry data and market share information to choose the applications we directly test on as part of our compatibility efforts, below you can find the a sample of the Consumer Scenarios we focus our testing on:
To view a full list of Application Categories covered please take a look at the Appendix section at the end of this post. While we would love to provide the detailed list of products, several of the sources of this information are based on proprietary research from third parties.
Another very important set of technologies we test on are the middle tier technologies like Java, the .Net Framework, etc. to make sure the applications that use these technologies continue working as expected
In addition to 3rd party stand alone applications we test a subset of OEM pre-installed software and their inbox applications for compatibility. The software tested come from the engagements we have with our OEM partners and their submitted installation images. These images are tested on clean installations of Windows 7 and upgrades from Windows Vista on OPEM standard hardware. This level of coverage allows us to best replicate the initial experience with Windows 7 for many of our customers. Because many of these applications are closely aligned with the OS, hardware and drivers, it is not unusual for an OEM to provide updates to this software with a new OS release.
In addition to the above mentioned testing approaches, Microsoft IT maintains a software portfolio of approximately 1,500 applications. These applications must be tested prior to software deployments inside Microsoft.
Microsoft IT developed an application-tracking method that simplified the process of selecting applications for sample-based testing. By identifying groups of applications that have similar data processing, controls, underlying technology, and methods, Microsoft IT is able to test approximately 4 to 6 percent of the total applications and gain a reasonable assurance of compatibility for all.
For more information you can read the LOB Application Compatibility Technical White Paper.
Part of our testing process includes the creation of scenarios that we validate on 3rd party applications. We approach this by verifying the intended functionality of the application while focusing some of our attention to changes in the OS, new functionality and risky integration areas. Manual and automated test passes are scheduled to cover the identified scenarios and to verify the user experience. We cover the applications on different sets of hardware and make sure that that we cover a lot of different configurations, x86, x64, Intel, AMD, touch and multi -touch machines, etc.
We use specific categories when measuring the compatibility of the applications we test on:
We will cover these categories in more detail in subsequent blog posts about how we manage application compatibility.
As part of our testing we do find issues that may need to be resolved by the 3rd party developers at the companies that develop and sell these applications. We collect all of the relevant technical information, resources and guidance and engage in a conversation with those external partners to help them fix the issues. Of course we also engage them in our technical beta programs so they can test Windows 7 compatibility with their products at the same time we do.
Application compatibility is very important to the Windows team. We are constantly working to improve your experience with applications as you move from one release of Windows to the next. We encourage you to try our Windows 7 beta release to experience the improvements in the application compatibility space and we want to hear your feedback.
It is worth mentioning the work we have done from an end-user perspective to assist in application compatibility. Many failures of application compatibility happen at install time and to assist with this, in Windows 7 we have improved the detection of failed installations and provided a step by step wizard which will help to get an application’s “compatibility mode” correctly set. We also provide real-time problem reports and solutions that can help you locate an updated version or patch. Many of you might have experienced this when trying to run Skype after an upgrade or install the current version, and in both cases you were automatically referred to the Beta download site.
Microsoft Covered Consumer Scenarios & Application Categories
Our list includes the top 50% best selling applications in the past 24 months, some of this data is collected and aggregated from several well-known third party research information providers.
Even with the pictures and videos so commonplace on PCs, many of us spend most of our time looking at and interacting with text. Yet few of us stop to think about the depth of technology required to render text well and that this is an area that continues to benefit from improved technology in displays, graphics cards, as well as the APIs available to developers. In Windows 7, The support for text and fonts in GDI continues to provide the foundation for compatibility and application support. Building on the foundation of the modern DirectX graphics infrastructure, Windows 7 enhances the text output available to developers with DirectWrite. This is a new API subsystem and one that over time you will see adopted more broadly by applications from Microsoft, independent software developers, and within Windows itself. This post will also talk about improvements to ClearType and the Fonts, both available as part of the improvements to the GDI-based text APIs. This work was introduced at the PDC (pointers towards the end of the post). This post is by Worachai Chaoweeraprasit, a development lead on our Graphics feature team. --Steven
One of the high-level goals of Windows 7 is to have even better graphics – graphics with higher fidelity. To that end, my team is looking into how to improve one of the most basic graphic elements in Windows, and that is text – the thing that’s always right in your face, but we hope you’ll never actually see it.
About 80% of the time people spend with their PC is to either read or write. This should come as no surprise when you realize that text is essentially how the machine talks back to you, and until we have a technology that would allow it to interject thought directly into our brains, text would probably continue to be the way we receive information from the computer screen.
Studies have shown that good text leads to better productivity. Essentially we are wired as human to be incredibly good at capturing words and making a smooth, rapid transition between them – the basis of reading. We’re so good at it that we can do it unconsciously with incredible speed given that the text is optimized for that process. This might explain why many can sink in to a good book for hours, but some quickly become tired after staring at the computer screen for a while. Any visual-related factor that could disrupt the reading process effectively slows us down. Good text, therefore, is text that is tuned to support the human reading process with minimal distraction possible.
The evenness of the white surrounding each letter, word, line, and paragraph plays a huge role in keeping the pace of reading while the black elements holds our attention together. A line too long, a word too tight, a paragraph too uneven, any of these conditions take us farther and farther away from the message being delivered but closer and closer to the mere medium delivering it. The art of text is essentially to make the actual text itself disappears before your eyes, so that the ideas it delivers reappear in your head. The study of how to prepare proper text is known as typography. And, as a typographer would say: good typography is not to be seen; only the bad ones are. As a platform, the role of Windows is to deliver great presentation of text and offering software developers great tools for creating the best presentation possible in the context of the software they develop.
People tend to develop habits and often over time these become the preferred way of getting things done. The more mundane the activity is, the easier we become attach to it, and the harder we’re willing to change. When it comes to text on your screen, the same screen you look at days in and days out. It could quickly become awkward if that completely changes overnight – even for the better. So, how do we go about improving on what we all become so used to? We want to make sure to support what is there and improve it, while supporting existing methods. But, before we get to understand the improvement, let’s first take a closer look into the current implementation really is and what challenges it presents over the years.
The current implementation is the product of text rendering design based on device pixel. The dimension of text at a certain size eventually translates into a fixed number of pixels in horizontal and vertical direction on the device surface. A 10-point text would translate to roughly 80 pixels height on a typical printer device of 600 dpi, while the same text would merely acquire 13 pixels on a 96 dpi monitor. This physical screen condition was hardly adequate for the quality we’re seeking for good text on screen.
Fortunately, the advent of ClearType during the past decade has largely improved the clarity aspect of quality. ClearType leverages the anatomy of the LCD pixel structure and takes advantage of the human visual system to distribute the energy typically emit to a whole display pixel, across the neighboring sub-pixels in the LCD’s typical 3-color channels making up each individual pixel, to create the visual illusion of higher resolution raster quality on a lower resolution device. As the result, ClearType text looks significantly sharper than the typical text on an LCD display, mitigating a large portion of the quality problem on a display technology that would become hugely popular a few years later.
Another pleasant design of the original ClearType in Windows was that it has improved the clarity of text without breaking application compatibility – that is, it doesn’t change the actual size of each individual glyph in either direction, nor did it change the distance between the two adjacent ones. This is the reason one could turn it on or off at will without having to “store” the selected option in the document or application. It is entirely per-user rendering preference. In Windows 7 we also improved the ClearType Text Tuner in keeping with our theme of being in control of your PC experience, by providing even more granular choices when tuning ClearType (and of course you can still turn it off).
But like many other things in the world, the coin comes in two faces. While it is able to preserve backward compatibility, it is limited by its own leverage unable to advance the state of the art. The width and height of the individual glyph and the nominal distances between the adjacent two remain fixed to the rounded number of screen pixels at a given size.
One of the graphics improvements we made in Windows 7, therefore, is to move from the physical pixel model of the past, and instead creating a new design around what we call the “device independent pixel” unit (or “DIP”), a “virtual pixel” that is one-ninety-sixth an inch in floating-point data type. In this model, a glyph (or any other geometric primitive for that matter) can size to fractional pixels, and be positioned anywhere in between the two pixels. The new ClearType improvement allows sizing and placement of glyph to the screen’s sub-pixel nearest to its ideal condition, creating a more natural looking word shape and making text on screen looks a lot closer to print quality.
The following figure shows the side-by-side comparison of the same word between the original or today’s ClearType (above) and the Windows 7 improvement – Natural ClearType (below), which does require calling the new APIs to render. Notice the width of the letters in the word and the spacing between them, as well as how the more consistent width and spacing improves the overall appearance of the entire word. Note that all the letters are placed with its nominal spacing and there is no kerning adjustment being applied here. A great article by Kevin Larson – a researcher in the Advanced Reading Technology team, discusses in details the scientific aspect of word recognition.
The ability to be more precise in approximating the screen placement of natural text also lends itself to a very nice side-effect, and that is the fact that text can now be placed on the line with no regards to the actual display device’s resolution. It means a UI designer can design an application UI knowing it’ll look the same on all other screens as it appears on his or her screen regardless of what type of display device the users might have. This fact is also particularly handy for software localization where the translated text produces the same layout everywhere.
This improvement could also offer a more realistic view of a print document on screen, or make the screen document looks closer to its print counterpart. It could also improve the quality of document zooming. Imagine document zoom that could go in and out in the same manner as what you would see when pulling the actual print page closer and farther away from your sight. It could mean a more joyful experience for online reading.
The Font is the heart and soul to typography, much like photo is to photography. A lot more fonts are shipped with Windows these days while even more are developed around the world. Windows Vista shipped with 40% more fonts comparing to Windows XP. Windows 7 is expected to ship with 40+ new fonts, just to underscore this trend. We’ve also added some additional viewing/categorization capabilities using the Windows 7 Explorer to improve working with a large library of related fonts.
The default common controls’ font dialog and the font chunk in Windows 7 Ribbon are also updated to be more intelligently selective of what fonts to be present to the user of the current user’s profile. Depending on a number of settings including the current UI language, the user locale, and the current set of keyboard input locales, the font list hides fonts of languages not typically used by the user of different culture and locale. For example, all the international fonts are automatically hidden away from a typical English user to reduce clutter and promote better productivity in common system applications such as NotePad, WordPad and Paint. Third-party application utilizing the Ribbon or the common controls’ font dialog could also have the same benefit. The user still retains the option of selecting any desired font back to the view by explicitly marking it in the Windows 7 Control Panel’s Font applet.
This growth, however, introduces some new opportunities for improvement. We’ve long treated fonts as system-wide resources. It gets “installed” on the machine and kept in a single flat namespace managed by the core part of the operating system. It may be interesting to some that the font named “Arial Black” isn’t really in the same grouping as “Arial Narrow” or “Arial”. This is because as far as the operating system is concerned, they are just different fonts with different names. And because font is uniquely identified by its name, you can’t have multiple versions of the same font at the same time.
Because font is system resource, non-traditional usage of font such as font embedded within the document, and font used exclusively in an application is done through the mechanism known as private installation, which involves making sure the font name is unique before installing it programmatically but doing so by hiding it from others to see. Private font is just like font installed publicly as far as the operating system internal is concerned.
An important improvement in Windows 7’s new font system is the notion of “font collection” which allows partitioning of fonts sharing the same usage into a separate namespace. The system collection is similar to what exists today and is created and managed by the system whereas custom collection can be created and managed, as many as needed, entirely by the application program. This allows document to have its own set of fonts local to it, and third-party application or plug-in to ship with its own font used exclusively within the program. This partitioning not only reduces unnecessary system-wide font update and allows update to happen only locally as needed, it also allows access to multiple versions of the same font in different collections.
The new font system also improves the way fonts are organized within the collection. It supports the notion of weight-width-slope variation where fonts with the same stylistic root but vary in its weight (thin, light, bold, black, etc.), width (wide, narrow, etc.), or slope (italic, oblique) are grouped together in the same font family. For instance, “Arial Narrow” becomes a variation or face in the “Arial” family. This grouping model is advocated by the CSS recommendation.
Fonts also represent art and artistic expression. The technology helping create font is therefore the artist’s tool of expression. An important technology called OpenType emerged during the past decade. It enables new ways type design can be realized. OpenType allows designer to define how glyphs interact and transform in stages. The designer then exposes this function as an executable unit known as the “font feature” for application programmable access.
OpenType was an offshoot of the TrueType Open technology Microsoft developed in 1994-95. The TrueType Open technology added the GSUB, GPOS, BASE, JSTF, and GDEF tables to the TrueType format. The primary usage at the time was to help with the creation of Arabic font due to the inherent complexity of the task. Microsoft chose to rename the technology to OpenType in 1996 and Adobe added their CFF glyph outline format to the technology in the same year. Today OpenType is used to improve readability of text as well as to express new and exciting type design in various languages.
However, despite its long-time presence and availability, the usage of OpenType in the Windows world remains largely in specialized programs. The Windows native graphics system has not fully embraced OpenType for its mainstream usage of text. This absence discourages many designers as there is no standard way in Windows to test the feature they produce. Likewise, its limited exposure doesn’t encourage discoverability for mainstream application developers. Improving this and transitioning to this improved rendering technology is a multi-step and multi-release investment done so as to maximize the benefit while minimizing the disruption that might be introduced as incompatibilities. Windows 7 takes another step on this path. We know for many that care deeply about this area there is a strong desire to move faster. We are doing our best to balance the speed of transition with the desire to maintain compatibility.
Windows 7 new text system not only uses available OpenType features internally but also allows access to any feature made available in the font in the high level programming interface, making it easier for application developer to discover and exercise the font feature in mainstream scenario. Windows 7 also ships with a brand new OpenType font “Gabriola” developed by a well-respected typographer John Hudson. Gabriola makes heavy use of contextual letterforms and offers an unprecedented number of stylistic sets for different usages of the font in different occasions. The figure below enumerates all stylistic sets available in this font; notice the subtleties and not-so-subtle way to distinguish each stylistic set.
The figure below also demonstrates the power of contextual letterforms in the eighth rendition of Gabriola’s stylistic set (“ss07”) to produce different ways the same word is rendered depending on where it’s at in the line.
Rendering text is complex and involved, even though it seems like something that should be straight forward. There are probably hundreds of ways to format text in a document and often many paths that ultimately yield the same results. HTML/CSS is a complex standard and is a great example of the richness of how text may be formatted and typeset. Underneath the formatting logic lies the language requirement – the rule of writing for the language. Windows has long been supporting Unicode – another complex standard for global data interchange. Windows supports an increasing number of Unicode script in every single release. The mapping from the input text to the final glyphs in the font requires intricate transformation, which involves parsing of font data and analyzing the language writing pattern. Once the glyph is finalized, it is then rasterized, merged and filtered into the final visual on the display device.
Due to this staging nature, different types of applications require different support from the text system. While a typical application such as the legendary “Hello world” type application may be satisfied with only the ability to get some text out showing to the user. The same level of support is hardly adequate for document preparation system such as Microsoft Word and Adobe InDesign. Some of the more mature application code bases may also have to deal with different graphics systems. This makes it harder in practice for a text system that tie to a particular graphics model to really be widely useful across the wide variety of applications in the Windows ecosystem.
It became obvious to us early on during the planning stage of Windows 7 that text processing is not homogeneous, and different types of applications have different needs and requires different levels of support. The appropriate level of programming access to the text functionality is as important as the functionality itself. The new text system in Windows 7 is assembled into a self-sufficient system called DirectWrite. The API is provided in four layers – the interfaces for font data, rendering support, language processing, and typesetting, each built upon the others with the lower layer makes no requirement to the upper one, and none depends on a specific graphics model. To illustrate the latter point, the figure below shows a sample application that uses the new typesetting interface and language processor while the final rendering happens as an extruded filled 3D geometry from the 2D graphics environment also new to Windows 7 called Direct2D. Both systems were introduced in PDC 2008 as the new graphic foundation in Windows 7.
DirectWrite preserves developer’s investment in existing technologies such as GDI and GDI+ in three important aspects. First, the previously described layering design of DirectWrite allows for the clean separation between the two fundamental processes of placing and rendering of text. It enables applications to use DirectWrite to place text while having it rendered onto traditional graphic surfaces such as GDI and GDI+. The reverse scenario in which the application may use GDI to place text while having it rendered through DirectWrite is also naturally supported. The second aspect of compatibility comes from the fact that DirectWrite also supports all existing methods for placing and rendering text found in GDI. A DirectWrite application can use DirectWrite to place and render text in the same manner as GDI does without actually using GDI. Text placed and rendered under this compatibility mode is indistinguishable from GDI text from the user’s point of view, and as such preserving existing layout of application UI and text document. Lastly, DirectWrite exposes a set of APIs that interoperate with GDI. An application selecting a GDI font object can turn it into a DirectWrite’s font object and vice versa. Since the font system is at the low end of the DirectWrite API layer, it provides a natural interoperability point that is fundamental enough to ensure high degree of data preservation and correctness. Once the application is able to acquire a DirectWrite’s font object, it can in turn use it in any other DirectWrite API requiring a DirectWrite font from that point onward. The conversion from a DirectWrite’s font object back to a GDI font object allows the rest of the GDI-based application to function with no change while still being able to reap the benefit of using DirectWrite’s new and improved font model. As in some real world examples, the XPS print rasterizer in Windows 7 is implemented on top of DirectWrite and utilizes DirectWrite’s interoperability API to convert back to a GDI font as part of the conversion of an XPS-based print job for a non-XPS printer driver. The Windows 7 XPS Viewer also uses DirectWrite alongside the GDI+ graphic rendering for its onscreen display.
There’s a lot more to the details of the API. In the PDC session linked to above, Leonardo Blanco and Kam VedBrat go into the details of DirectWrite and Direct2D and how to develop applications such as this.
The world has changed a lot since the first text APIs of Windows GDI, such as TextOut or ExtTextOut in Windows NT 3.1 (or the subsequent API additions). The evolution of support for text is a critical part of the underpinnings of Windows 7. We continue to improve this most “basic” element of a graphical operating system so that regardless of the language, script, or device used to render text, Windows will offers a great set of tools and APIs for developers and a great experience for end-users.
Excellent! What a fun discussion we’ve been having on High DPI. It has been so enriching that Ryan wrote up a summary of even more of the discussion. Thanks so much! --Steven
There have been quite a few comments posted regarding high DPI, along with some lively discussion. Most of what has been said has been good anecdotal examples which are consistent with the data we have collected. For the areas where we didn’t have data, the comments have helped to validate many of our assumptions for this group. It is also clear that there are some areas of this feature which are confusing, and in some cases there is a bit of “myth” around what is ideal, what is possible, and what is there. This follow up post is mostly to summarize what we have heard, and to provide some details around the areas where there has been a bit of confusion.
Here is a list of our top “assumptions” which have been echoed by the comments posted:
There have also been a number of areas where there has been a bit of confusion:
Most of these topics have been covered to some degree in the comments, but since there has been so much interest, we decided to go into a bit more details around a few of the top issues/concerns.
With PCs, there is always the hope of a “silver bullet” technology which solves all problems making life easy for users, developers, and designers across the board. As an example, some of the comments to the original posting suggested that if we just made the OS fully vector based, these scaling problems would go away. It turns out that there are several issues with using vector graphics which are worth explaining.
The first issue is that oftentimes when an icon gets to be small in size, it is better to use an alternate representation so that the meaning is clearer. Notice the icons below. In this case, the designer has chosen a non-perspective view for the icon when it is rendered at it’s smallest size.
This is because the designer felt that the information expressed by the icon was clearer with a straight-on view at the smallest size. Here is another example illustrating this point:
Of course, this means that the designer must create multiple versions of the source image design, so there is additional complexity. The point here is that there is a tradeoff that must be made and the tradeoff is not always clear.
Even when one vector source is used, it is common to have size-dependent tweaking to make sure that the result is true to what the designer had in mind. Imagine a vector graphic which has a 1-pixel line at 128x128 that gets scaled down by 1/2 to 64x64. The display has no way of rendering a perfect 1/2 pixel line! In many cases the answer is that the renderer will “round” to a nearby pixel line. However, doing this inherently changes the layout of the sub-elements of the image. And there is the question of, “which pixel line to round to?” If the designer does not hand tune the source material, it will be up to the rendering engine to make this decision, and that can result in undesirable effects. One could say that this should therefore define rules about what elements should be use in a vector, but that only further restricts what concepts can be represented.
It turns out that even the TrueType fonts which we use in Windows are hand-tuned with size-dependant information in order to make the result as high quality as possible. Most of the TrueType rendering pipeline is based on algorithmic scaling of a vector source, but there are size-dependent, hand-coded “hints” in TrueType which the designer specifies to direct the system how to handle edge cases, such as lines falling between pixel boundaries. Here is a link describing this in more detail: http://blogs.msdn.com/fontblog/archive/2005/10/26/485416.aspx
It is not even true that vector graphics are necessarily smaller in size (especially for small images). Most designers create graphics using an editor which builds up an image using many layers of drawings and effects. With bitmap based graphics, designers will “flatten” the layers before adding it to a piece of software. Most designers today pay little attention to the size of the layers because the flattening process essentially converts it to a fixed size based on the image resolution. With vector graphics, there is no such flattening technique so designers need to carefully consider the tools that they use and the effects that they add to make sure that their icon is not extremely large. I spent some time with one of our designers who had a vector graphic source for one of our bitmaps in Windows and the file was 622k! Of course that file size is fixed regardless of the resulting resolution, but that huge file flattens into this 23k PNG bitmap.
Of course, a hand-tuned vector based representation of this could be probably made smaller if the size constraints were part of the design time process. But that would be an additional constraint put on the designer, and one which they would need to learn how to do well.
For applications that need to carefully control the layout and graphics, or scale the fidelity of the images based on the available resolution, having a way of specifying specific pixel locations for items is important to get the best result. This is as true on the Mac as it is on the PC (see http://developer.apple.com/releasenotes/GraphicsImaging/RN-ResolutionIndependentUI/). There is often a belief that if we just provided the right tools or the right framework then all these problems would go away. We all know that each set of tools and each framework have their own set of tradeoffs (whether that is Win 32, .net, or HTML). While there is no silver bullet, there are things we can do to make writing DPI aware applications easier for developers. As an example, there are two important upcoming ecosystem events in which we will be talking in detail about High DPI. We will also have materials which we will be making available to developers which will help educate them on how to convert existing applications to be DPI aware. The first event is Microsoft Professional Developer Conference, where we will talk about High DPI as part of the talk “Writing your Application to Shine on Modern Graphics Hardware (link)”. The second is the Windows Hardware Engineering Conference, in which we will be discussing high DPI as part of the “High Fidelity Graphics and Media” track (link).
There have been several posts on app compat and high DPI (for example bluvg’s comment). There have also been comments talking about the complexity and understandability of the High DPI configuration. In some cases the app compat issues can be mitigated by enabling or disabling the automatic scaling feature. This can be changed globally by going to the DPI UI, clicking the button labeled “Custom DPI” and changing the checkbox labeled, “Use Windows XP style DPI scaling”. When this checkbox is unchecked, applications which are not declared to be DPI aware are automatically scaled by the window manager. When it is checked, automatic scaling is disabled globally. It is interesting to note that for DPI settings < 144 DPI, this box is checked by default, and for DPI settings >= 144 it is unchecked by default. In some cases, changing the default settings can result in a better experience depending on the applications that you use and your DPI setting. It is also interesting to note that automatic scaling can be turned off on a per application basis using the Vista Program Compatibility properties. Here is a link for more info on how to do that: http://windowshelp.microsoft.com/Windows/en-US/help/bf416877-c83f-4476-a3da-8ec98dcf5f101033.mspx. (Look at the section for “Disable Display Scaling on high DPI settings”.)
A typical application UI is made up of a content window and a frame UI. The frame UI is where the menu items and toolbar buttons are. The content window is the “document view”. For example, in IE the content window is the actual webpage. It turns out the code required to support high DPI scaling for the content windows is the same code required to do the zooming feature. In fact, for the content window, IE8 simply uses the high DPI setting to configure the default zoom factor (see DPI Scaling and Internet Explorer 8 for more details). However, high DPI has the additional side effect of scaling the size of the frame UI. Since most people use the scaling feature to make text larger to be more readable, it makes sense to scale the frame UI as well, since the text in the menu items and toolbar tooltips will also scale. In a sense if there is per-application scaling that is really about the content area, and applications will support that as developers see the customer need. DPI scaling makes the UI elements of all applications render similarly.
Some have suggested that we should just use high DPI as a way to calibrate the screen so that the physical size of an object is the same regardless of the display. This makes a ton of sense from a logical perspective. The idea would be to calibrate the display so “in inch is an inch”. We thought about doing this, but the problem is that it does not solve the end user need of wanting to have a way to configure the size of the text and the UI. If we then had a separate “global scale” variable, this would mean that application developers would need to pay attention to both metrics, which would add complexity to the developer story. Furthermore, if a user feels that the UI is too small, should it be up to the developer or the user to set the preference of how big the UI should be? In other words if the designer wants the button to be an inch, but the user wants the button to be 1.5 inches to make it easier to use, who should decide? The way the feature works today, it is up to the user to set their preference, but it is up to the application developer to make sure that the user preference is honored.
Once again, it is really great to see so much interest in high DPI. We certainly have some challenges ahead of us, but in many ways it seems like the ecosystem is ripe for this change. Hopefully this follow up post helped to consolidate some of feedback which we have heard on the previous post and explain some of the complexities of this feature in more detail.
One of the areas of any release of Windows that receives a significant amount of testing and scrutiny is the performance of graphics—desktop graphics all the way to the most extreme CAD and game graphics. The amazing breadth of hardware supported for Windows and the broad spectrum of usage scenarios contributes to a vibrant ecosystem with many different goals—from just the basics to the highest frame rates on multiple monitors possible. In engineering Windows 7 we set out to improve the “real world” performance of graphics as well as continue to improve the most extreme elements of graphics. This is work we do in Windows 7 and work our partners do as they work to improve the underlying hardware/software combination through drivers (note: Windows Vista drivers continue to work as they did in Windows Vista, but we've also been working with partners on updated drivers for Windows 7 which many of you have been testing through Windows Update downloads). This post looks at this spectrum of engineering as well as the different ways performance is measured. Ultimately we want to inform you about what we have done in engineering Windows 7, while we leave room for the many forums that will compare and contrast Windows 7 on different hardware and in different scenarios. This post is written by Ameet Chitre, a program manager on our Desktop Graphics feature team. --Steven
If you have gone online to check out or purchase a new PC, you must have noticed that “faster graphics” and “great performance” are always some of the key selling points. People have come to expect faster systems which enable them to edit photos, do email, watch high-definition videos and play the latest 3D games all with greater ease, often shuffling between these tasks seamlessly. Quite a few of these users refer to the enthusiast community blogs and various review sites which run graphics benchmarks and report results evaluating how fast the graphics of new hardware or software performs. Traditionally graphics performance has been measured and analyzed through 3D games but it also impacts what we call “desktop scenarios” - such as when you are using the Windows UI, moving or maximizing windows, or scrolling within Word or IE etc. The performance requirements for these desktop scenarios are quite different from3D games. In fact, this is the reason in Windows Vista Experience Index (WinEI) we give you rating for these two scenarios separately, highlighted in the image below:
Figure 1. WEI sample with Graphics capabilities highlighted.
Graphics performance is usually assessed through many benchmarks. These can be classified into 2 broad categories:
However, there are plenty of things that we all do on our PCs that don’t have benchmarks tracking them that are still quite critical to make fast. In these cases we use the instrumentation within Windows to obtain timing information and then analyze the performance.
This blog entry discusses various aspects of graphics performance - both gaming and desktop graphics performance. It calls out the changes we made in Windows 7 to address user feedback as well as to take advantage of modern hardware to improve graphics performance.
Many have experienced scenarios where an application, or Windows itself, stops responding momentarily. This is type of a performance issue that can be impacted significantly by the performance of graphics in the PC. We categorize these as desktop responsiveness issues. Improving responsiveness, both in real terms and by avoiding non-responsive moments, is one of the key ways that performance is improved in the system. It is also hard to measure.
Measuring desktop responsiveness is a hard problem since a number of issues which affect responsiveness aren’t easily reproducible and there is a great variety of them. They are rarely caught by either kind of benchmark as these issues are dependent on real-world combinations of factors. For Windows 7 we spent a great deal of time looking at these performance glitches using a mechanism in test versions of Windows 7 which has the ability to record key OS events and when they occurred. During real-world testing when we encounter a responsiveness problem, the tester can hit a record key and enter a small description of the issue encountered. The event history with diagnostic information called a “performance trace” is written out to a file and uploaded to a server where a team of performance analysts parse the data to figure out the cause of the responsiveness issue. This process has been successful to the extent that today most responsiveness issues can be quickly tracked down and root-caused.
Using this methodology, we analyzed thousands of desktop responsiveness traces where the tester experienced a frozen desktop anywhere from 100msec to several seconds. The type of issues ranged from an antivirus blocking disk access for all applications while updating itself on the vendor’s website to an application doing network access from a UI thread. In a significant portion of these traces, we found that a GDI application is waiting on another GDI application which is experiencing slowdowns due to excessive paging activity. This was the single-most frequently occurring cause of all desktop responsiveness issues, which without this data we probably would not have assumed. Based on these investigations, we worked to improve the architecture in these two key areas:
These are described in more detail below.
A number of performance traces we investigated in the context of desktop responsiveness pointed us to the design of a key synchronization mechanism in GDI. The performance challenge happens because the design of GDI in Windows Vista allows only a single application to hold a system-wide exclusive global lock. While this seems obvious in hindsight, when this decision was originally made the performance characteristics of different parts of the system made this optimistic implementation perfectly reasonable.
Figure 2. Existing architecture of GDI concurrency.
GDI applications running simultaneously vie for this global lock in order to render on the screen. The application that accesses the global lock prevents other applications from rendering till it releases the global lock. The situation often gets exacerbated when the application that is holding the lock needs to page in a large amount of memory from the disk since moving the memory from the disk to RAM takes a relatively long time. The above picture shows two GDI applications running simultaneously, contending for the global lock. If App X gets hold of the lock, it can render to the screen while App Y is unable to do so and waits for App X to finish.
Figure 3. Windows 7 architecture of GDI concurrency.
The solution to the problem was therefore to reduce the lock contention and improve concurrency by re-architecting the internal synchronization mechanism through which multiple applications can reliably render at the same time. Contention due to the global exclusive lock is avoided by implementing a number of fine-grained locks which are not exclusive but aid parallelism. The increased number of fine-grained locks adds a small overhead for scenarios where only a single application is rendering at a time.
Special attention was paid to GDI application compatibility as changing internal synchronization mechanism in the most widely used API stack could potentially give rise to timing issues such as deadlocks and rendering corruption.
This work also resulted in better rendering performance of concurrent GDI applications on multi-core CPUs. Multi-core Windows PCs benefit from these changes as more than one application can now be rendering at the same time.
After the GDI concurrency work was implemented in the Windows 7 builds leading to the Beta, we saw a large reduction in the number of desktop responsiveness issues reported by our testers which were caused by one application blocking another one due to GDI. To further validate the scalability of our new implementation, we wrote tests that draw 2D GDI primitives and measured the rendering throughput by launching simultaneously multiple such applications. The throughput is measured by adding together the frame rate (FPS) of each application window. Below is a sample of these results on a quad-core CPU system.
Figure 4. GDI Concurrency and Scalability.
Without the Windows 7 GDI concurrency, the rendering throughput of these applications is effectively limited to the performance of a single CPU core. Since only a single application can acquire the global exclusive lock while the others are waiting, this scenario doesn’t benefit from multiple CPU cores. This demonstrates that GDI applications in Windows 7 are now much less dependent on one another. This benefit will not need any new display drivers; it will work on any Vista (WDDM 1.0) and newer display drivers.
Another area which affects system responsiveness is memory usage. Simply put, increased system memory (RAM) usage leads to an increased paging activity which directly leads to reduced system responsiveness. Thus, for the best responsiveness, all applications, processes and OS components need to use as little system memory as possible.
In Windows Vista, the amount of memory required to run multiple windows scales linearly with the number of windows opened on the system. This results in more memory pressure when there are more windows or if the monitors have higher resolution. It gets worse if you have more than one monitor. As part of investigating various means to improve system responsiveness, we saw a great opportunity in reducing the usage of system memory by DWM. In Windows Vista, every GDI application window accounts for two memory allocations which hold identical content – one in video memory and one in system memory. DWM is responsible for composition of the desktop through the graphics hardware. Hence, it requires a copy of the same allocation in video memory, which is easily accessible by the graphics hardware. The duplicate copy present in system memory is required because GDI is being rendered utilizing the CPU completely in the operating system without any assistance or “acceleration” by the graphics hardware. As the CPU performs all the tasks for rendering GDI applications, it requires an easily accessible cacheable copy of memory.
Figure 5. Existing memory allocations.
Windows 7 saves one copy of the memory allocation per application window by getting rid of the system memory copy entirely. Thus, for a GDI application window visible on the desktop, the memory consumed is cut in half.
Figure 6. Windows 7 memory allocations.
We achieved the reduction in system memory by accelerating the common GDI operations through the graphics hardware - the WDDM drivers accelerate these to minimize the performance impact of the CPU read-back of video memory. This was necessary as performing these operations otherwise on the CPU would incur a heavy performance penalty. In order to decide which GDI operations to accelerate, it was important to understand the usage pattern of various GDI applications. We profiled the top 100 GDI applications to learn more about their calling patterns and frequency and nature of the GDI operations.
Figure 7. Calling patterns and frequency of GDI operations for 100 GDI-based applications.
Based on real-world application statistics, a tiny snapshot of which is seen above, we worked with our graphics IHV partners to provide support in their drivers to accelerate the most commonly used GDI operations. Windows 7 systems with these updated drivers, called “WDDM v1.1” will thus benefit from this memory savings work. Please note that WDDM 1.0 drivers continue to function and are fully supported on Windows 7. You might have seen Windows Update providing these 1.1 drivers during the Beta—these drivers are themselves in Beta.
Figure 8. Desktop Window manager memory consumption comparison using WDDM 1.1 v. WDDM 1.0.
The above data shows that the memory savings become more and more pronounced when you have multiple application windows visible on the desktop. Since you save a lot of system memory, the paging activity gets reduced – as a result, your system responsiveness improves for the same workload.
Certain trade-offs had to be made for the desktop responsiveness improvements which benefit a wide range of systems. For example – the elimination of the duplicate system memory copies which “speed up” certain operations introduced slightly reduced performance as the CPU now has to read data back from the video memory. An analysis of real-world application statistics showed that these operations were rare. However, certain GDI micro-benchmarks which issue these operations show some performance degradation. This is something to note if you are running existing benchmarks that stress specific GDI operations repeatedly, which isn’t necessarily a reflection of real-world performance. Our observation has been that these slow-downs do not impact the end-user functionality directly and that the memory savings directly result in Windows 7 being much responsive overall. The improvements overall are definitely noticeable on memory constrained PCs with shared memory graphics.
No article on graphics performance is complete without talking about gaming, which is still the most widely analyzed and discussed aspect of graphics performance. There are a number of popular benchmarks such as 3D Mark as well as in-game benchmarks which are really a mode in which you can run your game where it renders the game scenes and animations without any user interaction. This area has thus been well tracked by the gaming industry through various industry benchmarks, which are pretty realistic and representative of actual games. The different benchmarks and tests are widely discussed and gamers all well-versed in the subtleties of these measurements and translating them into recommendations depending on their hardware, drivers, and gaming expectations.
For Windows 7, we have worked closely with our Graphics IHV partners, helping them improve the WDDM drivers’ gaming performance with specific changes to how Windows 7 works under the hood, while maintaining the same driver model and compatibility. Our continued investments in performance tools has helped us and our IHV partners track down and analyze various gaming performance bottlenecks and fix them in subsequent driver releases. The fundamentals of the Windows Display Driver Model remain unchanged in Windows 7. Some policies around GPU scheduling and memory management were changed to enable better performance in certain scenarios.
Because these benchmarks are very sensitive to the specific hardware, firmware, drivers, and overall system and because these are so widely measured and discussed elsewhere we are going to leave these comparisons to third parties. Like many areas in Windows 7, our commitment is to engineer even better performance across many dimensions. We believe it is better for you to experience these efforts directly. In comparing Windows 7, we would encourage the comparison using Windows Vista SP1 and keep in mind the difference you might see in WDDM 1.0 v. 1.1 and that the 1.1 drivers are still under development.
As you can see, in engineering Windows 7 we have worked hard to improve the architecture for graphics for real-world performance. It benefits both ends of the hardware spectrum – by enabling low physical memory systems to run a leaner and faster Windows and at the same time enabling multi-core PCs render multiple graphics applications much more efficiently.
As you can imagine, our team is quite busy working through this next phase of Windows 7. We definitely appreciate the millions of downloads and installs of the Windows 7 RC. Things are going as we expect at this point. On a personal note, I wanted to thank all the folks who have been sending me mail. I’ve received a lot of kind words and support regarding the RC and quite a few people saying “hurry up and just release it”. We outlined the steps we’re taking for this next milestone and aren’t going to rush things. We’ve got a lot of work for sure! Not that I’m counting, but I just crossed over 3,000 emails sent via the contact link in this blog. While I haven’t answered all of them, I’ve done the best I can, and appreciate each and every exchange.
Windows 7 includes a set of features for safeguarding your PC when used by children. This post is by Vladimir Rovinsky, a program manager on our Safety Team, who details the features in Windows 7 specifically around Parental Controls. This work is in addition to the safety of the OS itself and of course the features built into Internet Explorer to provide safety and security while browsing. You might also want to check out Windows Live Family Safety which is part of Windows Live Essentials (http://download.live.com) which provides even more for safety and parental controls. --Steven
Today, children are exposed to digital hazards more easily than any time in the past. Especially with the help of powerful search tools, convenient social networking applications, low cost tools and services for publishing videos and photographs, the web is awash with content that’s inappropriate for children, and full of people that parents want to bar from contacting their children.
These digital hazards are accessible to children through a variety of applications, including web browsers, instant messaging applications, media players, games, and email applications. Many of these applications have attempted to offer parental control features. However, they offer this functionality through variety of user interfaces, locations and include varied terminology. The duplication and inconsistency of parental control settings management can make it difficult for parents to maintain the correct settings across multiple applications.
Windows Vista Parental Controls provided a framework to solve these problems by offering:
To get a quick demo of Windows Vista Parental Controls in action, check out this video.
For more information about developing software for Windows Vista Parental Controls, see Using Parental Controls APIs.
Responding to customer feedback and evolving nature of the web and challenges it poses to the parents, we strive to provide families with flexible and effective safety features. Our efforts for the Windows 7 release of Parental Controls were focused on the following objectives:
1. Further developing the extensibility of the Parental Controls platform to enable third-party developers to create richer Parental Control capabilities that integrate well with Windows 7 Parental Controls.
The Windows 7 Parental Controls platform was modified to allow multiple independent providers of Parental Controls functionality to be installed on the system and augment or fully replace the parental controls provided by Windows 7. Windows Vista allowed partial replacement of Windows Parental Controls; the web filter was replaceable. In Windows 7, in addition to the web filter components, the entire Windows 7 Parental Controls user interface can be replaced by third-party providers. The underlying enforcement of the offline restrictions will still be performed by Windows Parental Controls platform. Allowing a third party provider to replace the entire Windows Parental Controls user interface creates a consistent user experience that seamlessly combines existing Parental Controls functionality with the new ones introduced by the third-party provider.
The Windows Control Panel Parental Controls screen still remains the central location and launching point on Windows 7 for Parental Controls functionality regardless of whether it is provided by default (system) or by a third-party provider.
2. Removal of web content restrictions and activity viewing functionality from default (system) Parental controls provider and reliance on Windows Live or third-party providers for these capabilities.
The web is changing much faster than we can update the Windows operating system. For example, when Vista was released Social Networking was barely known. Now it has a thriving web presence. We need to keep web focused parental controls up with innovation. Because of this, we have moved them into Windows Live.
Web filtering and activity viewing capabilities can be more efficiently provided by Windows Live or a third-party solution that implement web based delivery of this functionality. For instance, Microsoft’s Windows Live Family Safety free application provides web content filtering, file downloads restrictions, and activity monitoring. It also provides online contact restrictions for children using Windows Live online applications (Windows Live Hotmail, Windows Live Messenger, etc).
You can learn more about Windows Live Family Safety solution here.
More information about Windows 7 changes to the Parental Controls platform can be found here.
Elements new to Windows 7 Parental controls top-level screen can be seen on the following screen shot:
Figure 1 Windows 7 Parental Controls screen
Additional control providers can still rely on the default’s (system) provider UI for the configuration of in-box offline restrictions. If a provider chooses to do so, the User Controls screen can be presented to configure a user’s Parental Controls settings.
If an additional provider is selected and configured, the following new user interface elements are shown on the Windows 7 User Controls screen:
Figure 2 Windows 7 User Controls screen. Additional controls provider is installed and configured.
If a Windows Vista PC which has parentally managed user accounts with enabled web filtering restrictions is upgraded to Windows 7, parents (administrators) are warned during the upgrade as well as when opening the Windows 7 Parental Controls screen, that web filtering and activity reporting functionality is not part of Windows 7 Parental Controls.
Figure 3 Windows 7 Parental Controls screen. Some users have web filtering restrictions. No additional provider is installed.
Windows Vista Parental Controls settings (including web filtering and activity logs information) are preserved unchanged when upgrading from Windows Vista to Windows 7. Although web filtering settings and activity logs information are not used by Windows 7 Parental controls, their preservation allows third-party provider to honor these settings.
As you start using Windows 7, we hope these changes to Parental Controls capabilities will make you feel more confident and in control of how your family members are using computers and experiencing the web.
About every decade we make the big decision to update what we refer to as the applets (note we’ll use applet, application, program, and tool all interchangeably as we write about these) in Windows—historically Calc (Calculator), Paint (or MS Paint, Paint Brush) and WordPad (or Write), and also the new Sticky Notes applet in Windows 7. As an old-timer, whenever I think of these tools I think of all the history behind them and how they came about. I’m sure many folks have seen the now “classic” video of our (now) CEO showing off Windows to our sales force (the last word of this video is the clue that this video was done for inside Microsoft). Windows 7 seems like a great time to update these tools. The motivation for updating the applets this release is not the 10-year mark or just time to add some applet-specific features, but the new opportunities for developers to integrate their applications with the Windows 7 desktop experience. While many use the applets as primary tools, our view of these is much more about demonstrating the overall platform experience and to provide guidance to developers about how to integrate and build on Windows 7, while at the same time providing “out of box” value for everyone. There’s no real “tension” over adding more and more features to these tools as our primary focus is on showing off what’s new in Windows—after all there are many full-featured tools available that provide similar functionality for free. So let’s not fill the comments with request for more bitmap editing features or advanced scientific calculator features :-).
The APIs discussed in this post are all described on MSDN in the updated developer area for Windows 7 where you can find the Windows 7 developer guide. Each of the areas discussed is also supported by the PDC and WinHEC sessions on those sites.
This post was written by several folks on our applications and gadgets team with Riyaz Pishori, the group program manager, leading the effort. --Steven
This blog post discusses some of the platform innovations in Windows 7 and how Windows 7 applications have adopted and showcased these innovations. This post details some of the platform features that developers and partners can expect in Windows 7 and how Windows 7 programs have showcased these innovations. This post also discusses how applications have been given a facelift both in terms of their functionality as well as their user experience by focusing on key Windows design principles and platform innovations. Finally, this post can serve as a pointer or guide to application developers and ISVs to get themselves familiar with some of the key new Windows platform innovations, see them in action and then figure out how they can build on these APIs for their own software.
The post is organized by each subsystem, and how Windows applets are using that particular subsystem.
The Windows Ribbon User Interface is the next generation rich, new user interface for Windows development. The Windows Ribbon brings the now familiar Office 2007 Ribbon user interface to Windows 7, making it available to application developers and ISVs.
There are several advantages of adopting the Windows Ribbon user interface, many of which have been talked about in the Office 2007 blogs. The Ribbon provides a rich, graphical user interface for all commands in a single place, without the need to expose various functions and commands under different menus or toolbars. The Ribbon UI is direct and self-explanatory, and has a labelled grouping of logically related commands. While using an application that is built on the Ribbon UI platform, the user only needs to focus on his workflow and the context of his task, rather than worry about where a particular function is located or accessible. The Ribbon UI also takes care of layout and provides consistency as compared to toolbars which the user can customize in terms of their sizes, location and contents. It also has built-in and improved keyboard accessibility, and making the application DPI and theme aware becomes easier by using the Ribbon. Finally, development and changes to the user interface is very quick and rapid due to the XML-markup based programming model for the Ribbon User Interface.
Paint and Wordpad are two of the first consumers of the Windows Ribbon UI Platform. In Windows 7, both these applications are enhanced with a set of new features, and the user interface of these applications also required to be brought up to the Windows 7 experience and standards. The Windows Ribbon UI is a great fit for these applications to revamp their user experience and make it consistent, and make these applications rich, fun and easy to use. The tasks and commands in these applications were amenable to be applied to the Ribbon UI framework, and it also served as an opportunity for popular native Windows applications to showcase the Windows Ribbon UI platform to consumers, as well as developers and ISVs. Many has asked about the Windows Explorer and IE also using the ribbon, which we did not plan on for Windows 7. Our Windows 7 focus was on the platform and demonstrating the platform for document-centric applications such as Paint and Wordpad.
Both these applications showcase several elements of the Windows UI Ribbon. The Application Menu of both Paint and Wordpad exposes Application related commands that are typically available thru the ‘File’ menu of an application. Both the applications have a core tab set that consists of ‘Home’, which exposes most of the commands in the application, and ‘View’ which exposes the image or document viewing options in the application. The commands in both these tabs are laid out logically in groups of related functionality.
A quick access toolbar (QAT) is provided by both Paint and Wordpad, which comes with certain defaults like Save, Undo and Redo that are meaningful to the application. The user can customize the QAT by using the QAT drop-down, or right-click on any command or group in the ribbon and add it to the QAT.
Several ribbon commands are used in both these applications, like command buttons, split buttons, galleries, drop-downs, check boxes and toggle buttons.
Further, both applications provide a ‘Print preview’ mode which shows a print preview of the image or the document in context. In a mode, all the core tabs are removed and only the mode is displayed for the user to interact with. On exiting a mode, the user is returned to the core tab set.
Paint also exposes a contextual tab for the Text tool, which is displayed only when a text control is drawn on canvas. A contextual tab shown next to the core tab set when the text tool is in focus, and removed when the text is applied to the image on the canvas. The contextual tab set contains the tools that are specific and relevant only to the text tool.
Both the applications provide live previews through ribbon galleries, for example the font size and font name for Wordpad and Paint while formatting text, bullets and lists in Wordpad, and color selection, outline size selection and outline and fill styles for shapes in Paint. A live preview allows the user to see the changes instantaneously on mouse hover, and then apply those changes on a selection. These previews are one of the key elements of the ribbon UI and demonstrate why the metaphor is much more than a “big toolbar” but a new interaction style.
By adopting the Ribbon User Interface, both the applications inherit built-in keyboard accessibility support using ribbon Keytips, have tooltips on all commands, and have ready support for DPI and Windows themes.
Paint and Wordpad can serve as examples of how the Ribbon UI can be easily used in MFC applications. The Windows Ribbon presents new opportunities and options for developers and ISVs to develop applications with the Ribbon User Interface. The Windows Scenic Ribbon programming model and architecture emphasizes the separation of the markup file and the C++ code files to help developers decouple the presentation and customization of the UI from the underlying application code. The platform also promotes developer-designer workflow, where the developer can focus on the application logic, while the designer can work on the UI presentation and layout. The ribbon UI is a significant investment for us and you should expect to see us continue to use it more throughout Microsoft, including an implementation in the .NET Framework as was demonstrated by Scott Guthrie at the PDC, which will be built on Windows 7 natively in the future.
Windows 7 provides support for multi-touch input data, as well as supporting multi-touch in Win32 via Windows messages. The investments in the multi-touch platform include the developer platform that exposes touch APIs to applications, enhancing the core user interface in Windows 7 to optimize for touch experiences, and providing multi-touch gestures for applications to consume. Developers on Windows 7 can build on these APIs decide on the appropriate level of touch support they would like to provide in their software.
Wordpad enhances the document reading experience by using the multi-touch platform and using the zoom and pan gestures. Zooming, panning and inertia lets the user get to a particular piece of content very quickly in an intuitive fashion. By using the zoom gesture, the user can zoom in or zoom out of the document which is akin to using the zoom slider at the right of the Wordpad status bar. On multi-touch capable hardware, the user can zoom in and out of the document by placing his fingers anywhere within the document window and executing the zoom gesture. Wordpad also supports the pan gesture to pan thru the pages of a document that is open in Wordpad. By executing the pan gesture, the user can scroll-down or scroll-up a document similar to using the scroll bar of the Wordpad application.
In Paint multi-touch data is used to allow users to paint with multiple fingers. It is an example of an application that allows multi-touch input without the usage of gestures. For Paint’s functionality, providing multiple finger painting ability was more compelling and enriching than allowing for zoom, pan, rotate or other gestures that act on the picture in a read-only mode and not in an edit-mode. New brushes in Paint are multi-touch enabled, and they handle touch inputs via multiple fingers and allow the user to simultaneously draw strokes on canvas on finger drag. These brushes are also pressure-sensitive, thereby providing a realistic experience with touch by varying the stroke width based on the pressure on the screen. While adopting the multi-touch platform to enhance the end-user experience in Paint, conscious design decisions were made to preserve the single touch experience for functionalities where a multi-touch scenario does not apply such as the color picker, magnifier and text tool.
By building with the multi-touch APIs, Paint and Wordpad have created more natural and intuitive interfaces on touch-enabled hardware and show “out of the box” how different capabilities can be exposed by developers in their software.
Sticky Notes (or just Notes) is an extension of a TabletPC applet available in Windows 7. One of the things which was key to the Notes experience on the desktop was to have the ability to quickly take all the notes away and get them back, but still making sure it is really easy to create a new note. We achieved this by having a single top level window for the sticky notes application. You can minimize all your notes and view a stack of notes in the preview on the command bar with a single click. The stacked preview has been achieved using the new thumbnail preview APIs that enable apps to override the default taskbar previews that are essentially a redirected snapshot of the top level application window, and provide their own. This enables applications to decouple their previews from the top level application window and provide a more productive preview based on the scenario. For example, this was very valuable in Sticky Note scenarios where a quick peek at a note that was last touched provides for quite a productive workflow. Taskbar also caches the preview thumbnail images so once the preview is given to the Taskbar, the application does not need to keep it around – the application does however need to send an updated preview whenever it changes.
Another nifty customization end-point on the task bar is the destination menu (aka jumplist). This menu comes up when a user right clicks on the application in the taskbar or hovers over the application icon in the Start Menu. The Sticky Notes application does not have a single main application window – this makes the application feel really light weight and fits in well into the Windows 7 philosophy of creating simple and powerful user experiences. The challenge then was exposing functionality such as the ability to create a new note from a central location or potentially other custom “tasks”. The destination menu helped exposing these scenarios in a simple yet discoverable way.
The new taskbar functionality and extensibility built in that has the potential to make it a lot easier for people to work with applications/scenarios in a more productive and efficient manner when developers integrate their software with the Windows desktop.
Building on the long history of Search in Windows and the significant enhancements in Windows 7, there are APIs available to developers to deeply integrate their content types with the desktop search user experience affordances in Windows 7. Sticky Notes shows one example of how these APIs can be used.
The Sticky Notes application now allows users to get back to their notes by simply searching for content through Windows Inline search within the start menu. This is in line with allowing users to reach the relevant note as quickly as possible even when the application is closed. Even though search could be done for both Text and Ink content, it is restricted to text because of lower success rates with varied handwriting styles in ink. The application registers a protocol handler that generates a URL for each Note. The Sticky Notes Filter handler gets asked for the content associated with each note that is then indexed by the Search infrastructure. These indexes are then used to perform a quick lookups when the user searches the Search interfaces provided by the Windows Shell. When a user clicks on a result, Search invokes the associated application with the URL corresponding to the one that the protocol handler had generated that the Filter handler associated with the content it sent to the Search indexer.
The search platform also has the ability to enable the filter handler to specify the language of each chunk of content passed on to it that overrides the default Search heuristics used to compute the language - this increases Search accuracy manifold and thereby enhances internationalization support of the entire ecosystem.
The reason Sticky Notes implemented a protocol handler in addition to a Filter handler was because it implements its own integrated storage schema on top of the Windows File system - all the notes are represented by a single .snt file. The protocol handler generates URL's to individual entities (in this case - notes); the filter handler picks out content for each of these URL's and gives it to Search for indexing.
This demonstrates the ease in which applications can plug into the search platform in Windows 7, and add search handlers which can enhance the overall user experience from the App as well as the platform.
Real-Time Stylus (RTS) is infrastructure that provides access to the stylus events coming from pen or touches digitizers. It provides information about strokes and points and provides access to ink-related events. Using RTS, applications can get access to stylus information and develop compelling end-user scenarios and experiences.
Sticky Notes now allows the users to Ink and Type on notes depending on the availability of inking hardware. Users can use keyboard input to type on notes and use the stylus to ink on notes. Though the experience has been designed keeping in mind that users will either use either ink or text on a particular note, it does allow users to ink and type on the same note. However these surfaces are maintained agnostic to each other. Sticky Notes also auto-grows the note while inking on the note, providing a real-time experience of the note adjusting its size to fit the inked content.
Real Time Stylus (RTS) is used for inking features provided in Sticky Notes. Inking gestures are also available to applications, and the scratch out gesture has been implemented in Sticky Notes to delete content.
In addition, Paint uses RTS to get a stream of positional input from mouse, stylus or touch which are used for drawing strokes on canvas. Paint also captures additional input variables like pressure and touch surface area when such input is available from the digitizer, and maps these inputs into the stroke algorithms that are used to generate Paint strokes on canvas. Using this algorithm, the user is able to modulate stroke width and other parameter based on the pressure or touch area on canvas.
Using RTS allows the development of applications and software that can build on the inking platform and provide ways to interact with the application that go beyond mouse or keyboard. Using stylus, inking and gestures, developers can create interactive experiences for end-users.
The Windows Error Reporting (WER) infrastructure is a set of feedback technologies that is built into Windows 7 and other earlier versions of Windows client and server. WER allows applications to register for application failures and capture this data for end-users who agree to report it. This data can be accessed and analyzed and can be used to monitor error trends and download debug information to help developers and ISVs determine the root cause for application failures.
WER can add value to software development at various stages: during development, during beta testing by getting early feedback from end-users, after the release of the product by analysing and prioritizing the top fixes, and at end of life of the product.
Related to failure recovery, Applications can also register with WER for restart on application of a Windows patch that terminates the application and on application of an update that reboots the computer, as well as failure caused due to an application crash or hang or not responding state. Applications can optionally register for recovery of lost data, can develop their own mechanism for recovery.
Several Windows applications adopt the WER infrastructure to collect and analyze data. Calculator, Paint and Wordpad register for restart and additionally recover the current data in the sessions of the application that were running. Sticky Notes also registers for restart and recovery, and returns the user to the set of notes open on the desktop. Using WER, end-users would allow Windows to capture and collect problem data and then would be returned to the applications in the same state that they were in earlier.
As you can see, our primary effort for the applets in Windows 7 is to showcase some of the new platform APIs and innovations available to developers. As you get to try out these applications you will see that while showcasing the Windows 7 platform innovations, we have also added some commonly requested features and functionality. Some of them are: Check and correct, calculation modes and templates in Calculator, New brushes, shapes and multi-touch support in Paint, Open standards support in Wordpad and Ink and text, taskbar and search integration in Sticky notes. Maybe we won’t wait 10 years to update these again :-)
--Riyaz Pishori and team
Microsoft has been working on handwriting recognition for over 15 years going back to the Pen extensions for Windows 3.0. With the increased integration and broad availability of the handwriting components present in Windows Vista we continue to see increased use of handwriting with Windows PCs. We see many customers using handwriting across a wide variety of applications including schools, hospitals, banking, insurance, government, and more. It is exciting to see this natural form of interaction used in new scenarios. Of course one thing we need to continue to do is improve the quality of recognition as well as the availability of recognizers in more languages around the world. In this post, Yvonne, a Program Manager on our User Interface Platform team, provides a perspective on engineering new recognizers and recognition improvements in Windows 7. --Steven
Hi, my name is Yvonne and I’m a Program Manager on the Tablet PC and Handwriting Recognition team. This post is about the work we’ve done to improve recognition in handwriting for Windows 7.
Microsoft has invested in pen based computing since the early 1990s and with the release of Windows Vista handwriting recognizers are available for 12 languages, including USA, UK, German, French, Spanish, Italian, Dutch, Brazilian Portuguese, and Chinese (Simplified and Traditional), Japanese and Korean. Customers frequently ask us when we plan to ship more languages and why a specific language is not yet supported. We are planning to ship new and improved languages for Windows 7, including Norwegian, Swedish, Finnish, Danish, Russian, and Polish, and the list continues to grow. Let’s explore what it takes to develop new handwriting recognizers.
Windows has true cursive handwriting recognition, you don’t need to learn to write in a special way – in-fact, we’ve taught (or “trained” as we say) Windows the handwriting styles of thousands of people and Windows learns more about your style as you use it. Over the last 16 years we’ve developed powerful engines for recognizing handwriting, we continue to tune these to make them more accurate, faster and to add new capabilities, such as the ability to learn from you in Vista. Supporting a new language is much more than adding new dictionaries – each new language is a major investment. It starts with collecting native handwriting, next we analyze the data and go through iterations of training and tuning, and finally the system gets to you and continues to improve as you use it.
The development of a new handwriting recognizer starts with a huge data collection effort. We collect millions of words and characters of written text from tens of thousands of writers from all around the world.
Before I describe our collection efforts, I would like to answer a question we are frequently asked: “Why can’t you just use an existing recognizer with a new dictionary?” One reason is that some languages have special characters or accents. But the overriding reason is because people in different regions of the world learn to write in different ways, even between countries with the same language like the UK and US. Characters that may look visually very similar to you can actually be quite different to the computer. This is why we need to collect real world data that captures exactly how characters, punctuation marks and other shapes are written.
Setting up a data collection effort is challenging and time consuming because we want to ensure that we collect the “right kind of data”. We carefully choose our collection labs in the respective countries for which we develop recognizers.
Before we start our data collection in the labs, we configure our collection tools, prepare documentation, and compile language scripts that will guide our volunteers through the collection process. Our scripts are carefully prepared by native speakers in the respective language to ensure that we collect only orthographically correct data, data from different writing styles, and data that covers all characters, numbers, symbols and signs that are relevant to a specific language. All of our scripts are proofread and edited before they are blessed to be used at the collection labs.
Once our tools and scripts are ready, we open our labs and start to recruit volunteers to donate their handwriting samples. Our recruitment efforts ensure that we have balanced demographics such as gender, age, left handiness, and educational background that represent the majority of the population for that country.
A supervisor at the lab instructs the volunteers to copy the text as it is displayed in the collection tool in their own writing style. What is important to note is that we want to collect writing samples that accurately represent the person’s natural way of writing. We therefore encourage volunteers to treat “pen and tablet” like “pen and paper”. If one of the volunteers tends to writes in big, curvy strokes, then we want to collect his/her big, curvy strokes during the collection session. High quality data in this context refers to data that was naturally written.
Here is a snapshot of what our collection tool looks like:
Figure 1: Collection Tool
A collection session lasts between 60-90 minutes at which point our volunteer has donated a significant amount of handwritten data without feeling fatigued. The donated data is then uploaded and stored in our database at Microsoft ready for future use. The written samples contain important information like stroke orders, start- and end points, spacing, and other characteristics that are essential to train our new recognizer.
Let’s take a look at some of our samples in our database to illustrate the great variation among ink samples:
Figure 2: Ink samples illustrating different stroke orders.
The screenshot shows how three different volunteers inked the word “black”. The different colors are used to illustrate the exact stroke orders in which the word was written. Our first two volunteers used five strokes to write the word “black”; our third volunteer used four strokes. Please also note how our third volunteer used one stroke only to ink the letters “ck”, while our first volunteer used three strokes for the same combination of letters. All of this information is used to train our recognizers.
Once we have collected a sufficient amount of inked data, we split our data into a training set, used by our development team, and a “blind” set, used by our test team. The training set is then employed to train the Neural Network, which is largely responsible for the magic that is taking place during the recognition process. Good, naturally written data is essential in developing a high quality recognizer; the recognizer can’t be any better than its training set. The more high quality data we feed into our Neural Network, the more equipped we are to handle sloppy cursive handwriting.
Our Neural Network is a Time-Delay Neural Network (TDNN) that can handle connected letters of cursive scripts. A TDNN takes ink segments of preceding and following stroke segments into consideration when computing the probabilities of letters, digits and characters for each segment of ink. The output of the TDNN is powerful but not good enough when handwriting is sloppy. In order to come within reach of human recognition accuracy, we have to employ information that goes beyond the shape of the letter: we call this the Language Model context. The majority of this Language Model context comes in form of the lexicon, which is a wordlist of valid spellings for a given language. For many languages, this is the same lexicon that the spellchecker uses. The TDNN and the lexicon work closely together to compute word probabilities and output the top suggestions for the given input.
Training the Neural Network is an involved process that takes time. We often experiment with borrowing data from other languages to increase the size of the training data with the ultimate goal to boost recognition accuracy. Borrowing characters from other languages does not always lead to success. As I mentioned above, stroke order, letter shape, writing styles and letter size can differ significantly from country to country and can have a negative impact on the performance of the TDNN. It often takes us several rounds of training, re-training and tuning before we find “the right formula” that will lead to high recognition accuracy.
How do we know if we are headed in the right direction when we build a new recognizer? This is an important question that the test team and native speakers answer for us. The test team is responsible for generating our recognition accuracy metrics that reflect how good our recognizer is. These accuracy metrics are based on our blind test set which is the collected data that development could not use for training. In addition to our accuracy metrics, we work with native speakers in house and at our world-wide subsidiaries to get feedback and further input.
In the previous paragraphs I have outlined how we develop high quality recognizers that can handle a wide variety of different writing styles. But there is more as each person can also train the recognizer his/her unique writing style. The training that is done to teach the recognizer a personal writing style is the same training that happens before Microsoft ships the product. The only difference is that we are now collecting unique training data from a specific person (and not that of thousands of people). We call this process “Personalization”.
Figure 3: Personalization Wizard (Sentence module).
As the screenshots of our Personalization wizard illustrates, a person is asked to write the requested sentence to provide his/her ink samples. The more data a person donates during the personalization process, the better the recognizer will become. In addition to providing writing samples based on specified sentences, a person can target specific recognition errors, shapes, and characters that will all be used for training. Our Personalization feature is complex and offers a variety of different modules that enable a person to optimally tune the recognizer. We are proud to announce that Personalization will be available for all Vista languages and all new Windows 7 languages. We encourage you to use this feature to improve your recognition accuracy.
We continue to work on improving our recognizers which also means that we are incorporating our customers feedback through online telemetry (anonymously, privately, voluntary, and opt-in). In Windows Vista we released a new feature called “Report Handwriting Recognition Errors”, which gives people the opportunity to submit those ink samples that the recognizer did not recognize correctly. After the person has corrected a word in the Tablet Input Panel (TIP), we enable a menu that allows a person to send the misrecognized ink together with its corrected version to our team.
Here is a screenshot of what our error reporting tool looks like:
Figure 4: With “Report Handwriting Recognition Errors” people can choose which of the misrecognized ink samples they want to submit.
We receive approximately 2000 error reports per week. Each error report is stored in our database before we analyze it and use it to improve our next generation of recognizers. As you can imagine, real world data is extremely helpful because it is only this type of data that can reveal shortcomings of our recognizers.
We value and appreciate every single error report. Keep sending us your feedback, so that we can use it to improve the magic of our present and future recognizers.
– Yvonne representing the handwriting recognition efforts
Hi Jon DeVaan here.
Steven wrote about how we organize the engineering team on Windows which is a very important element of how work is done. Another important part is how we organize the engineering project itself.
I’d like to start with a couple of quick notes. First is that Steven reads and writes about ten times faster than I do, so don’t be too surprised if you see about that distribution of words between the two of us here. (Be assured that between us I am the deep thinker :-). Or maybe I am just jealous.) Second is that we want do want to keep sharing the “how we build Windows 7” topics since that gives us a shared context for when we dive into feature discussion as we get closer to the PDC and WinHEC. We want to discuss how we are engineering Windows 7 including the lessons learned from Longhorn/Vista. All of these realities go into our decision making on Windows 7.
OK, on to the tawdry bits.
Steven linked last time to the book Microsoft Secrets, which is an excellent analysis of what I like to call version two of the Microsoft Engineering System. (Version one involved index cards and “floppy net” and you really don’t want to hear about it.) Version two served Microsoft very well for far longer than anyone anticipated, but learning from Windows XP, the truly different security environment that emerged at that time and from Longhorn/Vista, it became clear that it was time for another generational transformation in how we approach engineering our products.
The lessons from XP revolve around the changed security landscape in our industry. You can learn about how we put our learning into action by looking at the Security Development Lifecycle, which is the set of engineering practices recommended by Microsoft to develop more secure software. We use these practices internally to engineer Windows.
The comments on this blog show that the quality of a complete system contains many different attributes, each of varying importance to different people, and that people have a wide range of opinions about Vista’s overall quality. I spend a lot of time on core reliability of the OS and in studying the telemetry we collect from real users (only if they opt-in to the Customer Experience Improvement Program) I know that Vista SP1 is just as reliable as XP overall and more reliable in some important ways. The telemetry guided us on what to address in SP1. I was glad to see one way pointed out by people commenting about sleep and resume working better in Vista. I am also excited by the prospect of continuing our efforts (we are) using the telemetry to drive Vista to be the most reliable version of Windows ever. I add to the list of Vista’s qualities successfully cutting security vulnerabilities by just under half compared to XP. This blog is about Windows 7, but you should know that we are working on Windows 7 with a deep understanding of the performance of Windows Vista in the real world.
In the most important ways, people who have emailed and commented have highlighted opportunities for us to improve the Windows engineering system. Performance, reliability, compatibility, and failing to deliver on new technology promises are popular themes in the comments. One of the best ways we can address these is by better day-to-day management of the engineering of the Windows 7 code base—or the daily build quality. We have taken many concrete steps to improve how we manage the project so that we do much better on this dimension.
I hope you are reading this and going, “Well, duh!” but my experience with software projects of all sizes and in many organizations tells me this is not as obvious or easily attainable as we wish.
Daily Build Quality
Daily quality matters a great deal in a software project because every day you make decisions based on your best understanding of how much work is left. When the average daily build has low quality, it is impossible to know how much work is left, and you make a lot of bad engineering decisions. As the number of contributing engineers increases (because we want to do more), the importance of daily quality rises rapidly because the integration burden increases according to the probability of any single programmer’s error. This problem is more than just not knowing what the number of bugs in the product is. If that were all the trouble caused then at least each developer would have their fate in their own hands. The much more insidious side-effect is when developers lack the confidence to integrate all of the daily changes into their personal work. When this happens there are many bugs, incompatibilities, and other issues that we can’t know because the code changes have never been brought together on any machine.
I’ve prepared a graph to illustrate the phenomenon using a simple formula predicting the build breaks caused by a 1 in 100 error rate on the part of individual programmers over a spectrum of group sizes (blue line). A one percent error rate is good. If one used a typical rate it would be a little worse than that. I’ve included two other lines showing the build break probability if we cut the average individual error rate by half (red line) and by a tenth (green line). You can see that mechanisms that improve the daily quality of each engineer impacts the overall daily build quality by quite a large amount.
For a team the size of Windows, it is quite a feat for the daily builds to be reliable.
Our improvement in Windows 7 leveraged a big improvement in the Vista engineering system, an investment in a common test automation infrastructure across all the feature teams of Windows. (You will see here that there is an inevitable link between the engineering processes themselves and the organization of the team, a link many people don’t recognize.) Using this infrastructure, we can verify the code changes supplied by every feature team before they are merged into the daily build. Inside of the feature team this infrastructure can be used to verify the code changes of all of the programmers every day. You can see in the chart how the average of 40 programmers per feature team balances the build break probability so that inside of a feature team the build breaks relatively infrequently.
For Windows 7 we have largely succeeded at keeping the build at a high level of quality every day. While we have occasional breaks as we integrate the work of all the developers, the automation allows us to find and repair any issues and issue a high quality build virtually every day. I have been using Windows 7 for my daily life since the start of the project with relatively few difficulties. (I know many folks are anxious to join me in using Windows 7 builds every day—hang in there!)
For fun I’ve included a couple pictures from our build lab where builds and verification tests for servers and clients are running 24x7:
Whew! That seems like a wind sprint through a deep topic that I spend a lot of time on, but I hope you found it interesting. I hope you start to get the idea that we have been very holistic in thinking through new ways of working and improvements to how we engineer Windows through this example. The ultimate test of our thinking will be the quality of product itself. What is your point of view on this important software engineering issue?
We’ve seen some comments recently posted on a previous post on accessibility and a member of the User Interface Platform team wanted to offer some thoughts on the topic. Brett is a senior test lead who leads our efforts testing the Accessibility of Windows 7. --Steven
Hi, my name is Brett and I am the test lead for the Windows 7 Accessibility team. Back in November my colleague Michael wrote a blog post about the work our team is doing for Windows 7, I’m following up to that and some recent comments about our new screen Magnifier. On a personal note I would like to mention that I’m a person with low vision and depend on some of the technologies that my team produces to help me in my work.
I’ve been using Windows 7 for my day-to-day work for several months, this is something we call “dogfooding”, which is using our own pre-release products long before the public ever sees a beta. I’ve been using Windows 7 as my primary operating system and have found our new Magnifier to be very useful to me.
Now, about our Magnifier, as you can imagine, the appeal of the many features in Windows varies from person to person, we often say that it is like making pizza for a billion people. The same is true for the features my team owns. I’ve read many comments since we released our Windows 7 beta about magnifier, some are from people that have really benefited from our new work, some have suggestions, and others have concerns. I will say thanks for the feedback, we appreciate all types. Those of you that have benefited are mostly people that need basic magnification and appreciate the easy ability to zoom in and out as needed; I fall into this category myself. Those of you that need magnification in combination with custom colors, high-contrast or some screen readers probably haven’t been able to benefit from the new Magnifier, for you we’ve made sure that the Vista magnifier continues to work. Let me explain a little more about what we’ve done in Windows 7.
To go into more detail about our implementation I need to start with our graphics system in Windows. Over the last several years GPU technology has made huge advances and in Vista we finally made the leap to a modern hardware accelerated graphics system, what we call Aero, which takes advantage of the GPU. We often use the term Aero to refer to the specific elements of Windows visuals, such as transparency and gradients. In practice it is more than that, the modern graphics rendering (technically the desktop window manager along with the DirectX APIs) is not just for aesthetics but for all forms of rendering including text, 2D, and 3D all using modern hardware assisted graphics and a much richer API. It takes time, however, for the diverse ecosystem to adopt this technology, perhaps even over the course of several OS releases. It also takes time for Windows and time for software developers and hardware manufacturers to adopt new technologies; so for a time we will have (and fully support) a mix of both old and new. For example, some screen readers do the great things they do by capturing the data that goes through the original Windows graphics system (GDI) and building their off-screen UI models which is why they need to turn off the new rendering. On the other hand, our new Magnifier is integrated deeply into the desktop window manager (“Aero”) to leverage this graphics horsepower and deliver smooth full-screen multi-monitor magnification.
While, as this demonstrates, these advances aren’t seamless, in Windows 7 my team has worked to make sure that we maintain Vista functionality and compatibility while making new investments. Magnifier is an example of this, we utilize the power of the GPU where we can to bring new capabilities to a broad spectrum of customers, and when Aero needs to be off, whether for screen readers, high-contrast or other needs, we maintain the existing capabilities in the product. And by maintaining compatibility as much as possible, many of the tools you depend on today will continue to work with Windows 7.
So, is Magnifier better for everyone? Not everyone, but certainly for many people, but more than that I can honestly say that we have made advances to accessibility for everyone in Window 7. As Michael noted in his posting, we invested in several areas, there’s not only the Magnifier and on-screen keyboard work, there is also significant work to the underlying accessibility APIs. We also actively support the community and recently made a grant to NV Access to help them improve their open source screen reader support for Windows 7 and Internet Explorer 8.
Thanks for reading, and thanks for your comments,