Notes on comments.
Welcome to our blog dedicated to the engineering of Microsoft Windows 7
This post is from Michael Bernstein, a development lead on the User Interface Platform team where he focuses on accessibility. Accessibility is the term we apply to the APIs and features that enable Windows to be used, to be accessible, by as many people as possible so that, regardless of physical or cognitive abilities, everyone has the ability to access the functions of Windows. To enable this, Windows includes both built-in accessibility utilities as well as APIs used by third party assistive technology aids and by application developers to make sure their software is also accessible. This is a topic that is extremely important to Microsoft and one that is a key tenet in the engineering of Windows 7. Microsoft also has a corporate-wide group dedicated to making sure that PCs are easier to see, hear, and use. You can read more about Microsoft’s accessibility initiatives on http://www.microsoft.com/enable/. --Steven
Hi, I’m the development lead for Accessibility and Speech Recognition experiences in Windows 7, and I wanted to write about how we thought about Accessibility in Windows 7.
We wanted to make Windows 7 the most accessible operating system that Microsoft has ever produced. It became clear as we planned this release, however, that the notion of Accessibility is not as simple as it may appear. It is tempting to think about Accessibility like Security: either you have a known failure, or your system is believed to be secure/accessible. This definition turns out to be limited, though. How do you deal with the fact that the needs of customers who are blind are very different from the needs of customers who are deaf? The needs of customers who are blind are even different from those of customers with reduced vision: a magnification tool is useless for one group and crucial for the other. And what do we make of cases where something is technically accessible but practically frustrating, like a common user scenario that takes 36 keystrokes to execute? Clearly, Accessibility wasn’t going to boil down to a simple yes/no question. It is really more like a particular kind of usability, but usability for a specific set of audiences with individual needs.
Since the questions we were asking were complex, the answers ended up being complex, too. We chose a four-part strategy to improve Accessibility in Windows 7.
In Windows Vista, Microsoft delivered a new core component for Accessibility called UI Automation. UI Automation enables a user’s assistive technology (AT) to programmatically drive the UI of an application, and allows applications to expose their accessible functionality in a richer way than was possible in previous versions of Windows. More questions can be asked about a piece of UI, and that UI can be manipulated in richer ways. UI Automation also introduced the idea of Control Patterns: any given piece of UI can decide how it should be controlled. Buttons expose the Invoke pattern, indicating that they can be pushed; Combo Boxes expose ExpandCollapse, indicating that they can be opened and closed. We let different controls be different, instead of trying to force them all into the same mold. All this was introduced in Windows Vista and adoption is still ongoing.
In Windows 7, we invested in improving the performance of the UI Automation system and created a new, native-code API for UI Automation to make sure that it can be used effectively by a wide range of assistive technology software. Now applications written in C++, as well as those written using the .NET Framework, can take advantage of UI Automation.
We also did a bunch of work to make sure that the UI Automation system was integrated even more closely with the legacy Microsoft Active Accessibility (MSAA) system and developed new bridging techniques between the best of the new and the old technologies. UI Automation Clients can read Accessibility information from MSAA applications, and vice versa, to ensure maximum Accessibility regardless of which accessibility API an application used originally. Since the UI Automation and MSAA systems cooperate closely in many scenarios, we decided to name the combination of the two, calling it the Windows Automation API. This architecture forms the foundation for the rest of our Accessibility effort, and we’re pleased to have this Accessibility foundation Windows 7.
We also improved the Accessibility utilities that we include in the box with Windows. Microsoft works closely with many different AT software companies who deliver software to make Windows more accessible to customers with disabilities, but we also include a set of utilities to make sure that our customers’ early experiences are accessible, even before installing any other software. We decided to enhance two of those utilities in Windows 7: the On-Screen Keyboard and the Magnifier.
The most noticeable change to the On-Screen Keyboard is the improved look and feel, but there are also more subtle enhancements. The appearance of this utility had not changed since Windows XP; our customers were also asking for it to be resizable. We addressed both of these by working closely with Tablet developers to share a common code base between the Tablet Soft Keyboard and the On-Screen Keyboard. Both keyboards now have an attractive appearance that is in tune with Windows 7 and both are now resizable. The keyboards still are distinct, though, because customers use them differently: Tablet users may want to switch dynamically between handwriting and typing, whereas On-Screen Keyboard users may need modes where they can hover or scan to keys, if they have disabilities that prevent them from clicking. Along these lines, we also added basic text prediction to help customers with disabilities enter text more quickly. If you have ever tried typing with an on-screen keyboard, you can appreciate how significantly text prediction can improve text entry speed.
The Magnifier came in for a deeper overhaul. The Magnifier in Windows Vista and Windows XP was not an intuitive experience: when you pointed at part of the screen, the magnified content appeared in a separate window, usually docked at top of the screen. You had to point at one place and look at another. We considered two basic solutions to this problem: you could zoom into the entire screen or you could make the magnified area follow the pointer while leaving the rest of the screen the same. These became our two primary modes for the Windows 7 Magnifier: Full-screen mode and Lens mode.
Full-screen mode is great when you want to increase the size of everything on the screen at once. As you move the mouse or keyboard focus around the middle of the screen, the view stays still; if you move towards the edge, the Magnifier scrolls the view to keep up. One downside of this mode is that you can lose track of your context. To address that usability issue, we added a context animation that zooms out to show you where your work area is relative to the whole screen, and then zooms back in.
Lens mode, on the other hand, is nice when you just want to zoom in on one particular thing. In this mode, the lens centers on the mouse pointer, which feels much like using a magnifying glass. You can re-size the lens to be very wide and short, which can be nice if you are reading a document and want to magnify it line by line. We based our design on the popular Microsoft IntelliPoint magnifier, a design you can now enjoy with any mouse.
We also addressed customer feedback about the Magnifier window taking up too much space on the screen. We moved the most commonly used controls like zoom in/out to a small toolbar, which fades out to a semi-transparent watermark when you aren’t using it. The remaining options are available in an Options dialog when you need them. Last, we gave almost everything a keyboard shortcut, so if you really don’t want to see the UI, you don’t have to use it. Win-+ will zoom you in any time you are using Windows 7.
These tools directly improve Accessibility for customers with low vision and dexterity disabilities. It should be obvious, but making the PC easier to see or interact with benefits everyone and so these two examples also show the broad appeal of AT tools – at the PDC we showed both the On-Screen Keyboard and the Magnifier and I think it is fair to see everyone saw the benefit of using these tools themselves, regardless of abilities.
Windows APIs cannot provide Accessibility all by themselves; it is vital for Windows-based applications to do their part in providing Accessibility data for AT programs to use. For example, a screen reader may sound excellent, but if it can’t read your favorite web browser, what good is that? Assistive tools like screen readers and magnifiers are clients of the Accessibility system, while the applications that you want to use, like web browsers and word processors, are providers. It takes both to make the whole experience accessible--you need both a high-quality client and a well-written provider to have a good Accessible experience. There are more providers in the software ecosystem, so it is hard for us to work one-on-one with every provider to make sure they are well-written.
To address this challenge, our team developed the UI Accessibility Checker (AccChecker for short) and UI Automation Verify (UIA Verify) utilities, which can scan an application (a provider, really) and report on common Accessibility problems. Software developers can use AccChecker and UIA Verify to detect problems in their provider code before a customer ever uses it. Quality assurance engineers can use them to verify the quality of their firm’s work. We believe this is so important that we released AccChecker and UIA Verify as open-source software to make it available to the widest possible audience. If you are not a programmer, you may never use these utilities directly, but you may well benefit from the bugs they helped to eliminate before they ever reached you.
To make sure that Windows features themselves were good providers, we borrowed an idea from the Software Development Lifecycle, risk assessment. Before a line of code was written, each planned Windows 7 feature was rated on its Accessibility risk. Features that use more basic, off-the-shelf common controls are usually more accessible because Windows provides built-in providers for off-the-shelf components; features that do fancy, custom drawing have more work to do. This planning process made each team aware of how much accessibility risk it was taking on, so that they could plan appropriately. Once the features were all rated, the list was sorted by risk so that our team could reach out to teams with high-risk features and make sure that they had the resources and tools they needed to make their feature properly accessible. We also ensured that they received more hands-on testing and validation. As a result, most Windows features are more accessible than they have been in previous releases, making for a better overall customer experience.
To wrap up, we've emphasized Accessibility in engineering Windows 7. We’ve made good progress on improving the core architecture for Accessibility and enhancing the included tools like On-Screen Keyboard and Magnifier. The AccChecker and UIA Verify tools have made it much easier to validate applications to ensure that they will be compatible with current assistive tools as well as future tools based on the Windows Automation API. Our approach to Accessibility for the features and providers in Windows itself has become more thorough, consistent and integrated, thanks to the hard work of hundreds of engineers across the company. We’re proud of what we have accomplished in Windows 7 and hope that it will help customers with disabilities to realize their full potential and have a more enjoyable experience with Windows.
We are increasingly excited
It's great that Microsoft have been working hard to fix all of the problems in Windows. They've made a great new operating system that's going to make life great for all end-users. Well done!
This I look forward to -- in my job I work with (almost!) every blind and low vision assistive program out there. Having Windows 7 contain usable basic assistive products would be a godsend for my clients who don't need the full power of the commercial programs. MacOS has done very well with their magnifier, and you can use LINUX Compiz to very good effect as a full screen option. Kudos on not abandoning assitive tech!
The maghifer of Windows 7 6801 is the BEST!!!!
right Windows+ or Windows - keyboard
There needs to be a way to make each of these items modular.
No one in my family nor myself has any need for any of the options available for the deaf, blind or otherwise incapacitated.
So allowing this to be a modular addition is a strong desire of the community.
A simple spot in the Control Panel, as well as the option during the Windows installation itself. Something like this as you are installing Windows or as an OEM product, when Windows is started up the first time, not some buried "custom install" menu buried under 30 clicks:
"Windows 7 allows for increased functionality for people with disabilities including the deaf, blind as well as other handicaps. Do you want to install the Accessibility Module?
1) Yes, install everything
2) Yes, install but keep it inactive for later use"
3) No, do not install"
(but allow this to be installed at a later time from the installation media or downloaded).
This will not only allow minimalists to keep a slimmed down Windows install but also keep the option available in the future if the computer is sold or something happens to the person.
I would suggest and prefer each and every portion of Windows, including those mentioned in these blogs such as the taskbar and everything else to be modular, this way we can use a minimalist install on an AthlonXP or netbook system and the full blown install on a 8 core 4GHz Intel i7 (future) CPU.
Otherwise XP support will need to continue through 2014 and linux will continue to make gains (not like thats a bad thing as it is already happening)...
I know I'm going to be stoned to death for even just mentioning this, but I think the default shortcuts for keyboard accessibility aren't well thought-out.
The shortcuts I'm talking about are Sticky Key's five-times-shift, and Filter Key's shift-for-8-seconds. Toggle Keys' num-lock-for-5-seconds is much less likely to be invoked accidentally, likewise Mouse Keys is almost impossible to trigger accidentally.
This could be solved by avoiding to use common keys for the shortcuts, like I mentioned above the ones using num-lock are mostly safe from accidental invocations. You could have a single shortcut that pops up a window with all 4 keyboard accessibility features with big toggles (both clickable and keyboard-selectable), this way it's also easier to remember since you reduce the number of shortcuts to one.
Just a thought.
1. While speech was a major advancement, I feel Vista took large steps away from accessibility for normal people by making it difficult to do tasks by requiring more keystrokes and more mouseclicks than ever. At so many places in the UI (e.g. Default Programs), there is only one way of navigation, either the mouse or the keyboard but not both. Lots of 'tabbing' is required if using the keyboard.
2. What features does Windows have for customers who are color-blind? The ability to output a grayscale display would be nice. Is is technically possible on a color monitor?
3. It would be awesome if the Windows 7 accessories were made available on at least Windows Vista.
4. How about the ability to pin/temporarily stick the magnifier lens to the desktop when right-clicked so it only temporarily doesn't follow the mouse?
5. A better voice than Anna, why not license a cool one from partners such as NeoSpeech?
6. I too agree the keyboard shortcuts for sticky, filter, toggle keys need to be changed, first thing I do when I install Windows is turn off the shortcuts to prevent accidentally triggering accessibility features.
7. For FilterKeys, why are users limited to fixed predefined repeat rates in seconds? Why is it not possible to type in any value for seconds?
8. Correct me if I am wrong but AFAIK, the MouseKeys feature is impossible to use on some laptops because they don't have a numeric keypad.
9. Windows' keyboard layout viewer/IMEs should be universally accessible/have keyboard shortcuts. MS should also update Character Map to make inserting special character easier and update it to support the OpenType font typography features.
10. The new onscreen keyboard seems to have lost the numeric keypad, the ability to customize font and whether to keep it always on top.
I hope all these are addressed.
Here are my thoughts on acessibility (a "legally-blind" person talking, with a very low vision):
- Remember the Zoom'S setting in IE according to the url and/or a setting, if for example I want all my msdn webpages to be zoomed at 125% with a "full page zoom" (versus text zoom) it should be zoomed automaticly each time the browser starts and remember the type of zoom.
- Make higher-DPI available in a per-application basis. Not all legacy aplication respond well to high-dpi, but I still love it and would like to use it wherever I can (and as the population gets older, more and more people will love that feature which should be made even more acessible on each program setting). It should be possible to run a programe twice, with low and high dpi, because some part of an application may respond well to high-dpi and others not.
- When using high-dpi, I've encountered many pop-up error messages where I can't read the whole error message (the end of the sentences are simply "cut"), you could certainly do something about it in the way windows handles api...
I really love the fact that microsoft is taking acessibility seriously.
PS: I've dicussed this in the windows 7 taskbar comments, but I thought this might be a better place.
I would really love to see some thought given to accessibility when in safe mode, or while repairing/reinstalling Windows. I am a blind computer users, and it is frustrating to either create an answer file, or get help to repair/reinstall windows, or to just use safe mode. Perhaps some audio support for safe mode at least?
Thanks for any thoughts you may have alon these lines, and I appreciate what you are doing within windows itself to make it more accessible.
Didn't have the chance to already use any Win 7 Builds so I don't know this: Is there a Windows + Scroll Wheel option alike the Ctrl + Scroll Wheel on OS X for the magnifier? This is really handy because you don't have to move your hand away of the mouse to use the magnifier shortcuts on the keyboard.
@csm120> It would be nice to have some high-dpi or audio support when booting into safe mode indeed.
The problem is when you're pressing F8 and choosing in which mode to start, it is the motherboard constructor who manage that, my friend has a motherboard that talks to him through the whole boot process or when error popup, that's really great even for me because I can check other thing without having to wait on the screen for the next "message/error"
@GRiNSER> As far as I can see there is no such thing, that is a really important feature that is missing. Maybe there is some obscure shortcut but it isn't showed in the properties menu, which should be easily visible and customizable.
All the acessibility option should be available as custom shortcut as I don't want them "on" all the time and I hate the navigation in the new start menu, it's simply aweful (can't find the old one, which is way less "click", it works with hover only).
This looks even better and better!
One of the functions that I am excited about, is the speech recognition program. But the minus with this function is that it is with Windows ass default, and therefore, it uses a lot of diskspace. When you install Windows, you should have been asked if you want to install some of these functions.
But in the future, I can see us wring on a "paper" that is connected to the PC, and the PC recognize the letters we are writing on the paper. A familiar function is in the Windows 7 build 6801. If I had a touchscreen, I had used this function a lot, like when i write an essay.
But you guys in Redmond are doing a great progress on Windows 7, and keep doing it! But, please do not go in the same trap as you did with Windows Longhorn that the half OS was too complex, and the release date where placed many years after the release date you tough the Longhorn platform was finished.
Keep up the good work!
There's the usual conflict of priorities to ponder with the shortcuts - whether to make them memorable/easy to guess or make them easy to use. I can't press Winkey-+ with one hand and I'd like it to be a one-hand shortcut. But nothing is ever going to be as discoverable and memorable as Winkey-+ for zooming in.
I'm thinking the next rainy afternoon I get bored, I shall try cycling through the Winkey and the rest of the keyboard, because I only know 9 Winkey shortcuts (+,-,E,R,P,S,D,F,L) and I bet there's a few more...
I just am so excited about what has be and will be the most progressive and user friendly OS that is on the Market!
Thanks for all the hard work.
I discovered these features to be just great, I use when without reading glasses, I also really have to give the thumbs up to the font/display tuning for ease of use for older user such as my self!