Engineering Windows 7

Welcome to our blog dedicated to the engineering of Microsoft Windows 7

Accessibility in Windows 7

Accessibility in Windows 7

  • Comments 64

This post is from Michael Bernstein, a development lead on the User Interface Platform team where he focuses on accessibility. Accessibility is the term we apply to the APIs and features that enable Windows to be used, to be accessible, by as many people as possible so that, regardless of physical or cognitive abilities, everyone has the ability to access the functions of Windows. To enable this, Windows includes both built-in accessibility utilities as well as APIs used by third party assistive technology aids and by application developers to make sure their software is also accessible. This is a topic that is extremely important to Microsoft and one that is a key tenet in the engineering of Windows 7. Microsoft also has a corporate-wide group dedicated to making sure that PCs are easier to see, hear, and use. You can read more about Microsoft’s accessibility initiatives on http://www.microsoft.com/enable/. --Steven

Hi, I’m the development lead for Accessibility and Speech Recognition experiences in Windows 7, and I wanted to write about how we thought about Accessibility in Windows 7. 

We wanted to make Windows 7 the most accessible operating system that Microsoft has ever produced.  It became clear as we planned this release, however, that the notion of Accessibility is not as simple as it may appear.  It is tempting to think about Accessibility like Security: either you have a known failure, or your system is believed to be secure/accessible.  This definition turns out to be limited, though.  How do you deal with the fact that the needs of customers who are blind are very different from the needs of customers who are deaf?  The needs of customers who are blind are even different from those of customers with reduced vision: a magnification tool is useless for one group and crucial for the other. And what do we make of cases where something is technically accessible but practically frustrating, like a common user scenario that takes 36 keystrokes to execute?  Clearly, Accessibility wasn’t going to boil down to a simple yes/no question.  It is really more like a particular kind of usability, but usability for a specific set of audiences with individual needs.

Since the questions we were asking were complex, the answers ended up being complex, too.  We chose a four-part strategy to improve Accessibility in Windows 7.

I. Build a firm foundation with UI Automation

In Windows Vista, Microsoft delivered a new core component for Accessibility called UI Automation.  UI Automation enables a user’s assistive technology (AT) to programmatically drive the UI of an application, and allows applications to expose their accessible functionality in a richer way than was possible in previous versions of Windows.  More questions can be asked about a piece of UI, and that UI can be manipulated in richer ways.  UI Automation also introduced the idea of Control Patterns: any given piece of UI can decide how it should be controlled.  Buttons expose the Invoke pattern, indicating that they can be pushed; Combo Boxes expose ExpandCollapse, indicating that they can be opened and closed.  We let different controls be different, instead of trying to force them all into the same mold.  All this was introduced in Windows Vista and adoption is still ongoing.

In Windows 7, we invested in improving the performance of the UI Automation system and created a new, native-code API for UI Automation to make sure that it can be used effectively by a wide range of assistive technology software.  Now applications written in C++, as well as those written using the .NET Framework, can take advantage of UI Automation. 

We also did a bunch of work to make sure that the UI Automation system was integrated even more closely with the legacy Microsoft Active Accessibility (MSAA) system and developed new bridging techniques between the best of the new and the old technologies. UI Automation Clients can read Accessibility information from MSAA applications, and vice versa, to ensure maximum Accessibility regardless of which accessibility API an application used originally. Since the UI Automation and MSAA systems cooperate closely in many scenarios, we decided to name the combination of the two, calling it the Windows Automation API. This architecture forms the foundation for the rest of our Accessibility effort, and we’re pleased to have this Accessibility foundation Windows 7.

II. Improve our included Accessibility utilities

We also improved the Accessibility utilities that we include in the box with Windows.  Microsoft works closely with many different AT software companies who deliver software to make Windows more accessible to customers with disabilities, but we also include a set of utilities to make sure that our customers’ early experiences are accessible, even before installing any other software. We decided to enhance two of those utilities in Windows 7: the On-Screen Keyboard and the Magnifier.

The most noticeable change to the On-Screen Keyboard is the improved look and feel, but there are also more subtle enhancements.  The appearance of this utility had not changed since Windows XP; our customers were also asking for it to be resizable.  We addressed both of these by working closely with Tablet developers to share a common code base between the Tablet Soft Keyboard and the On-Screen Keyboard.  Both keyboards now have an attractive appearance that is in tune with Windows 7 and both are now resizable.  The keyboards still are distinct, though, because customers use them differently: Tablet users may want to switch dynamically between handwriting and typing, whereas On-Screen Keyboard users may need modes where they can hover or scan to keys, if they have disabilities that prevent them from clicking.  Along these lines, we also added basic text prediction to help customers with disabilities enter text more quickly.  If you have ever tried typing with an on-screen keyboard, you can appreciate how significantly text prediction can improve text entry speed.

The Magnifier came in for a deeper overhaul.  The Magnifier in Windows Vista and Windows XP was not an intuitive experience: when you pointed at part of the screen, the magnified content appeared in a separate window, usually docked at top of the screen.  You had to point at one place and look at another.  We considered two basic solutions to this problem: you could zoom into the entire screen or you could make the magnified area follow the pointer while leaving the rest of the screen the same.  These became our two primary modes for the Windows 7 Magnifier: Full-screen mode and Lens mode.

Full-screen mode is great when you want to increase the size of everything on the screen at once.  As you move the mouse or keyboard focus around the middle of the screen, the view stays still; if you move towards the edge, the Magnifier scrolls the view to keep up.  One downside of this mode is that you can lose track of your context.  To address that usability issue, we added a context animation that zooms out to show you where your work area is relative to the whole screen, and then zooms back in. 

Lens mode, on the other hand, is nice when you just want to zoom in on one particular thing.  In this mode, the lens centers on the mouse pointer, which feels much like using a magnifying glass.  You can re-size the lens to be very wide and short, which can be nice if you are reading a document and want to magnify it line by line.  We based our design on the popular Microsoft IntelliPoint magnifier, a design you can now enjoy with any mouse.

We also addressed customer feedback about the Magnifier window taking up too much space on the screen.  We moved the most commonly used controls like zoom in/out to a small toolbar, which fades out to a semi-transparent watermark when you aren’t using it.  The remaining options are available in an Options dialog when you need them.  Last, we gave almost everything a keyboard shortcut, so if you really don’t want to see the UI, you don’t have to use it.  Win-+ will zoom you in any time you are using Windows 7.

These tools directly improve Accessibility for customers with low vision and dexterity disabilities. It should be obvious, but making the PC easier to see or interact with benefits everyone and so these two examples also show the broad appeal of AT tools – at the PDC we showed both the On-Screen Keyboard and the Magnifier and I think it is fair to see everyone saw the benefit of using these tools themselves, regardless of abilities.

III. Make it easier to build Accessible software

Windows APIs cannot provide Accessibility all by themselves; it is vital for Windows-based applications to do their part in providing Accessibility data for AT programs to use.  For example, a screen reader may sound excellent, but if it can’t read your favorite web browser, what good is that?  Assistive tools like screen readers and magnifiers are clients of the Accessibility system, while the applications that you want to use, like web browsers and word processors, are providers.  It takes both to make the whole experience accessible--you need both a high-quality client and a well-written provider to have a good Accessible experience.  There are more providers in the software ecosystem, so it is hard for us to work one-on-one with every provider to make sure they are well-written.

To address this challenge, our team developed the UI Accessibility Checker (AccChecker for short) and UI Automation Verify (UIA Verify) utilities, which can scan an application (a provider, really) and report on common Accessibility problems.  Software developers can use AccChecker and UIA Verify to detect problems in their provider code before a customer ever uses it.  Quality assurance engineers can use them to verify the quality of their firm’s work.  We believe this is so important that we released AccChecker and UIA Verify as open-source software to make it available to the widest possible audience.  If you are not a programmer, you may never use these utilities directly, but you may well benefit from the bugs they helped to eliminate before they ever reached you. 

IV. Plan for Accessibility from Day 1

To make sure that Windows features themselves were good providers, we borrowed an idea from the Software Development Lifecycle, risk assessment.  Before a line of code was written, each planned Windows 7 feature was rated on its Accessibility risk.  Features that use more basic, off-the-shelf common controls are usually more accessible because Windows provides built-in providers for off-the-shelf components; features that do fancy, custom drawing have more work to do.  This planning process made each team aware of how much accessibility risk it was taking on, so that they could plan appropriately.  Once the features were all rated, the list was sorted by risk so that our team could reach out to teams with high-risk features and make sure that they had the resources and tools they needed to make their feature properly accessible.  We also ensured that they received more hands-on testing and validation.  As a result, most Windows features are more accessible than they have been in previous releases, making for a better overall customer experience.

To wrap up, we've emphasized Accessibility in engineering Windows 7.  We’ve made good progress on improving the core architecture for Accessibility and enhancing the included tools like On-Screen Keyboard and Magnifier.  The AccChecker and UIA Verify tools have made it much easier to validate applications to ensure that they will be compatible with current assistive tools as well as future tools based on the Windows Automation API.  Our approach to Accessibility for the features and providers in Windows itself has become more thorough, consistent and integrated, thanks to the hard work of hundreds of engineers across the company.  We’re proud of what we have accomplished in Windows 7 and hope that it will help customers with disabilities to realize their full potential and have a more enjoyable experience with Windows.

--Michael

Leave a Comment
  • Please add 4 and 7 and type the answer here:
  • Post
  • Can you make the 5 key of the numeric keypad (which currently sort of simulates a left mouse click) behave exactly like the left mouse click like the button we have on laptops? For example, it is currently not possible to drag or hold the 5 key down and simulate a drag or hold. Also, there is no real key for right click (though most keyboards have a context menu key however that really brings up the context menu where the keyboard focus indicator is, not where the mouse pointer/cursor is).

  • PS.

    Windows 7 Beta is coming for Developer !!!

    http://www.msdndevcon.com/Pages/start.aspx

    WOOOWW!!

    OPS!! Im Not Developers :(

  • @Domenico

    And also at January 13 for you others...

  • @bluefisch200

    I saw the article Ed Bott where he hopes the Beta for 13 January,

    I have also seen that $ 99 can sign up to receive the DVD

    http://blogs.msdn.com/bobfamiliar/archive/2008/12/02/windows-7-beta-1-for-mdc-attendees.aspx

    If it were possible from Sign from Italy to receive the DVD in Italy (although im not developers)

    would i immediately register.

    I can not resist

  • What I would like to see is Microsoft address one of the major complaints from the Beta's Vista, the ability to move and edit the tool bars, address bars, icon bars, menu bar.  For the life of me I can't not understand why Microsoft still puts up a wall when it come to this small aspect of the UI. It was and is still a major complaint by many, Microsoft ignored it in the beta's of Vista and still to this day has continued to ignore this simple and modest request. Come on Windows Seven team, stop locking down the UI, is really that difficult to add a few lines of code back in to Windows? Will it really hurt the operating system to allow users a choice? It such a simple request, Windows Seven team, Why can't you do it????? so disappointing, so very very disappointing....

  • I'll assume you've carefully thought through all of the implications of UI Automation. Sounds to me like a potential avenue for malware to exploit this API.

    I'm not a user that needs this functionality, but I see it as very important for MS to think really hard about this. If you can make it easier for someone with a limitation, its going to make it easier for all the rest of us as well. I understand the desire to have multi-key press activation of commands (Win + X key), but surely this has to be very difficult for people with limited hand dexterity or only a single usable digit. This may also be a problem with multi-touch on touch screens. Maybe some thought needs to be given to gesture-type interfaces (I've seen some stuff with cameras that follow hand movements), as well as the speech interface. Thinking about how these things might be used, even if the functionality isn't there built into the OS, may lead you to some new ideas.

    Please make sure you have as many physically-challenged users as possible on your usability review teams. I'm constantly surprised by the enterprising and innovative ways people have of overcoming their limitations, and finding a way to get stuff done. They come up with ideas that us able-bodied folks don't and for good reason they're passionate about it.

  • @Eghost

    I think removing the ability to totally customize the toolbars in explorer was done on purpose.  It makes it easier to support the software if you know someone hasn't hidden or squished the address and search bars down to nothing.  You might just want to suggest what icon you need to be quickly accessible.

  • I still don't see any valuable information about new system on this forum + I have found some new articles (like "Windows 7: Simple vs. Easy" and "Microsoft exec: Windows 7 is no service pack"). I decided to comment all these facts on my page: http://www.gammu.org/wiki/index.php?title=HydePark:Thinking_about_Windows_7

  • @marcinw

    Pls stop your SPAM!!

  • @Domenico,

    You're commenting for example fact, that testing beta of Windows 7 costs 100$ now and I'm commenting some other facts.

    If you think, that my last post was spam, I can start writing comments again directly on this forum.

  • @marcinw

    I dont think that your posts will help anything...wy not just say...i hate you, Microsoft...every post of you, in your wiki, is about how bad Vista, maybe the best OS on Earth, and 7 are, and that XP is the only good OS...do you know wy everyone likes XP...7 Years of working with the OS, and not becouse its easy...

  • @blargsoup

    I actually want to be able to squish and change tool bars around, by the way I never have had to call support.:)  It's just options and choice. I remember this was a major complaint in the Betas of Vista, and again its really a minor request, but it's a suggestion for Microsoft, I would love to see them allow access to the tool bars, icon bars menu bars, address bars it is the reason why I don't use Vista on a regular basis, and for many it's a major flaw in Vista, I just hate to see history repeat itself.

  • @bluefisch200

    Actually Vista has not been well received,

    "7 Years of working with the OS, and not becouse its easy..."

    I actually find XP a lot more intuitive than Vista. Almost every major complaint about Vista was major complaints in the betas of Vista, Microsoft ignored them,that was a mistake.

    I think saying sums it up.

    "Those who ignore history are doomed to repeat it."

      --George Santayana

  • @bluefisch200,

    Vista hasn't been received well by market, because system has got some disadvantages. This is not only my opinion.

    Microsoft starts to repeat some mistakes from Vista in Windows 7 and I'm writing about it and only about it (in my opinion system can be wrong received by market, if development will be done like now). Microsoft need to change some rules and go into some other direction - if they will say "we have excellent Vista codebase" only, they will fail again. As Eghost said it would be good to return to history:

    there were Windows 98 SE and NT 4.0 quite good received by the market. Microsoft proposed Windows ME, which wasn't excellent. They had seen it and resigned from it. Concentrated on successful NT 4.0 and we had 2000, XP, etc.

    now there was XP very good received by people. Microsoft proposed Vista, which isn't excellent. They don't resign from its' solutions, but say: we know better, what market need. And we will give something looking and working similar. And I'm writing about it.

    I don't say, that I hate Microsoft - there are many nice and hard working there people and they're making very good job there (for example with Visual Studio). I know, that Steven is making his job as good as he can. But some decisions (decisions, not people) are wrong and I'm writing about it.

    Windows 7 will be maybe received a little better, because our computers are better (faster). But it can be failure too. If you think, that's different, please show concrete technical reasons. Not marketing words about excellent taskbar or touch screen support. I'm really opened for discussion and I'm waiting for it.

    As Eghost said similar critic comments were ignored in beta stage of Vista. Will be good for Microsoft to ignore them now ? We will see.

  • @marcinw

    NO!Beta is TOTAL FREE!!!!

    I wanted to pay  $ 99 only for a DVD collector.

Page 2 of 5 (64 items) 12345