Finding out when WSR shows a window (or, a concise guide to IUIAutomationRegistrar)

Eric Brown — Tue, 25 Aug 2009 21:53:30 GMT

When WSR shows a window (for example, the alternates dialog, the Disambiguation Numbers UI, or the dictation scratchpad), that window does not take focus.

These windows do not take focus because these windows could alter the state of the application that WSR is talking to. For example, if you’re renaming a file in Windows Explorer, when the focus goes away, the file gets renamed.

Unfortunately, though, accessibility applications often have problems finding these windows, even though these windows fire the standard WinEvents (several types of windows are plain dialogs, for example).

In Windows 7, WSR fires a custom UI Automation event when it shows or hides one of its custom windows (Disambiguation Numbers, dictation scratchpad, Alternates, Spelling, Pronunciation, Switch Windows, or Launch Application).

To do this, WSR uses the IUIAutomationRegistrar interface.

IUIAutomationRegistrar::RegisterEvent defines a new event ID given a GUID and a name. If multiple applications register an event using the same GUID and name, they all get the same event ID. The event ID can also be used in a call to SetWinEventHook, if your application is using WinEvents.

Beware: all registrations are invalidated when you release the IUIAutomationRegistrar interface pointer! So you must retain the interface pointer until you no longer wish to process events. (MSDN doesn’t make this clear.)

So, if you want to find out when WSR is showing one of its custom windows, call IUIAutomationRegistrar::RegisterEvent with these two event declarations, and check for these events in your event processing code.

const GUID GUID_ShowEvent = {
0x3891149e, 0x7190, 0x47d0, {0xa5, 0x18, 0xca, 0x1c, 0xdb, 0x40, 0xf7, 0xe3}
};
#define UI_SHOW L"Microsoft.Speech.UI.Shown"

const GUID GUID_HideEvent = {
0x987a1c35, 0x597b, 0x4947, {0x9e, 0xf8, 0xe7, 0xa6, 0x81, 0x42, 0xfd, 0x11}
};
#define UI_HIDDEN L"Microsoft.Speech.UI.Hidden"

What’s new in Windows Speech Recognition?

Eric Brown — Thu, 29 Jan 2009 18:00:00 GMT

Now that the Beta of Windows 7 is out, it’s time to talk about the improvements and new features in Windows Speech Recognition.

For Windows 7, we focused primarily on improving the user experience and removing the “rough spots” that we did not have time to fix in Vista.

First and foremost, we focused on performance.

We rewrote the logic that builds the “say what you see” grammar to use the new native UI Automation API (instead of the MSAA IAccessible API). This dramatically reduces the number of cross-process COM calls (by an order of magnitude), and speeds up the grammar generation by about 5-6 times.
The document harvester also has substantial performance improvements;
Building the “start application” grammar also runs much faster, as well.

Second, we focused on usability.

Dictation into TSF-unaware applications works much better than it did before. Now, when you dictate into an unaware application, the dictation scratchpad appears.
- You can use the scratchpad as a temporary document, and it is even voice, mouse, and keyboard-enabled; you can type, use the arrow keys for navigation, or use the mouse or voice commands to select and correct text before inserting the finished text into the unaware application.
- If you don’t like the scratchpad, you can turn it off, and your dictations will be directly inserted into the unaware application.
Sleep mode works much better than it did before; false recognitions of “start listening” have been greatly reduced.
We simplified the transitions between OFF and SLEEP mode; for security reasons, we now default to OFF after “stop listening”; although users can change the default to SLEEP mode. (We call this “voice activation” in the Control Panel and First Time User Experience.)

Third, we looked at accuracy.

The SR engine now uses the WASAPI audio stack, so we support array microphones and echo cancellation; this vastly improves WSR’s accuracy when used without a headset.
Document harvesting runs periodically, rather than just at startup; this lets the harvester pick up new documents as you create them, rather than having to wait for you to reload speech.
You can upload your training data to Microsoft, so that we can improve the recognizers in the future. (You have to initiate this, incidentally; we will not upload any data without your explicit consent.)
The Chinese recognizer has substantial accuracy improvements as well.

Lastly, we did a few tweaks to the recognizer. In Vista, 3rd party applications couldn’t tell whether the shared recognizer was ON or SLEEPing. For Win7, there are new APIs that expose SLEEP mode.

TSF Aware : WSR

Finding out when WSR shows a window (or, a concise guide to IUIAutomationRegistrar)

What’s new in Windows Speech Recognition?