All the Cool Developers use Speech APIs : Microsoft

Cool Developers STILL use Speech APIs

Charles Oppermann — Tue, 11 Aug 2009 01:12:02 GMT

My apologies that there have been no new posts on this blog for nearly a year. Because my personal and professional lives have taken a new direction, my Microsoft blog will be closing.

I highly recommend the Speech @ Microsoft blog for more up-to-date information on Microsoft’s speech technologies.

Also, Rob’s Rhapsody is a good blog maintained by Microsoft’s Rob Chambers.

You can to email the Speech product team at Microsoft as well: listen@microsoft.com for speech recognition questions, and speak@microsoft.com for speech synthesis and Text-to-Speech (TTS) queries.

The best place for developers to get the Speech SDK samples and documentation is the Windows SDK. Download all or just part of it. You can email your SAPI related questions to sapitech@microsoft.com.

If you’re interested in contacting me personally, you can email me at chuckop@gmail.com or chuckop@live.com. Search the internet for Charles Oppermann OR ChuckOp and you’ll generally find me. I hope to hear from you!

Speech technology is incredibly enabling and is now mainstream and accepted by users. The technology is easily available to developers, so go forth and add the power of the human voice to your application!

Thank you,
Charles Oppermann
Program Manager, Microsoft Corporation
Microsoft Bob, Windows 95, Internet Explorer 3.0, Active Accessibility, Internet Explorer 4.0/4.01, Windows 98, Windows 2000, Windows 2000 Active Directory Programming (author), Windows Vista, Office Communication Server 2007, Windows Speech Recognition Profile Tool, Windows 7

“A computer on every desk and in everyone home…
…usable by everyone!”

Our Users Are Leading Authorities

Charles Oppermann — Tue, 26 Aug 2008 11:26:53 GMT

Throughout my career at Microsoft, I've eagerly participated in mailing lists, newsgroups, and web forums to engage customers and learn more about their needs and foster direct communication.

One of the better forums for speech recognition is run by Professor Itamar Even-Zohar of Tel Aviv University, where he teaches Culture Research. Itamar has been a long time user of speech recognition and vocal in feedback regarding Windows Speech Recognition. His web site on speech recognition contains useful information on WSR and speech recognition included in Office XP and Office 2003. In particular, his ms-speech forum is invaluable.

Recently when David Pogue of the New York Times wrote about the newest version of NaturallySpeaking, Itamar was quick to write David and set him straight on a few matters, including a plug about Windows Speech Recognition Macros!

David wrote of Itamar, "Clearly, I’ve unearthed the world’s leading authority on speech-recognition foreign-language versions,"

If you read the links I'm providing, you'll see that Professor Even-Zohar is not enamored of all that we do. He's critical of several aspects of WSR and while he "gets it" regarding WSR Macros, he's quick to point out flaws and features.

It's users like this that we need more of; people who are highly experienced and unafraid to share their opinions. The information provided is valuable to me and the rest of the product teams. On the flip side, we have to be careful regarding users expectations. Bending our ear doesn't mean you'll get whatever feature you asked for, and within a particular timeframe.

Oftentimes we'll have more features than time or people available. We have to be very choosy about where to spend our resources. Even things that are a number #1 priority sometimes have to take a backseat to a lesser feature because it was one that we could do in the time or resources available.

Having the feedback from experienced users though help us make the most of the resources we have. We can prioritize better and have confidence that what we're doing will have the greatest impact.

To everyone who writes us at listen, speak and sapitech - we thank you and keep the feedback rolling!

The "Mojave Experiment"

Charles Oppermann — Wed, 30 Jul 2008 20:12:24 GMT

Check out the "Mojave Experiment", where Microsoft brought in people to show them a un-released version of Windows.

Having been part of the Windows 95 team, and then shipped components in Windows 98, Windows 2000 and Windows Vista, I'm used to people complaining that the newest version of Windows is not as good at the previous version.

There are a lot of misconceptions about Windows Vista - that's it's slow, that feature X is not as good as it was in Windows XP, or any number of possible excuses. Many of the people doing the complaining haven't used Vista, or installed it on hardware that didn't meet have the recommended system requirements.

If you have Windows XP and are concerned that Windows Vista won't work with your hardware of existing applications, check out the very useful "Windows Vista Upgrade Advisor" tool.

What's your favorite Windows Vista feature? Of course, I'm partial to Speech Recognition and the Microsoft Anna TTS voice, but there are many new features, tell me yours.

Update: The New York Times collects some reaction.

Where can I get the Microsoft Bob SDK?

Charles Oppermann — Mon, 28 Jul 2008 23:32:24 GMT

My friend and colleague Karin Meier is the person I work with when putting updated speech content into the Windows SDK.

She recently blogged about some odd requests she's gotten for old software. One of them was "where can I get the Microsoft Bob SDK?". I've been feeling nostalgic about Bob recently, as I visited Dan Rose's exhibit on Microsoft Bob, part of Dan's 20th Century Abandonware site.

My first job at Microsoft in 1994 was working on Microsoft Bob which was code-named Utopia. My contribution to the product was minor, but in my opinion, it never deserved the trashing it got by the industry and pundits at the time and ever since. Someday I'll write more regarding what went right and what went wrong with Microsoft Bob.

In the meantime, and for a stroll down memory lane, check out Dan's website.

Display Context Menus Where The Cursor Is, Not Where the Mouse Is

Charles Oppermann — Thu, 06 Sep 2007 23:39:15 GMT

This is a little user interface rant of mine since I'm speech and keyboard-oriented. While editing text, when I say "Press Shift F Ten" or press the Application Key (to the right of the spacebar on Windows keyboards), I expect the context menu to appear at the text cursor location, since that's where the action is going to take place.

However, some applications assume the mouse activitated the functionality and positions the context menu wherever the mouse is. Since I'm using speech or typing and haven't touched the mouse in a while, the menu appears nowhere near where the cursor or selection is.

A more common variant of this is when the menu appears in the upper-left corner of the edit box when activated by keyboard.

The article titled Using Menus on MSDN contains sample code that always uses lParam for the X/Y location to display the menu. The documentation on WM_CONTEXTMENU is clear:

If the context menu is generated from the keyboard—for example, if the user types SHIFT+F10—then the x- and y-coordinates are -1 and the application should display the context menu at the location of the current selection rather than at (xPos, yPos).

That advice is ignored in the Using Menus topic, so I used the "Add Community Content" to add a note, and I'll file a bug on this so that it can be fixed in the future.

Using MSDN's Community Content feature, I added the following to the Using Menus article:

Remember when processing the WM_CONTEXTMENU message, that the X/Y coordinates might be -1/-1 which indicates that the keyboard generated the menu, thus, the menu should be shown at the cursor location or at the location of the selection - NOT at -1/-1 or the mouse pointer location.

The samples currently in this article do not account for this and will attempt to display the menu at -1/-1, which is confusing to the keyboard user. Pressing the Application Key on Windows keyboards (to the right of the spacebar) generates a VK_APPS virtual scan code which by default generates a WM_CONTEXTMENU. You can also get this if the user presses SHIFT+F10.
Never handle SHIFT+F10 or VK_APPS to popup a context menu, rely on the WM_CONTEXTMENU message and if the location given is -1/-1, revert to using the text cursor and/or selection information to place the menu.

The Desktop Is Not For Programs

Charles Oppermann — Tue, 04 Sep 2007 11:11:29 GMT

I'm constantly amazed that people think that putting shortcuts to programs on the desktop makes accessing that program "easier".

For the second time in about a week, I've encountered people asking how to put shortcuts to programs on the desktop.

The desktop is ill-suited for this. To start with, items located on it are often not visible because other windows are placed in front of the desktop. Depending on the current window layout, you might have to make one or more mouse or keyboard operations to select the desktop item you want.

To make matters worse, the location of the items will shift positions as screen resolutions change (because of games, connecting monitors, etc.) and items are added.

While oftentimes commercial software will litter the desktop with shortcuts, the purpose is to increase visibility, not ease of use.

Why not use the Start Menu? If WordPad is a program you use often, just "Pin" it to the Start Menu and it'll always be there, available in less keystrokes than trying to use it off the desktop.

If you really want quick access, pin the item to the Start Menu and then modify the item's properties to have a shortcut key assigned. ~~Only items in the Start Menu can have shortcut keys assigned to them.~~

Update 9/4/2007: My bad, shortcut keys can be assigned to shortcut file that are located on the desktop. My initial test of this failed, and since I knew that only certain locations respect shortcut keys, I figured that the desktop was not one of them. I'll try to find a definitive list, but it appears that any of the Start Menu locations and the Desktop are valid places for a shortcut file to have a shortcut key assigned. Interestingly, shortcut keys for items in the Quick Launch toolbar location are not respected.

How NASA Can Help Microsoft

Charles Oppermann — Wed, 13 Dec 2006 21:46:36 GMT

On an internal blog at Microsoft, I came across a posting by a Corporate VP on some books he was going read while on vacation. One of the books was a autobiography of one of my hero's, Gene Kranz, who was Flight Director for several flights in Project Gemini and Apollo.

Kranz lead his team of Flight Controllers (known as the White Team) during two of the most dramatic events in the space program at the time; the touchdown of the Apollo 11 Lunar Module, Eagle, and again when the Apollo 13 Service Module exploded.

Kranz's book is called Failure Is Not An Option and is wonderfully written and I recommend it highly. In my opinion, and echoed in the internal blog posting I read, the lessons of the space race of the late 1950's and 1960's are of value to any large organization. Kranz's management style is no-nonsense, and that constant practice through simulation kept everyone alert and allowed them to react quickly to unplanned situations.

Probably the worst thing in his Flight Control world was encountering a problem that wasn't already thought about. The idea being as anything went wrong, everyone has experience with the problem from simulation and could react very quickly. Absolutely there are numerous lessons for Microsoft in how NASA and it's contractors approached the space race to the Moon.

In this arena, anything could happen, so you assumed that from the start, planned for it, designed for it, and executed it with that in mind. Imagine a world in which software developers assume everything could fail and one in which simulation (testing) does fail everything. The result would be much more robust code.

I think Microsoft does more of this kind of testing than another major software maker. While software components, particular the interactions with the operating system platform are complex, the various systems of the Apollo missions where incredibly complex as well. There were several major companies and hundreds of small contractors providing pieces to the system and they all had to work together perfectly.

After reading his book, I got to meet Gene at a MoF dinner a few years ago. He's got a great personality and mentality for thinking through problems.

I also recommend Angle of Attack by Harrison Storms, who was a senior VP for North American Aerospace, personally responsible for Apollo Command Module. It too has lessons on how business can react to tragic failure (as what happened when the Apollo 1 command module caught fire during a ground test, killing three astronauts).

These are not business books, but biographies of head-strong people leading large organizations doing high-profile work.

Windows Speech Recognition Getting Some Respect

Charles Oppermann — Tue, 12 Dec 2006 18:46:28 GMT

When our new speech recognition for Windows Vista was demonstrated at the Microsoft Financial Analyst Meeting this past August, it went disastrously (read about it at my personal blog, along with links to the video).

As I said in that posting, I was angry at first because the high-profile failure didn't need to happen (Larry Osterman explains the technical details). But at the same time, I know our technology was good, and as a side benefit, the fallout from the demo would give WSR more recognition.

Now that Windows Vista is getting more attention in the mainstream technology press, people are trying out WSR and discovering that it works pretty well. Here's what Jupiter Research analyst Michael Gartenberg had to say:

"I’ve been using the integrated speech recognition in Windows Vista for the last few days, for a variety of tasks and in a variety of applications. I’m pleased to say it works well, and greatly improves the usability of my computer for entering text. It’s so darn good, it feels a little bit out of science fiction. But then again, isn’t that the way technology is supposed to work?"

Thanks to Todd Bishop's Microsoft Blog over at the Seattle Post-Intelligencer for pointing this out.

Microsoft Speech API SDK

Charles Oppermann — Fri, 24 Nov 2006 20:58:21 GMT

The Speech API Software Developers Kit (SAPI SDK) contains the documentation, samples, and header and library files to create applications and utilities that use speech recognition and voice synthesis. In addition, the SAPI SDK can be used to create speech recognition and voice synthesis engines that can be used by other applications.

Generally, the version of SAPI is determined by the platform that shipped it. SAPI 5.1 was included with Windows XP along with the Microsoft Sam TTS engine. The initial release of Windows XP did not include a speech recognition engine. The Tablet PC Edition of Windows XP did include version 6.1 of Microsoft's speech recognition engine. This was also shipped with Office 2003. Office 2003 also included SAPI TTS voices from Lernout & Hauspie, called LH Michael and LH Michelle. Also note that some vendors include SR and TTS engines with their products. For example, my laptop came with speech recognition and TTS engine provided by Toshiba.

With Windows Vista, the version of SAPI that is installed is 5.3. We have replaced the Microsoft Sam voice with next generation technology in a new female voice we call Microsoft Anna. We have also made major improvements to the speech recognition engine (now version 8.0) and that is included in all editions of Windows Vista.

For the SDK, you can download the SAPI 5.1 SDK to create applications and engines that work on Windows XP and Windows Server 2003. These applications or engines should also be forward-compatible with SAPI 5.3 on Windows Vista and beyond. The SAPI 5.1 SDK is a stand-alone package, separate from other Microsoft SDK's.

With SAPI 5.3, we integrated our SDK into the main Windows SDK (sometimes known as the Platform SDK). You can use the Windows SDK to create applications for Windows Vista, Windows XP, and Windows Server 2003. What OS version you target is done at compile-time, and that prevents features that only exist in latter versions from being available.

You can get an ISO image to burn the SDK to a DVD here:

http://www.microsoft.com/downloads/details.aspx?familyid=7614FE22-8A64-4DFB-AA0C-DB53035F40A0

To selectively download and install various components of the Windows SDK, go here:

http://www.microsoft.com/downloads/details.aspx?FamilyId=C2B1E300-F358-4523-B479-F53D234CDCCF

Something else that is new is our Managed Speech API's. Codenamed SpeechFX, the Managed Speech API is part of the Microsoft .NET Framework 3.0. The new System.Speech namespace provides managed classes for speech recognition and synthesis. This makes it much easier to write speech applications from managed code, such as C# or Visual Basic .NET.

The Managed Speech API documentation is included with the Windows SDK. Applications that use .NET Framework 3.0 will work on Windows Vista, Windows XP and Windows Server 2003. Note that you have to redistribute the .NET Framework 3.0 with your application for Windows XP and Windows Server 2003. The framework is already included with Windows Vista.

Internet Explorer, Screen Readers and Keyboard Access

Charles Oppermann — Wed, 08 Nov 2006 01:53:00 GMT

When I get around to posting a bio, you'll find out that I have a background in making technology accessibility to people with disabilities. I worked in this area for Windows 95 through Windows 2000 and Internet Explorer 2.0 through 4.01.

With the recent release of IE7, there have been the usual questions about whether adaptive aids will work with it, such as screen readers for the blind and visually impaired.

Kelly Ford, a tester in the IE group has produced a nice write up regarding this. Read it at the IE Team Blog.

Steve Wozniak at Microsoft

Charles Oppermann — Sat, 07 Oct 2006 05:35:00 GMT

One of the things I love about Microsoft is that you often get talk with, or listen to interesting people. Over the years, I got to shake hands with Steven Spielberg (wearing a Microsoft Bob baseball cap), Jay Leno (Windows 95 launch), Stevie Wonder (accessibility event), and just the other week, former NFL player Mike Utley. I just missed out on meeting Muhammad Ali once because I was heads down on shipping IE 4.01.

Today, Steve Wozniak, co-founder of Apple Computer came by the campus. "Woz" as his is commonly known, was a electronics geek in his teens and designed many little gadgets before Steve Jobs convinced him to sell his micro computer design that HP rejected. That became the Apple I in 1976.

Woz was giving a speech and signing copies of his book, "iWoz". He gave a well practicied, but informal talk about his early years of fiddling with electronic things and his early interactions with Steve Jobs.

Jobs and Woz considered themselves best friends, but in all the accounts of I've read of both men, they really couldn't be much different. Woz was grounded and knew he wanted to be an engineer, while Jobs was a free spririt. In his talk, he didn't dispell that image. Woz describes Jobs as having a lot of strange friends, doing strange things, and being a "free thinker" several times. It makes me wonder what they saw in each other in the first place.

In all, a nice talk by Steve.

UPDATE on Monday, October 9, 2006: Port 25 has a video interview with Woz on the day he visited.

UPDATE on Monday, October 9, 2006: Reworded portion on Apple I for clarity. Thanks Josh.

InterSpeech 2006

Charles Oppermann — Fri, 22 Sep 2006 11:45:00 GMT

[This posting originally appeared on my personal blog. I'm copying all my speech related blogging to this new MSDN hosted blog. I'll be doing an introduction post soon.]

I'm in Pittsburgh this week, attending the InterSpeech 2006 conference. Actually, I shouldn't say I'm attending it; I'm just staffing the Microsoft booth, giving demonstrations of Windows Speech Recognition. This is an academic conference, mainly for speech scientists and researchers to present their published papers. For example, one of the poster sessions is entitled "A Novel Framework of Text-Independent Speaker Verification Based on Utterance Transform and Iterative Cohort Modeling" which has Microsoft's own Zhengyou Zhang as one of the authors. The poster sessions which remind me of some early science fair projects because it's posted on a wall, with the research data and conclusions neatly shown.

Since Microsoft Research is one of the sponsors, they get a booth in which to demonstrate technology and products. A week ago, the Speech Research Group asked my group, Speech Components, if one of the program managers could come out and give demonstrations. I volunteered. The demos went well, and for the most part were trouble-free. I choose to use the Release Candidate 1 of Windows Vista for the demo machines, because I didn't want to risk problems with an unknown, random build. There was a small issue with the audio gain on the microphone that would set the gain at the maximum after the computer resumed from standby, or the USB headset unplugged and plugged back in. The gain is supposed to be set at 15, so when it went to a 100, recognition accuracy would plummet, but not too badly.

Usually, it was difficult at times to show the correction dialog, used when some phrase was dictated incorrectly. Even when there were hundreds of people milling about the vendor booths, and the ambient noise level very high, the system did very well.

The most often comment was similar to "this is a amazing".