Rob's Response Point : Speech

SP2 has been updated to support speech barge-in

RobertBrown — Sat, 21 Feb 2009 04:50:00 GMT

You wanted barge-in back. We heard you. It's back. Response Point SP2 now supports speech barge-in as a checkbox in the Configure Automated Receptionist dialog box: To install it.... Rather than supply a separate patch that would need to be applied every...(read more)

Streamlining your Automated Receptionist Greeting

RobertBrown — Sat, 14 Feb 2009 02:32:00 GMT

(Don't ping me on barge-in right now - or at least, if you do, I don't have any more info yet. Hopefully soon. UPDATE 3/14/09: barge-in is supported in SP2 ) In the meantime, here's a nice technique for streamlining your Automated Receptionist greeting....(read more)

Trouble-Shooting Speech Recognition

RobertBrown — Thu, 15 Jan 2009 06:22:00 GMT

(UPDATE 3/14/09: take a look at the 10 ways to optimize speech recognition quick guide.) There have been lots of posts all over the web about how cool Response Point ’s speech recognition is. But every now and then somebody tells us it’s not working out...(read more)

New Speech Server video on Channel 9

RobertBrown — Thu, 29 Jun 2006 20:04:00 GMT

http://channel9.msdn.com/Showpost.aspx?postid=208891

Albert, Mithun and Dave talk us through some of the great new capabilities of the next version of Speech Server.

I worked on the initial version of Speech Server (2004), as well as some of the plumbing of the server and API in the upcoming version. It's great to see what the Speech Server team is delivering. They've made excellent use of the last couple of years to build an impressive upgrade.

EDIT: I forgot to link to the beta download instructions: http://www.microsoft.com/speech/preview/default.mspx

---

As an aside, one of the comments (from jsampsonPC) made me grin:

"I'm an outsider to Microsofts guts, but it seems like every smart programmer on the MS team has a foreign accent Perhaps channel nine should start each show with, "Who are who, what's your position, and what country do you hail from, mate?" It would be an interesting stat to the [ratio] of foreign/local employees at MS "

I like the suggestion. The development teams I've been a part of are very cosmopolitan. It's one of the things I find enjoyable and enriching about working here.

Updating speech samples for beta 2

RobertBrown — Thu, 22 Jun 2006 20:31:00 GMT

When beta 1 shipped, we published this article in MSDN Magazine.

We've made some minor tweaks to the API for beta 2, and I figure what better way to illustrate them than to walk through the samples in that article and update them for beta 2.

First, the install. I installed two pieces of software:

Vista beta 2.
Visual C# Express Edition

I didn't need to install WinFX (it's part of Vista and installed by default in beta 2). I didn't need to install the SDK.

One of the most frequently asked questions in beta 1 was "how on Earth do I add a reference to System.Speech?". The answer then was to first install the SDK, then add a reference to "Speech", not "System.Speech". Ack! Sorry about that. It now works as it should. I just went into the "Add Reference" dialog in VC# and double clicked on System.Speech.

Today's updated sample is the first sythnesis sample from the article. The only real change is that we dropped "SpeakText()" and overloaded "Speak()" to take a string. Pretty simple.

using System;
using System.Collections.Generic;
using System.Text;
using System.Speech.Synthesis;
namespace Synth_sample_1
{
   class Program
   {
      static void Main(string[] args)
      {
         SpeechSynthesizer synth = new SpeechSynthesizer();
         synth.SelectVoice("Microsoft Anna");
         synth.Speak("Speak text has been replaced by an overload on speak.");
      }
   }
}

Well, that's all for today. More soon...

Recent ZDNet article on speech

RobertBrown — Fri, 16 Jun 2006 00:03:00 GMT

http://insight.zdnet.co.uk/software/applications/0,39020466,39274587,00.htm

A high-level tour around speech synthesis and recognition.

MSDN Magazine article on speech in Vista

RobertBrown — Tue, 15 Nov 2005 21:56:00 GMT

Okay, so I haven't been completely idle: http://msdn.microsoft.com/msdnmag/issues/06/01/speechinWindowsVista/#void

(Thanks to Robert Stumberger, Rob Chambers, and the other folks here at Microsoft who helped put this together).

Building a speech telephony app

RobertBrown — Wed, 14 Sep 2005 22:07:00 GMT

Bosky wrote:

"basically i want my speech server to be able to accept a call, quickly port it say to another 'something' . so is this possible ? i hear MS offices themselves at some places have replaced the phone opperator with spech recognition based speech servers . so how to tackle this situation of :

having multiple calls coming in to the speech server concurrently.
what changes would i need to make in program considering that right now i am not using remoting.
and what additional hardware apart from the telephony card ?"

You could build this from the ground up with SAPI (or WinFX), but that's a *LOT* of work. Telephony apps need to interface with telephony hardware, implement a lot of call control logic, and do some pretty sophisticated UI built out of question and answer dialogs. The recognition and synthesis APIs we provide are such a small piece of this, that you will still have a lot of work ahead of you.

My recommendation is to use Speech Server, since it takes care of the telephony stuff for you, and provides a lot of tools for building the dialog UI.

WinFX speech API changes in PDC CTP

RobertBrown — Wed, 14 Sep 2005 21:28:00 GMT

Joseph Kilada wrote "Robert, can you give any hints as to whether the WinFX Speech APIs have changed at all in build 5219 (or whatever the PDC build is) compared to Beta 1?"

Sometime soon I'll post updates to all the samples I've put in this blog, to make them work with the PDC CTP.

The major functionality hasn't changed. But we renamed some functions, changed some parameter lists, and rearranged some namespaces, primarily to address scenarios and feedback raised to us since we published the beta 1 release candidate. Thank you to all of you who posted and mailed feedback.

We also added a new set of classes for simplifying grammar building. We retained the SRGS classes from beta 1. But these are overkill for most grammars. So the GrammarBuilder and its associated classes were added to address mainstream scenarios.

The idea is that you'll be able to do things like "I'd like a <size> <topping> pizza" very easily:

Dim pizzaBuilder As New GrammarBuilder

pizzaBuilder.AppendPhrase("I'd like a")

pizzaBuilder.AppendChoices(New Choices("small", "regular", "large"))

pizzaBuilder.AppendChoices(New Choices("pepperoni", "cheese"))

pizzaBuilder.AppendPhrase("pizza")

'load it into the recognizer

_reco.LoadGrammar(New Grammar(pizzaBuilder))

Introducing SAPI 5.3

RobertBrown — Thu, 25 Aug 2005 02:32:00 GMT

Vista (a.k.a. Longhorn) has a new version of SAPI: 5.3.

SAPI 5.3 is an incremental update to SAPI 5.1. The core mission and architecture are unchanged across all 5.x releases. Among a variety of tweaks, SAPI 5.3 has these overall improvements:

Support for W3C grammar and TTS formats (SRGS and SSML, as well as support for one of the SML drafts)
Support for semantic interpretation script within grammars.
Performance improvements.
An overall scrub for quality and security.

The Vista beta 1 docs have an early draft of the documentation: http://windowssdk.msdn.microsoft.com/library/default.asp?url=/library/en-us/SAPI53/html/Welcome.asp

Speech recognition of interviews & videos

RobertBrown — Thu, 14 Jul 2005 19:52:00 GMT

Jonathan Tregear recently posted some comments/questions on speech recognition of interviews (in response to a brief discussion I had with Scoble in my Channel 9 interview a couple of months back). I looped in Frank Seide from Microsoft Research Asia to answer these, since he's done a lot of work in this field.

Jonathan:

Late in your interview with Robert Scoble, he asked you about the possibility of using speech rocognition to produce transcripts of his interviews for example. Your answer was that the results he would get would not be very good unless the the speech engine was trained to each of the speaker's voices.

Frank:

This is true according to our experience. In interviews, a number of problems come together. The first is spontaneous speech style -- speech is irregular with interspersed self-corrections, stutters, uhms, etc..

This can put off a recognizer. Second, recognizers always have a predefined vocabulary. For generic interviews, topics can be very broad and important words like names and special terminology might not be included in the vocabulary.

Jonathan:

I've heard about ASR engines produced by companies like Autonomy/Virage

that claim to be able to do a decent job of speaker independent and unconstrained domain voice recognition for similar uses like indexing and and searching newscasts etc. Do you have experience with or an opinion about how good those engines are?

Frank:

If the transcript should be searchable but not necessarily readable, lower accuracies will be sufficient. Errors happen overproportionally in short words like "is" that are needed for readability but you don't care about for searching. The downside is, however, that for searching, the topic-specific words like names and terminology are often the ones you are interested in. If your vocabulary contains them, you are fine, otherwise it will not work. Then you will need to resort to phonetic search techniques.

I do not know how good the mentioned engines are. State-of-the-art research systems achieve ~90% accuracy for broadcast news transcription.

News is, however, a rather narrow domain, and a broadcast news recognizer will not work well for say entertainment, sports, or talk shows.

Jonathan:

Related to this is another question I've been wondering about: Suppose you pointed an engine at a video like the interview example above, but instead of using it to produce a transcript of the interview you were only interested in finding instances of a well defined list of keywords.

This would be useful in indexing and searching libraries of audio content also. Would that be an easier problem to solve for speaker independent (i.e. untrained) speech recognition?

Frank:

You need to distinguish between frequent (common) and (uncommon) infrequent keywords. Frequent words are often rather short, and recognizers can only reliably recognize them considering also the words surrounding them. For those, a keyword spotter as you describe would not work well. Infrequent words, on the other hand, are also often longer, and surrounding word context is less helpful as those words are rare and thus reliable statistics is not available anyway. For those, the method can work well.

Speech interview on Channel 9

RobertBrown — Sun, 29 May 2005 06:21:00 GMT

Channel 9 just posted an interview with me.

Some links to stuff I talk about:

The new speech API I posted about a few days ago
The app I demo when I dial 0 is running on Speech Server, and the case study I mention is here.

You may also be interested in some of our research web pages, since I mention them at one stage in the interview: Synthesis research; Speech research in Redmond; Speech research in Asia.
I also mentioned the Voice Command app for Windows Mobile.