Working the Spoken Word : Speech Server 2007

What a year for speech recognition at Microsoft

Stephen Potter — Mon, 31 Dec 2007 23:45:00 GMT

Yeah, yeah, the year in review, what a crushingly unoriginal idea for a post. But wait - this is worth it. 2007 was a huge year for speech recognition products at Microsoft. I think we'll look back on it as a real turning point. Here's how it shaped up.

(Going into the year, Exchange Server 2007 had just shipped with Unified Messaging, including Outlook Voice Access that gives you access over the phone to email, calendar and other useful features. It's a significant integration of speech technology into the heart of a high-volume server product. Huge posters had been up on campus for months, and inside Exchange, they called it 'the sizzle on the steak'.

Meanwhile the teams in Windows, Speech Server, automotive and core technology are hard at work... )

January 2007

Windows Vista ships with Windows Speech Recognition built into the operating system in eight different languages. Now this is a significant investment in the voice user interface as a means of commanding and dictation for desktop users. The entire desktop is speech-enabled under the 'say what you see' metaphor; correction and selection are easy; and the system adapts to your voice and your typical word usage as time goes on. Since the release of WSR, many media reviews have been overwhelmingly positive - check out Rob Chambers' blog (one of the driving forces behind speech in Vista) for links and discussions.

March 2007

Microsoft announces intent to acquire Tellme Networks. Steve Ballmer says it all:

“Speech is universal, simple and holds incredible promise as a key interface for computing. Tellme brings to Microsoft the talent, technology and proven experience in speech that will enable us to deliver a new wave of products and revolutionize human-computer interaction.”

(Incidentally, CNET has a nice inside look at the discussions in Building 34 on Superbowl day between Steve Ballmer and Mike McCue, Tellme CEO, that led up to the deal.)

Also in March - Microsoft Response Point is launched out of Microsoft Research. Response Point is a new way for small business to manage their phone systems - inexpensive, easy to set up and easy to use. All thanks to VoIP and the speech technology that underlies the user interface.

May 2007

The acquisition of TellMe closes.

September 2007

In the mobile space, Tellme announces a deal with Sprint to incorporate Tellme's voice search technology with Live Search into certain phones.

Meanwhile, the first Ford cars hit the market in the USA with Sync - hands-free speech technology for voice dialing, messaging and media control within the car.

October 2007

Office Communications Server 2007 is released as the flagship of Microsoft's Unified Communications strategy. Bundled with OCS 2007 is the latest version of Microsoft Speech Server - now called Office Communications Server 2007 Speech Server (oh yes). It's a significant upgrade from Speech Server 2004, including native VoIP support, graphical dialog editing, conversational grammars, and rich data mining and tuning tools.

And - what a month - Live Search for Windows Mobile goes live with speech recognition. The speech team blog has more details of the kinds of searches possible. And you don't even need a mobile phone to make free 411 calls using the Live Search speech technology. Insider details from Long Zheng's interview with Program Manager Oliver Scholz.

So what's to come in 2008?

Let me say only that we have not been sitting around (well, actually, that's not quite true, I have been sitting around for the last month, since I was out on paternity leave. Only it wasn't really sitting around, there was a lot to do in terms of coping with the newborn's data streams and all that, but I wasn't building software, that's what I meant, now let me rescue my point) - all the teams behind these releases have been planning and executing on the next waves since even before the dates above, so huge momentum has already built in a number of areas, old and new, and we'll start to see evidence of this as the year progresses.

And - did I mention that we're hiring in a number of speech technology-related areas? Please contact me for details if you're interested.

Extracting session audio from OCS 2007 Speech Server logs

Stephen Potter — Fri, 31 Aug 2007 04:35:00 GMT

The ability to extract the audio for an entire call (both prompts and recognitions) from the Speech Server 2007 logs is a really useful feature for a number of analysis and tuning scenarios. Since the topic has surfaced a few times on the Beta newsgroups, here's a summary of how to do it.

1. Ensure your logging parameters for the server are set (via the Trace Logging tab in the Administration Console) to log Application Events and All audio for: 100% of calls.

2. When you import the log data into the database with the MssLogToDatabase utility, be sure to specify the flag /audio:session.

3. In Analytics and Tuning Studio, connect to the database and in the Session List View or Session Detail View select the call that you want to hear. Hit the Play Session Audio button to play it back using your default .wav player, or hit the Export Session Audio button to export to file.

Some notes:

1. The logging of prompt audio is not available with TIM deployments of Speech Server (i.e. you need to be running a VoIP gateway).

2. The prompt audio as recorded does not take account of any dynamic changes to timing or volume. (So if, for example, the caller accelerated or decelerated playback while the prompt was underway, you won't hear the changes, and you may notice timing mismatches on the concatenated output file. You can check for these cases by looking for the relevant event signalling the prompt rate change in the session details.)

3. Some scenarios may call for access to the audio data in environments outside Analytics and Tuning Studio. For some sample code that illustrates how to extract the audio programmatically from an OCS 2007 database, see this article. (Extracting session audio is not possible with the command line utility MssContentExtract.exe, which be used only for extracting recognition audio.)

Speech Server 2007 Public Beta available

Stephen Potter — Mon, 26 Mar 2007 23:16:00 GMT

(Tap-tap. Is this working? OK. Hh-hmm.)

We have shipped the Speech Server 2007 Public Beta!

It's available here: http://www.microsoft.com/downloads/details.aspx?FamilyId=4F4D3AA4-8223-406C-B74F-DB2DE928D8B2

Since Speech Server is now technically part of Microsoft Office Communications Server 2007, its shiny new name is Office Communications Server 2007 Speech Server. (I know. Let a marketeer post on that one. OCS2K7SS for short? Um, nice. Let's stick with Speech Server 2007, you and me.) In any case, this is a "Beta Refresh" of Speech Server that adds new features and bug fixes since the last (private) Beta in 2006.

More to come - especially on the analytics and tuning features that my team delivered in a milestone of intense coding and round-the-clock testing (literally), fuelled by team dinners, a high-latency caffeine machine and hourly refreshes of the bug glide path...

Speech in Office Communications Server - the word on the street

Stephen Potter — Sat, 19 Aug 2006 02:51:00 GMT

Now that the dust has settled on the announcement to integrate speech services into Office Communications Server, here are some choice words from the street:

Avery Glasser, Opus Research: "This is an ecosystem changing event"
Ian Rae, Silent Software: "Communication is becoming a key part of every organization. Communication across devices. Synchronous and ayschronous communication. Features such as: presence, IM, VoIP, video, Find-Me, Follow-Me, and ad-hoc conferencing. Think EMail++. For speech rec to be bundled into such a strategic product is a huge win."
Terry Gold, Gold Systems: "...even [for] most of Microsoft’s competitors, this is going to be good for business because it is going to extend speech recognition throughout the enterprise. The world of communications in general is going to grow and change in fundamental ways, and a lot of people will benefit from Microsoft’s massive investment in this world. " (also check out Terry's post for a nerve-wracking, blow-by-blow insider view of the demo at SpeechTek, and Rob's too)
Marshall Harrison, developer of MSS apps: "It will give us the ability to do IM, have presence awareness; speech enable audio and video conferencing and better voice and call management. I'm really stoked about the communications possibilities that this will open up."

And if I may put my marketing hat on for a moment (what? it's already on? oh. thanks) - I think there's a massive role for speech recognition services in today's communications landscape. We're only just beginning to get a sense of it. And by not only integrating automatic speech recognition capabilities into the core of the communications stack but also investing significantly in tools and APIs for developers to make the most of it - I think we've got a platform here that will see in a whole new wave of speech recognition apps.

More details of the MSS/OCS integration:

Grammar Advisory

Stephen Potter — Thu, 18 May 2006 02:00:00 GMT

Busy time lately, what with shipping the Beta, preparing for the partner events (see some partners here and here), and spec'ing the next round of tuning and reporting features... but let me not leave the blog dark a minute longer.

You saw this from the MSS 2007 Beta press release, right:

Advisor Technology in Speech Server 2007 provides best-practice guidance during authoring and refinement of the grammar, which is the vocabulary of a speech-enabled application.... The Grammar Tuning Advisor intelligently clusters unrecognized words, giving contact centers the information they need to improve call completion rates. After tuning an application, developers can validate their improvements against actual caller responses to proactively identify any new issues.

I really like this feature. I demo'd it at the partner events this week, and got some great feedback - and I'll be demoing it again tomorrow in the technical sessions. I'll try and blog more on the details next week. We used to call this the Grammar Doctor internally, but the medical associations didn't quite do it for Clint and his marketing team, so they went for the warmer Advisor tone. I don't know. I prefer to think of untuned applications as sick things in need of a doctor rather than unmentored things in need of an advisor, but hey, I'm the engineer...

Speech Server 2007 Analytics Tools - a preview

Stephen Potter — Thu, 20 Apr 2006 01:03:00 GMT

The recent announcement of Speech Server 2007 included a promise of "Deep Caller Behavior Insight". Heck of a tag line that, so what's it all about? In a phrase: better analysis of what real callers are doing with your system. We've built a tool called the Speech Server Analytics and Tuning Studio that's integrated with the design tools in Visual Studio. It provides graphical reports of key metrics such as call volume, task completion rates and recognition accuracy, and from these you can drill through to sets of calls that have particular properties, then isolate individual sessions, tasks or dialog turns for audio playback or other analysis.

Here's a screenshot:

The tool is based on a SQL Server database (2000 and 2005 both supported), and in addition to preset filtering queries, you'll be able to write custom SQL directly for custom data reports or views of the items that you want to see. You can also use the SQL Reporting Services tools Report Designer and Report Builder to create custom reports from our database schema. (We'll be shipping reports in RDL format at final release time, but they're not in the Beta release.)

And we called the tool a "Studio" because there's more than just manual analysis capabilities. We've included some automatic problem finding tools and also an offline batch re-recognition capability to evaluate grammar and configuration changes on real data before going live.

We also announced the Speech Server Business Intelligence Tools, for management and reporting on large scale deployments. These are based on SQL Server 2005, and include packages to migrate Speech Server data into a data warehouse and OLAP cube for rapid and efficient report querying. They will use a lot of the SQL Server 2005 stack: Integration Services for the data migration (the ETL operations), Analysis Services for the multi-dimensional component and Reporting Services for the actual reports. Again, each component is customizable with the standard SQL tool set. (One particularly nifty thing about Analysis Services is the built-in data mining algorithms that will help anyone who's interested in doing deeper trend analysis or other data mining on their customers' speech behaviour.)

The MSS 2007 Beta is available next month, sign up here.

Microsoft and VoiceXML

Stephen Potter — Wed, 05 Apr 2006 19:47:00 GMT

Well, for once the rumours are true. Microsoft issued a Press Release this morning announcing that Speech Server 2007, due later this year, will support not only SALT (and some new .Net speech APIs), but also VoiceXML. We have also joined the VoiceXML Forum. As former chair of the SALT Forum Technical Working Group, I have only this to say: "Bravo!"

This is an example of us listening to customers - many customers in many different camps. In some cases, the standards question was getting in the way of the issues of real importance. And with that question now removed, the benefits of Speech Server in resolving the real issues should become much clearer.

So what's our position on SALT? As strongly behind it as ever. We have a strong and growing SALT customer base that we'll actively support for years to come. And all the points I've previously made about the technical differences between SALT and VoiceXML still apply - now they apply more specifically to your choice of standards on MSS 2007...

When the dust settles on this one, more about the other feature announcements in the PR to come.