You Talkin' to Me

POSTED BY: JOHN CLARKSON, MSS Programmer Writer

In cases where prompts need to be changed frequently, it is not necessary to recompile and redeploy a .prompts file following every change. Since a .NET speech application can use more than one .prompts file simultaneously, it is possible to put the prompts that change frequently in separate, smaller databases. When a prompt changes, recompile and redeploy just the one file containing that prompt. Use the Manage Prompt Databases... link in the Properties window, or the Manage this application's prompt databases link on the Voice Output pane in the property builder to add additional prompt databases.

Alternatively, use the SALT content tag to set a frequently changed prompt to reference a single wave file, which can be updated as necessary. To do this, enter

<salt:content href='/MyWav.wav' \>

as the text of the inline prompt, where MyWav.wav references the audio file.

POSTED BY: RENAUD LECOEUCHE, MSS Software Dev Engineer

Some voice applications consider the user being silent as a valid answer. For example, an application could say: “Please choose one of the following products or stay on the line to speak to a sales representative” followed by a list of products that are available for phone order. If the user stays silent for a second or so, the call continues and a transfer is made.

As you probably know, the way to ask a question and get an answer in MSS 2007 Beta is to use a QuestionAnswerActivity. However the QuestionAnswerActivity will keep repeating until it gets a valid recognition. The question then is: How to model this kind of dialog with MSS?

The answer is easy. Here is how to proceed:

1. Create a QuestionAnswerActivity that recognizes the product names.
2. Create a SpeechSequenceActivity.
3. Add the QuestionAnswerActivity to the SpeechSequenceActivity.
4. Go to the SpeechEvents view of the SpeechSequenceActivity.
5. Add a ConsecutiveSilencesSpeechEventActivity and set its MaximumSilences property to 1.
6. Add a GotoActivity to the ConsecutiveSilencesSpeechEventActivity that redirects to the part of the dialog you want to run when the user remains silent.

You are done! Adding a ConsecutiveSilencesSpeechEventActivity allows you to take arbitrary actions when the user stays silent. Because SpeechEvents are associated with SpeechSequenceActivity activities, you can localize their actions to a single QuestionAnswerActivity or a group of related activities.

How speech events work
Speech events provide a powerful way of controlling your application flow. Let me clarify how they work. Speech events are scoped to a SpeechSequenceActivity activity. When the speech sequence is executing, its associated speech events track the changes in the workflow’s History property. This is how they know what is happening in the dialog and when they can execute.

For example, a ConsecutiveSilencesSpeechEventActivity activity increments its count when a SilenceHistoryItem is assign to the History property. SpeechEvents count the number of events globally. The ConsecutiveSilencesSpeechEventActivity activity increments its count when a SilenceHistoryItem from any activity is assigned to the History property. It does not matter whether the first silence and second silence come from the same or different activities. Similarly, it resets its count when a recognition from any activity is detected. The same is true for other speech events.

Activating speech events
Now that we know how a speech event tracks what happens in the dialog, we can look at how the speech events are activated. The workflow decides whether to run a speech event when a QuestionAnswerActivity activity completes a turn. This is the only point where a speech event can be started. If the speech event is active because its count matches the maximum allowed number of events, the workflow suspends the QuestionAnswerActivity activity and runs the speech event. Once the speech event completes, the QuestionAnswerActivity activity restarts, unless it was cancelled. I’d like to emphasize the point: you cannot force a speech event to execute only by changing the History property of the workflow. Changing the History property will change the speech events counts, but it won’t force activation. Activation is decided by the workflow.

Bail-out logic
Let’s go back to the solution presented above. There is a caveat worth mentioning. SpeechEvents are often used to trigger a bail-out: if the user stays silent three times in a row, the dialog is transferred or terminated.

In our example above, the silence that is used to move to another part of the dialog will count towards any bail-out limit you may have. If you have a SpeechEvent that triggers after three silences, the user can only stay silent twice now. This is due to the global nature of the speech event count: recall that it does not matter where silence comes from.

In this case, one comes from the initial question answer activity and the next ones come from the following question answer activity. In order to reset the count, you need to assign a RecognitionHistoryItem to the History property of the workflow. This makes sense: the first silence that was detected is not really a silence but a valid way for the user to interact with the system. By setting the history, you let the speech events know that. Setting the History is easy:

Workflow.History = new RecognitionHistoryItem(QAThatTriggeredTheSilence, false /*no bargein*/);

does the job. Changing the History property in this way will ensure that your bail-out logic works as expected.

POSTED BY: AHMED STEWART, MSS Software Dev Engineer

Today, I’m going to talk about the MenuActivity of MSS 2007 Beta.

The Menu, like the name implies, allows easy selection of an item from a menu. Under the covers the activity creates a grammar consisting of the items supplied to it.

Menu, like the other high level activities, supports confirmation, but since I’ve already talked about in the post about the GetAndConfirm Activity, I won’t go over it again here.

Now, let’s get into using the MenuActivity:

The code below creates a string array that contains the options that the user can choose from. The Executing handler then assigns the array to the DataSource of the MenuActivity and Binds the data to the activity. The binding process just creates the required grammars from the options.

     private string[] _choicesList = new string[] { "one", "two", "three", "four", "five" };
     private void menuActivity1_Executing(object sender, ActivityExecutionStatusChangedEventArgs e)
     {
          menuActivity1.DataSource = _choicesList;
          menuActivity1.DataBind();
     }

Below is the TurnStarting handler which is pretty self explanatory.

     private void menuActivity1_TurnStarting(object sender, TurnStartingEventArgs e)
     {
          //The actual text of the prompt
          string optionText = "This is option {0}. ";

          //Clear the prompt
          menuActivity1.MainPrompt.ClearContent();

          //For each option, add a bookmark and a prompt for it.
          foreach (string option in _choicesList)
          {

menuActivity1.MainPrompt.AppendBookmark(option);

menuActivity1.MainPrompt.AppendText(String.Format(optionText, option));
          }
     }

The code above that constructs the prompt text inserts a bookmark before text that is associated with an option. When the default grammar recognizes “that one”, the last encountered bookmark is set as the selected option. For the curious, the constructed SSML looks like this:

<speak version="1.0" xmlns="http://www.w3.org/2001/10/synthesis" xml:lang="en-US"><mark name="one" />This is option one. <mark name="two" />This is option two. <mark name="three" />This is option three. <mark name="four" />This is option four. <mark name="five" />This is option five. </speak>

Custom grammars can also be used for the “that one” recognition by adding them to the Grammars collection of the activity. If you decide to go this route, you’re going to have to set the selected option yourself. The function below is a sample grammar recognized handle that calls the appropriate method:

     void customTimedRecoGrammar_Recognized(object sender, RecognizedEventArgs e)
     {

menuActivity1.SetSelectedOptionToMostRecentBookmark(sender, e);
     }

After the activity completes execution, the selection is available via the SelectedOption property.

POSTED BY: AHMED STEWART, MSS Software Dev Engineer

Today I’d like to talk about one of the new activities that we’ve created in MSS 2007 Beta to handle common tasks, the Get and Confirm activity. Like the name suggests, this activity simple gets an utterance from the user and based on the confidence of the recognition, confirms it with the user or not.

The GetAndConfirmActivity, G&C for short, allows the developer to specify a grammar and based on the confidence of the recognition confirm the recognition. If the confirmation is successful, the activity exits, otherwise the get turn is started again to get a new recognition.

The control has default confirmation grammars that can recognize, yes and no as well as accept an updated reply. E.g.:

     App: Did you say blue?

     Caller: No, Red.

     App: Did you say red?

The example below will walk through a very simple usage of the Activity and shows how to add and use a custom Dtmf grammar during the confirmation phase of the recognition. I prefer to give examples and describe what they do so hopefully everything is nice and easy to understand.

I like to use the Executing handler to do things that are not going to change for the duration of the control’s execution so in the handler below I assign the grammar and prompt for the “get” turn to the control. The next section of code generates and assignes the Dtmf grammar for the confirmation phase as well as specifying a handler for the Grammar’s recognized event where the determination is made as to accept or deny the recognition if Dtmf is entered. By default, the confirmation threshold is set to one so the recognition is always confirmed, to alter it simply change the value. The confirmation threshold is the starting point where confirmation will not occur.

     private void getAndConfirmActivity1_Executing(object sender, ActivityExecutionStatusChangedEventArgs e)
     {
          string[] grammarChoices = new string[] { "one", "two", "three", "four", "five" };

getAndConfirmActivity1.Grammars.Add(GenerateGrammar(SrgsGrammarMode.Voice,grammarChoices));
          getAndConfirmActivity1.MainPrompt.ClearContent();
          getAndConfirmActivity1.MainPrompt.AppendText("Say a number between one and five now.");

          string[] grammarOptions = new string[] { "1", "2" };
          Grammar confirmationDtmfGrammar = GenerateGrammar(SrgsGrammarMode.Dtmf, grammarOptions);
          confirmationDtmfGrammar.Recognized += new EventHandler<RecognizedEventArgs>(confirmationDtmfGrammar_Recognized);

getAndConfirmActivity1.ConfirmationDtmfGrammars.Add(confirmationDtmfGrammar);

     }

The Confirmation Turn starting handler is pretty basic, it just specifies the prompt that will be played during the confirmation turn.

     private void getAndConfirmActivity1_ConfirmationTurnStarting(object sender, TurnStartingEventArgs e)
     {

getAndConfirmActivity1.ConfirmationMainPrompt.ClearContent();

getAndConfirmActivity1.ConfirmationMainPrompt.AppendText("The last recognition result was, " + getAndConfirmActivity1.RecognitionResult.Text +
          ", if this is correct say yes or press 1. If not say no or press 2.");

     }

The last piece of the puzzle is the handler for the confirmationDtmfGrammar_Recognized. It simply checks the value of the recognized Dtmf digits and either accepts the recognition or denies it.

     void confirmationDtmfGrammar_Recognized(object sender, RecognizedEventArgs e)
     {
          if (1 == int.Parse(e.Result.Text))
          {
               getAndConfirmActivity1.Accept(sender, e);
          }
          else
          {
               getAndConfirmActivity1.Deny(sender, e);
          }
     }

In this example the default grammars provided with the control are still active. To disable them, just set the value of getAndConfirmActivity1.UseDefaultGrammars to false.

Since I know some of you are curious about what it contains here’s the GenerateGrammar function.

     /// <summary>
     /// This function generates a grammar with the members of the provided string array as choices.
     /// </summary>
     /// <param name="grammarMode">The mode of the grammar, either Dtmf or voice.</param>
     /// <param name="grammarChoices">The string array containing the choices in the grammar.</param>
     /// <returns>The constructed grammar object.</returns>
     public static Grammar GenerateGrammar(SrgsGrammarMode grammarMode, string[] grammarChoices)
     {
          if (0 >= grammarChoices.Length)
          {
               throw new ArgumentNullException(grammarChoices.ToString());
          }

          SrgsDocument grammarDoc = new SrgsDocument();
          grammarDoc.Mode = grammarMode;

          //The name of the rule that will be created
          SrgsRule rule = new SrgsRule("GrammarOptions");
          rule.Scope = SrgsRuleScope.Public;
          SrgsItem item = new SrgsItem();
          SrgsOneOf choice = new SrgsOneOf(grammarChoices);
          item.Add(choice);
          rule.Add(item);
          grammarDoc.Rules.Add(rule);
          grammarDoc.Root = rule;
          return new Grammar(grammarDoc);
     }

POSTED BY: MARK PARKER, MSS Programmer Writer

If you are developing a Microsoft Speech Server 2007 Beta voice response application that writes data to a disk file, you will need to grant write access to the file (or the directory that contains the file) to either the ASPNET local account or the NETWORK SERVICE local account. MSS 2007 Beta uses one of these accounts whenever it creates or writes to a file.

If you are running MSS 2007 Beta on Microsoft Windows XP, grant write access to the ASPNET local account. If you are running MSS 2007 Beta on Microsoft Windows Server 2003, grant write acces to the NETWORK SERVICE local account.

To grant write access to the ASPNET local account in Windows XP:

1. In Windows Explorer, navigate to the directory that contains the file(s) you want to write to.
2. On the right-click menu, select Sharing and Security.
3. On the property page for the directory, select the Security tab.
4. Click Add…
5. On the Select Users, Computers, or Groups page, enter MachineName\ASPNET in the textbox, and then click OK. (For MachineName, use the name of the machine as found in the My Computer properties.) On the property page for the directory, an entry for ASP.NET Machine Account (MachineName\ASPNET) now appears in the Group or user names: pane.
6. In the Allow column in the lower half of the property page, check the Write box, then click Apply, and then click OK.

To grant write access to the NETWORK SERVICE local account in Windows Server 2003:

1. Perform steps 1 through 4 in the previous procedure.
2. For step 5, enter NETWORK SERVICE in the textbox. You do not need to prefix this with the machine name.
3. Perform step 6 in the previous procedure.

POSTED BY: KEN CIRCEO, MSS Lead Tech Writer

Now that I'm over the initial jolt of learning that Speech Server will be integrated with Office Communications Server (nee LCS), I thought I'd take a minute to write down my reaction and how I expect the integration to impact my work. (I'm just talkin' here. It's therapeutic. Bear with me.)

I don't mind change. If I did, I wouldn't last long in this industry. Making software isn't like making aluminum. Things don't just stay the same year after year, decade after decade. So when someone above me decides to integrate my product with another product, I accept it. In fact, at my pedestrian level, accepting the change is critical; understanding the reason behind it is far less important.

After last week's announcement, one of our partners said that adding MSS to OCS will give his company "a more extensive set of capabilities on which to build compelling unified communications solutions for our customers." Wow. It's hard to read that without marvelling at the jargon. I wonder if that was off the top of his head.

One of our VPs (I forget which one) calls it tipping over the siloes. I don't doubt that, but I also don't get it. I'm going to need Tillman to explain to me exactly what "tipping over the siloes" means. Maybe it has something to do with merging the API sets, or at least letting them hook into each other. Whatever it is, Tillman will straighten it out. He has a good bird's eye view on the software industry. I bet he can instantly see how integrating MSS with OCS will affect our customers, their companies, and the MSFT stock price. Also, he's more left-brained than I am, which is why he's got a far better shot at becoming a VP than I do. But, trust me, I can live with that. Both Tillman and I know that I'm not the brightest guy in the world (see picture).

So as far as the whole integration goes. Hey, if it's ok with Microsoft, it's ok with me. I'm still running the video project, doing the whole Community Lead thing, and cranking out Help docs and whitepapers. And I'm sure in the weeks ahead, even the technical details will start making more sense to me.

Really. It's all good.

POSTED BY: KEN CIRCEO, MSS Lead Tech Writer

Today from the SpeechTEK 2006 conference in New York, Microsoft Corp. announced that the full capabilities of Microsoft(R) Speech Server 2007 will be integrated into Microsoft Office Communications Server 2007, extending the company's commitment to unified communications and breaking down today's silos of instant messaging, Internet Protocol telephony, voice response, audioconferencing and videoconferencing. More...

Read the GotSpeech guy's response

POSTED BY: KEN CIRCEO, MSS Lead Tech Writer

The year's biggest speech conference, SpeechTEK 2006, will be held next week at the Marriott Marquis in NYC. I wanted to go, but Microsoft turned me down. Probably with good cause. I just wanted the free T-shirts. It doesn't matter. I'm already over it. (Really.)

SpeechTEK is now in its twelfth year. It started in Boston in 1995, then moved to the Big Apple the next year and it's been there ever since. If you're one of the lucky ones who gets to go this year, look for Microsoft in Booth 502. You can hardly miss it. It's one of the large booths that hits you just as you walk into the Expo Hall.

Also, here's a list of MSS sessions and speakers. If you haven't heard these speakers before, I know most of them pretty well. They're all good speakers. (If they weren't, I'd tell you.) Rich Bray is giving the keynote on Tuesday at 8:30AM. Have a great time. And don't worry about me. I'm fine. (But if you get an extra T-shirt, you can send it to me at One Microsoft Way, Redmond, WA 98052. No obligation or anything.)

Mon, Aug 7

VOX 2006-Service Provider Edition: Expanding Your Reach in Real-Time: SOA or SOL: Leveraging Web Infrastructure Panel Discussion

ALBERT KOOIMAN, Senior Business Development Manager

1:30PM–2:15PM in the Astor Ballroom

Tue, Aug 8

KEYNOTE: Unified Communications - The Next Wave of Speech Technology Applications

RICHARD BRAY, General Manager in Unified Communications

8:30AM–9:45AM in the Broadway Ballroom

Advanced Speech Technology Symposium (AVIOS): Advanced Speech and NL Technology Panel Discussion

ALEX ACERO, Research Area Manager

11:00AM–12:30PM in the Plymouth/Royale Room

VUI Track (Authoring & Tuning): Tuning the VUI, Not Tuning The Parameters Panel Discussion

STEPHEN POTTER, Program Manager

4:00PM–5:00PM in the Gramercy/Olmstead Room

Consumer Electronics & MM Track: Devices Panel Discussion

ALEX ACERO, Research Area Manager

4:00PM–5:00PM in the Uris/Shubert Room

Wed, Aug 9

Developing and Tuning Applications with MSS 2007

DAVID OLLASON, Lead Program Manager

STEPHEN POTTER, Program Manager

10:00AM–12:30PM in the Wintergarden/Palace Room

Improving Customer Loyalty: Building An Effective Self-Service Strategy Panel Discussion

ABHIJIT SARKAR, Senior Business Development Manager

11:00AM–12:00PM in the Music Box/Majestic Room

Consumer Electronics & MM Track: Overcoming Barriers to Multimodal Application Development Panel Discussion

ALBERT KOOIMAN, Senior Business Development Manager

2:00PM–3:30PM in the Uris/Shubert Room

Strategic & Management Insights Track: Mobilizing Your Workforce Panel Discussion

CLINT PATTERSON, Director of Product Management

2:00PM–3:30PM in the Soho/Herald Room

Are These Technologies Ready for Primetime Track: You Decide – Unifying Your Communications Panel Discussion

CLINT PATTERSON, Director of Product Management

3:45PM–5:00PM in the Chelsea/Gotham Room