Rob's Rhapsody : Q&A

Macro of the Day: Send Email to [OutlookContact]

robch — Mon, 09 Jun 2008 17:32:14 GMT

Today's macro let's you say things like "Send email to Rob", or "Send email to Rob Chambers", and as long as I'm a contact in your Outlook Contacts, you can send me mail! It also allows you to say "Send email to Rob Chambers about That cool outlook macro", it'll open up Outlook, create a new email, put me as the recipient, and it'll set the subject to "That cool outlook macro". Neat, eh?

Today's macro was inspired by Brad, over in the ms-speech Yahoo! Group, who asked me if WSR Macros could dynamically create rule content last week. Well, Brad, yes it can! Here's an example.

The most interesting part of today's macro, is the ruleScript element that allows you to define a rule programmatically. The key to this script is that it will find content to add to the rule by hooking up to Outlook's object model (using Create Object("Outlook.Application")), and then call the Rule.Items.AddItem method for each email address it finds.

That ruleScript looks like this:

<ruleScript name="OutlookContact" propname="EmailAddr" language="VBScript">
    <![CDATA[
      Call AddOutlookContacts

      Sub AddOutlookContacts

        REM Debug.DebugBreak
        Debug.Trace chr(13) & "Adding OutlookContacts..." & chr(13)

        Const olFolderContacts = 10
        Const olMSG = 3

        Set objOutlook = CreateObject("Outlook.Application")
        Set objNamespace = objOutlook.GetNamespace("MAPI")

        Set colContacts = objNamespace.GetDefaultFolder(olFolderContacts).Items

        For Each objContact in colContacts
          If TypeName(objContact) = "ContactItem" Then
            If objContact.FullName <> "" Then
              If objContact.Email1Address <> "" Then Rule.Items.AddItem objContact.FullName, objContact.Email1Address, True
              If objContact.Email2Address <> "" Then Rule.Items.AddItem objContact.FullName, objContact.Email2Address, True
              If objContact.Email3Address <> "" Then Rule.Items.AddItem objContact.FullName, objContact.Email3Address, True
            End If
          End If
        Next
        For Each objRuleItem in Rule.Items
          Debug.Trace objRuleItem.Phrase & " (" & objRuleItem.Property & ")" & chr(13)
        Next
        Debug.Trace "--" & chr(13) & "Added " & Rule.Items.Count & " items!" & chr(13)

      End Sub
    ]]>
</ruleScript>

There are a couple other interesting things in today's macro:

I used a Debug.DebugBreak method (currently commented out above), to allow me to debug the macro interactively in Microsoft Visual Studio Express. This is a handy way of single stepping over each and every line in the code. If you have VS installed, when the macro hits that line (if you uncomment it), you'll be asked what debugger to use to debug the macro's script. You pick VS, and then you have F10 single-step control over the macro as it executes. You can do fun things like inspect variables, change flow, etc... All really cool stuff that developers using Microsoft Development tools have grown accustomed to doing.
I also used Debug.Trace to output to VS what's going on in the macro, so even if I'm not single step debugging it, I can see what contacts got added.
I used the disambiguate tag to allow the user to disambiguate what email address to use if it's ambiguous. So, for example, if I have 3 people named Rob in my contacts, it'll allow me to pick which one I wanted to send email to. In fact, it'll even show me the email addresses in parenthesis after the person's full name.

Here's the entire macro:

<command priority="2">
    <listenFor>send email to [OutlookContact]</listenFor>
    <listenFor>send email to [OutlookContact] about [subject...]</listenFor>
    <disambiguate title="Send email to whom?" prompt="Who would you like to send email to?" timeout="30" propname="OutlookContact"/>
    <script language="VBScript">
      <![CDATA[
        Set outlookApp = CreateObject("Outlook.Application")

olMailItem = 0
Set newMessage = outlookApp.CreateItem(olMailItem)

        newMessage.Recipients.Add("{[OutlookContact.EmailAddr]}")
        newMessage.Subject = "{[subject...]}"
        newMessage.Display
      ]]>
    </script>
</command>

<command priority="1">
    <listenFor>send email</listenFor>
    <listenFor>send email to *</listenFor>
    <listenFor>send email to * about [subject...]</listenFor>
    <script language="VBScript">
      <![CDATA[
        Set contacts = CommandSet.RuleGenerators("OutlookContact")
        For Each item in contacts.Rule.Items
          ChooseFromList.Items.AddItem item.Phrase, item.Property, True
        Next
        ChooseFromList.Items.Sort
        chose = ChooseFromList.Choose("Send email to whom?", "Who would you like to send email to?")
        If chose > 0 Then
          Set outlookApp = CreateObject("Outlook.Application")

          olMailItem = 0
          Set newMessage = outlookApp.CreateItem(olMailItem)
          newMessage.Recipients.Add(ChooseFromList.Items(chose).Property)
          newMessage.Subject = "{[subject...]}"
          newMessage.Display
        End If
      ]]>
    </script>
</command>
<command>
    <listenFor>Refresh ?Outlook Email Contacts</listenFor>
    <listenFor>Refresh Outlook ?Email Contacts</listenFor>
    <script language="VBScript">
      <![CDATA[
        Set contacts = CommandSet.RuleGenerators("OutlookContact")
        contacts.Rule.Items.RemoveAll
        contacts.Script.AddOutlookContacts
        contacts.Rule.Commit
      ]]>
    </script>
</command>

<ruleScript name="OutlookContact" propname="EmailAddr" language="VBScript">
<![CDATA[
Call AddOutlookContacts

Sub AddOutlookContacts

REM Debug.DebugBreak
Debug.Trace chr(13) & "Adding OutlookContacts..." & chr(13)

Const olFolderContacts = 10
Const olMSG = 3

Set objOutlook = CreateObject("Outlook.Application")
Set objNamespace = objOutlook.GetNamespace("MAPI")

Set colContacts = objNamespace.GetDefaultFolder(olFolderContacts).Items

        For Each objContact in colContacts
          If TypeName(objContact) = "ContactItem" Then
            If objContact.FullName <> "" Then
              If objContact.Email1Address <> "" Then Rule.Items.AddItem objContact.FullName, objContact.Email1Address, True
              If objContact.Email2Address <> "" Then Rule.Items.AddItem objContact.FullName, objContact.Email2Address, True
              If objContact.Email3Address <> "" Then Rule.Items.AddItem objContact.FullName, objContact.Email3Address, True
            End If
          End If
        Next
        For Each objRuleItem in Rule.Items
          Debug.Trace objRuleItem.Phrase & " (" & objRuleItem.Property & ")" & chr(13)
        Next
        Debug.Trace "--" & chr(13) & "Added " & Rule.Items.Count & " items!" & chr(13)

End Sub
]]>
</ruleScript>

</speechMacros>

A new Microsoft Speech Blog

robch — Tue, 04 Sep 2007 18:16:00 GMT

The Speech Components Group (the team I work on) just created a new team blog for all things Speech @ Microsoft.

This is the spot that people in my team that don't have their own blog will answer questions we get from the community, and where we'll let you all know about the neat new things our team is working on.

Check it out: http://blogs.msdn.com/speech/.

In Mike's first post, he talks about language support in Vista.

Windows Speech Recognition - ExactMatchOverPartialMatch

robch — Mon, 07 May 2007 17:57:27 GMT

A user wrote in over the weekend with a question about how to automatically choose an item when prompted with our clarification user interface in Windows Speech Recognition (WSR). But ... Before I answer that question, I have to give a little background.

What is our clarification user interface?

When the recognizer hears the user say something ambiguous and we're not sure what to do, the system will ask the user for clarification (or confirmation). For example, if the user says "File" in an application that both has a "File" menu and a button somewhere that has the word "File" in it (like a toolbar button that is for creating a "New File"), little numbers will fly out and hover over the two items on the screen. The user can then choose, "1", or "2", and then the user can say "OK".

This is related to the feature that David Pogue wrote up as one of 2006's top 10 features of the year. That feature treats everything as ambiguous when the user says "Show Numbers".

There is another form of the clarification user interface, as well, that we use for items that don't have an on screen representation, like applications to start. In this example, if the user says "Start Internet Explorer", the system sees that there are two possible matches, "Internet Explorer" and "Internet Explorer (no add-ins)". Since we're not 100% sure which one the user wanted, we'll ask the user to clarify.

How do I make it automatically pick?

Well, in general, we can't automatically pick the right one because we still haven't perfected "brain wave recognition" yet. :-) But ... Seriously, though ... You can make it better.

There's a feature called ExactMatchOverPartialMatch. It's turned off by default, but you can turn it on by changing a value in the registry. If you're not comfortable changing the registry, don't do this. But if you are, here you go:

key = HKEY_CURRENT_USER\Software\Microsoft\Speech\Preferences
value name = ExactMatchOverPartialMatch
value type = DWORD
value = 1 (default is 0)

After changing this registry setting, you'll have to restart speech. Once that's done, now in the two examples I gave earlier ("File", and "Start Internet Explorer") no clarification user interface will be displayed, because there will be one and only one exact match. "File" == "File", and "Internet Explorer"=="Internet Explorer".

ExactMatchOverPartialMatch wont' solve all your problems, but it will make the system better for a lot of people. Try it out. Let us know what you think.

SpeechSaver -- kinda like a screen saver, but for speech recognition's listening state

robch — Sat, 28 Apr 2007 20:33:00 GMT

I was talking to a customer recently, Greg Duncan, and one of the things that he came up was an idea for a utility to automatically turn the speech recognition engine's state off after a certain amount of idle "speech" time.

A couple emails later, this morning, Greg had already posted the full source code and a link to an installer for SpeechSaver on his blog. I downloaded it, installed it, and checked it out. It does exactly what it's supposed to.

Nice job, Greg.

If you want a solution that automatically turns the microphone off after a set period of time of inactivity, you might want to check it out.

Windows Speech Recognition in Vista: Dictation Everywhere

robch — Mon, 23 Apr 2007 17:53:00 GMT

Here's a question from a reader about the Dictation Everywhere feature in Windows Speech Recognition in Windows Vista:

This feature is thus very important to me and my future as a PC user.
I have found that in many form fields (such as the one I'm using to send this message) dictation is impossible unless I switch to "Enable dictation everywhere" (otherwise I get "what was that?" messages). There is no short command for switching, and even in that mode I have to confirm every fragment of text I dictate with "one, ok." And then I have to switch out of that mode again for navigation.
It is very important to me that I find ways around this, or at least that a way around it will soon exist. Would you be so kind as to let me know how I can stay up to date with Vista SR improvements, and whom I can contact about its development?
Thank you very much for your time; I really appreciate your efforts to communicate to the public about Vista SR.

Now, for those of you that don't already know, Windows Speech Recognition allows users to dictate into any text field that supports a few standard platform APIs. These APIs (Application Programming Interfaces) are used by the speech recognition system, the handwriting recognition system, and input method editors for foreign languages.

However, some text fields in custom applications don't support what's needed. In those cases Windows Speech Recognition has a fall back input method. It's called Dictation Everywhere. You can turn it on by doing this:

Say, "Show Speech Options"
Say, "Options"
Say, "Dictation Everywhere"

That will toggle it on. Doing those three steps again will turn it off.

When Dictation Everywhere is turned on, we'll listen for the user to speak dictated text, even when they're not in an text field that doesn't support those APIs I mentioned. But ... Because the field doesn't support the right APIs, instead of just sticking the text in there, we're going to have a miniature correction experience with the user first. Otherwise, the user wouldn't be able to correct the text if there was a recognition error.

So ... If you're in one of those fields, use the three steps above to turn on dictation, then you can say, "Hello <period> This is a test <period>". The correction dialog will pop up and allow you to pick an alternative from the list. If you don't see what you really said, simple say it again. If you still don't see what you said, you can say "Spell it", and spell what you wanted.

This is the feature that the user was asking about. They'd like to be able to turn the command on and off easily by using a single voice command.

Unfortunately, since we don't have an end user feature either included directly into Vista at this time, nor do we offer one for download (yet!) for creating macros, end users can't really simulate the same impact of turning this feature on and off with a single voice command.

At least not easily...

If you're a programmer (or don't mind dabbling), you certainly could though. In fact, you can create a simple shell script that does this like this:

set Recognizer = CreateObject("SAPI.SpSharedRecognizer")
Recognizer.EmulateRecognition ("show speech options")
WScript.Sleep(1000)
Recognizer.EmulateRecognition ("options")
WScript.Sleep(1000)
Recognizer.EmulateRecognition ("dictation everywhere")
set Recognizer = Nothing

When you run this it will connect to the shared recognizer that Windows Speech Recognition uses, it will pretend that the user spoke "show speech options", then wait for 1 second, then pretend that the user said "options", again wait for another 1 second, then again, pretend the user said "dictation everywhere".

In fact, you can even save this text as a file called "Dictation Everywhere Toggle.vbs" in your start folder (e.g. "c:\documents and settings\{your user name goes here}\start menu\dictation everywhere toggle.vbs") and you'll be able to say to Windows Speech Recognition, "Start Dictation Everywhere Toggle".

Unfortunately, for all this to work, you actually have to turn Access Control (UAC) off. Otherwise, the shell script can't communicate with the shared recognizer.

In the future, though, we'll have a true end to end macro facility to deal with this in a secure way. Stay tuned for more info on that front...

Keeping speech grammars up-to-date

robch — Thu, 29 Mar 2007 20:24:00 GMT

I've been having a good conversation with some guys that are working on making it easier to program by voice using Python. Their project is called "VoiceCode" and they've been working on it for quite some time. Currently it only supports Dragon NaturallySpeaking as a speech recognizer, but I'm helping them to understand how easy it would be to port to Vista.

One issue that came up recently was: How can they keep their grammars up to date if their grammars change based on what application is running, or what the title is in any given application. I was trying to describe to them in email how they could use the SetWinEventHook method, but I wasn't being as successful describing it to them as I wanted to... So ... I just wrote about 500 lines of code for them to show them.

Then, I thought ... Hey, other people might want to see how to do this too. So I posted the code here. Let me know if you have any questions!

Installing additional speech recognizer languages...

robch — Tue, 20 Feb 2007 18:57:00 GMT

Here's a question that I get from time to time...

Q: How do I install speech recognition for a language other than the language Windows Vista is running in?

A: By installing a language pack.

Language packs are the Windows way of installing additional language support for features included in the OS. By default, when you install Windows Vista (or purchase a PC with Windows Vista already on it), it comes with a single language.

In Windows Vista Enterprise and Windows Vista Ultimate, you can go to the control panel, search for "display language", and then click on the "Install or uninstall display languages" link.

Follow the on-screen instructions, and in no time you'll have additional recognizer languages as long as it's a Windows Speech Recognition supported language. In Windows Vista we support US English, UK English, German, French, Spanish, Japanese, Simplified Chinese, and Traditional Chinese.

Transfering profiles programatically

robch — Tue, 20 Feb 2007 08:28:00 GMT

Last week I wrote about how to transfer profiles using the Easy Transfer Wizard. That generated some email and some questions in the community about how to do this programmatically.

That's a good question. But unfortunately, the answer isn't that straight forward currently.

In the SAPI documentation you can read about how some of this works, but there's not a single overview of how all this works. So ... I'll try to summarize how it works here.

Here goes ...

In SAPI, recognition profiles are accessed by the engine (and by the application) using the Resource interfaces of SAPI. Specifically, the ISpObjectTokenCategory, IEnumSpObjectTokens, ISpObjectToken, and ISpDataKey interfaces.

To get a list of recognition profiles configured by the user you can use the SpEnumTokens helper function (defined in sphelper.h). You'll need to specify the category you want to enumerate. That's the SPCAT_RECOPROFILES category. This will give you an IEnumSpObjectTokens interface, which is a standard derivative of the IEnum* class of interfaces. Using this interface, you can iterate thru the individual "tokens" by calling IEnumSpObjectTokens::Next -- each of the ISpObjectTokens you get back will represent a single recognition profile.

Given a specific token you can QueryInterface for the ISpObjectToken interface, which derives from ISpDataKey. Then you can ask for the name of the reco profile, by calling ISpDataKey::GetStringValue(NULL, &pszCoTaskMemAllocedProfileName). Be sure and call CoTaskFree on the string returned when you're done with it.

To back up the profile, there are no special helpers, so you'll just have to do a few things the hard way.

First, you should make a backup of the ISpDataKey settings. The easiest way to do this is to use the token's ID to open up the registry location directly. You see, typically the object token is simply a wrapper on top of registry access functions. The token ID is typically actually just nothing more than a path into the registry. For example, for my reco profile that I'm using right now, the token id (retrieved by calling ISpObjectToken::GetID) is: "HKEY_CURRENT_USER\Software\Microsoft\Speech\RecoProfiles\Tokens\{90E13428-A7A5-419B-8034-EA6453E31906}"

So, you'll have to write your own code to iterate thru the registry recursively and back up all the keys and values (with their names) that you find.

There are a few special things that you have to take into consideration when doing this:

1.) If the token id doesn't live directly under "HKEY_CURRENT_USER\Software\Microsoft\Speech\RecoProfiles\Tokens\", you shouldn't assume that the procedure I'm describing here will actually work to back up the recognition profile. If you see a token ID outside of this area, you should abort the backup.

2.) While backing up the profile, you will encounter one or more key names called "Files". Under keys named "Files", the values represent "pointers" to files on disk. You'll have to also back up the files that these "file names" point to.

3.) These files may start with character sequences that look like "%1c%\". If they do, you can replace the values between the percent signs with the text as returned from SHGetFolderPath. For example, one of that data files in my registry under a "Files" key has a "filename" of "%1c%\Microsoft\Speech\Files\TrainingAudio\SP_E18DB2EFB76A432B97889A3DB7210406.wav". The "%1c%" corresponds to a "real" filename of "C:\Documents and Settings\Rob\Local Settings\Application Data\Microsoft\Speech\Files\TrainingAudio\SP_E18DB2EFB76A432B97889A3DB7210406.wav". Again, you can use SHGetFolderPath with the hex value of 0x1c to get the path of "C:\Documents and Settings\Rob\Local Settings\Application Data" and you can put all this together.

4.) When you restore the profile, you should take special care to restore files with different new filenames. If you don't, there could be some collisions. You can get a new filename by calling the ISpObjectToken::GetStorageFileName method.

That's basically it.

Yeah, I know... It's not necessarily the easiest thing you'll ever do, but it is possible, and this is basically how you do it. Hopefully, we'll be able to release an actual example that shows how a real end to end backup and restore works for profiles in the future. But until then, hopefully this post will illuminate some of the magic behind profile management in SAPI.

Transferring Windows Speech Recognition profiles from one machine to another

robch — Thu, 15 Feb 2007 18:26:00 GMT

Today a customer wrote in and asked: "I was wondering if you knew if Vista stores its data files of a users speech pattern in such a way so it can be transferred to other computers; in particular if I was to re-install my operating system."

Good question!

The good news is that not only does Windows Speech Recognition (WSR) store the data files in just such a way, if you can actually use a feature in Windows Vista to do it for you automatically. The feature: the Windows Easy Transfer wizard. This wizard allows you to specify exactly what data you want to copy from one machine to another.

To use the Windows Easy Transfer Wizard:
1. Click Start
2. Type "Easy Transfer" into the search box
3. Click on "Windows Easy Transfer"
4. Click "Next"
5. Click "Start a new transfer"
6. Click "My old computer"
7. Pick a way to transfer the files (transfer cable, using a network, or removable media)
8. Answer the specific questions about that mode of transfer
9. Click "Advanced options" on the "What do you want to transfer" page
10. Uncheck all the items in the list
11. Expand the node in the tree that corresponds to your log on user name
12. Expand the "Windows Settings" node
13. Expand the "Sound and Speech Recognition" node
14. Check the "Speech Recognition" node
15. Click "Next"

That's it... All your WSR profiles will be backed up/moved.

Now, this isn't as easy as I'd like it to be for WSR users, and that's why I hope to update a tool on microsoft.com/downloads in the near future to make this 2-3 steps, instead of 15 steps... Stay tuned for more info!

Windows Speech Recognition -- Where's the API documentation?!

robch — Tue, 13 Feb 2007 22:19:00 GMT

I get a lot of email from customers every day. And some days I end up answering the same questions again and again. Don't get me wrong, I love hearing from customers, and I love enabling them to do great things with Microsoft products.

The real problem isn't that customers ask questions ... it's that the information that they need isn't always easily found. Well ... I'm hopeful that all this blogging is really going to help users. So I'm going to try an experiment and see how it works.

I'm going to try and start answering customer questions by sending them a link to my blog. If the answer is not already on my blog, I'll put it there, and then send them the freshly coined link... Sound good?

OK... So what's the first question that I get asked frequently? Well, it's probably not the most common question, but it's one near and dear to my heart as a platforms kind of guy...

Q: Where can I find Speech API documentation?

A: On MSDN of course. Try these different locations depending on how you want to call into the speech APIs:

C# and managed code: http://msdn2.microsoft.com/en-us/library/system.speech.recognition.aspx
VB and automation clients: http://msdn2.microsoft.com/en-us/library/ms720592.aspx
C++ and native code: http://msdn2.microsoft.com/en-us/library/ms723627.aspx

Do you have a question? Send it in and I'll try to get it answered...