Creating a Windows Phone 8 Translation App with Microsoft Translator and Speech Synthesis

In this Walkthrough you’ll learn how to create a Windows Phone 8 app that uses the Microsoft Translator API to translate user-input text, and speech synthesis to provide a spoken output of the translated language with a native voice.

Before beginning, you’ll need to sign up for the Microsoft Translator API in the Windows Azure Marketplace. There are a number of different offerings, including a free one, and you can see how to sign up for the free service, register your app and get your credentials here:

http://blogs.msdn.com/b/translation/p/gettingstarted1.aspx

Step 1. Getting the Windows Phone 8 Tools and SDK

The tools and SDK for developing Windows Phone Apps are available at: https://dev.windowsphone.com/en-us/downloadsdk.

Be sure to check out the system requirements before downloading and installing. For the emulator, Hyper-V is necessary, and it has requirements that your PC needs to meet, so be sure to have a system that supports them. Of course if you test on an actual physical device, you won’t need the Hyper-V dependencies.

Step 2. Creating the App with Visual Studio Express for Windows Phone

When you launch Visual Studio, and use the New Project wizard, you’ll see a number of templates that can be used to create Windows Phone apps.

Select the Windows Phone App template, and call your app Transl8 as shown here:

clip_image002

Press OK and Visual Studio will ask you which phone platform you want to target. This app will use Speech Synthesis, which is only available on Windows Phone 8, so select this platform as shown here:

clip_image003

Visual Studio will then create a project for you containing everything you need to get started with your app. In the next step, you are going to build the user interface.

Step 3. Creating the User Interface for the Translation App

The user interface for this app will be built using the MainPage.xaml file. This uses XAML (XML Application markup language) to define the user interface elements such as text boxes, text blocks and buttons etc.

Double click on MainPage.xaml in Visual Studio, and you should see something like this:

clip_image005

The designer window shows a preview of what the user interface will look like. Underneath that is the XAML code that defines this.

Change the code to look like this:

<phone:PhoneApplicationPage
    x:Class="Transl8.MainPage"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:phone="clr-namespace:Microsoft.Phone.Controls;assembly=Microsoft.Phone"
    xmlns:shell="clr-namespace:Microsoft.Phone.Shell;assembly=Microsoft.Phone"
    xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
    mc:Ignorable="d"
    FontFamily="{StaticResource PhoneFontFamilyNormal}"
    FontSize="{StaticResource PhoneFontSizeNormal}"
    Foreground="{StaticResource PhoneForegroundBrush}"
    SupportedOrientations="Portrait" Orientation="Portrait"
    shell:SystemTray.IsVisible="True">

  <Grid x:Name="LayoutRoot" Background="Transparent">
    <Grid.RowDefinitions>
      <RowDefinition Height="Auto"/>
      <RowDefinition Height="*"/>
    </Grid.RowDefinitions>

    <StackPanel x:Name="TitlePanel" Grid.Row="0" Margin="12,17,0,28"/>

      <Grid x:Name="ContentPanel" Grid.Row="1" Margin="12,0,12,0">
        <TextBox x:Name="txtToTrans" HorizontalAlignment="Left" Height="72" 
                 TextWrapping="Wrap" VerticalAlignment="Top" Width="446" 
                 AcceptsReturn="True"/>
        <Button Content="Translate" HorizontalAlignment="Left" Margin="10,131,0,0" 
                VerticalAlignment="Top" Width="436" Click="Button_Click_1"/>
        <RadioButton x:Name="optFrench" Content="French" HorizontalAlignment="Left" 
                Margin="10,59,0,0" VerticalAlignment="Top" IsChecked="True" 
                Checked="optFrench_Checked"/>
        <RadioButton x:Name="optSpanish" Content="Spanish" HorizontalAlignment="Left" 
                     Margin="150,59,0,0" VerticalAlignment="Top" 
                     Checked="optSpanish_Checked"/>
        <RadioButton x:Name="optGerman" Content="German" HorizontalAlignment="Left" 
                     Margin="296,59,0,0" VerticalAlignment="Top" 
                     Checked="optGerman_Checked"/>
        <TextBlock HorizontalAlignment="Left" Height="31" Margin="28,208,0,0" 
                   TextWrapping="Wrap" Text="Translation:" VerticalAlignment="Top" 
                   Width="117"/>
        <TextBlock x:Name="lblTranslatedText" HorizontalAlignment="Left" Height="208" 
                   Margin="28,257,0,0" TextWrapping="Wrap" VerticalAlignment="Top" 
                   Width="402"/>
        <Button x:Name="btnSpeak" Content="Speak" HorizontalAlignment="Left" 
                Margin="28,487,0,0" VerticalAlignment="Top" Width="402" 
                Click="btnSpeak_Click"/>
        </Grid>
    </Grid>
</phone:PhoneApplicationPage>

 

This will define a user interface containing everything you need for the simple translation app.

· A text box where the user enters their text

· Three radio buttons allowing them to select French, Spanish or German as the target languages

· A button to trigger the translation

· A text block to contain the translated text

· A button to trigger the speech synthesis of the translated text.

When you’re done, the user interface will look like this:

clip_image006

This markup specifies the code that will run in response to use actions, such as selecting the radio buttons or pushing the command buttons.

In the next section, you’ll configure this code.

Step 4. Writing the Code-behind

We’ll start with a couple of the shared variables that will be used across functions. These should be defined at the top of your code prior to the MainPage() constructor. You can find this in the MainPage.xaml.cs file.

        string strLngTo = "fr";
        private static string strTextToTranslate = "";

The strLngTo variable will hold the code representing the language that you want to translate to. This is set by pressing the radio buttons. Here’s the code:

private void optFrench_Checked(object sender, RoutedEventArgs e)
  {
    strLngTo = "fr";
  }

private void optSpanish_Checked(object sender, RoutedEventArgs e)
  {
    strLngTo = "es";
  }

private void optGerman_Checked(object sender, RoutedEventArgs e)
  {
    strLngTo = "de";
  }

Next up, let’s take a look at the code-behind for the ‘translate’ button. Pressing this will kick off the translation process, which takes a number of steps.

- Make the call to the Windows Azure Marketplace using your credentials to get an access token.

- When you get the token, set up the call to Microsoft Translator to do the translation

- When the call is ready, make it, and await the translation

- When the translation is done, update the user interface

The first task, upon clicking the button is to create the call to the Windows Azure Marketplace to get an access token. The first part of this is to create the HTTP request.

private void Button_Click_1(object sender, RoutedEventArgs e)
{
  // Initialize the strTextToTranslate here, while we're on the UI thread
  strTextToTranslate = txtToTrans.Text;
  // STEP 1: Create the request for the OAuth service that will
  // get us our access tokens.
  String strTranslatorAccessURI = 
        "https://datamarket.accesscontrol.windows.net/v2/OAuth2-13";
  System.Net.WebRequest req = System.Net.WebRequest.Create(strTranslatorAccessURI);
  req.Method = "POST";
  req.ContentType = "application/x-www-form-urlencoded";
  // Important to note -- the call back here isn't that the request was processed by the server
  // but just that the request is ready for you to do stuff to it (like writing the details)
  // and then post it to the server.
  IAsyncResult writeRequestStreamCallback = 
    (IAsyncResult)req.BeginGetRequestStream(new AsyncCallback(RequestStreamReady), req);

}

 

This sets up a function called ‘RequestStreamReady’ to be called when the HTTP Request is ready to go.

Here’s the code for that function:

private void RequestStreamReady(IAsyncResult ar)
{
 // STEP 2: The request stream is ready. Write the request into the POST stream
  string clientID = "<<Your Client ID>>";
  string clientSecret = "<<Your Client Secret>>";
  String strRequestDetails = string.Format("grant_type=client_credentials&client_id={0}&client_secret={1}&scope=http://api.microsofttranslator.com", HttpUtility.UrlEncode(clientID), HttpUtility.UrlEncode(clientSecret));

  // note, this isn't a new request -- the original was passed to beginrequeststream, so we're pulling a reference to it back out. It's the same request

  System.Net.HttpWebRequest request = (System.Net.HttpWebRequest)ar.AsyncState;
  // now that we have the working request, write the request details into it
  byte[] bytes = System.Text.Encoding.UTF8.GetBytes(strRequestDetails);
  System.IO.Stream postStream = request.EndGetRequestStream(ar);
  postStream.Write(bytes, 0, bytes.Length);
  postStream.Close();
  // now that the request is good to go, let's post it to the server
  // and get the response. When done, the async callback will call the
  // GetResponseCallback function
  request.BeginGetResponse(new AsyncCallback(GetResponseCallback), request);
}

 

This will make the make the call to the service using an HTTP Post, and when the response comes back it will be caught by the ‘GetResponseCallback’ function.

Below the code for ‘GetResponseCallback’. It parses the token out of the stream returned from the call to Azure, and then uses this token to set up an authenticated call to the Microsoft Translator API. Before you get there, you’ll need to create a helper class for the access token. This class should be called ‘AdmAccessToken’, and it should be built like this:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace Transl8
{
    public class AdmAccessToken
    {
        public string access_token { get; set; }
        public string token_type { get; set; }
        public string expires_in { get; set; }
        public string scope { get; set; }
    }
}

 

Note that you need to use the exact same names for the public strings when you create this class or the JSON parser will get confused as to how to map the response from the Azure service to this class.

private void GetResponseCallback(IAsyncResult ar)
{
// STEP 3: Process the response callback to get the token
// we'll then use that token to call the translator service
// Pull the request out of the IAsynch result
  HttpWebRequest request = (HttpWebRequest)ar.AsyncState;
// The request now has the response details in it (because we've called back having gotten the response
  HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(ar);
// Using JSON we'll pull the response details out, and load it into an AdmAccess token class (defined earlier in this module)
// IMPORTANT (and vague) -- despite the name, in Windows Phone, this is in the System.ServiceModel.Web library,
// and not the System.Runtime.Serialization one -- so you will need to have a reference to System.ServiceModel.Web
  try
  {
    System.Runtime.Serialization.Json.DataContractJsonSerializer serializer = new 
    System.Runtime.Serialization.Json.DataContractJsonSerializer(typeof(AdmAccessToken));
  AdmAccessToken token = 
    (AdmAccessToken)serializer.ReadObject(response.GetResponseStream());

    string uri = "http://api.microsofttranslator.com/v2/Http.svc/Translate?text=" + System.Net.HttpUtility.UrlEncode(strTextToTranslate) + "&from=en&to=" + strLngTo;
    System.Net.WebRequest translationWebRequest = System.Net.HttpWebRequest.Create(uri);
    // The authorization header needs to be "Bearer" + " " + the access token
    string headerValue = "Bearer " + token.access_token;
    translationWebRequest.Headers["Authorization"] = headerValue;
    // And now we call the service. When the translation is complete, we'll get the callback
    IAsyncResult writeRequestStreamCallback = (IAsyncResult)translationWebRequest.BeginGetResponse(new AsyncCallback(translationReady), translationWebRequest);
    }
    catch (Exception ex)
    {
      // Do nothing
    }
  } 

 

When the API finishes the translation, it will call back to the ‘translationReady’ callback. This reads the response from the translator API and parses the results out of it. It then updates the lblTranslatedText textblock with the results. Note that because the code is asynchronous, with all the callbacks, it doesn’t run on the UI thread, and cannot update the UI directly, so it uses the dispatcher to do so. You can see that on the final line in this function.

private void translationReady(IAsyncResult ar)
{
  // STEP 4: Process the translation
  // Get the request details
  HttpWebRequest request = (HttpWebRequest)ar.AsyncState;
  // Get the response details
  HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(ar);
  // Read the contents of the response into a string
  System.IO.Stream streamResponse = response.GetResponseStream();
  System.IO.StreamReader streamRead = new System.IO.StreamReader(streamResponse);
  string responseString = streamRead.ReadToEnd();
  // Translator returns XML, so load it into an XDocument
  // Note -- you need to add a reference to the System.Linq.XML namespace
  System.Xml.Linq.XDocument xTranslation = 
    System.Xml.Linq.XDocument.Parse(responseString);
  string strTest = xTranslation.Root.FirstNode.ToString();
  // We're not on the UI thread, so use the dispatcher to update the UI
  Deployment.Current.Dispatcher.BeginInvoke(() => lblTranslatedText.Text = strTest);

}

And that’s it. Now, when you run the application, you’ll see the translation. Here’s an example:

clip_image007

In the next step you’ll add code to the Speak button to take the translated text and read it out with a native voice.

Step 5. Adding Speech Synthesis

First of all, you’ll need to add a reference to the speech synthesis namespace at the top of your code behind file. It’s here:

using Windows.Phone.Speech.Synthesis;

You’ll also need to set up the app manifest to declare that you will use speech. To do this, open WMAppManifest.xml from your properties folder. On the Capabilities tab, you’ll need to select the ID_CAP_SPEECH_RECOGNITION capability as shown here:

clip_image008

Now you can implement the code for the ‘Speak’ button. This will take the language that you desire, and go through the installed voices to find one which matches the language. When it finds a voice, it configures the synthesizer with that voice and reads out the translated text. Note that because the speech synthesis is asynchronous, and needs to be awaited, this function needs to be created as an async.

Here’s the code:

private async void btnSpeak_Click(object sender, RoutedEventArgs e)
{
  string filterLanguage = "";
  SpeechSynthesizer speech = new SpeechSynthesizer();
  switch (strLngTo)
  {
    case "fr":
    {
      filterLanguage = "fr-FR";
      break;
    }
    case "es":
    {
      filterLanguage = "es-ES";
      break;
    }
    case "de":
    {
      filterLanguage = "de-DE";
      break;
    }
    default:
    {
      filterLanguage = "fr-FR";
      break;
    }

  }
  // Query for a voice that speaks French.
  IEnumerable<VoiceInformation> voices = from voice in InstalledVoices.All
                                         where voice.Language == filterLanguage
                                         select voice;

  // Set the voice as identified by the query.
  speech.SetVoice(voices.ElementAt(0));

  // Count in French.
  await speech.SpeakTextAsync(lblTranslatedText.Text);


}

And that’s it. You’ve now created an app that uses the Microsoft Translator API to translate from English to a different language, and added speech synthesis so you can hear the language pronounced with a native speaker’s voice!