· In celebration of International Mother Language Day, we are pleased to announce the addition of the Hmong language to our list of supported languages, made possible by a close partnership with the Hmong community. Anyone can now try out the new language on the Bing Translator site, or call it via the Microsoft Translator web service (Hmong Daw, language code mww). Hmong Daw is the dialect of Hmong the system supports, also known as White Hmong.
Instrumental to this effort were members of the Hmong community, who were able to leverage new tools from Microsoft Translator to help preserve and revitalize their language online. These new tools, currently in beta, enable automatic translation support for additional languages, or building higher quality systems for specific terminology and style in the established languages.
The addition of the Hmong language is an example of the first scenario: Members of the community utilized existing translated material and new features of Microsoft Translator to train a new translation engine. This leveraged Microsoft Translator’s learning abilities, which can learn how to translate from a set of parallel documents (same document in two languages), dictionaries and texts in the language to translate to (Hmong in this case). In addition to teaching the engine a new language, they also involved members of the community, partners and collaborators to create and review improved versions of the automated translation system, and collect qualitative feedback about each “trained” system. Deploying a system that reaches a certain level of quality allows seamless use with the standard Microsoft Translator APIs, and many scenarios powered by the API, like the web translation widget. Feedback that is generated through these scenarios can be utilized again in the training process – creating a virtuous loop for improving the translation quality. Stay tuned for more details about these new tools/features.
Once again, on International Mother Language day, we congratulate the Hmong community on their accomplishment. We are looking forward to working with many more partners and language communities in the near future.
Did you know that MSN messenger recently became* is the number one instant messenger in the world? Last summer, thanks to the efforts of Helvecio on our team, the MTBot prototype project quietly launched – to provide a glimpse to the community of 28.6 million unique messenger users what might be possible when you combine machine translation technology with instant messaging.
The MTBot prototype project was released in May 2007 with the main goal to try to understand how useful machine translation would be in IM conversations. The bot acts as a human translator, participating in conferences and translating messages as they are sent by all parties.
A typical usage scenario would be something like this: let's assume you have a friend in Japan that does not speak English... Well, you would add MTBotemail@example.com to your Live messenger buddy list, wait until the bot accepts your request (by switching status to Online) and then you would start your conversation by sending the "Hello" message... The Bot is going to wake up, and display a list of languages - enter "ja" for Japanese. Once it gets a valid connection the Bot will tell you to invite your friend to join the conversation. That's it... From this point on, everything you type will be translated from English to Japanese, and everything your friend types will be translated from Japanese to English.
Another typical use is as portable translator: using Messenger from any Smartphone a user can translate simple sentences when traveling to other countries.
As with any prototype effort, do keep in mind that this is experimental and there is a possibility the bot might be offline from time to time. The usual caveats about the quality of machine translation also apply.
We always appreciate your feedback and suggestions – so feel free to do so on this thread.
* Update: The link pointing to the data on Messenger becoming the most used IM client is from around when (2003-04) it first claimed that crown. Messenger has continued that trend since then. (Thanks to our keen eyed readers for catching that one!)
In the current crisis in Haiti there are a number of initiatives to rapidly build software to assist in humanitarian aid. Responding to community requests for a machine translation (MT) system to translate between English and Haitian Creole, our team has been hard at work over the last few days. I am glad to announce that an experimental Haitian Creole MT system is now publicly available via several services and APIs powered by Microsoft Translator technologies. We will continue working on improving the system, but we hope meanwhile that in spite of the experimental nature – it will be of use in the relief efforts.
1) What is being announced today?
Responding to requests from the community involved in Haitian relief efforts, Microsoft Research is making available today an experimental machine translation system for translating to and from Haitian Creole. You can try it at http://translate.bing.com or http://www.microsofttranslator.com.
2) How is it significant?
With the devastating disaster that struck Haiti, we have all been individually pitching in to help the efforts. This is our effort, as a team, to respond to the needs of communities such as Crisis Commons by delivering a Haitian Creole translator which can be of help to individual users, as well as other technology projects that could use a scalable translation system in their relief endeavors. Further, the usage of our API is completely free and it can be built into any application or website for immediate use. We hope that this might help the many applications being developed (such as those on crisiscommons.org) to aid the humanitarian efforts.
3) How can I use this system?
The Haitian Creole translator is now part of the Microsoft Translator web service enabling many of the user scenarios powered by the service. Users can access the service through the Microsoft Translator web site. Developers would be interested in looking at our APIs – and choose from SOAP or HTTP (Support for Haitian in our AJAX API will be rolled out in the coming days).
4) How is it different from other efforts?
There have been some great efforts in quickly building dictionary and rule-based Haitian Creole translation tools. The statistical machine translation system behind Microsoft Translator allows for a continuous improvement in the quality of translations (by adding more training data). Also, by delivering this as part of our web service we can ensure scale and performance and open up the possibility of using our many scenarios (Bing Translator, Internet Explorer 8, Messenger Bot etc.) with Haitian Creole, as well as using our extensive API set to add such support to other software and web sites at no cost.
5) What was involved in getting this out of the door in record time?
The process involved identifying parallel (translated) data between English and Haitian Creole, and training the MT engine to create the requisite language models. We would also like to acknowledge the great work being done the Crisis Commons folks, the dictionary builders at haitisurf.com, the folks at CMU that made available parallel data and the Microsoft volunteers who challenged our team to action.
6) What should I expect in terms of quality?
This is an experimental system put together in record time. While our typical approach to adding new languages involves significantly larger amounts of training, a higher threshold for quality testing – we decided that the upside warranted making the system available to the community at the earliest, and continue improving it subsequently. We are working diligently to keep improving the quality, but bear with us if you encounter problems. You can always contact us at firstname.lastname@example.org with feedback. Our user and developer forums are also available to discuss any issues you encounter.
7) How can I help improve the system?
The best way you can help improve the system is by helping us find more training data. This is typically sentences or words translated between English and Haitian Creole. We intend to make available to the larger community (via tausdata.org) data that we collect (as license restrictions permit) for training purposes. If you know of dictionaries, translated sentences, or websites that have such translations we urge you to contribute it to TDA’s TAUS data sharing initiative. TDA is a non-profit organization providing a neutral and secure platform for sharing language data. If you have any concerns or questions feel free to contact us at email@example.com.
8) How can I help the broader Haiti relief efforts?
Go here to learn more about how you can help those devastated by the earthquake.
9) Where can I get more information?
Please stay tuned to our blog for further announcements. You can learn more about Microsoft Translator and the services we offer here.
10) What can we expect next?
In the coming days expect to see support for Haitian Creole added to even more of our scenarios (Translation Bot, Translator widget, Office etc) as well as the AJAX API. Known issues and announcements can also be found on our forums.
We hope that this contribution proves useful to the various humanitarian efforts underway, and please stay tuned to this blog for further news on the Haitian Creole language support. If you have any questions feel free to contact us at firstname.lastname@example.org.
Update (2:53 PM PST): The Messenger Translation Bot can now speak Haitian Creole. Add email@example.com to your messenger buddy list. Try the group conversation feature with a Kreyol speaker!
-Vikram Dendi, Senior Product Manager, Microsoft Translator
Every now and then I look at visitor logs on the various personal and professional sites/blogs that I administer. It makes for a fascinating experience to see the many places worldwide that visitors come from. I have often wondered about non English speakers and how I could make my writing more accessible to them. While some professional and company web sites have translated versions available, in many user forums and communities across the web there have been requests for a translated version of the pages/posts. Today, on many sites, I have to copy the text on the site, paste it into a translator and look at the translation. It is cumbersome and not very seamless in an otherwise smooth navigation experience.
I am very pleased to say Windows Live Translator solved this problem with the latest feature addition that rolled out this week. Now on the Live Translator home page you will find a new link "Add the web page Translator to your site". By clicking on this link you go to a page that offers snippets of code that can be added to individual web pages for which you wish to offer translations.
The code generator will create the appropriate widget depending on the source language of your site. Refer to the Live Translator introduction post where Andrea listed the language pairs that we currently support.
So here is what you do to have a link on your web page to translate it:
Step 1: Click on the Add the web page Translator to your site link
Step 2: Select the language your web page is written in (source language)
For example: Since all the articles on my blog are in English, I choose English as the source language
Step 3: The code that you need to copy and paste into your web page's HTML is generated in the box
For example: Since I chose English, the code that is generated looks like this
Step 4: Copy that code and paste it into the page that should offer translation.
For example: On my blog say I want the blog post I wrote about Live Translator to be translated, I go into the blog editor and paste it like so:
If the blog or web page uses templates, one could also paste the code into a template - thereby providing the Translate This Page widget on all pages
Step 5: Enjoy an expanded (and hopefully more appreciative) audience!
The end result on my blog looks like this in the case of a single post translation:
The end result looks like this if I put it in the template (this allows for translation of every post):
For the more technically minded here is some more information on the parameters that the Live Translator accepts:
where lp is the language pair (such as en_fr for english to french) for source and target languages. a is the URL you want translated.
where lp is the language pair (such as en_fr for english to french) for source and target languages. a is the URL you want translated.
The Windows Live focused community site ViaWindowsLive is making creative use of the Live Translator to make their site available in multiple languages (look on the left bottom of the page). I would love to check out how you might be utilize this new feature. Feel free to post a link to your site in the comments.
Edit: Updating the parameters link
Will Lewis is a program manager on the Microsoft Translator team, working on language quality and data acquisition. Today's guest blog is a high level explanation of how the engine works:
As many of you know, under the hood Microsoft Translator is powered by a Statistical Machine Translation (SMT) engine. Statistical systems are different than rule-based ones in that the “rules” mapping words and phrases from one language to another are learned by the system rather than being hand-coded. Training an SMT requires amassing a large amount of parallel training data—hopefully of good quality and from heterogeneous sources—and training the engine on that data. (By parallel, we mean a source of data where the content for one language is the same as the content for the other.) The engine learns the correspondences between words and phrases in one language and those in another, which are often reinforced by repeated occurrences of the same words and phrases throughout the input. For instance, in training the English-German system let’s say, if the engine sees the phrase All rights reserved on the English side and also notices Alle Rechte vorbehalten on the German side, it may align these two phrases, and assign some probability to this alignment. Repeated occurrences of the source and target phrases in the training data will only reinforce this alignment.
Generally, having parallel data for a language pair means we can train engines in both directions (i.e., both the English-German and the German-English systems can be trained on the same input sentences). Some of you had some questions regarding why it was that we released the English-Spanish system before we released Spanish-English. There were really two reasons. First, English-Spanish was the first general domain language pair we released. Releasing one language pair allowed us to test the infrastructure before we started releasing more. Second, the technology for Spanish-English was slightly different than that used for English-Spanish, and we need some additional time to do the necessary infrastructural changes to accommodate. In the future, we plan to release new translation systems in pairs (with a couple of exceptions). I can’t reveal what languages we have planned next, but do expect some new ones soon!
For those of you interested in technical discussions regarding our engines and how they work, please refer to some of the papers by the researchers who developed them. Three recent papers of note are:
Chris Quirk, Arul Menezes. Do we need phrases? Challenging the conventional wisdom in Statistical Machine Translation May 2006 New York, New York, USA Proceedings of HLT-NAACL 2006
Chris Quirk, Arul Menezes. Dependency Treelet Translation: The convergence of statistical and example-based machine translation? March 2006 Machine Translation 43-65 (Attached file)
Chris Quirk, Arul Menezes. Using Dependency Order Templates to Improve Generality in Translation July 2007 Association for Computational Linguistics