Most of you know that we released the first publicly available Haitian Creole statistical machine translation engine last week and have been hard at work making it even better. I am pleased to announce since last night we rolled out two updates to the system and our site which bring several improvements:
1) More training data = better translations. We trained the system on even more training data (including data that we hand translated) which should reflect in better translations. We are nowhere near done yet, and we will continue to work on this.
2) Updating the AJAX API and widget. The Translator widget (and the underlying AJAX API) now accurately reflect “Haitian Creole” as the language selected in their UI. This was primarily a user interface fix (the Haitian Creole translation itself worked fine). You can use the widget to deliver any webpage in any of the languages we support (including Haitian Creole).
3) Please don’t forget the broad set of APIs and webmaster resources that are available for those that are building applications and websites to help with the relief efforts. There are several efforts underway to develop mobile apps (using the SOAP or HTTP API) and websites (using the AJAX API). If you are working on something along those lines, leave a link to your app/site in the comments and I will make sure to surface them up here so people can find them more easily.
We will continue to work on improving the system and we wish to thank everyone in the community that has been instrumental in helping us get this much requested translation engine out of the door. Stay tuned for more announcements!
Also, let me once again point to a resource where you can help with the broader Haiti relief efforts. Please help in any way you can!
Update (1/31): The DIPLOMAT project at CMU in the 1990s was an earlier project to create a Haitian Creole system for DOD/DARPA. As I mentioned in our earlier blog post, our system makes use of CMU’s data from that project.
- Vikram Dendi, Senior Product Manager, Microsoft Translator
In the current crisis in Haiti there are a number of initiatives to rapidly build software to assist in humanitarian aid. Responding to community requests for a machine translation (MT) system to translate between English and Haitian Creole, our team has been hard at work over the last few days. I am glad to announce that an experimental Haitian Creole MT system is now publicly available via several services and APIs powered by Microsoft Translator technologies. We will continue working on improving the system, but we hope meanwhile that in spite of the experimental nature – it will be of use in the relief efforts.
1) What is being announced today?
Responding to requests from the community involved in Haitian relief efforts, Microsoft Research is making available today an experimental machine translation system for translating to and from Haitian Creole. You can try it at http://translate.bing.com or http://www.microsofttranslator.com.
2) How is it significant?
With the devastating disaster that struck Haiti, we have all been individually pitching in to help the efforts. This is our effort, as a team, to respond to the needs of communities such as Crisis Commons by delivering a Haitian Creole translator which can be of help to individual users, as well as other technology projects that could use a scalable translation system in their relief endeavors. Further, the usage of our API is completely free and it can be built into any application or website for immediate use. We hope that this might help the many applications being developed (such as those on crisiscommons.org) to aid the humanitarian efforts.
3) How can I use this system?
The Haitian Creole translator is now part of the Microsoft Translator web service enabling many of the user scenarios powered by the service. Users can access the service through the Microsoft Translator web site. Developers would be interested in looking at our APIs – and choose from SOAP or HTTP (Support for Haitian in our AJAX API will be rolled out in the coming days).
4) How is it different from other efforts?
There have been some great efforts in quickly building dictionary and rule-based Haitian Creole translation tools. The statistical machine translation system behind Microsoft Translator allows for a continuous improvement in the quality of translations (by adding more training data). Also, by delivering this as part of our web service we can ensure scale and performance and open up the possibility of using our many scenarios (Bing Translator, Internet Explorer 8, Messenger Bot etc.) with Haitian Creole, as well as using our extensive API set to add such support to other software and web sites at no cost.
5) What was involved in getting this out of the door in record time?
The process involved identifying parallel (translated) data between English and Haitian Creole, and training the MT engine to create the requisite language models. We would also like to acknowledge the great work being done the Crisis Commons folks, the dictionary builders at haitisurf.com, the folks at CMU that made available parallel data and the Microsoft volunteers who challenged our team to action.
6) What should I expect in terms of quality?
This is an experimental system put together in record time. While our typical approach to adding new languages involves significantly larger amounts of training, a higher threshold for quality testing – we decided that the upside warranted making the system available to the community at the earliest, and continue improving it subsequently. We are working diligently to keep improving the quality, but bear with us if you encounter problems. You can always contact us at email@example.com with feedback. Our user and developer forums are also available to discuss any issues you encounter.
7) How can I help improve the system?
The best way you can help improve the system is by helping us find more training data. This is typically sentences or words translated between English and Haitian Creole. We intend to make available to the larger community (via tausdata.org) data that we collect (as license restrictions permit) for training purposes. If you know of dictionaries, translated sentences, or websites that have such translations we urge you to contribute it to TDA’s TAUS data sharing initiative. TDA is a non-profit organization providing a neutral and secure platform for sharing language data. If you have any concerns or questions feel free to contact us at firstname.lastname@example.org.
8) How can I help the broader Haiti relief efforts?
Go here to learn more about how you can help those devastated by the earthquake.
9) Where can I get more information?
Please stay tuned to our blog for further announcements. You can learn more about Microsoft Translator and the services we offer here.
10) What can we expect next?
In the coming days expect to see support for Haitian Creole added to even more of our scenarios (Translation Bot, Translator widget, Office etc) as well as the AJAX API. Known issues and announcements can also be found on our forums.
We hope that this contribution proves useful to the various humanitarian efforts underway, and please stay tuned to this blog for further news on the Haitian Creole language support. If you have any questions feel free to contact us at email@example.com.
Update (2:53 PM PST): The Messenger Translation Bot can now speak Haitian Creole. Add firstname.lastname@example.org to your messenger buddy list. Try the group conversation feature with a Kreyol speaker!
-Vikram Dendi, Senior Product Manager, Microsoft Translator