Download Research Tools
Sign language has long been a tool used by many of the world’s estimated 360 million people with severe hearing loss. But since the majority of hearing individuals do not understand sign language, the hearing world does not always have the capability to engage in real-time, unwritten communication with people with hearing loss. Now, technology is poised to make such interactions more feasible.
Hoping to facilitate communication between the signing and non-signing communities, Microsoft Research in February 2012 initiated the Kinect Sign Language project in collaboration with the Chinese Academy of Sciences (CAS) and Beijing Union University. The Kinect Sign Language Translator enables real-time conversations between signing and non-signing participants by turning sign language into words spoken by a computer and simultaneously changing spoken words into sign language rendered by an avatar.
Some of the attendees at the Kinect Sign Language Working Group's inaugural event
Early last month, the Kinect Sign Language Working Group, a research community that includes a website for sharing data and algorithms, was established at the Institute of Computing Technology, CAS in Beijing. P. Anandan, managing director of Microsoft Research Outreach, attended this inaugural event, as did other dignitaries representing the community’s founding members: the CAS, Beijing Union University, and Microsoft Research. We are encouraging experts from other research institutions, schools for the deaf and hard of hearing, and non-government organizations to join the Kinect Sign Language Working Group.
The community’s vision is to advance research in sign-language recognition.
The community’s vision is to advance research in sign-language recognition. As a first step, we are opening to academia the DEVISIGN, Chinese Sign Language Database. Compiled by the Visual Information Processing and Learning (VIPL) group of the Institute of Computing Technology, under the sponsorship of Microsoft Research Asia, the DEVISIGN covers about 4,400 standard Chinese Sign Language words based on 331,050 pieces of vocabulary data from 30 signers (13 males and 17 females). The vocabulary data comprises RGB video (in AVI format), and depth and skeleton information (in BIN format). The DEVISIGN thus provides sign-language researchers with a rich store of data for training and evaluating their algorithms and for creating state-of-the-art practical applications, such as solutions for training the system to adapt to an unknown signer.
In the near future, we hope to expand the sign-language database with contributions from new community members, which will help advance the research and development progress for this and potentially other sign language translations. In addition, we intend to organize workshops and to post sign-language-recognition algorithms from researchers worldwide.
Microsoft Research Asia Director Tim Pan expects the Kinect Sign Language project to provide cost-effective and reliable communication between deaf and hearing users.
No single field of expertise can fulfill such an expansive mission. Doing so requires “the collaboration of experts in such diverse fields as machine learning, sign language, social science, and more,” noted Microsoft Research Asia Director Tim Pan during the community’s inaugural event. “In the long run,” he added, “the community will work together to turn ideas into reality, and we fully expect the Kinect Sign Language project to provide cost-effective, easy, and reliable communication between deaf and hearing users.”
—Guobin Wu, Research Program Manager, Microsoft Research Asia
The Third International Women’s Hackathon is now in full swing, having launched on October 11, 2014. A unique crowdsourcing event designed to empower young women leaders in computer science, the hackathon provides a fun and safe environment in which participants explore computing as a means of solving real-world problems. This year’s hackathon should draw more participants than ever, because, in response to requests from several universities, worldwide local events can participate through December 12, 2014. This means that groups who couldn’t join the virtual event on October 11 can still get in on the action.
This year, hackers are devising solutions for two worthy challenges—the Climate Data Challenge (PDF, 291 KB) and the Disaster Response Challenge (PDF, 291 KB)—sponsored, respectively, by the nonprofit organizations The Nature Conservancy and Direct Relief.
At the hackathon kickoff (which took place in Phoenix, Arizona, during the 2014 Grace Hopper Celebration of Women in Computing), participants around the world worked on these challenges, connecting virtually with one another. Those of us in Arizona were excited to link up with female hackers in India, Japan, Nepal, England, South Africa, Turkey, Pakistan, Bangladesh, Brazil, Kenya, and Trinidad & Tobago. (You can see the conversations on our Facebook page.)
I was extremely impressed by the solutions produced by our local winners in Phoenix—Team Recovery and Team Cosmos.
Other teams around the world came up with equally impressive solutions, and now, with the extended deadline, we look forward to even more innovative ideas from women hackers worldwide. We encourage you to find an event near you or start an event of your own. As an added benefit, hackathon participants can now submit their finished solutions to the Imagine Cup World Citizenship or Gaming challenges. If you have any questions, feel free to contact Microsoft Research diversity.
—Rane Johnson-Stempson, Principal Research Director, Microsoft Research
It seems that everybody today wants to employ machine learning in their research. And why not? Machine learning promises to transform our ability to understand, model, and predict the world around us. It is pushing boundaries in areas as diverse as image recognition, cardiac patient risk management, and real-time speech translation.
Despite an explosion in algorithm development, libraries, and services that has made machine learning more accessible, most researchers are not machine learning experts. It’s one thing to write a one-off model and tune it to your application—it is quite another to make that model repeatable and reliable for others to use. If only it were easier to build models and publish them straightaway, then maybe machine learning would be less impenetrable to the majority of researchers.
The recently launched Microsoft Azure Machine Learning (Azure ML) service hosted in the cloud aims to make it easier to build and deploy models for use in limitless applications. To explore this brave new world of machine learning, 50 researchers from 15 universities in France, Italy, the Netherlands, and the United Kingdom recently met at the Microsoft Research lab in Cambridge, England, eager to see how their work could be transformed.
Lab director Andrew Blake kicked off the day by highlighting some of the machine learning techniques that Cambridge lab researchers have used to develop body tracking with Kinect, TrueSkill player matching for Xbox, game and movie recommendation systems, and medical imaging methods that measure tumor growth. These achievements entailed hundreds of person years of effort, from fundamental research to applied algorithm development and implementation. Blake concluded by showing how this depth of knowledge is now available in Azure ML, for anyone to use in their own applications.
Participants then took on a hands-on journey through the web-based Azure ML Studio. Much of the work in building a predictive model involves the nitty-gritty of handling data ingress and the cleaning, slicing, and dicing of data sources of all kinds. Azure ML Studio makes this pipeline clear and easy to build, using a combination of graphical workflow and R scripts.
The real magic came at the end, when we published our best predictive model online with a mouse click—exposing it through a web application programming interface (API) instantly. This aroused great interest, as participants realized they could rapidly deploy and share models with no web API expertise.
Needless to say, we want the Azure ML platform to reach far more than 50 researchers, which is why we are offering substantial grants for researchers and data-science educators via the Microsoft Azure for Research Award program. The next deadline for grant proposals is November 15, 2014, with subsequent deadlines every two months thereafter. And, if you’re not sure but interested in testing this out, we offer a free tier of Machine Learning Studio where you can use up to 10GB of your own data. Whether you just want to get your feet wet or you are ready to dive in completely, we invite you to join the machine learning revolution. Azure ML might just transform your research.
—Kenji Takeda, Solutions Architect and Technical Manager, Microsoft Research