An estimated 360 million people worldwide suffer from hearing loss. Because the majority of hearing individuals do not understand sign language, communication between the hearing and the deaf can be challenging. While other communication methods exist, researchers hope to make translation even easier with a cost-effective, efficient prototype that translates sign language into spoken language—and spoken language into sign language—in real time.

ASIA ANSWERS THE CALL

Dedicated researchers in China have created the Kinect Sign Language Translator, a prototype system that understands the gestures of sign language and converts them to spoken and written language—and vice versa. The system captures a conversation from both sides: it displays the signer and renders a written and spoken translation of the sign language in real-time, and it also takes the non-signer’s spoken words and turns them into accurate, understandable sign language. An avatar on the screen represents the non-signer and makes the appropriate sign languages gestures.

This project was a result of collaboration, facilitated by Microsoft Research Connections, between the Chinese Academy of Sciences, Beijing Union University, and Microsoft Research Asia, each of which made crucial contributions.

Professor Xilin Chen, deputy director of the Institute of Computing Technology at the Chinese Academy of Sciences, has spent much of the past decade studying sign language recognition, hoping to devise a way to enable signed communication between people with hearing loss and their hearing neighbors. “We knew that information technology, especially computer technology, has grown up very fast. So from my point of view, I thought this is the right time to develop…

Read more | research.microsoft.com