Science

New glove translates sign language to speech in real time



Bioengineers have designed a glove that is able to translate American Sign Language to speech in real time.

The glove has thin, stretchable sensors inside which run to the fingertips. These sensors are able to pick up and motions and finger placement through electrically conducting yarns.

Those sensors are then connected to a small piece of circuit board – approximately the size of a coin – that users wear on their wrists.


As such, when people move their hands and fingers to sign words, the glove is able to translate the individual letters, numbers, words and phrases into spoken words.

Extra sensors can be added to the face, between the eyebrows and on the sides of the mouth, to capture facial expressions.

The actual translation is done through a smartphone app, which uses a custom custom machine-learning algorithm to turned gestures into the letters, numbers and words.

The system recognises 660 signs, which includes each letter of the alphabet and the numbers from zero to nine.

It can translate at a speed of one word per second.

A commercial version of this technology would require more vocabulary and a faster translation time. UCLA has filed a patent on the glove.

“Our hope is that this opens up an easy way for people who use sign language to communicate directly with non-signers without needing someone else to translate for them,” said Jun Chen, the principal investigator on the glove.

“In addition, we hope it can help more people learn sign language themselves.”

Zhihao Zhao, Kyle Chen, Songlin Zhang, Yihao Zhou and Weili Deng, from the University of California, Los Angeles (UCLA), authored the study that developed the glove, alongside Chen.

Jin Yang, of China’s Chongqing University, also authored the study.

The benefits of the glove are its portability and weight. Previous wearable devices offered similar capabilities, but were heavy and impractical to wear.

This new design is far lighter, and the polymers are inexpensive – as are the electronic components.

However, some deaf researchers have criticised the development.

“The tech is redundant because deaf signers already make extensive use of text-to-speech or text translation software on their phones, or simply write with pen and paper, or even gesture clearly,” said Gabrielle Hodge, a deaf researcher from the Deafness Cognition and Language Research Centre at University College London, to CNN. “There is nothing wrong with these forms of communication.”

“It would be so much easier if tech focused on user-driven and user-centered design in the first instance, rather than dreaming up ‘solutions’ they think will fix all the problems in the world.”



READ SOURCE

Leave a Reply

This website uses cookies. By continuing to use this site, you accept our use of cookies.