Translating Sign Language Into Speech With Wearable-Tech Glove
Most people who are deaf, mute, or hard of hearing use American Sign Language (ASL). They communicate among themselves, their family, and teachers through ASL. But communication becomes complicated when they have to talk with other people who can speak and hear normally. Typically, they have to use a sign language interpreter, especially in healthcare and medical settings or in court settings.
For people who use sign language, everyday communication is a struggle. Therefore, hearing about technological innovations and research to help them conduct verbal interchange is always welcome.
Glove-like translator
Bioengineers from the University of California – Los Angeles (UCLA) have developed a wearable-tech device to translate ASL into English speech. With the help of a smartphone app, the translation is in real-time. Their research is published in Nature Electronics Journal in June 2020.
A user will wear a pair of gloves equipped with thin, stretchable sensors. The sensors are made from electrically conducting yarns. Each of them runs the length of every finger in each hand. They pick up the hand motions and the placement of the fingers representing phrases, words, numbers, and individual letters.
Not only for people who sign
UCLA Samueli School of Engineering assistant professor of bioengineering, Jun Chen, is the leader of the research and development of the wearable tech. He said that they want people who use ASL to communicate directly with people who do not sign without the help of an interpreter. Moreover, they want more people to learn ASL.
Translation process
When the user wears the gloves, the device will turn the fingers’ movements into electrical signals. The signals are sent to a coin-sized circuit board the user wears on their wrist. The board then wirelessly transmits the signals to a smartphone where the user can see the text translation. Currently, the rate of conversion is a word per second.
The researchers are capturing the signer’s facial expressions, which are part of ASL, by attaching adhesive sensors on one side of the mouth and between the eyebrows of test users.
Significant improvement from previous systems
This type of wearable system for American Sign Language translation is not new. However, the previous ones were uncomfortable to wear while others were heavy and bulky, Chen added.
The new device from the UCLA team uses cheaper, lighter, and stretchable polymers that last longer. The electronic sensors they use are likewise less expensive and more flexible.
In one of their tests, the researchers asked four deaf people who use ASL to wear the gloves. They repeated each hand gesture several times. The gestures were turned into letters, words, and numbers by their custom machine-learning algorithm. During the test, 660 signs, the numbers zero to nine, and all the letters of the alphabet were recognized by the system.
A similar wearable-tech prototype for ASL was developed by the University of California San Diego engineers in 2017. The smart glove, which they call ”The Language of Glove,” translates the ASL alphabet into text wirelessly by controlling the virtual hand that mimics sign language gestures. The device is Bluetooth-enabled.
Aside from ASL, they want to explore the possibility of wearable-tech gloves for other applications, such as defense, technical training, telesurgery and augmented, and virtual reality.