Wearable Device that Can Understand Nonverbal Prompts

April 2018 - Researchers at MIT have developed a wearable device called AlterEgo that can recognize nonverbal prompts, such as unspoken words, and can transcribe them for the user. The device consists of flexible electrodes that pick up neuromuscular signals from the user’s jaw and face vibrations, which are then passed through a machine-learning system that can connect these prompts with specific words. Also, the device uses bone-conduction headphones that transmit the audio signal into the bones of the inner ear, which makes the wearable silent to all but the user, allowing the user to have conversations at normal audio levels. Together, these technologies become a lightweight wearable device that can “translate” nonverbal prompts into words, with a silent user interface. According to Arnav Kapur, a graduate student who was involved with the development of the wearable system, “The motivation for this was to build an IA device — an intelligence-augmentation device. Our idea was: Could we have a computing platform that’s more internal, that melds human and machine in some ways, and that feels like an internal extension of our own cognition?”

 

The device, still in development, is capable of discerning twenty words with ninety-two percent accuracy. As to the future of AlterEgo, the team is hopeful that they will one day achieve full conversation capabilities with the device. Other researchers have taken note of this project and its potential in employment and assisting people with disabilities. One such researcher is Thad Starner, a professor at Georgia Tech’s College of Computing who said, “I think that they’re a little underselling what I think is a real potential for the work. Like, say, controlling the airplanes on the tarmac at Hartsfield Airport here in Atlanta. You’ve got jet noise all around you, you’re wearing these big ear-protection things — wouldn’t it be great to communicate with voice in an environment where you normally wouldn’t be able to? You can imagine all these situations where you have a high-noise environment, like the flight deck of an aircraft carrier, or even places with a lot of machinery, like a power plant or a printing press. This is a system that would make sense, especially because oftentimes in these types of or situations people are already wearing protective gear. For instance, if you’re a fighter pilot, or if you’re a firefighter, you’re already wearing these masks.” The team behind AlterEgo hopes that the device can also be helpful to people with disabilities to communicate more effectively in work and non-work environments. One example could be assisting people who use sign language to remain in communication with their colleagues even if their hand signals are not visible or if they were wearing movement restricting safety apparel such as gloves. 

 

For more information on AlterEgo please visit MIT’s News Office which features a video demonstration of the prototype: https://news.mit.edu/2018/computer-system-transcribes-words-users-speak-silently-0404

 

[Source: Sara Rice, Consumer Technology Association; Larry Hardesty, MIT News Office]