With TranscribeGlass, deaf & hard of hearing people can see the words they cannot hear

Indian teenager Madhav Lavakare has developed an affordable and comfortable assistive technology device for people who are deaf and hard of hearing called TranscribeGlass. This is a smart glass that allows people with hearing impairments to see the words they cannot hear. An early beta version of the device will be launched by August 2021.

Three years ago, Madhav Lavakare was in his junior year of high school when a friend who was hard of hearing dropped out of school because he could not understand what was being taught or follow conversations around him. That led Madhav, who is now 19, to think of building a solution.

“The idea was to build a solution to address the communication barriers faced by deaf and hard of hearing people who do not have access to, or cannot afford existing solutions such as hearing aids or cochlear implants”, says Madhav. Taking some inspiration from Google Glass, he set upon developing a heads-up display for people with hearing loss, including the deaf, hard of hearing and the elderly.

The result is TranscribeGlass, an assistive technology device that improves comprehension of spoken communication using captions.

How TranscribeGlass works

TranscribeGlass projects closed captions from a variety of sources onto a heads-up display in the user’s field of vision in near-real-time. The caption sources include Automatic Speech Recognition (ASR), live captioning (CART/stenography), and subtitle files. Users can watch the speaker and receive non-verbal communication cues and increase their environmental and situational awareness while simultaneously reading the captions. A companion app running on an Android mobile device connects to the caption source and sends them to the TranscribeGlass heads-up display via Bluetooth Low Energy. The captions are overlaid onto the user’s vision of the surrounding real world environment in real-time.

“It displays captions in the user’s field of vision, is independent of caption source, has a flexible optical system reducing eye-strain, and a retrofit design that clips on to most spectacle frames”, says Madhav. This makes TranscribeGlass an affordable and comfortable alternative to the existing solutions, many of which are unaffordable, uncomfortable, inaccessible or inconvenient.

Hearing aids score low on convenience as they require audiological examination, fitment, regular battery changes and maintenance. Cochlear implants require invasive surgery. Reading captions heads-down limits mobility. Live captioning and sign language interpretation is not always available.

There is a need for a solution for deaf/hard of hearing individuals that is affordable, effective, convenient and accessible. TranscribeGlass, at a target price of ₹4,000 ($55) fills in the void. When captions are displayed on mobile devices or screens, users miss out on crucial visual cues, engaging with the speaker and environmental awareness. The other benefits of a heads-up display include hands-free usage, improved mobility, increased perceived emotional connection to the speaker, and better social acceptance as an assistive technology device for communication. – Madhav Lavakare, Founder, TinkerTech Labs

Madhav, who took a gap year after graduating from school to build the device and his start-up, faced many challenges.

Early challenges

“Access to resources and facilities in the early days was a challenge as India’s DIY culture and maker-ecosystem is still developing. Also, raising funding for a part-sustainable start-up and part-impact venture. And given the fact that I am a teenager looking to raise serious funding!”, he says. Madhav started the venture alone and after nearly four years of working solo has been able to put together a team backed by experienced mentors across India, the United States and the United Kingdom.

The third prototype of TranscribeGlass was completed by early 2020. Madhav has demonstrated the version to users at the National Association of the Deaf in the United States, Gallaudet University, and the Massachusetts Institute of Technology (MIT) last year. Interviews were conducted with 30 deaf and hard of hearing individuals in India, the US and the UK to understand use cases, barriers to adoption, and design considerations. The fourth version of a working prototype completed in early 2021 was used as part of a 25-user usability study.

An early beta version called TranscribeGlass Beta will be launched by July-August 2021. This is a closed-group limited release. If all goes well, production will begin by late 2021/early 2022.

User testimonials available on the TranscribeGlass website are encouraging.

“I’ve tried a couple of heads up displays.. Yours’ probably had the least pain points”, says one user. “A heads up display offers a personal, powerful experience”, says another.

Globally there are over 460 million people with disabling hearing loss. That’s 6.1% of the world’s population. In addition one-third of people over the age of 65 are affected by disabling hearing loss, which underlines the critical need for a low cost device like TranscribeGlass.

Watch in Sign Language

Also Read:

• Prosthetic hand Grippy gives amputees sense of touch, grip control at affordable prices