Google’s Android Q to introduce Live Captions and Subtitles to all types of content

Android has a whole host of options that make the entire user interface easier to navigate for people with certain types of disabilities. Rival Apple’s iOS also offers similar features. Now, Google has announced plans to do more.

At I/O developer conference, Google unveiled an impressive feature on the forthcoming Android Q upgrade called Live Caption. When enabled, it will transcribe any video or audio in real time.

While closed captions are not a new feature, but reports are Google’s implementation is very accurate. Captions or subtitles are overlaid on top of whatever app is running, be it YouTube, Instagram, or anything else. It also supports video chat platforms including Skype and Google Duo. It will even transcribe video and audio that users record themselves.

In a blog post, Google said it worked closely with the deaf community to develop a feature that would improve access to digital media, pointing out that for the 466 million deaf and hard of hearing people around the world, captions are not just a convenience but make content more accessible.

Google CEO Sundar Pichai says that building for everyone means Google’s products must be easy to use even for those with certain impairments.

We believe technology can help us be more inclusive, and AI is providing us with new tools to dramatically improve the experience for people with disabilities. You can imagine all the use cases for the broader community too. For example, the ability to watch any video if you’re in a meeting or on the subway, without disturbing the people around you.- Sundar Pichai, CEO, Google, CEO

Live Captions make use of on-device machine learning instead of relying on an active connection. This means it can work even offline, and does not require sending data about the user’s activity to the cloud. Google is putting live transcriptions inside a black box onscreen, which can then be moved freely around. The feature will work even if the phone’s volume is toned down or silent as it will still analyse the audio. But the live captions cannot be saved for later reference.

However, it is important to note that Live Caption will be available only on select phones. Speaking to VentureBeat, a tech news website, Brian Kemler, Android accessibility product manager, said it would not be available on all devices but on some some select, higher-end devices. “This requires a lot of memory and space to run. In the beginning it will be limited, but we’ll roll it out over time, said Kemler.

Closer to the Android Q launch, Google plans to release a list of sanctioned devices that will offer Live Caption.

Watch in Sign Language