A small tweak to Amazon Alexa makes it accessible to deaf people
Now Amazon Alexa can respond to sign language! This is thanks to developer Abhishek Singh, who has created an app that allows Alexa to do this.
Singh's project uses a camera-based system to identify gestures and interpret them as text and speech.
Voice assistants run by Amazon, Google and Apple have seen a rise in popularity over the last few years. But for the deaf and hard of hearing community, this is a future they are left out of.
Speech recognition is rarely able to pick up the rhythms of deaf users. Lack of hearing poses a challenge to communicating with voice-based assistants.
Singh's project involves rigging Amazon's Alexa to respond in text to American Sign Language (ASL). He trained an AI using the machine-learning platform Tensorflow, which involved repeatedly gesturing in front of a webcam to teach the system the basics of sign language.
Once the system was able to respond to his hand movements, he connected it to Google's text-to-speech software to read the corresponding words aloud.
The Amazon Echo's vocal response is automatically transcribed by the computer into text, which is read by the user.
Singh says this shows that such a functionality can be built into any voice assistant.
Watch in Sign Language