New VizWiz database aims to teach computers to give blind people the information they need

Computers find it hard to answer questions about an image. This is an important skill as it could help blind people with daily life.

There are apps that blind people can use to take a photo, record a question like -‘What is the color of this shirt?’ or ‘When does this packet of milk expire?’ They can then ask volunteers for the answers. However, the images are often not well framed, or poorly focused, or lack the information needed to answer the question. This is because the photographers are not able to see.

Computer vision systems could help out by removing images that are not suitable. They can ask the photographer to take the image again. However, this is something that machines cannot do right now. This is due to the lack of a significant data set of real-world images that can be used to train them.

A team of researchers in Texas in the United States is working to change this. They have published a database of 31,000 images. The images come with questions and answers about them. The data set comes from an existing app called VizWiz. This has been developed to help blind people.

How VizWiz app works

With the app, a blind person can take a picture, record a question verbally, and send them both to a team of volunteers who answer as best as they can. The app VizWiz has many shortcomings. Volunteers are not always available. Also, the images do not always make an answer possible.

The researchers in Texas have tried to find a solution to these problems. They analyzed the photos gathered by VizWiz from users. The team removed photos that had personal details such as credit card information, addresses, or nudity.

They presented the images and questions to workers and asked them to give an answer made up of a short sentence. The team gathered 10 answers for each image to check for consistency.

These 31,000 images, questions, and answers make up the new VizWiz database, which has been made available to the public.

The team hopes to encourage others to develop more generalized solutions to help blind people. The aim is to offer an opportunity to develop assistive technologies that break down accessibility barriers for blind people.