Microsoft’s AI-powered assistant app for the visually impaired will now be able to describe scenes and read text in Spanish, Japanese, German, French, and Dutch
- Seeing AI is a virtual assistant that describes scenes and objects to the blind
- The iOS-exclusive app analyzes images and video from the iPhone cameras
- The new update is the first time the app will speak a language other than English
Today Microsoft announced an update to the Seeing AI app that will include new language output options, including Dutch, French, German, Japanese, and Spanish.
The iOS exclusive app was first released in 2017 as a free tool to help people with visual impairments navigate day-to-day life.
It’s built around a series of different channels, which users can select depending on their particular needs or circumstances.
For the first time Microsoft’s Seeing AI app will speak in languages other than English. Today’s update enables audio output in Japanese, German, Spanish, Dutch, and French
In one channel, the app will read out the text of any document the iPhone’s or iPad’s front facing camera is pointed at.
In another, it will describe the appearance of other people and users can flag close friends or family members so the app can recognize them in future encounters.
The app can also scan bar codes to identify products, read nutritional labels, translate currency, identify colors, describe the light levels of one’s surroundings, and give overall descriptions of a room or setting.
Users can save the faces of friends or family members so the Seeing AI will be able to immediately recognize them in the future.
Seeing AI is structured around channels, which users select to help the AI process the visual information being taken in through the phone’s front facing camera
‘More than the technology itself, the thing that has really touched me is the way that people have taken the features of Seeing AI and incorporated it into their personal lives,’ Microsoft’s Saqib Shaikh, co-founder of Seeing AI, said in a statement released alongside the app.
Shaikh had been especially touched by the story of a father who’d written to him to share how the app had helped him bond with his son during trips to the zoo.
‘The father was feeling a bit down before because he couldn’t really interact at the zoo, tell his son what was around him,’ Shaikh said.
‘But with Seeing AI, he was able to read the signs. And suddenly now, he could have that father-son moment of teaching his son where the monkeys live, and what they eat and all that kind of thing.’
Seeing AI was originally launched in the US, Canada, India, Hong Kong, New Zealand, and Singapore.
Today, the app is available in 70 countries.
The latest update marks the first time it will be able to output in a language other than English.
