This statue at the Seeing Eye, Morristown, NJ, commemorates the many guide dogs who have been trained there. Seeing AI is a play on an earlier trademark: Seeing Eye dog.
Assistive technology is finding more and more ways to see for the blind and convey what sighted people experience. In a November 2017 piece in this space, we discussed Aira, essentially a pair of dark glasses with a camera and microphone. Live agents use the video and audio transmissions to guide the blind user of the device and service. Since its April 2017 launch, Aira continues to grow, often mentioned in the news with acclaim.
Now, Microsoft has Seeing AI, a free app for iOS (Apple) that “narrates the world around you.” The manufacturer designed the app for persons with a visual impairment. Seeing AI is a research project “that harnesses the power of artificial intelligence to describe people, text, and objects.” Microsoft recently released version 2.2 of Seeing AI. Unlike Aira, the app has an internal “voice,” namely VoiceOver, the screen reader built into Apple’s portable iOS devices.
Need to find out more? Microsoft offers introductory tutorials, starting with an overview:
The videos that follow each cover the “channels,” each one providing a specific type of information:
Short Text Channel. Seeing AI can read any standard text. The ability of computers to use OCR has been around for a while, but Seeing AI is able to read the text right away.- Document Channel. This feature scans an entire page of text. Then, the phone goes a step further and detects the formatting (such as heads and subheads), allowing one to navigate the page easily with the commands of VoiceOver.
- Product Channel. This channel allows the user to scan barcodes to identify products. A series of beeps guides the user to where the barcode is on the package.
- Person Channel. The Person Channel recognizes and describes people, including an estimate of their age and, more important, their facial expressions. A guide helps the visually impaired person to center the face. (Centered photos of people smiling are also much more useful for posting to social media.) In recognizing people who are friends, the user takes the picture of the person and keys in or says his or her name.)
- Scene Channel. As research for Seeing AI is ongoing. One of these experimental features is the ability to describe a scene. An algorithm detects the objects in the photo frame and seeks to determine the relationship among them, describing the landscape in a single sentence. The AI component “learns” what the app detects and refines the scene description capability continually.
- Currency Bills Channel. US currency bills are all the same size and color. This feature differentiates each (even if the note is upside down).
- Recognizing Images in Other Apps. Among the options in the SHARE feature of an app. One of these actions is “Recognize with Seeing AI.”
From here, we’ll end on a humorous note, with a funny blog essay.
