Technology Blog #9 – Seeing AI
FREE: AiOS/Windows Phones 4/5
An App Review by Daniel Fuster
Person Centered Quality Facilitator, Arc of Westchester
Microsoft has recently developed a unique application for the visually impaired called Seeing AI. It runs on the iOS and Windows phone platforms. I have not tested the Windows Phone version but I have tested the iOS version on the iPhone 7, 10.5’ iPad Pro, and iPhone 6. It works effectively for both devices and I definitely recommend this application for the visually impaired and other disabilities that cause people to have difficulty remembering people by physical identification. This application is also useful for people who have difficulty remembering other people’s name.
I found that the application’s usefulness depends on the device it is on. There is a feature where a person is supposed to hold the device over a sheet of paper or anything else with text on it. The device will automatically read aloud what the is identified on the screen. If a person is trying to read something like a document, the iPad is more useful because its camera covers a larger area, whereas an iPhone does not. If a person is trying to cover a 8.5×11 sheet of paper with an iPhone, he/she will need to scroll the phone from left to right. The problem with this is that the camera is incredibly sensitive, and many times, it will jump back to the beginning of the sentence; which can be quite aggravating.
The iPhone, being more portable than the iPad, is more useful when using facial recognition. When I started using this feature, I was in the office and honestly I felt quite awkward asking my colleagues if they didn’t mind staring directly into the camera of my phone. Seeing AI will scan the face of the person almost immediately and determine the approximate age, facial expression, hair color, as well as any other outstanding features like glasses. The downfall for this application is camera quality. The iPhone 7 has a camera with higher pixilation, which means it can detect more details. The iPhone 6 has less pixilation, making the program slightly less accurate. My iPhone 6 identified me as being a 50-year-old male with grey hair. Now, I am a 31-year-old male with glasses and blonde hair. The iPad Pro and iPhone 7 both detected me as being a 31-year-old male with glasses and blonde hair.
Now here is a feature I find impressive. The application can use its facial recognition to indicate who the person is. I can ask John Doe to take 3 photos of himself and then the application grants me the option as to whether I would like to save the person by name. So in the future, the application will tell me exactly who the person is next time they appear on the camera. When using the app, I tried something different. I took three pictures – one photo for the right side of the person’s face, the second being the front view of the face, and the third being the left side of the face. When the photos were saved, I was able to detect who was in front without them looking directly at me. Once again, this app works better on devices that have better quality cameras.
There is one feature on this app which I believe can use substantial revision called “Document Mode”. In “Document Mode” a user can take a photo of text and the app converts it into a document which Apple’s VoiceOver feature can read aloud. As somebody who has a visual impairment, this feature was practically useless, but it sounded promising in the beginning. I find apple’s VoiceOver feature incredibly bothersome. So for me, “Document Mode” is absolutely useless. If Microsoft were to modify this feature where VoiceOver is not necessary to read the document aloud, that would be perfect. However, at this moment, this option isn’t available. On the other hand, for people who use “VoiceOver” this feature would be great.
The final feature for Seeing AI is the “Short Text” option. This is beneficial for people who want to read items on a menu or something quickly. In real-time, the application dictates the line of text as it is scrolled from side to side. The only downfall is that if you want to dictate an entire paragraph, it isn’t recommended because it will have difficulty determining where to begin; especially if the person using the application does not have steady hands.
All in all, this application is incredibly promising, especially for those with visual impairments. It even has the potential of changing lives. People with developmental disabilities can benefit from it as well in regards to reading facial details. As mentioned before, the application can describe the facial expression of the person. It also could be beneficial to people with low literacy abilities.
