The research work by two students at the Penn School of Engineering and Applied Science with help from the GRASP director resulted in providing the robot with reading capabilities.
Penn Engineering graduate student Menglong Zhu and Graspy
The robot (PR2) is called Graspy and it is the first time a robot has been programmed with reading abilities. The PR2 has been built by Willow garage and has been provided to institutes like the Penn for research purposes. Menglong Zhu, a graduate student, has worked on the creation of the reading ability. Another GRASP team is focussing on Graspy’s object manipulation capabilities.
Zhu’s research concentrates on cameras fitted as eyes of the robot and the programming, which is the brain of the robot. The cameras process the images of words and convert them into their digital equivalents. This conversion is termed as optical character recognition (OCR). Graspy goes about looking for groups of closely bunched lines having identical widths and spacing considering them as words. It performs OCR on these words and checks it against a customizable dictionary. Graspy uses an algorithm that is similar to that found in the spellchecker programs in order to ensure accuracy. It is important that the program coded by Zhu takes into consideration the angle, from which Graspy views the words. Once OCR is complete, Graspy spells the words aloud with a speech synthesizer. The ability to read helps the robot recognise its location by reading sign boards. Another application of this ability is in helping vision impaired people in reading signs in front of them by wearing a camera.