A Hierarchical Approach to Sign Recognition

Sighted individuals draw a significant amount of information from signs but this information is denied to the visually impaired. VIDI is an evolving system for detecting and recognizing signs in the environment and voice synthesizing their textual contents. The wide variety of signs commonly encountered and the uncontrolled nature of the real world add significant complexity to the problem. VIDI treats the recognition problem as one of matching an unknown sign image, obtained from the detection component as a hypothesized sign, to a database of known signs. A color based support vector machine classifier coarsely picks a group of sign classes that are the most likely matches to the query. A finer retrieval technique employing corners and shape contexts ranks the hypothesized sign classes and verifies whether or not the top ranked class is the true class of the query. The database includes a set of real images with a wide variety of sign classes, each containing multiple signs exhibiting not only illumination differences, but also rotational variations. Tested on over 1,200 images, our system correctly recognizes and identifies the sign class of a query, achieving a 94.75% accuracy.