Text detection and recognition in urban scenes

Text detection and recognition in real images taken in unconstrained environments, such as street view images, remain surprisingly challenging in Computer Vision. In this paper, we present a comprehensive strategy combining bottom-up and top-down mechanisms to detect Text boxes. The bottom-up part is based on character segmentation and grouping . The top-down part is achieved with a statistical learning approach based on box descriptors. Our main contribution consists in introducing a new descriptor, Fuzzy HOG (F-HOG), fully adapted for text box analysis. A thorough experimental validation proves the efficiency of the whole system outperforming state of the art results on the standard ICDAR text detection benchmark. Another contribution concerns the exploitation of our text extraction in a complete search engine scheme. We propose to retrieve a location from a textual query: combining our text box detection technology with OCR on georeferenced street images, we achieved a GIS system with a fully automatic textual indexing. We demonstrate the relevance of our system on the real urban database of [10].