Towards robust and efficient text sign reading from a mobile phone

Embedded applications on mobile phones are reaching impressive goals thanks to the current powerful smartphones. This work is focused on text recognition applications from mobile phone pictures. Optical Character Recognition (OCR) methods have been developed for a longtime, but they still have poor robustness to process text in general scene images. Our general goal is to study and improve their results, in particular when running locally on a phone. We present a realistic prototype running on iOS, with a light geometry based pre-processing step that helps detecting regions of interest in the image, i.e., likely to contain text-signs. Then, we show how to process and filter these hypothesis to facilitate text recognition by standard OCR methods. This initial version is aimed to rectangular shaped signs to easily take advantage of geometric cues. We demonstrate the performance improvements of including our proposal together with several available OCR libraries. All steps are run locally on the phone in the designed application, which can read or translate the text using additional standard services in the phone.