StreetOCRCorrect: An Interactive Framework for OCR Corrections in Chaotic Indian Street Videos

Obtaining a high-quality OCR output in smart cities, with human-in-the-loop, is an interesting problem for surveillance and other similar applications. Achieving high accuracy while reading license plates in the real world videos is cumbersome due to complexities like multiple vehicles, high-density traffic in spatial and temporal domains, varying camera angles and illumination, occlusions and multiple resolutions. We present a modular framework for OCR corrections in the chaotic Indian traffic videos that especially involve complex license plate patterns. Such patterns are obtained from a state-of-the-art deep learning model trained on video frames. Since such a model reads the text from videos (instead of images), we incorporate multi-frame consensus for generating suggestions in our framework. To ease the correction process, our human-interactive framework first breaks down the multi-vehicle videos into multiple clips, each containing a single vehicle from the video using an object detector and a tracker. Our framework then provides suggestions for an individual vehicle using multi-frame consensus. Our framework then selectively presents these extracted clips to the user to verify/correct the predictions with minimal human efforts via interactive suggestions. Such high-quality output can be used to continuously update a large database for surveillance and can be further used to improve the accuracy of deep models in the complex real-world scenarios.