Instantaneously responsive subtitle localization and classification for TV applications

This paper presents an algorithm for localization and classification of subtitles in TV videos. We extend an existing static-region detector with object-based adaptive temporal filtering, bounding box computation around blobs of refined static regions, bounding box categorization based on geometry and filling degree of static regions, and subtitle classification using text-stroke alignment features. On a test set of more than 5000 video frames, a Precision rate of 96% is achieved at 98% Recall rate. The system detects subtitles without frame delays, and uses techniques suitable for implementation in a TV platform. We also experimentally show that the picture quality of Motion-Compensated Picture Rate Conversion in televisions can benefit from our system.