Automatic Recovery and Verification of Subtitles for Large Collections of Video Clips

This paper describes an experimental system that can create good quality subtitle
ⁱ
files for video clips derived from broadcast content. The system is designed to run automatically without the need for human verification. The approach utilizes existing metadata sources, an off-air broadcast archive and an archive of original subtitle files along with audio fingerprinting and speech-to-text technology to identify the source programme. It then locates the position of the video clip, verifies the match between the video clip and the subtitles, and creates a new subtitle file. This paper also reports on the results of the work using a large corpus of over 7,000 video clips and, further, smaller sets of clips from different television genres, and explores where improvements might be made. It then looks at the limitations of the current approach discussing alternative methods for providing subtitles for video clips.
ⁱ

This paper uses the UK nomenclature of “subtitles” rather than “closed captions” as it was originally written for a European audience.