Subtitle Synchronization Using Whisper ASR Model

This paper introduces a novel approach to subtitle synchronization using the Whisper ASR (Automatic Speech Recognition) model from OpenAI. The primary aim of this research is to achieve accurate and robust synchronization of subtitles with audio content, even in the presence of uniform or non-uniform delays. The project leverages the capabilities of the Whisper ASR model, which is trained on diverse audio datasets and offers multitasking functionalities including multilingual speech recognition, speech translation, and language identification. Key features of the project include Whisper ASR integration, SRT modification, and accuracy enhancement. The system utilizes FFmpeg for audio extraction and preprocessing, followed by speech recognition using the Whisper ASR model. Preprocessing techniques are applied to enhance the precision of generated timestamps, ensuring precise synchronization. The methodology also involves timestamp adjustment based on text comparison between input and transcribed SRT files, resulting in accurately synchronized subtitles. The project offers a user-friendly interface for input acquisition and interaction, guiding users through the synchronization process. While promising, the system has limitations such as the lack of action detection, character name prediction, and challenges with repetitive word handling. Nonetheless, the Whisper ASR-based Subtitle Synchronization project presents a reliable solution for enhancing subtitle accuracy and accessibility in various video content scenarios.