Transformer Based Text Summary Generation for Videos

With advancements in the multimedia sector, abundant videos are being generated for several applications. Consequently, video summarization has emerged as a crucial endeavor, aimed at distilling key segments from videos while preserving essential information. Furthermore, the ability to comprehend video content and autonomously generate event synopses holds significant potential for streamlining the creation of video descriptions, thereby alleviating the manual efforts typically required for this task. In this paper, a Transformer-based Video-to-text generator is proposed that can generate a concise summary for videos containing frequent scene changes. The proposed pipeline involves the identification of keyframes from the input videos and the extraction of images. Evaluation is done on a custom dataset, and the average BLEU 4 and ROGUE-f-measures obtained are 50.69% and 37.78% respectively. These results outperform the CNN- LSTM video-to-text summarization approach and highlight the robustness of Transformers.