Shot-Level Semantic Reward-based Self-Attentional Network for Unsupervised Video Summarization

Video summarization is the task of condensing long videos into shorter versions while preserving the important content. This research introduces a novel self-attention-based technique for unsupervised video summarizing. To produce more realistic video descriptions, the suggested method combines deep reinforcement learning with a semantic reward function at the shot level. Minimizing the impact of user subjectivity is a significant challenge in video summarization, and the reward function is designed to achieve this objective. For recording videos, shot-level semantics are employed as rules that apply to all viewers. Our approach is based on deep reinforcement learning and differs from previous methods in this area with Convolutional Neural Network (CNN) in the encoding part extracting high-level features from the input video frames and a Gated Recurrent Unit (GRU) model in the decoding part, generating the summary by decoding the features extracted by the encoder is trained on a dataset of videos and their corresponding summaries. To get a key shot summary with no need for manual annotations and learn from the input video, the suggested model combines a bi-directional recurrent neural network, reinforcement learning, and self-attention methods. The approach outperforms video summarizing methods without supervision on the SumMe and TVSum datasets.