Video Summarization System using NLP and Latent Semantic Analysis

In recent years, automatic captioning of multimedia content, especially videos and audio, has garnered significant attention due to its potential applications in accessibility, content indexing, and human-computer interaction. The proposed paper presents a novel architecture with the integration of Neural networks and Long Short-term Memory (LSTM) via a disintegrator model, the process learns to map a video to a sentence, where encoding is done by the Xception model which is of prima facie and comprises of compression segments into its static representation in the form of feature vectors, Word2Vec function is implemented for better vectorization. The proposed model assigns the respective captions to the audio and the video segments and integrates them accordingly and represents the semantic relation between the words and the summarized captions. Caption's similarity, fluency, accuracy as well as precision are checked with parameters such as BLEU, METEOR, CIDER, and ROUGE. This approach advances the field of multimedia recommendation models also providing aid to the visually impaired. The model is effectively able to capture both the spatial and the temporal features from video frames and the audio segments. The model also showcases the use of sound effects provided in video to generate concise captions using a novel audio captioning architecture.