Enhancing Video Summarization Using Object and Action Detection

Video content consumption has surged in recent years, with over 500 million hours of video watched daily on YouTube alone. This trend underscores the growing dominance of video as a primary medium for information exchange. As textual content continues to yield ground to visual and video-based formats, the need for effective video management tools, such as video summarization, becomes increasingly evident. Video summarization is a vital process that condenses lengthy videos into keyframes or essential segments, preserving the video's main content. This paper redefines video summarization as “a technique to create a short skim of the original video while preserving the main contents,” akin to skimming through textual information. The applications of video summarization are diverse, with a primary focus on gauging user interest and enhancing content sharing on social media. This research project endeavors to harness the power of neural networks to comprehend the semantics of video content for improved video summarization. The core concept involves leveraging object detection and action detection algorithms to achieve a deeper understanding of video semantics. The proposed video summarization system integrates two key components: a PyTorch implementation of the YOLOv3 model for object detection and a pre-trained Epic Kitchens model for action detection. These components work in tandem to extract vital information from video frames. Following the detection of objects and actions, the system seamlessly merges and compresses the results, ultimately generating a text file that captures the temporal, verb, noun, and object attributes of all detected keyframes. This research sets the stage for enhancing video summarization techniques by harnessing the capabilities of object and action detection, marking a significant stride toward more efficient and informative video content management and sharing in an era where video dominates the information exchange landscape.