DF Sampler: A Self-Supervised Method for Adaptive Keyframe Sampling

Video understanding has become crucial in computer vision research due to the vast amounts of video data generated from various sources. Efficient video analysis, especially the extraction of keyframes from long video sequences, is crucial when combined with state-of-the-art deep learning algorithms for video understanding. This has numerous applications, including smart surveillance, autonomous driving, video summarization, and human activity monitoring in human-robot collaboration (HRC) settings. In the context of Industry 5.0, such analysis aids in enhancing productivity, safety, and ergonomics for industrial workers. However, there is a notable lack of research on adaptive frame selection for video-based Human Activity Recognition (HAR) systems in challenging, uncontrolled industrial environments, leading to high memory and computational demands. To address this, we introduce Dynamic Frame Sampler (DF Sampler), a novel adaptive keyframe sampling method designed to tackle the challenges of processing variable-duration action sequences in complex industrial environments. DF Sampler uses a self-supervised learning approach to adaptively select keyframes, prioritizing segments with significant motion while optimizing the computational efficiency of HAR systems. The method is evaluated on the publicly available HRI30 dataset.