Hybrid Self-Supervised and Semi-Supervised Framework for Robust Spatio-Temporal Action Detection

This paper presents a novel Hybrid Self-Supervised and Semi-Supervised Framework for Robust Spatio-Temporal Action Detection, which integrates the advantages of self-supervised transformation classification and semi-supervised temporal action proposal generation. The proposed approach addresses critical challenges in video action detection, specifically the requirement for spatially robust features and temporally consistent action proposals in contexts with limited labeled data. The self-supervised transformation classification branch enhances spatial generalization by training the model on a diverse set of transformations, while the semi-supervised temporal proposal branch employs a mean teacher framework with temporal perturbations to improve action proposal accuracy. The framework is evaluated on standard benchmarks, including UCF101, HMDB51, THUMOS14, and ActivityNet, where it consistently demonstrates superior performance compared to baseline models, achieving higher accuracy and mean Average Precision (mAP) across varying Intersection over Union (IoU) thresholds. An ablation study confirms the significance of each component, emphasizing the benefits of a hybrid approach in learning robust spatio-temporal representations. This research contributes an efficient, scalable solution for action detection, with potential applications in various video understanding tasks and domains.