Drone-Based Human Action Recognition for Surveillance: A Multi-Feature Approach

Understanding human actions is crucial for enabling machines to interpret human behavior, with applications spanning video-based surveillance applications, sports monitoring, and entertainment systems. However, the diversity in human motion and appearance poses significant challenges for action recognition, especially when using drones for video capture, which exacerbates complexities such as dynamic backgrounds, motion blur, varying capture angles, and exposure issues. In this study, we propose a system tailored to address these challenges, specifically in drone-recorded RGB videos. Our approach involves splitting the video into frames and applying some preprocessing steps to these RGB frames. In preprocessing, we focus on reducing the computational cost, resizing the image quality, and enhancing foreground objects by removing the background. Subsequently, we also detect humans from images by applying YOLOv5. By using this, we can detect humans from the original frames, from which we extract the human skeleton and identify key points representing various body parts. These key points, encompassing critical joints such as the head, wrists, elbows, thighs, knees, and ankles, are used to derive normalized positions, angular relationships, distance measures, and 3D point clouds. For feature optimization, we use the Linear Discriminant Analysis (LDA), followed by classification via a multi-class SVM. Experimental validation using the Drone Action dataset demonstrates notable action recognition accuracies of 83.2%, showcasing the efficacy of our proposed system.