Robust Activity Recognition Based on Human Skeleton for Video Surveillance

Activity recognition is an important task in video analysis, which can be used in accident monitoring and other daily applications. Traditional activity recognition methods are mainly based on the pixel-level analysis of 2D images. However, they achieve poor robustness in various complex environments, and are vulnerable to the perspective distortion caused by the fixed camera view. To address these challenges, we propose a robust skeleton-based human activity recognition method using a fixed monocular surveillance camera. We encode human skeleton with more critical motion information like pairwise distances between keypoints to capture high-level motion modality. Besides, we normalize skeleton data to eliminate the defects of 2D frames, such as the impact of distance on skeleton scale. Furthermore, we propose a skeleton calibration method based on perspective transformation to adapt our method to the deployment environment of surveillance cameras, especially different downward pitch angles. Experimental results show the recognition accuracy of our system reaches 91 percent with a frame rate of 10 FPS.