Multi-person Activity Recognition and Tracking Based on Skeletal Keypoint Detection

Currently, most action recognition networks have deep overall structures, large model parameters, and high requirements for computer hardware equipment. As a result, it is easy to over-fit in the recognition process for too deep network layers. Furthermore, it is also difficult to extract features because of the video's interference information, such as illumination and occlusion. To solve the above problems, we propose a multi-person action recognition and tracking algorithm based on skeletal keypoint detection. Firstly, the n network combining the improved dense convolutional network and part affinity field (PAF) are used to extract the skeletal information points of the human body. Then, we present an improved Deepsort network for multi-person target tracking, which contains a Hungarian matching algorithm based on the generalized intersection over union (GIoU) and a pedestrian re-identification network combining GhostNet and Feature Pyramid Network (FPN). Finally, we construct a deep neural network model to classify the extracted human skeletal information and realize action recognition. Experiment results show that the multi-person action recognition and tracking algorithm achieves an action recognition accuracy of 98%. In addition, The multi-target tracking accuracy of the proposed algorithm is improved by 4.2% on the MOT16 dataset. Compared to other common algorithms, the proposed algorithm can achieve high accuracy in detecting keypoints of the human body and improve the accuracy of multi-person action recognition with fewer parameters and complexity of operations.