Deep Learning-Driven Body-Sensing Game Action Recognition: A Research on Human Detection Methods Based on MediaPipe and YOLO

Body-sensing games are gradually becoming an important means of human-computer interaction, yet traditional recognition methods struggle to meet the demands of both high accuracy and low latency simultaneously. To address this, this paper proposes a real-time human action recognition method that fuses YOLOv11 and MediaPipe. The proposed method employs an improved YOLOv11 algorithm to achieve efficient human target detection, reaching an accuracy of 90.5% and a mAP50-95 of 73.3%, effectively solving the problem of insufficient detection accuracy in complex environments and for small targets encountered by traditional methods. Meanwhile, the MediaPipe algorithm is used to accurately localize key points in the human regions detected by YOLO, thereby enabling pose estimation and action classification, with a key point localization error (MPJPE) of only 8.5 mm and a real-time processing speed of 24 FPS. Experimental results demonstrate that this method significantly outperforms OpenPose and HRNet in both accuracy and real-time performance, effectively enhancing the interactive experience for users.