Gamified Rehabilitation and Physiotherapy Using MediaPipe for Posture and Hand Gesture Recognition

Introduction: Rehabilitation for conditions affecting mobility, such as stroke, muscle atrophy, and others, poses significant challenges in traditional physiotherapy practices. Engaging patients in enjoyable exercises is crucial for maintaining compliance with rehabilitation protocols. In response, this paper introduces a pioneering game-based approach to rehabilitation, leveraging posture detection technology. Materials and Methods: A game-based rehabilitation system was developed utilizing deep learning techniques through the MediaPipe framework for posture and hand gesture recognition. The system analyzes user posture in real time via a webcam and translates these movements into game controls, allowing patients to interact with popular games like Subway Surfers to simulate physical activities such as running and jumping. Latency between patient movements and in-game responses was measured across three hardware configurations (8GB, 12GB, and 16GB RAM) to evaluate system performance and responsiveness. Results and Discussion: Latency decreased with higher RAM, with the lowest measured at 430 ms on a 16GB configuration. This real-time responsiveness enhances patient engagement, making the rehabilitation process more immersive and effective. By leveraging posture detection technology, this innovative system bridges the gap between the necessity of rehabilitation exercises and the appeal of gaming, offering a promising solution for improving treatment outcomes across diverse age groups and conditions.