Holistic Pose Estimation and Dynamic Motion Analysis for Telerehabilitation of Physically Disabled Individuals

Telerehabilitation systems leveraging depth video analysis provide an effective solution for remote physiotherapy, particularly for individuals with physical disabilities. This study presents an advanced exercise classification framework that integrates multi-modal feature extraction and attention-based transformation to enhance rehabilitation monitoring. The proposed pipeline begins with depth image preprocessing, followed by human detection using a pre-trained Histogram of Oriented Gradients (HOG)-Support Vector Machine (SVM) model. The human silhouette is segmented using the GrabCut algorithm, enabling robust region-of-interest extraction. We propose a novel Lightweight Two-tier Key Body Point Detection (LT-KBPD) algorithm to efficiently and accurately identify key skeletal points, which are then used to extract both static and dynamic kinematic features. In parallel, silhouette-based analysis is performed, where shape descriptors, dense optical flow, Gaussian Mixture Model (GMM)-based body part segmentation, and contour analysis extract spatial and motion-related features. The extracted feature sets are fused into a comprehensive feature vector and further refined using an attention-based transformation mechanism to highlight salient features relevant to exercise classification. Finally, a Long Short-Term Memory (LSTM) network is employed to model temporal dependencies and classify exercises with high accuracy. The proposed approach is validated on three benchmark depth-video datasets: Kimore, K3DA and MEx - Multi-modal Exercise Dataset, achieving classification accuracies of 92.19%, 91.35%, and 85.51%, respectively. These results demonstrate the system’s effectiveness in accurately recognizing rehabilitation exercises for individuals with physical disabilities. Future work aims to enhance the adaptability of the system through personalized rehabilitation feedback and improved temporal modeling techniques.