Towards On-Demand Virtual Physical Therapist: Machine Learning-Based Patient Action Understanding, Assessment and Task Recommendation

In this paper, we propose a machine learning-based virtual physical therapist (PT) system to enable personalized remote training for patients with Parkinson's disease (PD). Three physical therapy tasks with multiple difficulty levels are selected to help patients with PD improve balance and mobility. Patients' movements are captured by a Kinect sensor. Criteria for each task are carefully designed by our PT co-author such that the patient's performance can be evaluated in an automated manner. Given the patient's motion data, we propose a two-phase human action understanding algorithm TPHAU to understand the patient's movements, and an error identification model to identify the patient's movement errors. To enable automated task recommendation, a machine learning-based model is trained from real patient and PT data to provide accurate, personalized, and timely task update recommendation for patients with PD, thereby emulating a real PT's behavior. Real patient data have been collected in the clinic to train the models. Experiments show that the proposed methods achieve high accuracy in patient action understanding, error identification and task recommendation. The proposed virtual PT system has the potential of enabling on-demand virtual care and significantly reducing cost for both patients and care providers.