Depth Sensor-Based In-Home Daily Activity Recognition and Assessment System for Stroke Rehabilitation

Stroke is a leading cause of long-term adult disability. Many stroke patients participate in rehabilitation programs prescribed by an occupational therapist to aid in recovery; however, occupational therapists rely on in-clinic assessments and often-unreliable self-assessments at home to track a patient's progress, limiting their ability to monitor how patients perform outside of a clinical setting. Our Daily Activity Recognition and Assessment System collects depth and skeletal data passively from within the patient's home to assess long-term recovery and provide metrics to an occupational therapist to allow for more individualized rehabilitation plans. Using data from a wall-mounted depth sensor, we adapt a hierarchical co-occurrence network to identify actions from pre-segmented skeletal data. We then perform assessments on the classified actions to track key recovery metrics: normalized jerk, speed of motions, and extent of reach. We also introduce novel filters to identify high quality data for analysis. Our sensor was installed in a stroke patient's kitchen for seven days, generating the first action recognition data set from a stroke patient in a naturalistic environment. We use this data in conjunction with the NTU-RGB-D data set to validate our recognition and assessment algorithms. We achieved 90.1% accuracy by replicating the results of the NTU-RGB-D data set and a maximum of 59.6% accuracy on our kitchen data set.