A Novel Combination Model of Convolutional Neural Network and Long Short-Term Memory Network for Upper Limb Evaluation Using Kinect-Based System

Nowadays, somatosensory devices were widely used for developing the rehabilitation systems for limb-injured patients. However, professional evaluation was rarely studied in this field. In our study, we presented a novel hybrid deep network combined long short-term memory (LSTM) network and convolutional neural network (CNN) for rehabilitation evaluation referred to Brunnstrom Scale. In the identification task of 3-class Brunnstrom stages (III, IV, V), the mean accuracy of our proposed model was up to 80% (84.1%). The experimental result validated the reliability of our proposed evaluation method. And the comparison result of three machine learning algorithms indicated that the superiority of our hybrid model for Kinect-based 3D data analysis.