A Combined Low-Rank Matrix Completion and CDBN-LSTM Based Classification Model for Lower Limb Motion Function

Assessment of lower limb motion function is crucial for clinicians to treat patients effectively. However, diagnosis accuracy is still affected by clinical experience. To reduce the influence of subjective factors, a new classification model using conditional deep belief network (CDBN) and long short-term memory (LSTM) is proposed. Firstly, the original data of gait are collected by motion capture system, and low-rank matrix completion is utilized to reconstruct missing data caused by the interruption of hands and clothes. Then CDBN is trained for extracting features and LSTM classifies the features from the completed data. Compared with deep belief network (DBN), CDBN is based on conditional restricted Boltzmann machine (CRBM) which improves the importance of past time-step to achieve higher accuracy for temporal sequences. And LSTM classifies features with time information. The matrix completion result verifies excellent recovery accuracy of the proposed method. Experimental results demonstrate that CDBN-LSTM is 4% higher than that of DBN. In addition, the proposed model outperforms traditional neural network on the train process and precision rate.


I. INTRODUCTION
In recent years, assessment of lower limb motion function has received significant attention in the medical field due to its consequence for clinicians to treat patients. Traditional methods are mostly based on the International Classification of Functioning, Disability and Health and Lower Extremity Motor Score [1], [2]. However, these methods are time-consuming and subjective. Since the accuracy of the assessment results rely on the experience of clinicians. It is necessary to develop an objective method to assess lower limb motor function.
To reduce the influence of subjective experience of clinicians, numerical studies based on biometric information have been carried out. On the basic of the energy changes in the mechanical and potential during the gait cycle, Robert Michnik et al. [3] have presented energy expenditure (EE) index to assess the conditions of subjects. By means of the data from patients with low back pain (LBP), a toe-heel The associate editor coordinating the review of this manuscript and approving it for publication was Siddhartha Bhattacharyya .
walking test is proposed in [4] to assess lower limb muscle weakness. According to the comparison in vertical ground reaction forces (VGRFs) between Parkinson patients and healthy people, a detection algorithm is presented in [5] to categorize gaits as balanced or unbalanced. Ming Tan et al. [6] have developed a Kinetic Index (K.I.) for assessing the conditions of stroke survivors by utilizing the surface Electromyography (sEMG) signals of both Tibialis Anterior (TA) and Gastrocnemius Lateral (GL). Xin Shi et al. [7] have proposed a scale unscented Kalman neural network (SUKFNN), which is based on the features extracted from sEMG via wavelet packet and Principal Components Analysis (PCA) to build a lower limb motion classification model.
Unfortunately, all the above works extract features from data directly, as a consequence of which the error of the collection system resulting in decrease of the assessment accuracy. Therefor the data should be processed before the feature extraction, including not only the elimination of the errors also the recovery of the missing data for completing the data matrix. In order to cope with the errors in data, Kim S. et al. [8] employ a deep neural network to recover matrix from partially observed distance information, which requires abundant data and time to train network. Ahmed H. I. et al. [9] have proposed a squared distances matrix completion through Nystrom algorithm, which is simple but only effective in the circumstance of large missing measurements. Low-rank matrix completion finds a matrix with the lowest rank from the observed data of the matrix. It can efficiently recover the missing data [10], [11].
As a powerful tool in the big data era, the deep learning network can simulate the learning process of human brain on the foundation of a deep model. With the development of the deep learning and the neural network technology, deep learning models are gradually introduced to the field of gait recognition. Nguyen T. N. and Meunier J. [12] collect point clouds of gait obtained from depth sensors, and propose Gait Normality Index (GNI), which is based on deep auto-encoder, to indicate human gait normality. In [13] a two-branch multistage Convolutional Neural Network (CNN) is trained by Skeleton Gait Energy Image (SGEI) to accomplish the gait recognition task. Classifier Oriented Gait Score (COGS) is a pathology specific score for the measurement of the gait quality. For the sake of feasibility in practical applications Josef Christian et al. [14] improve COGS by decomposing it into 15 interpretable sub-scores. Marwa E. Ehaimir et al. [15] extract features from gait sequences with Gait Pal and Pal Entropy (GPPE) and classify the gait by a neural network. Ma et al. [16] collect ground reaction forces and present the abnormal evaluation index (AEI) based on a semi-Markov process. In [17], the lower limb sEMG signals of the cerebral palsy patients are utilized to establish synergy comprehensive assessment (SCA) for quantitative evaluation motor dysfunction. According to the data of the activation patterns of lower limb muscles in Duchenne muscular dystrophy children through sEMG M. Rinaldi et al. [18] analyze the level of co-activation of agonist-antagonist muscles at the knee and ankle joints to evaluate the lower limb motor function. While, the parameters of the traditional neural network are obtained by random assignment, thus making it end in the local extremum some extent. Deep belief network (DBN) proposed in [19] utilizes an unsupervised pre-training strategy, which is based on Restricted Boltzmann Machines (RBM), to solve the problem and improves the training efficiency [20]- [22]. Conditional Restricted Boltzmann Machine (CRBM) adds connection from the last few time-steps to latent and visible variables, enhancing the importance of the past time-step to improve accuracy [23]. Moreover, in many neural networks, the recurrent neural network (RNN) improves the state-ofthe-art performance on sequence modeling tasks [24], [25]. Additionally, LSTM is a type of RNN, overcoming the problem of the long-term dependences such as vanishing and exploding gradients [26], [27].
Gait data have high temporal correlations, whereas traditional deep networks, like CNN and DBN, loss the correlations during the extraction process. In order to address the aforementioned problems, a novel classification model using low-rank matrix completion and CDBN-LSTM is proposed in this paper. Low-rank matrix completion reconstructs the missing data of gait. CDBN is employed for features extraction of gait time series data, which strengthens the influence of past data. Furthermore, LSTM network is utilized for features classification. Compared with traditional neural network for gait recognition, the proposed model strengthens the influence of past data, reduces time of training, and improves the accuracy.
The rest of this paper is arranged as follows. Section 2 introduces the process of data collection ans completion. Section 3 proposes the classification model based on CDBN and LSTM. Section 4 presents the experiment results of proposed algorithm. Section 5 summarizes the study and put forward future work.  Fig. 1. And Fig. 2 shows the interface of Vicon system.

II. DATA COLLECTION AND COMPLETION
The data of hip, knee and ankle angles collected by Vicon are shown in Fig. 3. Averages of angles are represented by red curves and standard deviations of data are represented by blue lines. And subfigure (d) shows a subject's integrated gait.
All subjects of this paper were divided into three groups by clinician, including middle-age stroke group (MG, VOLUME 8, 2020   26 subjects), elderly stroke group (EG, 26 subjects) and young healthy group (YG, 27 subjects). The conditions of the three group subjects are listed in Table 1, including age, gender and physical condition. The MG group consists of subjects with motor dysfunction aged between 40 and 59 years. The EG group consists of subjects with motor dysfunction aged between 60 and 70 years. Both the EG and MG groups are the test groups in this study and YG group is a reference group. The YG group consisted of subjects with motor dysfunction aged between 20 and 39 years. All the subjects could complete the gait collection without any help or assistance.

B. DATA COMPLETION
When a marker is captured by three inferred cameras, the position of marker is ensured in the space. However, markers are often interrupted by hands and clothes, which causes the data matrix is incomplete. The -rank matrix completion is used to recover missing data. An incomplete lowrank matrix is E ∈ R m×n and the reconstruction matrix is X . The problem can be described by: where ⊂ {1, 2, · · · , n}×{1, 2, · · · , m} is the set of sampled entries of E,and rank(X ) represents the rank of matrix X . The problem can be expressed as P (X ) = P (E). And P (·) is defined as: The rank of the matrix X is discontinuous and non-convex. To solve the problem, it is converted into the problem of solving Nuclear Norm Minimization (NNM). Thus, the problem is transformed into the following: where X * is the nuclear norm of X , the algorithm starts with Y 0 = 0 ∈ and updates as follows: The D τ (Y ) is the singular value shrinkage operator, it is denotes as D τ (Y ) = UD τ ( )V T where U and V are matrices with orthonormal columns and D τ ( ) = diag(max{σ i − corresponding to the singular values of matrix. The step size of the iterative algorithmic process is given by δ k .

III. CDBN THEORY AND LSTM NETWORK A. CDBN THEORY
The CDBN is formed by multiple layers of CRBM, which is mainly composed of neurons in the visible and hidden layers. And there is only a connection between the visible layers and the hidden layers, and no connection is between the layers. The structure of CRBM is shown in Fig. 4.
The energy function of the CRBM is: where W is the weight matrix between the visible and hidden layers, the input data is v = {v 1 , v 2 , · · · , v m } ∈ (0, 1) and the data of hidden layer is h = {h 1 , h 2 , · · · , h n } ∈ (0, 1). The bias value of each visible node b j , j ∈ m and c i , i ∈ n is the bias value of each hidden node. The joint probability density of v and h is: And the edge probability density of v and h can be described as: CDBN is a probabilistic generation model, which can extract characteristic features from high dimensional data. Based on the CRBM, CDBN can be constructed by adding layers. The structure of CDBN is shown in Fig.5. The training process of the CDBN is mainly divided into two steps: 1) Each layer of the CDBN is trained separately, the first CRBM's hidden layer is the next CRBM's visible layer. This structure can extract high level features from lower layer features.
2) The layer-by-layer training of CRBM can only achieve the optimal parameters of current layer network. The whole model needs to be trained by back propagation (BP) algorithm and fine-tune to get the optimal weight of the structure. Where v 0 t represents the visible variable at t and h 1 t represents the variable of hidden layer 1 at time t, the autoregressive weight is α × A and the weight of connection from the past to hidden units is β × B. And the energy function is reformulated as: where a i and b j are the bias of visible and hidden units, α and β are parameters which strengthen the past influence, and the distribution of h j t is: where β k B ki v k presents the contribution from past and β is the artificial differentiated parameter for the directed connection from past to hidden units, and the reconstruction distribution can be obtained as: where α k A ki v k represents the contribution from the past and α is the artificial differentiated parameter on the autoregressive weight, N (·) is normal distribution probability density.

B. LSTM NETWORK
As shown in Fig. 6, LSTM introduces oblivion gate f n , input gate i n and output gate o n to control the effect of the previous neuron on the current neuron. VOLUME 8, 2020  The LSTM forward calculation formulas are: The input at time n is X n , the cell state at time n is S n and the output at time n is Y n . The weights of forget gate, input gate, output gate and cell state are presented by W f , W i , W o and W s . And biases of forget gate, input gate, output gate and cell state are presented by b f , b i , b o and b s . After forward calculation training, back propagation algorithm through time (BPTT) is used to update the weights of LSTM.

C. RESEARCH METHOD
The architecture of CDBN-LSTM contains two parts: a multilayer CDBN and a LSTM network. Firstly, gait data are divided into train set and test set. The train set of sample gait data trains RBMs' parameters layer by layer, and then BP algorithm is used to optimize the entire network. After the training process of CDBN, the test set is input into CDBN to get the feature vectors from the last hidden layer of CRBM. Then the feature data from training set are used for the forward calculation of LSTM network, and BPTT is used to optimize the hidden units of LSTM. The architecture of CDBN-LSTM is shown in Fig. 7.

A. DATA COMPLETION ANALYSIS
Matrix completion simulation has been executed through MATLAB. The missing data are set to 0. And low-rank matrix completion reconstructs the incomplete matrix. Fig. 8 shows the comparison of the original gait and recovery gait, which verifies the algorithm's effectiveness.

B. CDBN-LSTM NETWORK ANALYSIS
Theoretically, with the increase of depth in CDBN, the learning ability becomes stronger and stronger. But in practice, the performance of network relies on the different sample size and the distribution of samples. Fig. 9 shows the training root means square error (RMSE) of different layers of DBN and Fig. 10 shows the testing classification results of different layers of DBN. Compared with RBM and DBN with three RBM layers, DBN with two RBM layers can extract features more efficient, and its accuracy is higher than the other methods. The results show that RBM's performance is preferable to DBN with three RBM layers, which implies the model is overfitting.    performance of CDBN with 2 layers of CRBM is best, and CRBM is better than the CDBN with 3 layers of CRBM.
The comparison of two structures of DBN is presented in Table 2. The results show that CRBM can achieve better performance, which demonstrate that strengthening the influence of past time steps truly improves the performance. So, this paper uses CDBN with two-layer CRBMs to extract features.    that increasing the hidden units of LSTM can improve the accuracy. When the number of hidden units is 800, the network performs best. In this paper, the number of hidden units is 800.
The classification results of DBN, CDBN and CDBN-LSTM are presented in Fig. 14. All models use two RBM(CRBM) layers as DBN(CDBN) structure. The results demonstrate that compared with DBN, CDBN just improves the accuracy because its features have time correlations. VOLUME 8, 2020 CDBN-LSTM can get information of sequence features and classify, which makes the model more efficient in sequence data classification.

V. CONCLUSION
This paper presents a novel lower limb motion function classification based on low-rank matrix completion and CDBN-LSTM. Lower limb kinematic data are collected by Vicon and reconstructed by low-rank matrix completion. Then, CDBN extracts features recovery data and LSTM classifies features. The completion result demonstrates the efficiency of low-rank matrix completion. Moreover, the proposed model extracts more information from historical sequence efficiently. The experimental results demonstrate that CRBM can improve RBM. The model, which has two layers of CRBM, shows the best performance. And then this paper discusses the number of hidden units of LSTM and chooses the best structure of LSTM. In addition, the final result shows that the proposed method improves the performance of DBN on accuracy and converge speed significantly.
The proposed method is verified offline. In the future, we will attempt to use this method to classify the condition of gait online. And we only focus on evaluating dysfunction of lower limb, which affects the motion function directly.