Rehabilitation Exercise Recognition and Evaluation Based on Smart Sensors With Deep Learning Framework

Exercise therapy is seen as one of the major treatments for the rehabilitation for patients, particularly using modern technologies, such as virtual reality or augmented reality. Computer-assisted physical rehabilitation training involves measuring performance by analyzing the movement data collected with a sensory system during prescribed rehabilitation exercises. Human activity recognition is a challenging topic for machine learning in the present area of research. Since the sensor-based activity recognition seeks deep knowledge from various low-level sensor readings concerning human activities. In this paper, the Smart Sensor-based Rehabilitation Exercise Recognition (SSRER) system has been proposed using a deep learning framework. For the recognition of rehabilitation exercise with sensor information, a convolutional neural network (CNN) has been used on dynamic platform(D-CNN) where it has sensory data for physical rehabilitation exercise body movement by Gaussian mixture models (GMM). The input signals and GMMs are in various segments contains shapes for many CNN routes. To retrieve the state transition likelihood of hidden states, the Sensor (S-CNN) utilizes the algorithm of improved lossless information compression as discriminant features of various movements. Therefore, the hybridized CNN of the Sensor (S-CNN) and D-CNN are combined with a deep learning classifier to assess every rehabilitation class exercise at different levels. The categorized deep learning methods show improved performance with best-learned features for any rehabilitation exercise. The difference between the best attribute and the test score analyzed mathematically with our collected data and a variety of activity recognition datasets has been illustrated in this article with test results.

Rehabilitation is one of the main steps towards recovery from operation, particularly after surgery with the joint disease [1]. In postoperative rehabilitation, a broad range of musculoskeletal conditions participated in rehabilitation programs and physical therapy is important and necessary [2]. However, for all rehabilitation sessions, it is not practicable and economically justifiable to provide patients with access to a clinician [3]. Accordingly, existing health services worldwide are centralized in a hospital with physicians under direct supervision, followed by an initial part of rehabilitation The associate editor coordinating the review of this manuscript and approving it for publication was Wei Wei . programs where patients do a series of controlled activities in their own homes in an outpatient setting [4]. Literature reports show that more than 92% of all treatment sitting in a home-based setup [5]. Patients are asked to report their daily process under such conditions and regularly visit the clinic for progress evaluation [6]. Several healthcare sources suggest lower patient adherence and motivation to the appropriate rehabilitation exercise regimes that lead to longer treatment times and higher healthcare costs [7]. Even though many various factors that contribute to fewer compliance rates have been identified, the main influence is a lack of ongoing feedback and promptly monitoring of patient exercises by the medical professionals in a home environment [8]- [10]. Figure 1 shows the human activities decomposition.  As discussed in Figure.1. In rehabilitation, a home exercise system is a general where a patient executes a series of physical activities in his or her house [11]. Nevertheless, these treatments do not always help patients to recover fully. Further, one of the major challenges is that patients fail to meet with the workout programs specified [12]. However, this system does not have medical care and monitoring. In turn, Action recognition has received increased attention in the fields of machine learning and computer vision. Recognition of activities can be divided into two categories-visual and sensor-oriented approaches. Here, Human behavior can be seen as a whole of Spatio-temporal shifts in expressions or gestures for vision-based strategies [13].
For the identification of motion in video and still pictures, approaches devoted to efficient visual representation which include movement analysis, models and space-time volume. While many visual techniques have been used in recent decades, large differences in human occlusion, anatomy, and change of perspectives often build this difficult. Sensor technology has made significant progress, especially with low power, wireless communications, high capacity, and data processing, creating it possible for sensors to developed from lower-level information gathering to transition on high-level deductions [14]. For information collection and analysis, wearable sensors can be implanted in strips, clothes, smartwatches, and mobile devices [15]. Several measures are included in the rehabilitation movement. Contrary to the recognition of actions that recognize the action as various classes, the action assessment objective to evaluate the action. This is especially significant for recovery, as it demonstrates whether or not the patient can perform the procedure. The score means that a particular injury recovers [16].
In this paper, the smart sensor-based rehabilitation exercise recognition (SSRER) and evaluation using deep learning framework has been designed and developed mathematically. The problem of data synchronization is one of the issues with sensor data for activity recognition. Where It can be completely different from the start and finish time with the speed required for the activities. The data includes noise and differences when different persons perform the activity [17]. D-CNN is proposed to overcome this problem. Furthermore, a State Probability Transition is proposed to show the transition likelihoods among states to capture the hidden states of sensory data. For test rehabilitation activities, the special matrix has been suggested and the learned classifier is utilized for determining the best features of every class at various levels. The article examines the efficiency of deep networks of autoencoders for reducing dimensional data captured. Besides, the scoring functions are given for the scale in the [0; 1] range of the output values tested. As the basis for training the proposed deep neural networks (DNNs) in the application of rehabilitation, the resulting movement quality scores are used [18], [19].
The major goal of the paper has been listed as follows, • The design of the Smart Sensor-based Rehabilitation Exercise Recognition System (SSRER) approach using a deep learning framework has been mathematically formulated.
• A comparison of deep neural network architectures in motion evaluation and the use of the auto-encoder neural network to minimize rehabilitation data dimensionality has been computed.
• The evaluation result is obtained by comparing the distance and the best feature of the current system has been analyzed with the dataset https://github.com/niemasd/ UI-PRMD-Analysis [20]. The remainder of the paper articulated as follows: Section 1 and section 2 discussed the introduction and existing methods of rehabilitation exercise recognition system. In section 3 the smart sensor-based rehabilitation exercise recognition (SSRER) mathematical model has been discussed. In section 4 the experimental results have been demonstrated. Finally, section 5 finalizes the research paper.

II. RELATED SURVEY ON VARIOUS COMPUTATIONAL AND MATHEMATICAL MODELS
Lun and Zhao [21], Capecci et al. [22] proposed the Hidden semi-Markov Model (HSMM) for the assessment of rehabilitation exercise. The method extracts clinically related motion features from the Red, Green and Blue depth (RGB-D) camera's skeleton joined trajectory and offers a result for the output of the subject. This technique combines various aspects of the law and design approaches. Here the Clinicians are identified as practice descriptors for features and tested them by an HSMM, trained in an exemplary sequence of motion. The efficacy of the proposed solution will be tested by analyzing the relationship between it and both a clinical evaluation and Dynamic Time Warping (DTW) algorithms.
Liao et al. [23] introduced the deep learning-based framework (DBLF) for the rehabilitation exercises assessment. The major parts of the system are the metrics for the quantity of motion output, the scoring of performance assessment functions for numerical motion quality ratings, and DNN models for quality regressions of input motion through supervised learning. A performance metric depends on the logic of an encoding model where the Gaussian mixture identifies with a deep neural network of the autoencoder which is suggested in the paper. Various architectures of the deep network are repurposed for the role and validated through a reinstruction data set.
Zhu et al. [24] initialized the Multi-path CNN for the recognition of rehabilitation exercise. Results of classification accuracy demonstrated by the experiments that an MP-CNN is highly efficient for sensor data acquisition. Strong ensemble classification is either required for training or powerful handmade feature depiction to meet a high level of recognition precision. Deep learning recognition of action reveals delegates characteristics and trains the classifier into a complete model. When compared to those identified section the other deep learning patterns, classification results are superior and the assessment results are efficient for practical applications.
Wang et al. [25] suggested Sensor-based activity recognition (SBAR) for utilizing deep learning. This research discusses the recent progress in sensor-based recognition in deep learning model where the summarize the current literature: deep model, sensory modality, and application. The detailed insights have been provided into current work and suggest significant future research challenges. Deep learning minimizes dependency on human-based features and achieves improved efficiency compared with traditional pattern recognition methods through automatic learning of high-level sensor data representations. Recent advances are underlined in three key groups: deep model, sensor mode, and application. Here, the review and analysis has been reported in detail. Ultimately, for future research, there are several big challenges and practical solutions.
Qi et al. [26] proposed the Physical Activity Recognition and Monitoring (PARM) monitoring for health care using IoT. This paper offers a systematic review of PARM studies objectively from a traditional IoT-funded viewpoint. Firstly, it will sum up the latest state-of-the-art PARM methodologies, including visual, feature-extraction and recognition strategies in the field of wellbeing. The paper further identifies some new trends in research and challenges in the IoT environments for PARM research and discusses some key techniques for dealing with these. Finally, this article examines some of the successful cases in the field and examines PARM's potential future industrial uses in smart health.
To overcome these issues, in this paper, a Smart Sensorbased rehabilitation exercise recognition system (SSRER) using a deep learning framework has been proposed. The recent development of deep learning allows high-level automated feature extraction to achieve promising performance in numerous areas. Deep learning approaches for sensor-based activity recognition operation have been widely adopted. Further, Deep learning can greatly reduce the strain on features that can acquire much higher and meaningful features by training a neural end-to-end network. Furthermore, the deep network structure is easier to perform uncontrolled and incremental learning.

III. SMART SENSOR-BASED REHABILITATION EXERCISE RECOGNITION (SSRER) SYSTEM
The process of rehabilitation includes more time-series activities. There may only be one action in other movements such as pushing the shoulders upwards. In this paper, the Smart sensor-based Rehabilitation Exercise Recognition system (SSRER) has been proposed with a deep learning framework for the collection and classification of the action. Here, SSRER consists of two sub-nets, S-CNN and D-CNN.

A. CONVOLUTIONAL NEURAL NETWORK: BASED ON THE TRANSITION PROBABILITY
Features may conventionally be depicted by the probabilistic changes between states with PFSA-probabilistic finite-state automata. The estimation of transitional probabilities is however highly complex. Therefore, the CNN model has been proposed the links between input signals and probabilities for the changes between states to provide a more discriminatory depiction of features. The PFSA-coded Lossless information compression utilizes the Lossless information compression coding to symbolize the PFSA and sensor information to calculate the probability of transition between hidden states. It contains 3 stages: LIC, Quantization and PFSA construction. Figure 2 shows the Lossless information compression coding flowchart with the PFSA method. The raw data are utilized as input for the training of S-CNN with two coding and pooling layers. The outcomes obtained from the last pooling layer are then utilized as input to the completely associated layers. To find discriminatory classification features, the S-CNN paradigm can be regarded as the regression method for a map of sensor signals for the State probability transient the Lempel-ZivWelch-coded Probabilistic Finite State Automata learns.

B. QUANTIZATION
Every training signals are first linked and sorted in ascending order into a single vector. Then the vector is divided into L parts representing L Levels Each part's boundaries reflect the standard boundaries. Next, the raw data is symbolized by every level limit in the level index so that raw data are processed in a less complex way.  encodes the sequence with the table. The initial step is to initialize the input stream form the coding table.
Corollary 1: Condition check: Corollary.1 displays the symbolic data encoded with modified Lossless information compression. A states in each class, the most often used for composing a state dictionary E, are chosen to reduce the number of states. Therefore, by distance has been calculated based on the state D and the dictionary has been measured the condition which is not in dictionary E. Apply the quantization as the following equation (1): As shown in the equation (1) where |w l | is the string of w l state, w l = dictionarystate and |w v |−stringvariation, lev w v ,w l (.)Levenshtein distance analyzer, w v is the state not in the dictionary E, and W map is the mapping function to measure the w v into w l .

D. PFSA CONSTRUCTION
The PFSA collects the probability of state change for the data symbolized by the LIC code. The Probabilistic finite-state automata is a state representation that records the possibilities for the transition between every state. The probabilities for state transition can be estimated as the following equation (2): As shown in equation (2) where M (p i , p j ) is the number of transitions from p i to p j , m is the number of states, and P is As shown in algorithm 1 the Lossless information compression coding with PFSA construction has been demonstrated. Our system provided the input signal from three-axis accelerometers and the transition probability which has been obtained by utilizing State probability transition Convolutional Neural Network. Features are extracted from raw input to train the second element, the Dynamic Convolutional Neural Network. A fixed sliding window is used for pre-processing the raw information. The signal is then standardized to[ 0, 1]. Here, A median filter (size: 4) is utilized for signal noise removal. The signal linked with the acceleration of gravitation is the gravitational feature. Without gravity acceleration, the signals respect to the body features.

E. DYNAMIC CONVOLUTIONAL NEURAL NETWORK
The problem of data alignment is one of the difficulties of utilizing sensor data to recognize the activity. The start and end times of the exercises can be varied in practical applications. The data includes noises and variations when different people perform the exercise. A Gaussian Mixture Regression-Gaussian Mixture Model (GMR-GMM) is, therefore, being proposed for a complex CNN. The GMM is used as a reference norm for each operation. During the research, two operations are used to match the data and the GMM, namely information and channel fitting to the proposed dynamic assignment process. The corresponding data will then be utilized as the input for the channel in the Convolutional Neural Network method. Figure 3 shows the D-CNN for rehabilitation exercise recognition.

F. GAUSSIAN MIXTURE REGRESSION -GAUSSIAN MIXTURE MODEL (GMR-GMM)
Gaussian Mixture Regression-Gaussian Mixture Model is utilized for modeling the various activities ' distributions. This overcomes the variations of the internal class when sensor data are obtained for one activity. Let's assume h t = t, h y (t) , h x (t) , h z (t) , h t ∈ T 4 , t = 1, . . . .R, where time size is R and a t = t, a y (t) , a x (t) , a z (t) , a t ∈ T 4 , t = 1, . . . R. L-components with GMM are utilized to model a t and h t . The covariance matrix and mean vector defined as l ∈ T 4×4 and µ l ∈ T 4 and for h t , where l = 1, . . . L. GMR is utilized to identify the covariance and mean matrix at time t for l-th GMM element. To divide the acceleration and temporal values in µ l and l as the following equation (4): The bb l andμ b l are combines by the likelihood α l of the lth element at time t to estimate the expected covariance matrix bb and acceleration µ b at time index t, as the following equation (6): As shown in the above equation where M is the Gaussian distribution function. Thus, the covariance and mean acceleration matrix at time t in the series R has been calculated to create the GMR-GMM model. The GMR-GMM model will be trained to fit the input signal in a sectoral way to the GMR-GMM model with the dynamic assignment approach. The dynamic assignment includes two levels: information and fitting the channel. In the database. For each activity class, the GMR-GMM model is trained. The model is then split into m parts, corresponding to the D-CNN M channels. Functions are divided into M parts as well. Features equal to the same model component go to the same channel on the Dynamic Convolutional Neural Network by channel fitting. In channel movement, distance can be calculated by the distance from the divided characteristics and the model component. The signal to Mahalanobis distance at time t to the paradigm component can be stated as equation (8) as the denotation y t as the acceleration of triaxle signal at time t: The Euclidean distance between two motion data X w,r and Y w ,r has been generally utilized for motion evaluation and it is stated as (x t,e w,r − y t,e w ,r ) 2 (9) As shown in the above equation where bb t and µ b t are covariance and mean matrix of the paradigm at time t and for w, w ∈ W and r, r ∈ T w . The Dynamic time warping (DTW) algorithm for aligning time series data with non-linear warping to minimize the spacing of the time series. The Euclidean distance is the most commonly used remote function of DTW. DTW calculates the optimal alignment path by reducing the cumulative distances of the two-time series from the minimum distances of the neighboring points.
e DTW X w,r , Y w r = e D X w,r , Y w r + h X w,r , Y w r (10) Thus, the distance between the model and separated features the section can be calculated as the following equation (11): As shown in the equation (11) where m is the partitioned feature size.

G. PREDICTION LOSS
The data has been labeled as several categories, including four classes of actions. Every class of acts had three levels of assessment: good, average and poor. The bilinear interpolation has been utilized, raw action signals has been resized to the same scale. Then a three-layer long short term memory has developed to predict the type, followed by the Softmax layer model and three fully connected layers. K q is known as the loss function expressed as (12):

H. CONDITION LOSS
The weight of the final Long short term memory layer is used as a classifier in this segment once the LSTM is trained. The feature depiction of every class can be reversed by the use of a classifier and a particularly intended loss of condition. Such a feature is called the general or good feature for every action class level. The feature denoted as f ∈ T M , and N is the number of action classes. Thus, total classes Q = N × K . The classifier dimension is M ×Q since the output is Q classes and the dimension of feature is M. K is the number of stages. Score matric can be defined as W ∈ T Q×Q .
This implies that the input of w t,t (equal to one) of the other entries (equal to null) should be the largest (equals to one) of the same row. The condition loss K a can be defined as the following equation (14): The goal is to reduce the distance between H * A and W, where W is the ground truth score. The class signal n and stage l must have the highest value in the input w t,t by minimizing the loss function. When the classifier training is completed, A and W are set to find H for one training iteration. This general feature to get the largest score in W. Figure 4 shows the assessment system of rehabilitation exercise using sensor input. The CNNs ' inputs are pairs of repeat data and quality values. The trained networks in a controlled way, with the output as a predicted movement quality value for a repetition of data. For network training, motion quality scores are used depends on the Gaussian Mixture Model log-likelihood features estimated with minimized auto-encoder information. Only the inter-subject case is reviewed as the number of iterative per subject is too low for CNN training in the internal subject cases. The output of the final layer in the long short term memory and the general feature is utilized to calculate the evaluation score. The cosine Angle is utilized to calculated two vector similarity and the angle to standardize is the range of 0 to 1. The expression can be formulated as (15):

I. EVALUATION LOSS
As shown in the equation (15) where feature LSTM is the last layer in the long short term memory feature extraction and feature Gen is the general feature. Then a scaling interval is multiplied to obtain the final assessment: The evaluation is scaled to 0 to 100 if the interval is set to 100.
To update the paradigm with cross-entropy, the score turns into a label. For each level, they typically set the same levels.  If the assessment is within the range, the label is marked. The formula is defined in (17): Now the assessment label has been obtained: The loss function K d can be determined using the assessment label and ground truth x true : The overall loss function for updating our model is provided by (19): As shown in equation (19) where δβ, and α and are three loss terms of balancing parameters. K q , K a , K d are evaluation loss, prediction loss, and condition loss correspondingly. The results of the evaluation model are the performance score after the model has been trained. To see their performance at three stages: good, mean or poor the users can match the range of values defined by Equation (17).
The scaled values of performance metrics comparison has been done with the proposed notion of separation degree. For any positive number y,x separation degree is stated as Separation degree values similar to 1 or -1 indicate that both sequences are well differentiated. The sequences do not separate well and nearly mix, on the other hand, for values of the degree of separation near 0. The separation grade indicates that the metric is better able to distinguish between correct and wrong exercise repetitions when applying to the values of the distance metrics.

A. PREDICTION RATIO MATHEMATICAL PREDICTION
The results are expressed as a distance from the configuration of the reference sensor consisting of five sensors used.
For exercise type, a positive distance is greater than the accuracy of the reference configuration. The numbers represent prediction errors in the case of strength prediction and should, therefore, be less and negative for better results. Figure 5 shows the different sensor combinations. Concerning training times, the average classification time has reduced substantially to millisecond predictions. In conjunction with active learning, a stacked denoising auto-encoder offers excellent for automated labeling and feature extraction for intense exercise recognition. To track daily living activities, the stacked denoising automatic encoder implementation is important for the morbidity prediction of the sensor. The feature extracted using the backpropagation neural network has predicted for energy expenditure. The prediction data set has however obtained from sensors mounted on the string, which do not indicate the direction of movement. Therefore, data from sensors mounted on the hand, chest or ankle should be tested to detect accurately and to track total body movements of human activity models by translating the sensor value into a binary number and extracting discriminatory features using a convolutional neural network. The sensor value must be calculated. Figure 6 shows the prediction ratio of the proposed SSRER method.

B. ACCURACY RATIO ANALYSIS
The test results showed that the SSRER approach offers a high classification of activity types and incorrect identification of the movement. The incorrect identification accuracy considers the misclassified types of exercise to be below the accuracy of the identification of the exercise type classification. The accuracy of the classification of exercise compassionate artificial errors that appeared during the user could not achieve the motion criteria would influence this result. Figure 7 shows the accuracy ratio of the proposed SSRER method. The proposed SSRER method achieves a high accuracy ratio for rehabilitation exercise classification.

C. PERFORMANCE RATIO FOR NUMERICAL CONSISTENCY
A CNN automated encoder is utilized to decrease the dimensionality of the skeleton information collected during recovery repetition exercises. Therefore, the low-dimensional representation of the data is probabilistically modeled by a GMM and the movement repetition logic is used as VOLUME 8, 2020   a performance measurement metric. A scoring function maps the values in movement quality values for the performance metric. A deep NN model is developed for each rehabilitation exercise to learn the relationship between motion data and quality outcomes and to produce qualitative results for unknown rehabilitation exercises. The performance ratio of the proposed SSRER system is high  when compared to other existing methods. Figure 8 demonstrates the performance ratio of the proposed SSRER method.    Table 1 demonstrates the performance evaluation of the SSRER system. The test results show that movements quality values derived from the proposed deep learning framework keeps the score of ground truth quality for motion closely and confirm the possibility of deep learning methods for rehabilitation exercises.

D. EFFICIENCY RATIO ANALYSIS
The program proposed proved efficient by enhancing user engagement and performance outcomes. The findings show that biomechanical criteria are useful for the understanding of movements in the rehabilitation exercises to direct patients. An automated method will make the administration of daily rehabilitation programs efficient and cost-effective. The proposed SSRER method has a high-efficiency ratio when compared to other existing HSMM, DBLF, MP-CNN, SBAR, and PARM methods (Figure 9).  Table 2 shows the efficiency analysis of the proposed SSRER method. A proposed algorithm has been used for the study and analysis of movement data by machine learning. The model has a prototype that offers comfortable and energy-efficient equipment used in standard processes. SSRER offers effective features for efficient training.

E. RECOGNITION RATE DETERMINATION
A movement recognition focused upon biomechanical concepts has a consequence of the possibility of direct a right and accurate movement. The system improved the engagement and quality of the exercise of users. A blind experiment has been conducted to verify the paradigm by iteratively the recognition process to retrieve new feature sets from the measuring information on the proposed exercises. The filtering criteria for calculating the enveloped spectrum has given by both diagrams. The suggestive features has then acquired via VOLUME 8, 2020 the enveloped spectrum, to improve the model and, consequently, to enhance recognition. Figure 10(a and b) demonstrates the recognition rate of the proposed SSRER method.
Hence based on the experimental results the S-CNN and D-CNN with a deep learning classifier are proposed to assess the general depicted of every rehabilitation class exercise at different levels. Then, test results show outperforms best-learned features for any rehabilitation exercise.

V. CONCLUSION
In this paper, the Smart Sensor-based Rehabilitation Exercise Recognition (SSRER) system using a deep learning framework. This paper proposes a CNN and an action assessment approach. The proposed system consists of S-CNN and D-CNN models. Different signal segments can be allocated by D-CNN to the same CNN. Therefore, the issue of data noises data convergence and other distinction can be better addressed. Results of classification accuracy demonstrated by these experiments have demonstrated that SSRER is very efficient for sensor data exercise recognition. Compared to new machine learning-based approaches, strong ensemble classifications or powerful characteristics are required to achieve high accuracy of recognition. Besides, a GMM is probabilistically used for the low-dimensional data representation and the dynamics of movements replicated are used as a metric for performance evaluation. A scoring function maps the results into motion quality values. A deep NN model is trained for each rehabilitation exercise to learn the relation between movement data and quality results and to generate quality scores for undiscovered rehabilitation exercises.
CAIXIA SU graduated in traditional Chinese medicine from Shanghai University, in 2008. She is currently working with the Department of Rehabilitation, The Affiliated Jiangsu Shengze Hospital, Nanjing Medical University. Her research interest includes pain treatment.
CHUAN HE graduated in psychiatry and mental health from Suzhou University, in 2005. He is currently with working with the Department of Rehabilitation, The Affiliated Jiangsu Shengze Hospital, Nanjing Medical University. His research interest includes stroke rehabilitation. VOLUME 8, 2020