Two-Branch Neural Network Using Two Data Types for Human Activity Recognition

Human activity recognition (HAR) consists of identifying and then analyzing a person’s behavior, using a motion capture device. It has been of interest to many authors since the 1980s. This study has proved useful in various fields such as video game animation, sports training or, for this work, health management. The project aims, more precisely, to recognize and evaluate the postures and movements to correct, improve or assist in care. We work particularly on tremor, dyskinesia or any other movement induced by Parkinson’s disease (PD). This type of HAR application is now possible thanks to the embedded sensors. These can be found in all our connected devices such as our phones, watches, sports, and health sensors. These tools provide 3-D time signals that can be interpreted by different algorithms. Deep learning has, moreover, proven its performance in HAR with data extracted from these sensors. This article presents a new technique for motion recognition, using a neural network model, named the CNN-BiLSTM-FCN (CBF) model. It is composed of two branches with different input data. As the name suggests, it is structured with three networks, namely a convolutional neural network (CNN), a recurrent neural network (RNN), and a fully connected network (FCN). This technique was tested using a benchmark considering the UCI-HAR dataset. It has already been used to compare this type of HAR method. The experimental results highlight the effectiveness of our approach, which differs significantly from those presented in the literature. Finally, this technique is applied to movement data, including Parkinson-type movements, collected in our laboratory.