HiHAR: A Hierarchical Hybrid Deep Learning Architecture for Wearable Sensor-Based Human Activity Recognition

Wearable sensor-based human activity recognition (HAR) is the study that deals with sensor data to understand human movement and behavior. In a HAR model, feature extraction is widely considered to be the most essential and challenging part as the sensor signals contain important information in both spatial and temporal contexts. In addition, because people often carry out an activity for a while before changing to another activity, the sensor data also contain long-term context dependencies. In this paper, in order to enhance the long, short-term and spatial features from the sensor data, we propose a hierarchical deep learning-based HAR model (HiHAR) which is constructed from two powerful deep neural network architectures: convolutional neural network (CNN) and bidirectional long short-term memory network (BiLSTM). With the hierarchical structure, HiHAR contains two stages: local and global. In the local stage, a CNN and a BiLSTM are applied on the window-data level to extract local spatiotemporal features. The global stage with another BiSLTM is used to extract long-term context information from adjacent windows in both forward and backward time directions, then performs activity classification task. Our experiment results on two public datasets (UCI HAPT and MobiAct scenario) indicate that the proposed hybrid model achieves competitive performance compared to other state-of-the-art HAR models with an average accuracy of 97.98% and 96.16%, respectively.