Multi-STMT: Multi-Level Network for Human Activity Recognition Based on Wearable Sensors

Human activity recognition (HAR) has gained significant interest in recent years, specifically in the context of wearable sensors. This approach utilizes the abundant sensory information provided by multimode embedded sensors like accelerometers and gyroscopes to deduce and identify human activities. How to extract rich representation features from these data to capture subtle differences between different activities is the key challenge of this technology. Although the HAR technology based on deep learning has achieved satisfactory results, most methods still have problems such as insufficient feature extraction and insufficient attention to key features. Therefore, this article designs a multilevel network called Multi-STMT, which is based on the spatiotemporal attention mechanism and multiscale temporal embedding. The network integrates a convolutional neural network (CNN) module and a bidirectional gated recurrent unit (BiGRU) module and introduces an attention mechanism in each module to fully capture the temporal and spatial dependence of multimodal signals. Comparing previous methods, we have achieved state-of-the-art recognition accuracy on four publicly available datasets. Specifically, on the DSADS, SisFall, HCI-HAR, and KU-HAR, the proposed method’s recognition accuracy has reached 99.48%, 91.85%, 96.67%, and 97.99%, respectively, hence indicating the advantage of the proposed method.