An Efficient Diverse-Branch Convolution Scheme for Sensor-Based Human Activity Recognition

Deep convolutional networks have recently achieved significant success in sensor-based human activity recognition (HAR). Existing works are mainly devoted to extracting multiscale activity features from sensor time series by increasing network depth or width, which are not friendly to mobile devices with limited computing resources. It still remains a challenging issue to strike an ideal tradeoff between activity recognition performance and inference-time costs, e.g., the latency and memory footprint. To address this issue, we propose a diverse-branch convolution (DBC) scheme, which could strengthen the representation capacity of vanilla convolution via exploiting diverse branches of different scales and complexities to enrich activity feature space. Then, DBC could be equivalently transformed into a single convolution layer after training for HAR deployment. In such a manner, DBC only complicates training-time microstructure while preserving inference-time macrostructure, where the model performance can be boosted to a higher level and then converted back to its original inference-time structure for activity recognition. Extensive experiments are conducted on three publicly available benchmark datasets, i.e., OPPORTUNITY, UniMiB-SHAR, Wireless Sensor Data Mining (WISDM), and a self-collected weakly labeled HAR dataset, which verify that our method can give a lift in performance while maintaining the same number of parameters and FLOPs as baselines. Finally, the actual inference-time cost is evaluated for HAR deployment on a Raspberry Pi platform.