Time-Frequency Fusion-Augmentation based Contrastive Learning for Self-Supervised Human Activity Recognition

Human activity recognition (HAR) using sensor data from wearable devices has significant applications in health assessment and accident monitoring. However, building a generalized recognition model requires large amounts of annotated time series data, which can be both costly and time-consuming. Contrastive learning has emerged as a promising approach for learning activity representations from raw sensor data without the need for human annotations. Therefore, we combine contrastive learning and HAR and propose a Time-Frequency Fusion-Augmentation based Contrastive Learning (FusionCL) method to realize self-supervised human activity recognition. FusionCL combines the time-domain augmentation method (named TNet) with the frequency-domain augmentation method (named FNet) to generate diverse augmentation samples. TNet aims to capture different temporal aspects of original sensor data with time-domain variations, such as jittering, scaling, and permutation. FNet aims to augment the original time-domain data with frequency-domain information, which is particularly useful for HAR tasks where different activities can have distinct frequency components. By augmenting the original data from both the time and frequency domains, the proposed FusionCL method can construct a more informative pre-train model, consequently enhancing the generalizability and robustness of the final HAR model. Extensive experiments on four benchmark HAR datasets show that the FusionCL method achieves state-of-the-art performance, demonstrating the effectiveness of our proposed method and the potential for constructing the unified HAR model with contrastive learning strategies.