MM-HAT: Transformer for Millimeter-Wave Sensing Based Human Activity Recognition

Human activity recognition (HAR) shows significant applicable values in health care. However, it has not been widely used due to some constraints, like privacy concerns of vision-based methods and the inconvenience of wearable devices-based methods. Recently, millimeter-wave (mmWave) based HAR has caught attention increasingly as it is able to provide a non-imaging, contactless and continuous approach for HAR. However, the existing mmWave-based HAR methods have limited ability and scalability to discriminate similar activities. To address existing problems, we present MM-HAT, an end-to-end Transformer network for mmWave point cloud based HAR. We design point-cloud-specific adaptations to Transformers that have gained success in natural language processing and vision. In addition, MM-HAT takes both mmWave point cloud data and target-data extracted from point clouds as input to mitigate the adverse effects of mmWave signals' vulnerability. We have collected a 7-activity mmWave dataset and carried out experiments on it. Evaluation results show that MM-HAT outperforms the existing methods by up to 34.76% (RadHAR) and 20.05% (MMPoint-GNN).