RoGER: Few-shot Learning Based Robust Gesture Recognition with Multi-modal Wi-Fi CSI Measurements

Wi-Fi based gesture recognition has raised emerging interest recently due to its high identification accuracy and low-hardware cost. However, its application is impeded by the severe performance degradation in cross-domain sensing, where the change of sensing target or sensing distance/orientation would significantly reduce the sensing accuracy. In this paper, we propose a domain-Robust GEsture Recognition (RoGER) system based on few-shot learning (FSL) and multi-modal Wi-Fi channel state information (CSI) measurement fusion. In particular, the proposed RoGER system combines the features derived from the CSI data received from multiple pairs of transceiver devices and uses FSL to achieve stable cross-scenario sensing performance with the minimum number of labeled samples. The proposed RoGER consists of two major stages: the source domain meta-training stage and the target domain meta-testing stage. In the meta-training stage, we design and train separate convolutional neural network (CNN) modules using the CSI data captured from different transceiver links to extract and fuse the implicit multi-modal gesture features. In the meta-testing stage, we use the source-domain CNN module as the embedding model to generate the feature map and train a domain-specific classifier using minimum labeled samples of the target domain. Experiment results show that the proposed RoGER can achieve on average 94.84% − 99.71% cross-domain recognition accuracy in a 6-class gesture classification task using only 1 labeled sample per class, which significantly outperforms the considered benchmark methods.