Multi-person device-free gesture recognition using mmWave signals

Device-free gesture recognition is an emerging wireless sensing technique which could recognize gestures by analyzing its influence on surrounding wireless signals, it may empower wireless networks with the augmented sensing ability. Researchers have made great achievements for single-person device-free gesture recognition. However, when multiple persons conduct gestures simultaneously, the received signals will be mixed together, and thus traditional methods would not work well any-more. Moreover, the anonymity of persons and the change in the surrounding environment would cause feature shift and mismatch, and thus the recognition accuracy would degrade remarkably. To address these problems, we explore and exploit the diversity of spatial information and propose a multidimensional analysis method to separate the gesture feature of each person using a focusing sensing strategy. Meanwhile, we also present a deep-learning based robust device free gesture recognition framework, which leverages an adversarial approach to extract robust gesture feature that is insensitive to the change of persons and environment. Furthermore, we also develop a 77GHz mmWave prototype system and evaluate the proposed methods extensively. Experimental results reveal that the proposed system can achieve average accuracies of 93% and 84% when 10 gestures are conducted in different environments by two and four persons simultaneously, respectively.