Toward Robust Device-Free Gesture Recognition Based on Intrinsic Spectrogram of mmWave Signals

Device-free gesture recognition is a potential noncontact human–computer interaction technique. It leverages the unique influence of the conducted gesture on surrounding wireless signals to accomplish gesture recognition. Existing methods usually leverage doppler spectrogram of the influenced wireless signals to characterize the motion pattern of gestures. These methods have achieved satisfactory accuracy when the gestures are conducted in a relatively fixed location, direction, and speed. However, when gestures are conducted in a different scenario, the recognition accuracy will drop dramatically. In this article, we try to solve this issue by characterizing the gesture motion pattern using a novel robust intrinsic spectrogram, which is independent of the conducted scenario. Specifically, we create a virtual coordinate system in which the coordinates of a gesture trajectory remain unchanged no matter where and how the gesture is conducted. Then, we design a coordinate transformation method to transform the raw doppler spectrogram into the robust intrinsic spectrogram to characterize the intrinsic motion pattern of the gesture. We further feed the intrinsic spectrogram into a deep network to realize gesture recognition. Extensive evaluations on a 77-GHz mmWave testbed show that the proposed method could achieve an average recognize accuracy of 88.4% with ten types of gestures.