Radar-Based Gesture Recognition Using a Variational Autoencoder With Deep Statistical Metric Learning

Radar-based gesture recognition systems are a promising concept for new human–machine interfaces. However, the systems are sensitive to user-dependent gesture patterns, sensor noise characteristics, background environments, and unknown gestures or background motions. To overcome such challenges, we propose a variational autoencoder architecture-based deep metric learning, which is optimized using a novel loss function combining a statistical distance triplet loss and the center loss. By learning the statistical distance between distributions, the proposed solution, compared with triplet loss, is able to better learn the nonlinear characteristics in the data, is less sensitivity to training strategy, and is capable of creating close knit embedding clusters. Moreover, it is experimentally demonstrated that the proposed gesture recognition system using radar spectrograms has improved classification accuracy and random gesture rejection accuracy compared with the state-of-the-art deep metric learning approaches. Furthermore, we proof the real-world capability of the system by implementing a real-time demonstrator, which allows playing “Tetris” with hand gestures.