Short-Range Radar-Based Gesture Recognition System Using 3D CNN With Triplet Loss

Gesture recognition is the most intuitive form of human computer-interface. Gesture sensing can replace interfaces such as touch and clicks needed for interacting with a device. Gesture recognition in a practical scenario is an open-set classification, i.e. the recognition system should classify correct known gestures while rejecting arbitrary unknown gestures during inference. To address the issue of gesture recognition in an open set, we present, in this paper, a novel distance-metric based meta-learning approach to learn embedding features from a video of range-Doppler images generated by hand gestures at the radar receiver. Further, k-Nearest Neighbor (kNN) is used to classify known gestures, distance-thresholding is used to reject unknown gesture motions and clustering is used to add new custom gestures on-the-fly without explicit model re-training. We propose to use 3D Deep Convolutional Neural Network (3D-DCNN) architecture to learn the embedding model using distance-based triplet-loss similarity metric. We demonstrate our approach to correctly classify gestures using short-range 60-GHz compact short-range radar sensor achieving an overall classification accuracy of 94.5% over six fine-grained gestures under challenging practical environments, while rejecting other unknown gestures with 0.935 F1 score, and capable of adding new gestures on-the-fly without an explicit model re-training.