Latern: Dynamic Continuous Hand Gesture Recognition Using FMCW Radar Sensor

Recently, hand gesture recognition systems have become increasingly interesting to researchers in the field of human–computer interfaces. Real-world systems for human dynamic hand gesture recognition is challenging as: 1) the system must be robust to various conditions; 2) there is a rich diversity in how people perform hand gestures, making hand gesture recognition difficult; and 3) the system must detect and recognize hand gestures continuously using unsegmented input streams in order to avoid noticeable lag between performing a gesture and its classification. In this paper, to address these challenges, we present Latern, a novel system for dynamic continuous hand gesture recognition based on a frequency-modulated continuous wave radar sensor. The radar system does not depend on lighting, noise, or atmospheric conditions. We employ a recurrent 3-D convolutional neural network to perform the classification of dynamic hand gestures. To enhance the processing performance, a connectionist temporal classification algorithm is used to train the network to predict class labels from inprogress gestures in unsegmented input streams. The experimental results show that Latern is able to achieve high recognition rates of 96%, which is higher than state-of-the-art hand gesture recognition systems. In addition, the conclusion in this paper can be used for a real-time hand gesture recognition system design.