Gesture Recognition System Using 24 GHz FMCW Radar Sensor Realized on Real-Time Edge Computing Platform

Affected by the global epidemic, non-contact unmanned control system provides people with safe human-computer interaction (HCI). This paper presents a radar hand gesture recognition system based on real-time edge computing platform. The system uses a low-cost 24 GHz commercial low bandwidth frequency modulated continuous wave (FMCW) radar as the detection source. For real-time gesture recognition, the background echo of radar is fitted by the exponential weighted average method which can reduce the hardware resource consumption when implementing on field programmable gate array (FPGA). A novel extracted feature named range-Doppler matrix focus (RDMF) is designed to reduce the data dimension and preserve the useful information. 3-D CNN + LSTM encoder classification framework is used to classify the features. A data set of 3200 samples for the basic 8 gestures and 1600 samples for the additional 4 gestures is designed and collected for training and testing the classification framework. The results show that the average accuracy of 3-D CNN + LSTM encoder on training and testing data set of 12 gesture classification exceeds 97.9%. The model size of this network is nearly 390 kB. In order to compress the size for recognition on edge-computing platform, a simplified framework for sending one antenna’s RDMF to the LSTM encoder is proposed. The model size is reduced to 20 kB, and the average accuracy of the simplified framework on basic 8 gestures is 95.9%. We deploy this gesture recognition framework on the FPGA platform to speed up computing.