RadarSNN: A Resource Efficient Gesture Sensing System Based on mm-Wave Radar

Radar offers a promising modality for enabling gesture recognition, which is a simple and intuitive alternative to click and touch-based human–computer interface. In this article, we propose a spiking neural network (SNN)-based hand gesture recognition with frequency-modulated continuous-wave 60-GHz radar. As preprocessing, the 2-D fast Fourier transform (FFT) is performed across fast time and slow time to generate a video of range–Doppler maps, which are then processed to generate range spectrograms, Doppler spectrograms, and angle spectrograms. The spike trains are fed into an SNN to classify the gesture that has been performed. We demonstrate that with few neurons, SNNs can achieve recognition accuracies close to 99.50% comparable to their deep learning counterparts for eight dynamic gestures. Moreover, the proposed model size is 75 kB, which is substantially smaller compared to the state-of-the-art models making it memory efficient. We also demonstrate using tSNE plots that SNNs can operate with lower embedding dimensions, implying that we can realize SNN with a small compute and memory footprint.