Real-time Radar Gesture Classification with Spiking Neural Network on SpiNNaker 2 Prototype

Neuromorphic hardware has been emerging in recent years, seeking various applications to explore its uniqueness, limitations, and possibilities. As a representative application and research area, gesture recognition is gaining wider popularity, while the conflict of spiking neural network (SNN) size and available memory of neuromorphic edge-AI can be a thorny issue, which is even intensified by the demand for continuously processing input data stream from the sensor in the real-world scenario since a certain amount of memory is required to ensure that no data loss or overwrite happens. In this paper, an SNN-based real-time radar gesture recognition closed-loop system is proposed, with Infineon's 60 GHz radar continuously capturing motion and the multi-core neuromorphic hardware SpiNNaker 2 serving as the backend to classify gestures with an SNN. A PC is used to preprocess the data and manipulate an actuator. This system requires less than 8 k operation cycles per processor for each radar frame and achieves a classification accuracy of 98.83% with an 8-bit quantized model, with only 134.2 kB memory usage on three processing elements (PEs) and low energy cost. Besides, it performs well in the real-time closed-loop test with 35 ms latency.