Optimized Multi-Radar Hand Gesture Recognition: Robust MIMO-CNN Framework with FPGA Deployment

Hand gesture recognition (HGR) utilizing radar sensors frequently encounters obstacles such as interference, clutter, and the restricted amount of radar datasets, which hinder the optimization of deep learning (DL) models. This research presents a novel framework for HGR that combines rigorous signal preprocessing, deep learning modeling, and hardware implementation. Signal preprocessing methods, including filtrating, computating of squared absolute values, and normalization, effectively reduce noise and clutter, resulting in improved binarized images appropriate for deep learning models. A tailored convolutional neural network (CNN) is employed to extract features individually from each radar. The combined features are processed via fully connected layers and a classifier with a softmax layer, attaining training and testing accuracies of 99.94% and 99.20%, respectively, for the Multi-input Multi-Output (MIMO)-CNN architecture. The model was quantized and deployed on a System-on-Chip (SoC) FPGA platform, KR260 evaluation kit, achieving a testing accuracy of 93.78%, compared with 95.43% accuracy on a PC environment using exactly the same testing dataset. In addition, the KR260 evaluation kit resulted in a 35% improvement in execution speed. This paper demonstrates the potential of integrating multi-radar data, optimized DL models, and FPGA hardware for real-time HGR with superior accuracy and efficiency.