Sparsity-Driven Gesture Recognition Using Lightweight TCNFormer Networks in Millimeter-Wave Radar

Gesture recognition with millimeter-wave radar has broad application prospects in human-computer interaction. However, the traditional recognition methods generate overly redundant features and construct large-scale networks, rendering them unsuitable for embedded devices with limited memory. To address this challenge, we propose a sparse-driven dynamic gesture recognition network in millimeter-wave radar, named time convolutional network and transFormer (TCNFormer). First, we employ a 2-D fast Fourier transform (2D-FFT) to obtain the range-Doppler maps (RDMs). These maps are then processed through incoherent integration of multiple frames to produce Doppler-time maps (DTMs). We subsequently use the orthogonal matching pursuit (OMP) algorithm to achieve a sparse representation of the Doppler-time trajectories and integrate the RDM to extract the range features of gestures, obtaining the multidimensional sparse sequences encompassing the range-Doppler-time feature. We then design a TCNFormer network tailored to these multidimensional sparse sequences. This network leverages a shallow TCN to learn local features, a Transformer network to capture global features and an adaptive weighting method to fuse these local and global features effectively. Experimental results demonstrate that our network fully exploits the sparse multidimensional sequences, achieving a recognition accuracy of 99.17% on a self-built dataset. The parameter size of the network is only 0.13 M, significantly outperforming the existing state-of-the-art models in relevant metrics, thereby proving its suitability for embedded applications in human-computer interaction.