Human-Centered Fully Adaptive Radar for Gesture Recognition in Smart Environments

Over the past decade, radio frequency (RF) sensing or radar has garnered great interest as an emerging modality to enable human–computer interaction via gesture recognition. Current approaches involve utilization of a radar system that transmits a fixed signal with predetermined frequency, bandwidth, and other waveform parameters. However, gesture recognition accuracy can be greatly impacted by radar transmission parameters, which affect computational load and performance. In this work, we introduce a human-centered, fully adaptive radar (HC-FAR) system for ambient gesture recognition in which a programmable, software-defined radar system dynamically changes its RF transmission in response to human behavior. We design and switch between different transmission modes for different human-computer interaction tasks—human presence detection, trigger detection, and command translation—as well as alter processing so as to minimize computational load. In this way, the proposed HC-FAR paradigm enables dynamic management of the tradeoffs between dimensionality of RF data representations and their resulting computational load with real-time classification accuracy. Our results show that HC-FAR significantly reduces the allocation of computational and spectral resources, while enhancing fine-grain gesture recognition via a joint domain multi-input deep neural network, which takes as input the RF micro-Doppler signature, range, and angle profile.