Radar-Based Gesture Recognition Under Ego-Motion for Automotive Applications

Sensing explicit communication signals like gestures provides autonomous vehicles (AVs) expressive information about pedestrians’ intentions. In automotive applications, gesture recognition algorithms have to provide high recognition accuracy under ego-motion, and at larger distances. In principle, radar-based methods add both accuracy and reliability to a vehicle’s gesture perception. However, while ego-motion has little impact on algorithms working with camera data, it significantly alters the radar responses of the pedestrians, e.g. due to the non-static appearance of clutter. To meet the resulting challenges, this paper proposes a radar-based gesture recognition algorithm for radar sensors in vehicles. By optimizing the signal processing chain to extract clutter-reduced radar observations of the pedestrians, the impact of the environment and the ego-motion on the recognition accuracy is minimized. Subsequently, a convolutional neural network (CNN) is trained for gesture classification. The algorithm is tested on a comprehensive gesture dataset recorded with an experimental vehicle in two different outdoor scenarios. The CNN achieves an average recognition accuracy under ego-motion of 91% for distances up to 15 m. In addition, it is shown that appropriate data augmentation strategies allow the classifier to be successfully trained on data from static measurements, with an accuracy of 85.9% at significantly less measurement cost and effort.