RETA: 4D Radar-Based End-to-End Joint Tracking and Activity Estimation for Low-Observable Pedestrian Safety in Cluttered Traffic Scenarios

Due to the small radar cross Section (RCS), pedestrians are typical low-observable traffic participants for radar-based automotive perception systems. The early detection and understanding of pedestrians’ activities are of great significance to automotive safety. To this end, this paper presents an end-to-end joint tracking and activity estimation (RETA) system based on 4D automotive radar, which deals in particular with pedestrian activity identification under cluttered real-world scenes. Firstly, a novel integrated detection and tracking algorithm is proposed to guarantee positioning accuracy, in which all unthresholded 4D radar measurements are incorporated to explore the spatial coherent information across multiple frames, avoiding weak target information loss. After that, to discriminate continuous activities with varying durations in sequential trajectories, this paper innovatively presents a decomposed connectionist recurrent convolutional neural network, which facilitates fused temporal-spatial motion feature extraction. Especially, the labor-consuming activity pre-segmentation problem is circumvented with the help of a connectionist temporal classification algorithm in the proposed neural network. At last, RETA can be implemented for real end-to-end perception applications. Extensive experiment results highlight its superiority and effectiveness by attaining a continuous recognition accuracy of 94.8%. To the best of our knowledge, this is the first end-to- end activity recognition system specific for low-observable pedestrians. A demonstration video recorded in challenging practical traffic scenarios has been uploaded in the supplementary materials.