Temporal-Based Multi-Sensor Fusion for 3D Perception in Automated Driving System

The 3D object detection task is a crucial subtask of the environment perception module in Automated Driving System (ADS). The accuracy of object detection directly impacts the effectiveness of downstream autonomous driving tasks such as tracking, prediction, and planning. Existing 3D object detection networks in ADS that rely on multi-sensor fusion lack the utilization of temporal information and fail to fully consider the dynamic nature of the surrounding traffic environment. Therefore, we proposed BEVTemporal for ADS, which fuses LIDAR point clouds and surrounding multi-channel images. Its unique temporal module establishes the correlation between historical data and current data, effectively leveraging the temporal information of surrounding objects. We train and validate BEVTemporal on the nuScenes datasets. After incorporating temporal module, the Average Precision (AP) metrics of the network improved by 0.3%~1.7%, and mean Average precision (mAP) achieves 0.87% higher, nuScenes Detection Score (NDS) increased by 0.46%. The validation results on subsets of occluded objects show that the model effectively alleviates the problem of missed detection caused by sample occlusion, with significant improvements observed for heavily occluded samples. In different scenarios (sunny, rainy, daytime, night), mAP improvement ranges from 0.75% to 1.18%. Notably, in challenging scenarios such as rainy and night, AP can be improved by up to 3.6%. The experimental results show that BEVTemporal not only improves the accuracy of the 3D object detection network, but also significantly enhances the robustness of model in various scenarios and recall of objects under low visibility conditions.