DEN: Depth Enhancement Network for 3-D Object Detection With the Fusion of mmWave Radar and Vision in Autonomous Driving

In the realm of autonomous driving, precise and robust 3-D perception is paramount. Multimodal fusion for 3-D object detection is crucial for improving accuracy, generalization, and robustness in autonomous driving. In this article, we introduce the depth enhancement network (DEN), an innovative camera-radar fusion framework that generates an accurate depth estimation for 3-D object detection. To overcome the limitations caused by the lack of spatial information in an image, DEN estimates image depth using accurate radar points. Furthermore, to extract more comprehensive and fine-grained scene depth information, we present an innovative label optimization strategy (LOS) that enhances label density and quality. DEN achieves an 18.78% reduction in mean absolute error (MAE) and a 12.8% decrease in root mean-square error (RMSE) for depth estimation. Additionally, it improves 3-D object detection accuracy by 0.8% compared to the baseline model. Under low visibility conditions, DEN demonstrates a 6.7% reduction in MAE and a 9.6% reduction in RMSE compared to the baseline. These improvements demonstrated its robustness and enhanced performance under challenging conditions.