4DRC-OC: Online Calibration of 4D Millimeter Wave Radar-Camera With Depth Map Assistance

The online calibration of 4D millimeter-wave radar and camera is crucial for advancing perception and SLAM technologies in complex environments. It eliminates the reliance on manual labeling, offering real-time and convenience. However, the sparse nature of 4D radar point clouds presents challenges in establishing correspondences with camera images. This letter proposes an online 4D radar-camera online calibration method (4DRC-OC) that utilizes unified depth map representations for auxiliary training, ensuring feature alignment and modal unification between the two sensors. Due to the limited useful information within sparse depth maps, 4DRC-OC uses dynamic convolution to adaptively capture detailed features. Furthermore, this letter designs a correlation module based on channel-wise fusion (CMCF) that computes correlations between error depth maps and RGB-derived depth maps, thereby enhancing features to facilitate extrinsic parameter regression. Experimental results on the Dual-Radar dataset validate the superiority of the proposed approach in extrinsic calibration.

I. Introduction

As An emerging sensor for autonomous driving [1], 4D millimeter-wave radar is increasingly becoming a key solution for environmental perception [2], [3], and simultaneous localization and mapping (SLAM) technology in complex environments [7]. Due to the sparsity of its point clouds [8], 4D radar faces challenges in distinguishing the semantic information of objects, whereas camera sensors offer rich texture and semantic details of scenes [9]. Therefore, the fusion of 4D radar and camera is considered a robust combination strategy. For multi-sensor fusion [10], [11], real-time and stable calibration is essential for efficient integration.