Depth-Vision-Decoupled Transformer with Cascaded Group Convolution Attention for Monocular 3D Object Detection

Monocular three-dimensional (3D) object detection is a challenging and important task in autonomous driving. In 3D object detection, depth information is crucial for determining the position, size, and posture of objects. However, in the two-dimensional (2D) images obtained by monocular cameras, depth information is compressed into a plane, which blurs the relative position and size relationships between objects. In addition, the pose changes of objects in 3D space (such as rotation, tilt, etc.) can cause changes in their projected shape, size, and position in 2D images, which increases the difficulty of detecting 3D objects from monocular 2D images. In this paper, we propose a depth-vision-decoupled transformer for monocular multi-class 3D object detection. The proposed scheme is constructed with the following novel components: (1) a Cascaded Group-Multi Scale Convolutional Attention (CG-MSCA) with multi-scale-perception and direction-perception capabilities to focus on the local characteristics of the complex scene; (2) a Decoupled-Feature-Aware TRansformer (DFTR) module, which globally decouples depth and visual features, encodes and decodes them separately to avoid the monocularly estimated inaccurate depth interfering with the model’s learning of visual information and alleviate the mismatch between stereo information and geometric shape of the object; (3) a Cross-Attention-Guided Fusion Module (CAFM) to rationalize the fusion of decoupled depth and visual features before prediction. Experiments on the KITTI, DAIR-V2X-V, and DAIR-V2X-I datasets show that our proposed method produces competitive performance compared with Transformer-based methods and other state-of-the-art (SOTA) methods. For the Car Category, our method achieves 17.04% Average Precision of three-dimensional (AP3D) for the moderate setting (the most important setting) of the KITTI dataset with an Intersection over Union (IoU) threshold of 0.7.