Beyond 3D: Generic IoU for 3D Object Detection

Object detection from point clouds is a fundamental task for 3D scene understanding and has a wide range of applications in the field of multimedia data processing and analysis, such as autonomous driving and virtual interaction. The IoU evaluates the overlap between the two bounding boxes to ensure consistency across network optimization and testing, becoming a recognized regression loss in the field of 3D object detection. However, there is a kind of error coupling between the IoU and the angle, i.e., the IoU does not decrease as the angle error increases and vice versa. This problem leads to sub-optimal solutions for the neural network model, which severely hampers the improvement of 3D object detection accuracy. In this paper, a novel 4DIoU method is introduced for detecting 3D objects from point clouds, which provides a comprehensive rethinking of IoU computation by integrating angular information as an additional dimension. 4DIoU not only solves the problem of error coupling between IoU and angular but also facilitates neural network optimization using angle information. Furthermore, to solve the different impacts of various object shapes on IoU variations, a special 4DIoU called TV4DIoU is proposed to fuse shape information based on three orthogonal projection views, which can adaptively learn the information of objects with different shapes. In addition, to enhance the generalization of the 4DIoU method, a high-flexibility anchor encoding method and a cyclic consistent computation formula for angular errors are designed to make 4DIoU a plug-and-play module for both anchor-based and anchor-free frameworks. Extensive evaluations conducted on the nuScenes, Waymo, and KITTI datasets have confirmed the effectiveness of the proposed method.