Frustum PointVoxel-RCNN: A High-Performance Framework for Accurate 3D Object Detection in Point Clouds and Images

Precise 3D object detection plays a pivotal role within the perception module of autonomous vehicles. Many approaches have shown promising results for 3D object detection with lidar. Voxel-based methods are computationally efficient but suffer from information loss, while point-based methods handle irregular data but require extensive computations and optimization processes. In this paper, a high-performance 3D object detection framework named Frustum PointVoxel-RCNN has been developed. Using 2D object detection with frustums effectively mitigates the presence of irrelevant point clouds associated with the objects. Afterwards, 3D Voxel CNN and PointNet-based set abstraction are used to learn the features of the frustum-processed point clouds. Our method significantly reduces the computation cost of frustum-processed point clouds in 3D Voxel CNN and PointNet-based networks. Extensive experiments on the KITTI dataset shown our proposed method surpassed stae-of-the-art 3D detection methods when accurate 2D detections are precise. Specifically, it yielded significant speed advantages and improved accuracy when the scene was at a hard level. Our proposed method achieves a 50% improvement in computational efficiencv comnared to PV-RCNN.