Painted PIXOR: A Real-Time Multi-Sensor Fusion Object Detection Pipeline for Autonomous Vehicles

Multiple sensors such as cameras and LiDARs play an essential role in perceiving the surrounding environment for autonomous vehicles (AVs) by providing complementary information. Fusing multi-sensor information and detecting traffic objects in an accurate and real-time manner remains a challenge. In this work, we propose a new fusion pipeline: Painted PIXOR, based on the improved Point Painting and PIXOR detector. We first improve the semantic correctness of images by fusing multi-view images in Point Painting through an efficient PCC fusion method. We then project the LiDAR points into the fused image semantic map and append traffic object class scores to the corresponding original point. The appended point is used as the feature encoder input to extract the suitable input for the PIXOR detector. Experiments tested on KITTI detection show that our proposed method performs better than the original PIXOR and other methods in terms of average precision (AP).