Multimodal Cooperative 3D Object Detection Over Connected Vehicles for Autonomous Driving

Having an accurate and comprehensive understanding of surrounding environment is the first step toward autonomous driving. For an autonomous car to plan its actions, for example, to make instant decisions about whether to proceed at an intersection or slow down to wait until a pedestrian crosses the road, it first needs to recognize its surroundings, including but not limited to vehicles and pedestrians. One of the challenges needed to be addressed is how to handle situations where objects are being occluded, especially for pedestrians since they are small and vulnerable. Cooperative perception is a promising approach in addressing this challenge. By sharing perceived information together, the detection range for each individual connected autonomous could be extended. Although existing works focusing on cooperative perception improve the accuracy and extend detection range for vehicles, they fail to consider the effect of small objects. In this article, we investigate cooperative 3D object detection targeting on improving the detection of small objects (i.e., pedestrians), which may be achieved by leveraging shared information extracted from image and LiDAR over connected autonomous vehicles. Evaluation results show that the proposed method outperforms state-of-the-art networks on 3D pedestrian detection.