Deep Learning Based 3D Object Detection for Automotive Radar and Camera

In this paper it is demonstrated how 3D object detection can be achieved using deep learning on radar pointclouds and camera images. A deep convolutional neural network is trained with manually labelled bounding boxes to detect cars. The results are compared to a deep neural network trained on lidar pointclouds and camera images. The average precision (AP) is used to evaluate the performance. For radar and camera the AP is 0.45, 0.48, and 0.61, whereas the AP for lidar and camera is 0.33, 0.35, and 0.46 for occluded, partially occluded and not occluded cars respectively. The performance of the network is significantly better with radar data compared to lidar data. Currently, the main limitation of the performance of the object detection with radar data and camera images is the dataset, which is until now rather small. However, the results show that deep learning is generally a suitable method for object detection on radar data.