LiDAR Data Noise Models and Methodology for Sim-to-Real Domain Generalization and Adaptation in Autonomous Driving Perception

In autonomous driving, object detection and semantic segmentation are critical tasks for path planning and control of an autonomous vehicle. Recent approaches are based on supervised learning methods, with large datasets sampled in the target domain. However, annotating training data for supervised learning methods is a high resource and time-consuming task. In this work, we propose to exploit artificial LiDAR data for object detection and semantic segmentation. We use the CARLA simulator [1] to generate artificial data of autonomous driving scenarios and propose ways to mitigate the differences between artificial and real-world data (domain generalization). We modeled both the noise and the missed reflections (denoted point dropout) that occur in real-world data collection, and show their effects in the detection and segmentation tasks. We assess the potential benefits of using pre-trained models on artificial data when fine-tuning with all, or a fraction, of the available real-world data (domain adaptation). We find clear improvements when using artificial data to pretrain a network, which allows to use a reduced amount of realworld data, and boost the performance of the trained models.