Validation Framework Foradas Deep Learning Based Object Detection Models

Deployment of autonomous cars on a wide scale requires large datasets and a variety of scenarios to train deep learning models on different circumstances encountered in real-life situations. Manual annotation of these datasets is expensive and requires a vast number of man-hours, making it unsuitable for large scale training and validation. These limitations highlight the need for exploiting automatic annotation in simulators. This work presents a framework for synthetic dataset generation for Lidar sensors with the same Lidar sensors setup of the Waymo ego vehicle used to generate the real-world Waymo-Open Dataset. The framework generates synthetic dataset with the exact format of the well-known Waymo-Open Dataset. In addition, this paper releases a new synthetic dataset named Cairo University-Carla Waymo (CUCW) dataset for Lidar sensors using the framework introduced in this paper. This dataset can be used to aid in training and validating deep learning models used in the advanced driver-assistance system (ADAS). The framework uses the CARLA urban simulator which is one of the best-suited urban simulators for end-to-end testing of the unique functionalities that self-driving cars offer, including perception, mapping, localization, and vehicle control. The generated dataset is compared against both Waymo-Open and KITTI-CARLA datasets. The framework is made publicly available. Lastly, the CUCW dataset is combined with the real-world Waymo-Open dataset to study the effect of the CUCW dataset on the deep learning model accuracy.