Object Detection using Synthetic Data and Generative AI

Autonomous driving object segmentation and detection training models generally rely on real-world data, which is costly to annotate, difficult to acquire, and heterogeneous. We present a new system based on deep learning and generative AI methods for producing high-quality synthetic data with the goal of overcoming these challenges. Our system produces photorealistic synthetic images using Stable Diffusion models and annotates them using the CVAT annotation tool and Segment Anything Model (SAM). We divide the obtained data into training sets and validation subsets upon pre-cleaning the given annotations into corresponding segmentation masks. We employ this dataset to train YOLOv8 on object detection and segmentation tasks in such a manner that we are able to check the quality of our produced fake data. In addition, we deploy the BLIP image-captioning feature on Salesforce to produce rich information-based descriptive product captions. Our models trained on our synthetic data are better, or even better, than models trained on real data, according to our experimental test. The test proves the huge potential of synthetic datasets as an economically sustainable and scalable way to train perception systems for autonomous vehicles, particularly for perceiving challenging or exotic driving situations.