GAN-Based Domain Adaptation for Creating Digital Twins of Small-Scale Driving Testbeds: Opportunities and Challenges

In recent years, small-scale driving testbeds have been developed as controlled physical environments for the evaluation of autonomous vehicle controllers. Such controllers are heavily dependent on computer vision algorithms that enable the vehicle to perceive its surroundings. To bridge the Sim2Real content and appearance gap between simulated and real-world image data for training these algorithms, we propose a novel transfer learning approach that performs domain adaptation using StyleGAN to generate style-mixed images that closely resemble real-world images. We explain our approach within the context of our small-scale driving testbed, CHARTOPOLIS, and demonstrate it on synthetic image data of two object classes, vehicles and buildings, from the driving simulator CARLA. Our results show that this approach works on the vehicle object class while failing on the building object class. This paper thus provides a plausible approach to bridging the Sim2Real gap through the use of custom pipelines that augment image datasets using a mix of techniques for domain adaptation and domain randomization.