Adverse to Normal Image Reconstruction Using Inverse of StarGAN for Autonomous Vehicle Control

Controlling autonomous vehicles in adverse weather conditions poses unique challenges, particularly in reconstructing images from diverse adverse environments—such as rain, snow, fog and night light—into clear, normal-appearing visuals. Although StarGAN has been successfully applied to multi-domain conversion, its design typically focuses on transforming a single image into multiple styles rather than converting images from multi-domains into a unique representation. To address this limitation, we propose an inverse version of StarGAN that processes images captured by the vehicle’s camera and transforms them into clear visuals simulating normal conditions. Initially, the model is trained with explicit weather domain labels and later with shuffled labels, a strategy that reduces dependence on specific label severities and enhances the model’s adaptability in dynamic environments. Since CNNs often overfit when dealing with multi-domain tasks, we incorporate Capsule Neural Networks to mitigate overfitting, capture spatial hierarchies and improve sensitivity to object orientation. Furthermore, a perceptual loss function is employed to preserve fine image details. Our extensive dataset comprises over 100,000 samples collected from the AVIS ENGINE simulator along with real-world validation from the BDD100k dataset under rain, snow, fog, and nightlight conditions. Results indicate an average PSNR of 22.21 and an SSIM of 0.92 in closed-loop simulations, demonstrating robust and real-time performance. This work represents a significant step forward in enhancing the safety, reliability, and overall robustness of autonomous systems operating in challenging and unpredictable weather conditions.