Enhancing the Domain Robustness of Self-Supervised pre-Training with Synthetic Images

We present a novel method for improving the adaptability of self-supervised (SSL) pre-trained models across different domains. Our approach uses synthetic images that are generated using an auxiliary diffusion model, namely InstructPix2Pix. More specifically, starting from a real image, we prompt the diffusion model to generate synthetic versions of that image in the style of the target domains. This allows us to generate a diverse set of multi-domain images that share the same semantics as real images. Integrating these synthetic images into the training dataset enhances the model’s capacity to generalize to other domains. We pre-trained different SSL methods on Imagenet-100 with and without the synthetic images and evaluated their performance on three multi-domain datasets, DomainNet, PACS, and Office-Home. Our results show significant improvements in all datasets and methods, encouraging new research in the direction of leveraging synthetic data to improve the robustness of pre-trained models. Code is available at https://github.com/has97/Diffusion_pre-training.