Out-of-Distribution Sample Selection Generated by Diffusion Model toward Model Generalization

Model generalization that prevents overfitting to the training data is critical to the robustness and reliability of image classification models. Diffusion models are rapidly developing in generating photorealistic images from text, yet it is difficult for a simple text to generate a variety of images that reflect the training data, and there is potential for model generalization. In this paper, we propose a novel method for model generalization with generated images via diffusion models. We make the diffusion model recognize the training data distribution in both the latent and the image space, and obtain generated images reflecting the distribution. In the latent space, we synthesize new features from the training data distribution and decode them into images using the diffusion model. In the image space, we perform selection of unintentionally generated images (out-of-distribution samples) based on the semantic information of the training data. By considering the training data distribution not only in the latent space but also in the image space, we eliminate out-of-distribution samples generated by the diffusion model. Experiments demonstrate that our generated images are faithful to the training data distribution and enhance model generalization performance.