Unleashing Stable Diffusion for Multi-Modal Facial Image Augmentation

In this paper, we propose a novel method that combines Stable Diffusion (SD), ControlNet, and ChatGPT to enhance the FFHQ-Aging dataset. Our approach leverages Stable Diffusion to generate realistic and high-quality synthetic images, while ChatGPT prompts enable the generation of diverse facial expressions, ethnicities, and lighting variations. The ControlNet plays a crucial role in controlling facial pose variations during the augmentation process. We carefully design the parameters of the diffusion model and ControlNet, and provide appropriate prompts to preserve desired facial attributes while introducing pose variations. Furthermore, we introduce an advanced model that extends our previous work by simultaneously performing age transformation and face reenactment. This model enables comprehensive facial transformation and provides a versatile tool for various applications. Our experimental results demonstrate the effectiveness of our proposed method in improving age transformation and face reenactment tasks.