DevilDiffusion: Embedding Hidden Noise Backdoors into Diffusion Models

Diffusion models represent state-of-the-art deep learning architectures behind many popular and powerful image-synthesizing generative Artificial Intelligence (AI) systems. Their underlying approach, which relies on scheduled noise addition and optimized noise removal, has been applicable to the syn-thesis of sophisticated and nearly photorealistic images across various domains; however, the potential impacts of compro-mised diffusion models remain an underexplored area in the literature. While prior studies have investigated the insertion of explicit triggers into the noise or prompt spaces of diffusion models, it is unrealistic to assume that benign users would intentionally input such triggers into their own diffusion process. In our DevilDiffusion approach, we demonstrate the capability to surreptitiously embed triggers that leverage uncommon but naturally -occurring characteristics of Gaussian noise directly into the noise space of conditional diffusion models. When these specific characteristics occur in naturally -occurring noise, the model will instead construct our target backdoor image. By adjusting the trigger size and the ratio of poisoned images, we can control the trigger rates of the specified target image, ranging from less than 0.01 % to 25 % of all generated images, while still maintaining performance on the target task.