Diffusion to Confusion: Naturalistic Adversarial Patch Generation Based on Diffusion Model for Object Detector

Many physical adversarial patch generation methods are widely proposed to protect personal privacy from malicious monitoring using object detectors. However, they usually fail to generate satisfactory patch images in terms of both stealthiness and attack performance without making huge efforts on careful hyperparameter tuning. To address this issue, we propose a novel naturalistic diffusion model-based (DM) adversarial patch generation method. Through sampling the optimal image from the pretrained DM model upon natural images, it allows us to stably craft high-quality physical adversarial patches without suffering serious mode collapse problems as other deep generative models. Moreover, the generated patches are not only visually pleasing but also robust against the state-of-the-art adversarial patch removal algorithm due to different image statistics from the traditional ones. In addition, to resolve the huge memory requirement of the diffusion model during backpropagation, we also utilize adjoint method for patch generation. With extensive experiments, the results demonstrate the effectiveness of the proposed approach to generate better-quality and more stealthy adversarial patches than other approaches against the adversarial patch removal algorithm while achieving comparable attack performance than other state-of-the-art patch generation methods.