Building Towards "Invisible Cloak": Robust Physical Adversarial Attack on YOLO Object Detector

Deep learning based object detection algorithms like R-CNN, SSD, YOLO have been applied to many scenarios, including video surveillance, autonomous vehicle, intelligent robotics et al. With more and more application and autonomy left to deep learning based artificial intelligence, humans want to ensure that the machine does the best for them under their control. However, deep learning algorithms are known to be vulnerable to carefully crafted input known as adversarial examples which makes it possible for an attacker to fool an AI system. In this work, we explored the mechanism behind the YOLO object detector and proposed an optimization method to craft adversarial examples to attack the YOLO model. The experiment shows that this white box attack method is effective and has a success rate of 100% in crafting digital adversarial examples to fool the YOLO model. We also proposed a robust physical adversarial sticker generation method based on an extended Expectation Over Transformation (EOT) method(a method to craft adversarial example in the physical world). We conduct experiments to find the most effective approach to generate adversarial stickers. We tested the stickers both digitally as a watermark and physically showing it on an electronic screen on the front surface of a person. Our result shows that the sticker attack as a watermark has a success rate of 90% and 45% on photos taken indoors and on random 318 pictures from ImageNet. Our physical attack also has a success rate of 72% on photos taken indoors. We shared our project source code on the Github and our work is reproducible.