TransCAB: Transferable Clean-Annotation Backdoor to Object Detection with Natural Trigger in Real-World

Object detection is the foundation of various critical computer-vision tasks such as segmentation, object tracking, and event detection, which can be deployed on pervasive Internet of Things (IoT) and edge devices. A large amount of data is often required to train an object detector with satisfactory accuracy. However, due to the intensive workforce involved with collecting and annotating large datasets, data curation task is often outsourced to a third party (e.g., Amazon Mechanical Turk) or volunteers. This work reveals severe vulnerabilities in this data curation pipeline. We propose TransCAB, the first work to craft clean-annotated images to stealthily implant the backdoor into the object detectors later trained on them by the data curator/user even when the data curator can manually audit the images and fully controls the training process. Existing clean-label poisoned images are only shown in classification tasks but not non-classification tasks, in particular, object detection due to unique challenges faced, generally owing to the complexity of having multiple objects within each frame (image), including the victim and non-victim objects. Furthermore, we demonstrate that the backdoor effect of both cloaking and misclassification are robustly achieved in the wild when the backdoor is activated with inconspicuously natural physical object as trigger (i.e., T-shirt). The efficacy of our TransCAB is ensured by constructively i) applying the image-camouflage attack that abuses the image-scaling function widely used by the deep learning framework (i.e., PyTorch), ii) incorporating the devised clean image replica technique, and iii) combining identified poison data selection criteria given constrained attacking budget. Extensive experi-ments on YOLOv3, YOLOv4, CenterNet, and Faster R-CNN affirm that TransCAB exhibits more than 90% attack success rate under various real-world scenes even when a very small (i.e., 0.14%) dataset fraction is poisoned. In addition, the small set of poisoned images crafted on one detector (i.e., YOLOv3) can be effectively transferred to insert a backdoor on another detector (i.e., CenterNet). A comprehensive video demo is at https://youtu.be/MA7L_LpXkp4, where a poison rate of merely 0.14% is set for YOLOv4 cloaking backdoor and Faster R-CNN misclassification backdoor. Our collected dataset with T-shirt as a natural trigger (about 11,350 frames in total) is open to the public at https://github.com/inconstance/T-shirt-natural-backdoor-dataset, which is the first relatively large-scale natural trigger backdoor dataset.