Palette: Physically-Realizable Backdoor Attacks Against Video Recognition Models

Backdoor attacks have been widely studied for image classification tasks, but rarely investigated for video recognition tasks. In this paper, we explore the possibility of physically-realizable backdoor attacks against video recognition models. Different from existing works that directly apply image backdoor attacks to videos, i.e., patch a visible trigger to each frame of a video, we carefully take into consideration the temporal interactions among frames in a video. Our proposed video backdoor attack, named
Palette
, features two special design choices. The first is to utilize natural-light-alike RGB offset as triggers rather than traditional patch triggers. Such triggers may be applied in the physical world through lighting without the need to modify video files. The second is to make the backdoored model more robust to temporal asynchronization between the trigger and the video samples by performing rolling operations during sample poisoning. Extensive experiments show that
Palette
outperforms existing video backdoor attacks, especially in the physical world. It is shown that
Palette
is also resistant to backdoor defense methods. We will open-source our codes upon publication.