Imperceptible Backdoor Attacks on Text-Guided 3D Scene Grounding

With the maturity of depth sensors, the vulnerability of 3D point cloud models has received increasing attention in various applications such as autonomous driving and robot navigation. Previous 3D adversarial attackers mainly focus on attacking naive 3D classification models by perturbing 3D objects. However, since real-world 3D applications generally rely on more complicated scene-based point cloud data, these attack methods are impractical to deploy in realistic scenarios. Therefore, in this paper, we attempt to introduce the adversarial attacks into a more practical yet challenging large-scale scene-based 3D task, i.e., text-guided 3D scene grounding. To make perturbations both effective and imperceptible in scene cases, we investigate the vulnerability of 3D grounding models to backdoor attacks, which implant backdoor triggers into 3D models via data poisoning so as to control the models' predictions at test time. Specifically, we propose a novel Joint Scene-Text Backdoor Attack (JSTBA) method to embed triggers in each of the input modalities and activate the malicious behavior only when both triggers are present. We further design a visual trigger optimization strategy to place the visual trigger appropriately in the 3D scene, aiming to make it natural and imperceptible. Extensive experiments are conducted on seven classic 3D grounding models and three datasets, showing that our JSTBA attack significantly degrades the performance of 3D models on the poisoned data while gaining comparable performance with the benign models on the clean data.