Probing Vulnerabilities of Vision-Lidar Based Autonomous Driving Systems

Autonomous driving systems rely on advanced perception models to interpret their surroundings and make real-time driving decisions. Among these, Bird's Eye View (BEV) perception has emerged as a critical component, offering a unified 3D representation from multi-camera and sensor inputs. However, in the meantime, the security vulnerabilities of BEV-based models are starting to be examined within the scope of adversarial machine learning research. This study provides a preliminary security analysis of BEV perception models, focusing on adversarial attacks employed in different modalities, including both visual signals from cameras and point-cloud signals from LiDAR. Specifically, we examine the vulnerabilities of state-of-the-art modelsincluding BEVDet, BEVDet4D, DAL, and BEVFormer-to different forms of adversarial attacks. In addition to the white-box setup, we also check the transferability of these attacks to black-box models. Our findings reveal that although multi-modal inputs significantly improve BEV models' detection performance, they also introduce new channels for adversarial attacks and hence increase vulnerability. As long as the adversarial attack is applied to all modalities that a model takes in, e.g., adversarial perturbation is added to both vision and LiDAR signals for a vision-LiDAR model, and the attack can always achieve an almost complete success rate, while utilizing incomplete modalities to attack results in sub-optimal outcomes, as this strategy allows detection models to still capture undisturbed information. Moreover, we show that the designed attack can transfer across totally different BEV architectures. For example, adversarial input trained with DAL, a CNN-based model, can still transfer to BEVFusion and significantly degrade its performance, despite BEVFusion using a transformer-based architecture. We hope that this work can help to raise the community's attention to the vulnerability of BEV-based autonomous driving systems.