When AI Meets Code Analysis: A Study of Adversarial Attacks on Deep Learning-based Code Models via Program Transformation

Semantics-preserving program transformations (SPTs) are widely used to generate adversarial example attacks against deep learning-based models for code analysis tasks. This work studies 34 recent representative papers that utilize SPTs to challenge the robustness of code models. We summarize trends in how SPTs are designed, evaluated, and combined in different architectures and tasks. Additionally, we identify core elements and consequences of adversarial attacks against models for code analysis tasks. We perform fine-grained and holistic qualitative analysis of our studied works across 19 metrics to provide a high quality overview of the studied field. Our findings reveal that, while single and combined SPTs are adequately studied separately, there is insufficient joint evaluation of their effects. We also identify critical issues such as inconsistencies in evaluation metrics and limited assessment of a specific category of deep learning-based model. Finally, we evaluate replicability and external validity of existing approaches by inspecting datasets and code availability. Based on the results of our study, we provide recommendations for how developers and researchers should defend against SPT-based attacks and outline serious gaps in the existing literature. Our work aims to guide future research directions to improve the overall robustness of deep learning-based code models against adversarial attacks utilizing SPTs.