Supply Chain Security: Pre-training Model for Python Source Code Vulnerability Detection

In this paper, we propose an innovative deep learning-based model designed to detect potential security vulnerabilities in source code. The model is pretrained on the open-source PyPI code repository and a specially curated malicious code dataset, allowing it to deeply learn the syntactic structure and semantic information of the code, thus building a deep knowledge representation. We further refine this knowledge representation through fine-tuning to form a classifier focused on malicious code detection. Evaluation results on an open-source malicious code dataset indicate that our model's performance not only surpasses traditional machine learning methods, recurrent neural networks (RNNs), and convolutional neural networks (CNNs), but also outperforms popular open-source security tools and significantly exceeds advanced detection models designed for other programming languages. Despite our model's smaller scale, limited by the amount of data and model parameters, it demonstrates exceptional performance on the test dataset, far exceeding traditional machine learning techniques. This achievement highlights the strong potential of our model in the analysis of source code security and provides new directions and possibilities for future research in this field.