Detecting Source Code Vulnerabilities Using Fine-Tuned Pre-Trained LLMs

Pre-trained large language models (LLMs) have advanced capabilities in feature extraction and pattern discovery. Utilizing fine-tuning techniques effectively adapts LLMs to specific scenarios. Detecting vulnerabilities during the coding phase is crucial. In this paper, we collected the Java CWE vulnerability from the SARD dataset and then supervised fine-tuned the open-source Qwen2-7B model using the LoRA technique. We compared the results with vulnerability detection models based on Graph Neural Networks (GNN) and Long Short-Term Memory (LSTM). It is demonstrated that the approach of fine-tuning for LLMs can effectively detect source code vulnerabilities.