Software Defect Prediction Method Based on Transformer Model

Aiming at the problem of grammar and semantic information understanding of the network structure of software system, this paper proposes a method of software defect prediction, which is based on Transformer model, which is completely dependent on self-attention mechanism, it can embed key information in the code semantics of the end-to-end learning software modules. Firstly, the software module is converted into an abstract syntax tree, and later it is traversed to extract word sequence of the software module. Then, thinking of the word sequence as input, acts on the self-attention layer to perform semantic feature embedding of the software module. Finally, the Softmax neural network is used to predict the software defect. The experimental results show that the software defect prediction method based on Transformer model has better defect prediction effects in the three open source software, and it has an average increase of 3.2% than the optimal model based on the Convolutional Neural Network (CNN).