CodeGen-Test: An Automatic Code Generation Model Integrating Program Test Information

Automatic code generation is to generate the program code of the corresponding language according to the given natural language description. At present, the mainstream methods basically use neural network to encode natural language description, form an abstract syntax tree in the decoder, and then convert the abstract syntax tree into program code. Although the generated code conforms to specific syntax rules, two problems are still ignored: one is the lack of program testing, which is an indispensable step in the process of complete code implementation; Second, it only focuses on whether the generated code meets the syntax requirements, ignoring the more important requirement - functional requirements. This paper constructs a CodeGen test model for integrating code testing, that is, on the basis of generating code that meets the syntax requirements, add program testing steps, integrate program testing information, and iteratively generate code that meets the functional requirements of the program, so as to improve the quality of code generation. This paper evaluates the effect of CodeGen test model on a python data set "hearthstone legend". The experimental results show that this method can effectively improve the quality of generated code. Compared with the existing optimal model, CodeGen test model improves the Bleu value by 0.4%, Rougel value by 0.4%.