Exploring the Capability of ChatGPT in Test Generation

The design of test is a crucial step in the field of software testing. The quality of test significantly impacts the effectiveness of software testing, with well-designed test cases improving the efficiency of bug detection. However, manual test case design and writing are often considered time-consuming and labor-intensive. With the emergence of large language models (LLMs), especially ChatGPT, the potential of LLMs in the field of test generation has become evident. Pretrained LLMs can learn and understand code in various programming languages and design test cases using multiple testing frameworks. In this paper, we used ChatGPT to generate tests for some tested projects. Through experiments, we found that ChatGPT has some gaps compared to traditional test generation tools, but its performance is closer to manual testing. However, the tests generated by ChatGPT exhibit higher readability. We believe that ChatGPT is better suited to serve as a manual testing assistant, helping understand the tested code and providing testing ideas.