Test Case Generation for Python Libraries Using Dependent Projects' Test-Suites

Software developers often leverage open-source libraries to efficiently develop their own software. The reliance on popular libraries makes libraries' quality increasingly important. Although most modern libraries have their own tests to maintain quality, these tests are not perfect, and writing effective tests can be challenging and time-consuming. Automatic test generation techniques have been studied to help generate effective tests with less effort, however, automatic test generation for dynamically typed languages, such as Python, is challenging due to the absence of type information. This paper proposes a test case generation method for Python libraries. By instrumenting both the target library and a project that is dependent on the library, we collect execution traces including what data is passed to the library, and what data is returned from the library during the test execution of the dependent project. These traces are then used to reconstruct how the library is used in practice, enabling the generation of test cases that closely reflect real-world usage of the library. We evaluate the proposed method on three open-source Python libraries, each with three dependent projects by measuring line coverage of the generated test cases along with the original test cases in the library. The proposed method achieved test coverage improvements for all target libraries. The evaluation result suggests that the proposed method can generate effective test cases for Python libraries.