Generating and Contributing Test Cases for C Libraries from Client Code: A Case Study

Software libraries are at the core of software development, and any bugs can affect a potentially large number of present and future client applications. Therefore, thorough testing of libraries is of key importance. Unfortunately, writing library test cases is often difficult, requiring awareness of complex data structures and preconditions. We report our experience implementing APISLICER, a technique (and tool) which starting from the client of a library under test, extracts self-contained library test cases that can be used to enhance the library's test suite. Such test cases provide a key benefit: they represent real-world usage scenarios of the APIs exported by the target library, which may not have been envisioned by the library developers. We have conducted a case study in which we have applied APISLICER on seven mature libraries, with a total of twelve clients. Our experience highlights that while APISLICER has successfully extracted compilable test cases for all these libraries, library developers are oftentimes cautious about accepting the test cases. Out of seven libraries, three (LIBUNISTRING, AMPLGSL, and GSL) accepted our contributions. We report on the reaction of the developers to our contributions and more generally on the opportunities and challenges facing this approach.