Invasive Software Testing: Mutating Target Programs to Diversify Test Exploration for High Test Coverage

Software testing techniques have advanced significantly over several decades; however, most of current techniques still test a target program as it is, and fail to utilize valuable information of diverse test executions on many variants of the original program in test generation. This paper proposes a new direction for software testing - Invasive Software Testing (IST). IST first generates a set of target program variants m₁, ⋯, m_n from an original target program p by applying mutation operations ₁, ⋯, μ_n. Second, given a test suite T, IST executes m₁, ⋯, m_n with T and records the test runs which increase test coverage compared to p with T. Based on the recorded information, IST generates guideposts for automated test generation on p toward high test coverage. Finally, IST generates test inputs on p with the guideposts and achieves higher test coverage. We developed DEMINER which implements IST for C programs through software mutation and concolic testing. Further, we showed the effectiveness of DEMINER on three real-world target programs Busybox-ls, Busybox-printf, and GNU-find. The experiment results show that the amount of the improved branch coverage by DEMINER is 24.7% relatively larger than those of the conventional concolic testing techniques on average.