Global Overviews of Granular Test Coverage with Matrix Visualizations

Existing IDE-based tools that are available to developers make understanding software testing difficult for a software system, for both granular tasks (e.g., answering questions such as, "which test cases execute this method?") and global tasks (e.g., answering questions such as, "what is the proportion of unit tests to system tests?"). IDE-based tools typically support local, file-based views of a project’s test suite, and rarely offer a global overview. Global overviews can provide a larger context for a method’s execution by test cases; help identify other similar, or related methods; and even reveal similarity between individual tests. This work approaches such challenges with a novel, interactive, matrix-based visual interface that provides a global overview of a software project’s test suite, specifically in the context of the methods available in the project’s codebase. Through a series of interactive functions to sort, filter, query, and explore a test-matrix visualization, we demonstrate how developers can effectively answer questions about their project’s test suite, and the code executed by such tests. Our evaluations, performed on four real-world software systems, show that the interactive visualization assisted developers to answer questions about software tests and the code they execute. Further, the visualization consistently outperforms traditional development tools, both in accuracy and time taken to complete software-engineering tasks.