Evaluating the testing ability of senior-level computer science students

Testing is a key skill for computer science students to acquire during their studies. To determine how well students are learning this skill, we conducted an empirical study in two offerings of a senior-level computer science course. The goal of the study was to determine whether students would be able to create a small, complete test suite for a simple program. The students created a test suite first without the aid of a coverage tool and then with the aid of a coverage tool. The results indicate that without a coverage tool, students achieved significantly less than 100% statement, branch or condition coverage. When provided with a code coverage tool, students increased coverage levels. Still, examination of the test suites indicated that they were significantly larger than the minimum required. These results indicate that students cannot conduct adequate testing of even a small program. To provide context for our results, we provide a literature survey summarizing various techniques proposed for teaching testing in the computer science curriculum. We discuss each technique, its strengths, and its weaknesses.