Analysis of Software Vulnerabilities Introduced in Programming Submissions Across Curriculum at Two Higher Education Institutions

This full research paper describes the analysis of common software vulnerabilities that are introduced by students enrolled in four-year computing and cybersecurity majors from two different higher education institutions in Georgia. As the demand for secure coding education continues to grow, pedagogical improvements need to be made in identifying key software vulnerabilities students commit during code development (from the first programming course to the exit senior design capstone) which in turn can be analyzed to inform the pedagogical interventions focused at preparing students with skill sets for writing secure code and entering the professional workforce. While code security is emphasized throughout the computing curriculum, this research is focused on training individuals to be aware of common vulnerabilities and tailoring programming concept knowledge that has been shown to have a positive effect on code security. Existing research has mainly focused on developing vulnerability analysis tools rather than collecting data (and subsequently analyzing) regarding the types of vulnerabilities produced by students at their institutions. In this paper, we analyzed student code across different courses and reported the types of vulnerabilities produced by students in their assignment submission code from two different higher education institutions across different levels of the four-year curriculum. The reported vulnerabilities are grouped by CWE-ID, which is a standard and common way to categorize and identify software vulnerabilities. The resulting CWE-IDs are then grouped per student submission and per semester (across curriculum levels) to discover the common types of software vulnerabilities committed across cross sections of students. Our results from the analysis of vulnerabilities (ranging from CS1 courses to capstone courses) are organized around the following research questions: 1) What are the most common software vulnerabilities produced by computing majors at different levels through the computing curriculum?; and 2) Do these vulnerabilities persist throughout their curriculum as they advance into higher-level courses? We report that students commonly make mistakes related to variable usage, null pointer checks, hard-coding sensitive information, and improperly validating input. Vulnerabilities such as CWE-489 (“ Active Debug Code”) and CWE-215 (“Insertion of Sensitive Information Into Debugging Code”) tend to persist across multiple course levels and may need to be focused in the computing curriculum. The number of vulnerabilities introduced in assignment code increases as course complexity increases. We also find that vulnerabilities produced by students have little overlap with what software vulnerability researchers commonly study, potentially leading to a mismatch in priority for secure coding topics. Our findings have implications for computer science and cybersecurity curriculum design and delivery.