A System-Level Testing Framework for Automated Assessment of Programming Assignments Allowing Students Object-Oriented Design Freedom

Automated assessment of programming assignments is essential in software engineering education, especially for large classes where manual grading is impractical. While static analysis can evaluate code style and syntax correctness, it cannot assess the functional correctness of students' implementations. Dynamic analysis through software testing can verify program behavior and provide automated feedback to students. However, traditional unit and integration tests often restrict students' design freedom by requiring predefined interfaces and method declarations. In this paper, we present SYSCLI, a novel testing framework for system-level testing of JAVA-based command-Line interface applications. SYSCLI enables test suites that evaluate the functional correctness of students' implementations without limiting their design choices. We also share our experience using SYSCLI in a second-year programming course at the University of Auckland, which focuses on object-oriented programming and design patterns and enrolls over 300 students each offering. Analysis of student assignments from 2023 and 2024 shows that SYSCLI is effective in automating grading, allows software design flexibility, and provides actionable feedback to students. Our experience report offers valuable insights into assessing students' implementation of object-oriented concepts and design patterns.