Reflection for Recovery: Exam Wrappers in an Object-Oriented Software Development Course Help Struggling Students Improve Future Exam Scores

Can student reflection play a beneficial role in pursuit of a career in software development? In this paper, we report on a study of the effectiveness of one reflective tool, “exam wrappers,” for teaching a software engineering (SE) and computer science (CS) fundamentals course focused on object-oriented design and programming. In general, reflection is an area of growing interest in SE. We see it, for example, in the use of retrospectives in Scrum. Reflection is increasingly being used in the science and engineering college courses, after a long history in liberal arts. We collected exam scores for nearly a thousand (957) students who took the course. This relatively large sample of exam scores allows us to perform statistical analysis to measure effects otherwise challenging to detect. Because exam wrappers are used toward a specific, measurable end, this is one case where statistics can be used to judge the effectiveness of a type of reflection. Having a way to measure the results of this effort contributes to the science of education. We compare student performance on assessments in several ways, focusing our analysis mainly on comparisons between the terms (quarters) where these wrappers were used, and prior years when they were not used. In terms when an exam wrapper was present, our results show a greater relative improvement from the first exam to the final exam for low-scoring students than in terms when the exam wrapper was absent. This result is statistically significant and lends support to the hypothesis that exam wrappers can improve student outcomes. We discuss potential confounding factors and also the implications of this work for SE and CS course development.