Characterization of Assessments on a First Programming Course in Higher Education

The present research introduces a characterization of the questions in summative assessments (tests) for an introductory programming course in higher education, developed for first-year engineering students. This characterization is a first approach that examines the disparity between the thematic contents declared in the course syllabus and those effectively present in the assessments. It also explores the difficulty level and context in which each question is presented. Additionally, a detailed review of the course's process for test building was performed, and its results allow one to conclude that the amount of time dedicated to the test building itself might be too scarce. Some of the results of this research expose that 36.8% of the declared contents in the course syllabus have been explicitly evaluated in tests, using mainly simulated situations as context for the questions (52.9%). The possibility that there might be a link between the difficulty of a question, its contents, and the given context also emerges; nonetheless, more research is required to properly assess this claim. The timeframe considered in this research includes from the second half of 2015 until the second half of 2019, in pre-pandemic conditions.