Large Programming Task vs Questions-and-Answers Examination in Java Introductory Courses

This study investigates two different forms of examination in introductory Java programming courses: computer-based examination where the students are given one relatively large programming task only, and paper-/computer-based examination where the students are given a number of smaller questions to answer. The study focuses on identifying how well the two different examination forms reveal the students' practical skills and theoretical knowledge in relation to the intended learning outcomes specified in the course syllabus. Course syllabuses from 8 Swedish universities are examined and from these, 4 specified learning outcomes, each shared by at least 5 of the selected universities, are identified. These learning outcomes are then used to analyze the exams from two of the universities, one practicing the large programming task examination form, and the other is using the questions and answer examination form. For both examination forms, two course phases are analyzed with respect to the specified learning outcomes: 1) exam (e.g. what do the questions capture and how are they formulated), and 2) execution (e.g. what do the students answer and how well are the intended purposes of the questions fulfilled). The results illustrate the strengths and weaknesses with the two examination forms. The results can be used as decision support when selecting examination form and, furthermore, to improve both forms to attain more comprehensive examinations, with respect to the specified learning outcomes.