Using Consistency to Measure CS1 Student’s Comprehension of User-defined Functions

This study aims to investigate novice programmers' comprehension of user-defined functions. We have explored whether students are consistent (1) in their function declaration and the value returned by its implementation and (2) the value returned and its usage in the main program. Function consistency appears to be a good proxy for how well novices understand the role of user-defined functions in the main program, regardless of the ability to write a function's body that calculates a correct result. We analyzed the solutions submitted by 249 students to a practical exam question from a CS1 course that includes C functions. In terms of program consistency, 64% of students could make a good use of the function result in the driver program. However, one third of them call the same function multiple times without taking into account its computational cost. Inconsistent programs did not include a call (5%), treated an int function as if it was void(8%), or called it correctly but ignored the result (4%). The outcomes of this analysis bring to the fore the need to teach and assess user-defined functions in the context of building larger solutions, so that students could understand and consistently use modular code.