Enhancing a Free-Text Adaptive Computer Assisted Assessment System with Self-Assessment Features

Willow is a Web-based application which automatically and adaptively assesses students' free-text answers written in Spanish and English. It is intended to help students review concepts outside of class, and provides an alternative assessment method. However, students tend to be greatly concerned by computer generated scores and, although they have been told that the system is not intended for summativepurposes and their teachers will not have access to that scores, they remain worried that the machine could make a mistake and this mistake could have a negative impact on their final course grade. Given that automatic scoring cannot assure 100% approximation to a human teacher's assessment, we have introduced self-assessment features in Willow. Thanks to the combination of automatic free-text scoring and self-assessment, students have more control over their evaluation and, whenever they do not agree with the automatic score assigned by the system, they can change it. This new feature has been tested by 22 students reviewing a course with Willow in the 2007-2008 academic year. We observed that the students did not abuse the self-assessment feature, only modifying the score assigned in 5% of answered questions.