Towards an Optimal Affect-Sensitive Instructional System of cognitive skills

While great strides have been made in computer vision toward automatically recognizing human affective states, much less is known about how to utilize these state estimates in intelligent systems. For the case of intelligent tutoring systems (ITS) in particular, there is yet no consensus whether responsiveness to students' affect will result in more effective teaching systems. Even if the benefits of affect recognition were well established, there is yet no obvious path for creating an affect-sensitive automated tutor. In this paper we present the first steps of the OASIS project, whose goal is to develop Optimal Affect-Sensitive Instructional Systems. We present results of a pilot study to develop affect-sensitive tutors of “cognitive skills”. The study was designed to: (1) assess the importance of affect to teaching, and also (2) collect training data with ecological validity that could later be used to develop an automated teacher. Experimental results suggest that affect-sensitivity is associated with higher learning gains. Behavioral analysis using automatic facial expression coding of recorded videos also suggests that smile may reveal embarrassment rather than achievement in learning scenarios.