EmoDetect: A Learning-Centred Affective Database for Detecting Student Frustration in Online Learning

The role of facial expressions in human interaction is essential, with significant strides being made in the development of systems designed to automatically recognise these expressions in the context of human-computer interaction (HCI). However, there is a noticeable lack of databases dedicated to training these automatic detectors for learning-centric affect detection in online learning. Existing annotated databases often draw primarily on images sourced from movies, the internet, or simulated expressions in controlled laboratory environments, and they fail to capture learners’ facial expressions at different stages and in varying contexts of genuine learning experiences. To address this gap, we present EmoDetect, a novel database comprising video recordings of 30 students’ facial expressions and screen-captured interactions with an online learning system as they engage in online learning tasks. By analysing these videos, we have identified facial action units related to frustration and discovered new action units and body language specific to learning that can aid in detecting learning-centred frustration. These findings enhance our understanding of automatic recognition of students’ affective states in online learning and offer valuable data for future research.