Multi-modalities Analysis In Profiled Learning

The advances in information technologies offer a promising approach to leveraging effective and engaging learning experiences. The diversity of sensor-based technologies, such as facial expression analysis and gaze tracking, has introduced the opportunity to capture students’ interactions with learning activities or assessments. In such context, captured data hold meaningful promise for gathering a deeper understanding of students’ learning experience and informing an adaptive frame to support individualized learning needs. This paper introduces an analytic approach that incorporates students’ eye tracking, facial expression, and mouse movement data to predict students’ performance while resolving an English exam. The proposed system examined the degree to which different modalities captured from 53 students (aged 18 to 22) in an authentic learning environment were predictive of the students’ exam scores. The analysis of the collected dataset shows that adding student features has effectively predicted his assessment score. The adoption of different regression models revealed that multi-modal data could accurately predict students’ exam scores and hold significant potential for guiding a real-time adaptive environment.