Towards Mobile Cognitive Fatigue Assessment as Indicated by Physical, Social, Environmental, and Emotional Factors

This research sought to establish which in-situ measures of cognitive fatigue, physical activity, social interaction, location, emotional state and facial landmarks, made using a smartphone application, could be used to indicate episodes of cognitive fatigue. This assessment was realised using cognitive tests (assessing memory, attention, reaction time, information processing speed and executive function), self-assessment, contextual factors and facial feature analysis. This study also investigated the use of an ensemble algorithm for the classification of cognitive fatigue utilising facial features and a Rotation Forest approach. Self-assessment of cognitive fatigue was shown to directly correlate with reaction time through a Psychomotor Vigilance Task (r = .643, p = .001), and self-reported increases in the level of social activity (r = .377, p = .001). Facial feature analysis revealed dominant emotions of sadness and anger when participants were cognitively fatigued. It also revealed underlying facial cues that indicated higher levels of cognitive fatigue including expressions of negative valence, and Facial Action Coding System units of increased brow furrow, eyelid tightening and lip suck. In addition, a Principle Component Analysis based Rotation Forest ensemble with a ternary output demonstrated a cognitive fatigue classification accuracy of 82.17%. The findings presented indicate that the inclusion of data relating to surrounding cognitive, social, physical and emotional factors can improve the accuracy of mobile in-situ cognitive fatigue assessment using our previously validated smartphone-based cognitive fatigue assessment approach. The findings further suggest gross-level fatigue status may be potentially classified to a reasonable degree of accuracy using facial features, which may give rise to personalised in-situ fatigue detection.