CoAffinity: A Multimodal Dataset for Cognitive Load and Affect Assessment in Remote Collaboration

Understanding the relationship between cognitive load and affective state in remote work is vital for designing intuitive collaboration. We present CoAffinity, a multimodal dataset encompassing eight structured remote-work tasks, during which 39 participants provided self-reported measures (arousal, valence, positive/negative affect, and cognitive-load) while being recorded via audio, video, and physiological signals (PPG and GSR). Spanning over 38 hours of annotated data, our approach involved precise timestamp alignment, short and long-session labelling, and subsequent machine-learning and deep-learning benchmarks. Key findings show that integrating multiple modalities, especially physiological data, significantly improves the detection of cognitive load and emotion, while group synchrony metrics highlight how physiological coherence shifts under varied task demands. By capturing complex cognitive-emotional dynamics in realistic remote settings, CoAffinity aims to advance affective computing, inform human-computer interaction research, and foster more empathetic remote collaboration tools.