CEAP-360VR: A Continuous Physiological and Behavioral Emotion Annotation Dataset for 360$^\circ$ VR Videos

Watching 360
$^\circ$
videos using Virtual Reality (VR) head-mounted displays (HMDs) provides interactive and immersive experiences, where videos can evoke different emotions. Existing emotion self-report techniques within VR however are either retrospective or interrupt the immersive experience. To address this, we introduce the
Continuous Physiological and Behavioral Emotion Annotation Dataset for 360$^\circ$ Videos (CEAP-360VR)
. We conducted a controlled study (N=32) where participants used a Vive Pro Eye HMD to watch eight validated affective 360
$^\circ$
video clips, and annotated their valence and arousal (V-A) continuously. We collected (a) behavioral (head and eye movements; pupillometry) signals (b) physiological (heart rate, skin temperature, electrodermal activity) responses (c) momentary emotion self-reports (d) within-VR discrete emotion ratings (e) motion sickness, presence, and workload. We show the consistency of continuous annotation trajectories and verify their mean V-A annotations. We find high consistency between viewed 360
$^\circ$
video regions across subjects, with higher consistency for eye than head movements. We furthermore run baseline classification experiments, where Random Forest classifiers with 2s segments show good accuracies for subject-independent models: 66.80% (V) and 64.26% (A) for binary classification; 49.92% (V) and 52.20% (A) for 3-class classification. Our open dataset allows further experiments with continuous emotion self-reports collected in 360
$^\circ$
VR environments, which can enable automatic assessment of immersive Quality of Experience (QoE) andmomentary affective states.