Using prefrontal cortex near-infrared spectroscopy and autonomic nervous system activity for identifying music-induced emotions

Physiological-based emotion identification systems may offer an alternative means of expressing emotions, particularly, for adults and youth with severe motor disabilities who may have little or no voluntary muscle control. The current study investigated inclusion of autonomic nervous system activity in combination with central nervous system activity in the form of a multi-modal emotion identification system. Prefrontal cortex hemodynamics were monitored using near infrared spectroscopy, and autonomic nervous system activity (ANS) was concurrently monitored using heart rate, skin temperature and electrodermal activity sensors, in a music-based emotion induction paradigm. Classifiers were trained using ANS or prefrontal hemodynamic features, in addition to dynamic modeling-based features. A combination of classifier decisions was applied for solving arousal (intense vs. neutral) and valence (positive vs. negative) classification problems. The classification accuracies of the ensemble varied substantially across participants (54.4%–85.1% for the arousal differentiation and 48.4%–76.8% for the valence differentiation). These results suggest the importance of individual specific detection algorithms in physiological-based emotion identification efforts. In addition, combining features from the autonomic and central nervous system resulted in a degradation of classification accuracies (68.3% in arousal and 58.5% in valence differentiation) compared to when prefrontal hemodynamic features were used exclusively (71.9% for both arousal and valence differentiation).