Decoding a Music-Modulated Cognitive Arousal State using Electrodermal Activity and Functional Near-infrared Spectroscopy Measurements

Biofeedback systems sense different physiological activities and help with gaining self-awareness. Understanding music’s impact on the arousal state is of great importance for biofeedback stress management systems. In this study, we investigate a cognitive-stress-related arousal state modulated by different types of music. During our experiments, each subject was presented with neurological stimuli that elicit a cognitive-stress-related arousal response in a working memory experiment. Moreover, this cognitive-stress-related arousal was modulated by calming and vexing music played in the background. Electrodermal activity and functional near-infrared spectroscopy (fNIRS) measurements both contain information related to cognitive arousal and were collected in our study. By considering various fNIRS features, we selected three features based on variance, root mean square, and local fNIRS peaks as the most informative fNIRS observations in terms of cognitive arousal. The rate of neural impulse occurrence underlying EDA was taken as a binary observation. To retain a low computational complexity for our decoder and select the best fNIRS-based observations, two features were chosen as fNIRS-based observations at a time. A decoder based on one binary and two continuous observations was utilized to estimate the hidden cognitive-stress-related arousal state. This was done by using a Bayesian filtering approach within an expectation-maximization framework. Our results indicate that the decoded cognitive arousal modulated by vexing music was higher than calming music. Among the three fNIRS observations selected, a combination of observations based on root mean square and local fNIRS peaks resulted in the best decoded states for our experimental settings. This study serves as a proof of concept for utilizing fNIRS and EDA measurements to develop a low-dimensional decoder for tracking cognitive-stress-related arousal levels.