EEG Microstates and fNIRS Metrics Reveal the Spatiotemporal Joint Neural Processing Features of Human Emotions

Emotions deeply influence human behavior and decision-making. Currently, the spatiotemporal joint neural processing pattern of human emotions remains largely unclear. This study employed EEG-fNIRS simultaneous recordings to capture the spatiotemporal neural processing characteristics of human emotions by presenting Chinese emotional video stimuli. 1) EEG microstates’ temporal dynamic: Compared to low emotional arousal, microstate C and D's activities and C $\rightleftharpoons$ D's transition probability significantly increased. 2) fNIRS spatial patterns: Compared to low arousal, the dorsolateral prefrontal cortex (DLPFC), inferior frontal gyrus (IFG), and temporoparietal junction (TPJ)'s oxygenated hemoglobin (HbO) concentrations significantly increased, along with the significant increase of DLPFC-IFG&TPJ's functional connectivity during high arousal. Compared to high valence, the DLPFC was significantly activated during low valence. 3) EEG-fNIRS spatiotemporal joint features: Compared to low arousal, there was a significant positive correlation between the occurrence of microstate D (corresponded to dorsal attention network, DAN) and HbO concentrations of DLPFC (DAN's key node) during high arousal, which consistently revealed the DAN's involvement for human emotional arousal processing. These results could provide not only multimodal complementary features for promoting the development of affective brain-computer interface, but also potential objective spatiotemporal neural markers for emotional disorders.