A Real-Time Emotion-Aware System Based on Wireless Body Area Network for IoMT Applications

The Internet of Medical Things (IoMT) stimulates the development of intelligent medical applications. As mental disorders become a global problem, emotion recognition has received widespread attention, as it can contribute to more comprehensive mental health monitoring and psychological assessment. Physiological signal-based emotion-aware monitoring is a particularly promising application due to its non-invasive and objective data collection. Recently, multi-modal emotion recognition has been enhanced with Wireless Body Area Network (WBAN) access to IoMT, where wireless medical sensors are interconnected and abundant signals are acquired conveniently. However, how to synthesize these multi-source physiological signals to facilitate emotion recognition is a challenging problem due to their heterogeneity and interference. To solve this problem, we propose a real-time Differential Multi-modal Transformer (Diff-MT), where the main components are the Differential Hyper-information Extraction (DHE) module, the Multi-modal Global Cross-attention Encoder (MGCE) and the Difference-augmented Feature Fusion (DFF). Ultimately, we endow the system with emotional awareness and distribute the state to IoMT devices. Extensive experiments demonstrate that the proposed Diff-MT exhibits superior performance compared to existing methods on the WESAD and DEAP datasets and is appropriate for IoMT-based healthcare.