Wearable Ear-Centric Physiological Sensing with On-Edge Physics-Informed Neural Networks for Negative Mental State Detection

Monitoring mental states is essential for understanding physiological and neurological health, offering insights that enhance diagnostics and therapy. However, current systems are often bulky, immobile, and reliant on external devices for deep learning (DL), limiting real-time applications. This study introduces a compact, real-time mental state monitoring system with four innovations. (1) First, a wearable ear-centric device captures multimodal biosignals—behind-the-ear (BTE) electroencephalography (EEG) and in-ear photoplethysmography (PPG)—enhancing portability. (2) Second, a high-resolution superlet transform (HRST) converts these signals into spectrogram images to enrich input features for DL models. (3) Third, the PhysAttenConvLSTM model, designed for edge deployment using Tiny Machine Learning (TinyML), integrates convolutional neural networks (CNNs), long short-term memory (LSTM), self-attention, and a physics-informed merge layer (FIML) to classify affective states efficiently. All processes—from signal acquisition to classification—occur directly on the edge device, enabling low-latency, real-time operation without external computation. (4) Fourth, an Internet of Medical Things (IoMT) framework is implemented for data storage, remote access, and system management. Multimodal data were collected from 25 participants during social media-based mental state induction tasks. The system achieved 92.07% accuracy in leave-one-out cross-validation (LOOCV) and 95.05% in 10-fold cross-validation (CV), with strong specificity, sensitivity, precision, and F1 scores. This research advances edge DL, multimodal biosignal fusion, and IoMT integration, offering an ultra-portable, energy-efficient solution for real-time mental state monitoring, with potential in affective computing and personalized mental health care.