WNOTNet: A Hybrid Wavelet Neural Operator and Transformer Framework for Enhanced EEG Denoising

Electroencephalography (EEG) signals play a vital role in brain science research and clinical diagnostics. However, these signals are often contaminated by physiological artifacts such as electromyography (EMG), electrooculography (EOG), electrocardiography (ECG), and nonphysiological noise such as white Gaussian noise (WGN), which substantially reduce the accuracy of analysis and decoding. In recent years, deep learning (DL) techniques have shown significant promise in EEG denoising tasks. However, existing methods struggle to capture the complex, nonlinear relationships inherent in EEG signals, particularly in multiscale feature extraction and time–frequency representation. To address these limitations, we propose WNOTNet, a novel model that integrates the wavelet neural operator (WNO) and transformer architecture. The WNO enables multiscale analysis and captures intricate time–frequency features, while the transformer effectively models long-range dependencies and global context. This integration significantly improves both denoising performance and generalization capabilities. Extensive experiments conducted on the publicly available EEGdenoiseNet dataset show that WNOTNet outperforms existing methods across multiple evaluation metrics, including time-relative root mean square error (T-RRMSE), spectral-relative root mean square error (S-RRMSE), correlation coefficient (CC), signal-to-noise ratio (SNR), and others. In real-world applications, WNOTNet was evaluated on practical EEG classification tasks, including sleep stage classification (MIT-BIH Polysomnographic dataset), driver fatigue detection, and emotion recognition. Notably, after denoising with WNOTNet, classification accuracy improved by 2.53% for sleep staging, 7.88% for driver fatigue detection, and 11.82% for emotion recognition. These experimental results demonstrate that WNOTNet can effectively enhance EEG signal quality and improve downstream classification accuracy. These findings indicate that WNOTNet provides robust and reliable denoising performance with strong generalization potential, making it suitable for practical EEG-based applications.