A CNN-Transformer-Based Multimodal Model for Cardiac Abnormal Signal Detection

Cardiovascular disease detection plays a crucial role in medical diagnosis, and deep learning models have shown promise in analyzing physiological signals such as electrocar-diogram (ECG) and phonocardiogram (PCG). In this study, we propose a multimodal model that combines a convolutional neural network (CNN) and Transformer. Our model harnesses the information embedded in both ECG and PCG signals, thereby enhancing the detection capability of cardiac abnormalities. A channel attention mechanism is introduced in the CNN module, while the Transformer module captures long-range dependencies. Experimental results on a real world dataset show the effectiveness of our approach. Our model is also designed to accommodate other modalities, such as EEG and tabular data. The proposed model offers a flexible and effective solution for multimodal cardiac abnormality detection.