An Improved ConvNeXt with Multimodal Transformer for Physiological Signal Classification

With escalating mortality rates associated with cardiovascular disease, the early detection of arrhythmias assumes ever-increasing significance. This study introduces a novel multimodal network that concurrently classifies electrocardiogram (ECG) and wrist pulse signal (WPS). Both ECG and WPS, as human physiological signals, share closely related distributions and characteristics, holding potential to accurately reflect underlying cardiovascular conditions. The proposed ICMT-Net utilizes continuous wavelet transform to partition 5-second ECG and WPS segments into spectrograms. It incorporates an improved ConvNeXt, a multimodal transformer layer, and a fused multi-layer perceptron to extract and fuse multimodal features for ECG classification. Subsequently, the network is adapted to WPS and coronary heart disease classification tasks through transfer learning techniques. In comparison to existing methods, our approach achieves heightened sensitivity in detecting supraventricular and ventricular ectopic segments, while also outperforming established WPS classification methodologies. Importantly, the proposed network adeptly handles multimodal signals and excels in classification accuracy, particularly within the realm of physiological signals.