Multimodal Neural Network for Recognition of Cardiac Arrhythmias Based on 12-Load Electrocardiogram Signals

Automatic classification of heart rhythm disturbances using an electrocardiogram is a reliable way to timely detect diseases of the cardiovascular system. The need to automate this process is to increase the number of electrocardiogram signals. Classification methods based on the use of neural networks provide a high percentage of arrhythmia recognition. However, known classification methods do not take into account patient characteristics. The work proposes a multimodal neural network that takes into account the age and gender characteristics of the patient. It includes a Long short-term memory (LSTM) network for feature extraction on twelve-channel electrocardiogram signals and a linear neural network for processing patient metadata such as age and gender. Extraction of electrocardiogram signal features occurs in parallel with metadata processing. The last unifying layer of the proposed multimodal neural network integrates heterogeneous data and features of electrocardiogram signals obtained using an LSTM network. The developed multimodal neural network was verified using the PhysioNet/Computing in Cardiology Challenge 2021 ECG database. The simulation results showed that the proposed multimodal neural network achieves a recognition accuracy of 63%, which is 2 percentage points higher compared to state-of-the-art methods.