Survey on DL - Driven Multi-Modal Data in Cardiovascular Disease Prediction

One of the primary reasons of illness and mortality across the world is cardiovascular disease (CVD), for the timely detection of which the need for accurate detection methods persists. The last years have seen the development of cutting aged multimodal machine learning methods to enhance diagnosis accuracy that consider multiple types of data. In such scenarios, Phonocardiogram (PCG) signals and Electrocardiogram (ECG) images form integrated data. This review presents the summary of recent research works published on heart abnormality detection models based on a multimodal dataset by combining PCG and ECG data. The PCG signals capture acoustic heart information, including murmurs and other mechanical abnormalities, while ECG images documentation of the heart's electrical functions, aiding in the identification of arrhythmias and other rhythm-related issues. We delve into the various deep learning architectures used in implementing Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and process of attention in a complementary way to extract and combine features within these modalities. Building on the insights gained from existing literature in heart abnormality detection using multimodal datasets, we suggest a hybrid deep learning model that effectively integrates ECG pictures and PCG signals. This approach leverages the complementary nature of the two modalities to enhance detection accuracy and reliability.