TransformerFusionNet: A Real-Time Multimodal Framework for ICU Heart Failure Mortality Prediction Using Big Data Streaming

This paper presents a real-time multimodal frame-work to enhance ICU mortality prediction for heart disease patients by integrating structured data, clinical notes, and big data streaming platforms. The proposed framework comprises two components: an offline multimodal model and a real-time pipeline. In the offline component, we proposed the Transformer-FusionNet model leverages BioBERT-Transformer for clinical notes and a Recurrent Neural Network (RNN) for structured data. The model integrates outputs through a concatenation layer to make predictions and is rigorously evaluated using the MIMIC-III dataset. It demonstrates superior performance across evaluation metrics and achieves an accuracy of 91.720%, precision of 91.85%, recall of 91.720%, and F1-score of 91.693%, outperforming other single and multimodal models. The real-time pipeline component incorporates Apache Spark and Apache Kafka for ingesting, preprocessing, and streaming structured data and clinical notes. This integration enables real-time mor-tality prediction with TransformerFusionNet, addressing critical gaps in the utilization of clinical notes in healthcare analytics. The framework significantly advances multimodal deep learning applications in healthcare by combining state-of-the-art models with scalable streaming technologies, offering a practical solution for improving clinical decision support systems and patient outcomes in real-time, especially in intensive care unit settings.