External validation of a deep learning prediction model for in-hospital mortality among ICU patients

With increasing hospital adoption of electronic health record (EHR) systems worldwide, a massive amount of EHR data are generated in intensive care practice, and deep learning models are increasingly applied in mortality prediction. However, due to the lack of external validation, it’s difficult to generalize the deep learning models in critical care settings. Our previous work proposed a routinely collected data based deep learning model for intensive care unit (ICU) mortality prediction. This study aimed to externally validate the model using a cohort from the published MIMIC III data set so as to examine the generalizability and feasibility of the model. With little changed in the modeling, the deep learning based model achieved a high accuracy (AUROC=0.90; AUPRC=0.70), and good calibration properties which was reflected by a brier score of 0.070, in the external validation database. The model’ excellent performance in our external validation cohort provides more evidence for the application of the deep learning model in clinical practice.