Comparison of Structured and Free-text Based Features for Rehospitalization Prediction among Patients with Severe Mental Illness

Background: Clinical risk prediction is to help identify the patients with high risk of poor outcomes. The creation of predictive models heavily relies on structured data. It is not well understood the benefit of adding free-text features in improving risk prediction. Objectives: We aimed to predict the 30-day and 3-year rehospitalization risk among patients with severe mental diseases and to compare predictive performances of using structured data alone, unstructured data alone, and combined structured and unstructured data. Methods: Veterans with ≥2 diagnoses of serious mental diseases including schizophrenia (SZ), schizoaffective disorder (SAD), bipolar disorder (BP), and major depressive disorder (MDD) were sampled from the Veteran Healthcare Administration (VHA) databases. Topic modeling methods were used to process unstructured data to identify stable topics. Correlation tests and automatic stepwise models were used to select features. Results: Among 139,830 patients with severe mental disease, 76.2% diagnosed with MDD, 23.4% for BP, 8.3% for SZ, and 3.5% for SAD. Regardless of 30-day and 3-year rehospitalization outcomes, patients with following characteristics were at higher risk of hospitalization: male, homeless, having a condition of SAD, primarily diagnosed with and treated for alcohol withdrawal, and diagnosed with cocaine abuse. Besides, patients who had topics related to homeless, suicide, and substance abuse were also at higher risk. In all models, the prediction accuracy based on the individual level measured by c-statistic estimate was in the range of 0.6-0.7 regardless of features only from structured data, only from unstructured data, or from both and unstructured data. Conclusion: Topics identified from the unstructured data made contribution to predicting short-term and long-term rehospitalization. Their importance was as good as the features identified from the structured data such as demographics, ICDs, CPTs, and medications.