Importance of Recalibrating Models for Type 2 Diabetes Onset Prediction: Application of the Diabetes Population Risk Tool on the Health and Retirement Study

A timely prediction of type 2 diabetes (T2D) onset is important for early intervention to prevent, or at least postpone, its incidence. Several models to predict T2D onset according to individual risk factors were proposed. However, their practical applicability is limited by the fact that they often perform suboptimally when applied to a different population. A solution to overcome this limitation is model recalibration, which consists in updating the model parameters. The aim of this work is to demonstrate the benefits of T2D predictive model recalibration. For the purpose, we considered as case study the Diabetes Population Risk Tool (DPoRT), originally tuned for the Canadian population, and we applied it to data collected in older Americans in the Health and Retirement Study (HRS). A subset of 30,274 subjects was extracted from HRS and divided into a training (N=24,219) and a test set (N=6,055) stratifying for sex and diabetes incidence. The DPoRT was recalibrated by re-estimating all model coefficients on the training set, and then assessed on the test set by comparing the performance of recalibrated vs original model. Model discriminatory ability and calibration were assessed by the concordance index (C-index) and the expected to observed event probability ratio (E/O), respectively. Results show that the recalibrated DPoRT presents similar discriminatory ability to the original model, with C-index equal to 0.68 vs. 0.67 in men, 0.73 vs. 0.73 in women, and better calibration than the original model, with E/O ratio equal to 0.75 vs. 4.57 in men, 0.81 vs. 2.53 in women. Results confirm that recalibration is a key step to be performed before the application of predictive models to different populations in order to guarantee an accurate prediction of diabetes incidence.