A Glucose-Specific Metric to Assess Predictors and Identify Models

In diabetes, the mean square error (MSE) metric is extensively used for assessing glucose prediction methods and identifying glucose models. One limitation of this metric is that, by equally treating errors in hypo-, eu-, and hyperglycemia, it is not able to weight the different clinical impact of errors in these three situations. In this paper, we propose a new cost function, which overcomes this limitation and can be used in place of MSE for several scopes, in particular for assessing the quality of glucose predictors and identifying glucose models. The new metric called glucose-specific MSE (gMSE) modifies MSE with a Clark error grid inspired penalty function, which penalizes overestimation in hypoglycemia and underestimation in hyperglycemia, i.e., the most harmful conditions on a clinical perspective. From a mathematical point of view, gMSE retains sensitivity of MSE and inherits some of its important mathematical features, in particular it has no local minima, simplifying the optimization. This makes it suitable for model identification purposes also. First, the goodness of it is demonstrated by means of three experiments, designed ad hoc to evidence its sensitivity to accuracy, precision, and distortion in glucose predictions. Second, a prediction assessment problem is presented, in which two real prediction profiles are compared. Results show that the MSE chooses the worst clinical situation, while gMSE correctly selects the situation with less clinical risk. Finally, we also demonstrate that models identified minimizing gMSE are more accurate in potentially harmful situations (hypo- and hyperglycemia) than those obtained by MSE.