The Ember Convergence Coefficient (ECC): A Proposed Statistical Framework for Evaluating Deep Learning Models

The evaluation of deep learning models typically relies on final-state metrics like accuracy and loss, often neglecting the crucial insights from the model's learning trajectory during training. Assessing training convergence is vital for ensuring model reliability, yet conventional methods like the visual inspection of loss curves lack quantitative rigor and often fail to capture complex training dynamics. To address this gap, this research introduces the Ember Convergence Coefficient (ECC), a novel metric designed to provide a quantitative, holistic assessment of the training process. The ECC is formulated by integrating four weighted statistical variables: (1) trainvalidation performance correlation, (2) the correlation between epochs and the train-validation performance gap, (3) the correlation between epochs and mean performance, and (4) the overall performance shift. The weights for these variables are derived from a distribution analysis of models trained on a cataract image classification dataset. The coefficient's efficacy is validated on a separate tuberculosis classification dataset, where the ECC scores demonstrate a strong alignment with conclusions drawn from the direct visual inspection of training graphs. Consequently, the ECC serves as a robust and practical indicator of learning quality, empowering engineers to make more informed model selections by supplementing final-state metrics with a standardized score for convergence behavior.