Re-Thinking Classification Confidence with Model Quality Quantification

Deep neural networks using for real-world classification task require high reliability and robustness. However, the Softmax output by the last layer of network is often over-confident. We propose a novel confidence estimation method by considering model quality for deep classification models. Two metrics, MQ-Repres and MQ-Discri are developed accordingly to evaluate the model quality, and also provide a new confidence estimation called MQ-Conf for online inference. We demonstrate the capability of the proposed method by the $3D$ semantic segmentation tasks using three different deep networks. Through confusion analysis and feature visualization we show the rationality and reliability of the model quality quantification method.¹¹This work is supported by the National Natural Science Foundation of China under Grant U22A2061 and High-performance Computing Platform of Peking University.