Computing Confidence Score for Neural Network Predictions from Latent Features

As machine learning (ML)-based systems become more commonplace in our daily lives, users should be able to trust the predictions made by the ML models. Inappropriate predictions may have disastrous effects depending on the context in which the ML model is applied. To decide whether to trust a classifier prediction or not, current methods use the probabilities associated with the model predictions or nearest neighbors of the predicted label. In this study, we present experiments that show how the distribution statistics of the latent space features can be used to accurately compute the output confidence of a model.Our method involves first mapping the latent space feature values from the penultimate layer of a model to a hypothesized prior distribution and then storing the parameters of the prior distributions for each node. The degree to which a trained model’s penultimate layer activations for a given sample match the previously stored distribution parameters for the predicted label determines the level of confidence associated with that prediction. We demonstrate the effectiveness of utilizing different statistical properties of the latent space activations in our confidence scoring algorithm and compare our algorithm’s results against Softmax probabilities based confidence scores on the CIFAR-10 and MNIST datasets. The results of the experiment show that the confidence score computation based on latent space activation statistics performs better than the Softmax based approach.