Exploring Uncertainty-aware Class-wise Thresholds for Recognition Model’s Uncertainty Detection

This study addresses the current limitations in artificial intelligence models, specifically their ability to accurately predict uncertainty, a crucial aspect for improving trustworthiness and safety. We introduce a technique for estimating uncertainty in zero-shot image classification using the vision-language model, CLIP. The standard method for quantifying uncertainty in classification models is to measure the entropy from normalized output value, which is then compared to a predetermined threshold to ascertain the certainty of the model’s output. We propose a method for adapting this approach for vision-language model-based zero-shot image classification to enhance uncertainty prediction through class-wise entropy thresholding. Experimental results reveal that class-wise entropy thresholding surpasses the previous approach. Additionally, it is demonstrated that our approach is successful for detecting out-of-distribution instances.