Estimation and Applications of Quantiles in Deep Binary Classification

Conditional quantiles obtained via regression are used as a robust alternative to classical conditional means in econometrics and statistics, as they can capture the uncertainty in a prediction, and model tail behaviors, while making very few distributional assumptions. In this work, we extend the notion of conditional quantiles to the binary classification setting—allowing us to quantify the uncertainty in the predictions, increase resilience to label noise, and provide new insights into the functions learnt by the models. We accomplish this by defining a new loss called binary quantile regression loss. We compute the Lipschitz constant of the proposed loss and show that its curvature is bounded under some regularity conditions. These properties are later used to characterize the error rates of the learning algorithms and to accelerate the training regime with using Lipschitz adaptive learning rates. We leverage the estimated quantiles to obtain individualized confidence scores that provide an accurate measure of a prediction being misclassified. We aggregate these scores to provide two additional metrics, namely, confidence score and retention rate, which can be used to withhold decisions and increase model accuracy. We also study the robustness of the proposed nonparametric binary quantile classification framework, and finally, we demonstrate that quantiles aid in explainability as they can be used to obtain several univariate summary statistics that can be directly applied to existing explanation tools.