Multi-Scale Uncertainty Calibration Testing for Bayesian Neural Networks Using Ball Trees

Bayesian neural networks (BNNs) offer an elegant and promising approach to quantifying the uncertainty of neural network predictions by providing predictive distributions. Although the potential of BNNs is considerable, established BNN training methods often result in inaccurate uncertainty estimation and local differences in quality depending on the considered input space region. To assess the efficacy of Bayesian models such as BNNs and gain insights into their predictive capabilities in distinct input space regions, we introduce a novel methodology that utilizes ball trees as a space partitioning data structure. Our approach enables the assessment of the predictive quality within specific regions of the input space across multiple scales in the input space, utilizing all nodes provided by the ball tree structure. Furthermore, our method allows the combination of results across different scales.