Evaluation of Post Hoc Uncertainty Quantification Approaches for Flood Detection From SAR Imagery

Deep neural networks are the current state-of-the-art for analysis of remote sensing imagery. While they often provide accurate results, they are usually prone to be overconfident with respect to their predictions. In particular when these predictions are used by human decision makers in high stake scenarios, e.g., during detection and monitoring of natural disasters, trustworthiness is a necessary feature. In the context of flood detection from SAR imagery, this work evaluates a variety of uncertainty quantification methods that are applicable to already trained models (i.e., post hoc approaches) and provides detailed experiments evaluating the quantification quality of the different methods. The results show typical and pronounced uncertainty features, such as class boundaries, inaccurate predictions, and out-of-distribution regions. In summary, the overall best epistemic estimations are (albeit with only a small margin) provided by student ensembles, followed closely by the well established Monte Carlo dropout quantifier. However, surprisingly nearly all evaluated methods (incl. using the original softmax estimate of the trained model) provide rather similar results.