Low-shot, Semi-supervised, Uncertainty Quantification Enabled Model for High Consequence HSI Data

In this work we introduce Bootstrapped Paired Neural Networks (BPNN), a semi-supervised, low-shot model with uncertainty quantification (UQ). BPNN can be used for classification and target detection problems commonly encountered when working with aerospace imagery data, such as hyperspectral imagery (HSI) data. When collecting aerospace imaging data, there is often large amounts of data which can be costly to label, so we would like to supplement labeled data with the vast unlabeled data (often > 90% of the data) available, we do this using semi-supervised techniques (Exponential Average Adversarial Training). Often, it is difficult and costly to obtain the sample size necessary to train a deep learning model to a new class or target; using paired neural networks (PNN), our model is generalized to low- and no-shot learning by learning an embedding space for which the underlying data population lives, this way additional labeled data may not be necessary to detect for targets or classes which weren't originally trained on. Finally, by bootstrapping the PNN, the BPNN model gives an uncertainty score on predicted classifications with minimal statistical distributional assumptions. Uncertainty is necessary in the high consequence problems that many applications in aerospace endure. The model's ability to provide uncertainty for its own predictions can be used to reduce false alarms rates, provide explainability to black box models, and help design efficient future data collection campaigns. Although models exist to contain two of these three qualities, to our knowledge no model contains all three: semi-supervised, low-shot, and uncertainty quantification. We generate a HSI scene using a high fidelity data simulator that gives us ground truth radiance spectra, allowing us to fully assess the quality of our model and compare to other common models. When applying PBNN to our HSI scene, it outperforms in target detection against classic methods, such as Adaptive Cosine Estimator (ACE), simple deep learning models without low-shot or semi-supervised capabilities, and models using only low-shot learning techniques such as regular PNN. When extending to targets not originally trained on, the model again outperforms the regular PNN. Using the UQ of predictions, we create “high confidence sets” which contain predictions which are reliably correct and can help suppress false alarms. This is shown by the higher performance of the “high confidence set” at particular constant false alarm rates. They also provide an avenue for automation while other predictions in high consequence situations might need to be analyzed further. BPNN is a powerful new predictive model that could be used to maximize the data collected by aerial assets while instilling confidence in model predictions for high consequence situations and being flexible enough to find previously unobserved targets.