Uncertainty Quantification in Neural Networks Using Stochastic Differential Equations

Uncertainty quantification plays a crucial role in reduction of uncertainties during optimization and decision making, however, it is not solved yet in deep learning. Bayesian network is often used for uncertainty estimation but is only applicable to cases with a small number of parameters because the prior probability of the parameters is needed. On the other hand, while some non-Bayesian models have solved the parameter limitation problem, they often fail to separate different sources of uncertainties. In order to address these issues, we propose the Reduce Bias Stochastic Differential Equation Network (RB-SDENet) by adding a stochastic term to the existing ODE network, which enables us to capture epistemic uncertainty through the variance of random motion. And the probabilistic distribution of output can represent the aleatoric uncertainty for the model. Additionally, we ensured the stability of the model through the mathematical analysis of uniqueness.