Neural Networks for Astronomical Data Analysis and Bayesian Inference

We present our generic neural network training algorithm, called Sky Net and the accelerated Bayesian inference algorithm, BAMBI. Sky Net combines multiple techniques already developed individually in the literature to create an efficient and robust machine-learning tool that is able to train large and deep feed-forward neural networks for use in a wide range of learning applications, such as regression, classification, density estimation, clustering and dimensionality reduction. Sky Net uses a powerful `pre-training' method, to obtain a set of network parameters close to the true global maximum of the training objective function, followed by further optimisation using an automatically-regularised variant of Newton's method that uses second-order derivative information to improve convergence, but without the need to evaluate or store the full Hessian matrix, by using a fast approximate method to calculate Hessian-vector products. This combination of methods allows for the training of complicated networks that are difficult to optimise using standard back propagation techniques. The blind accelerated multimodal Bayesian inference (BAMBI) algorithm implements the MultiNest package for nested sampling as well as the training of an artificial neural network by Sky Net to learn the likelihood function. In the case of computationally expensive likelihoods, this allows the substitution of a much more rapid approximation in order to increase significantly the speed of the analysis. Astrophysical examples are provided for both Sky Net and BAMBI.