Sequential Monte Carlo for model selection and estimation of neural networks

We address the complex problem of sequential Bayesian learning and model selection for neural networks. This problem does not usually admit any type of closed-form analytical solution and, as a result, one has to resort to numerical methods. We propose here an original sequential simulation-based strategy to perform the necessary computations. It combines sequential importance sampling, a selection procedure, variance reduction techniques and reversible jump Markov chain Monte Carlo (MCMC) moves. We demonstrate the effectiveness of the method by applying it to radial basis function networks. The approach can be easily extended to other interesting on-line model selection problems.