Recursive Bayesian Regression Modeling and Learning

This paper presents a new Bayesian regression and learning algorithm for adaptive pattern classification. Our aim is to continuously update regression parameters to meet nonstationary environments for real-world applications. Here, a kernel regression model is used to represent two-class data. The initial regression parameters are estimated by maximizing the likelihood of training data. To activate online learning, we properly express the randomness of regression parameters as a conjugate prior, which is a normal-gamma distribution. When new adaptation data are enrolled, we can accumulate sufficient statistics and come up with a new normal-gamma distribution as the posterior distribution. We therefore exploit a recursive Bayesian algorithm for online regression and learning. Regression parameters are incrementally adapted to the newest environments. Robustness of classification rule is assured using online regression parameters. In the experiments on face recognition, the proposed regression algorithm outperforms support vector machine and relevance vector machine for different numbers of adaptation data.