Recursive Bayesian Linear Regression for Adaptive Classification

This paper presents a new recursive Bayesian linear regression (RBLR) algorithm for adaptive pattern classification. This algorithm performs machine learning in nonstationary environments. A classification model is adopted in model training. The initial model parameters are estimated by maximizing the likelihood function of training data. To activate the sequential learning capability, the randomness of the model parameters is properly expressed by the normal-gamma distribution. When new adaptation data are input, sufficient statistics are accumulated to obtain a new normal-gamma distribution as the posterior distribution. Accordingly, a recursive Bayesian algorithm is established to update the hyperparameters. The trajectory of nonstationary environments can be traced to perform the adaptive classification. Such recursive Bayesian models are used to satisfy the requirements of maximal class margin and minimal training error, which are essential in support vector machines (SVMs). In the experiments on the UCI machine learning repository and the FERET facial database, the proposed algorithm outperforms the state-of-art algorithms including SVMs and relevance vector machines (RVMs). The improvement is not only obtained in batch training but also in sequential adaptation. Face classification performance is continuously elevated by adapting to changing facial conditions.