A dynamic model selection algorithm for mixtures of Gaussian processes

The mixture of Gaussian processes (MGP) is a powerful and widely used model in machine learning. However, it remains a challenging problem to determine the actual number of GP components in the mixture, i.e., the model selection problem. Synchronously Balancing Criterion (SBC) has been recently proposed and shown to be effective for the model selection of MGPs, but it is rather time consuming to use SBC directly since we need to repeat the conventional learning process on a large number of candidate models. In this paper, based on the convexity of the negative SB Criterion objective function, we propose a dynamic model selection algorithm under the framework of the hard-cut EM algorithm with the GP number dynamically changed step by step according to the increase of SBC. It is demonstrated by the experiments on some typical synthetic datasets and an artificial toy dataset that our proposed algorithm is not only much more efficient on implementation time, but also more effective on model selection, in comparison with the conventional SBC based model selection method.