Generating Pool of Classifiers with Hyper-Parameter Optimization for Ensemble

Ensemble is an effective method to integrate diverse classifiers and take advantage of their individual strengths. Given a test sample, ensemble selects the most competent classifiers from a pool of candidates and predicts the result by combining the outputs of the selected classifiers. One of the key factors that affects the performance of ensemble is the criterion defined to evaluate the competence of classifier. The commonly used criterion is the prediction accuracy in the local region around the test sample. However, if the competence of all classifiers in the pool are not qualified, the prediction of the ensemble will be inaccurate. In this paper, we propose a hyper-parameter optimization based strategy to generate high quality classifiers. In order to automatically identify the optimal hyper-parameters, we explored six optimization algorithms that are based on Gaussian Process and Evolutionary theory. The experiment results show that Bayesian Optimization is the best algorithm for hyper-parameters optimization in terms of time consumption and classification accuracy. Furthermore, cooperating with the pool of classifiers that are configured with the optimized hyper-parameters, Kullback-Leibler Divergence and Randomized Reference Classifier are the two outstanding dynamic ensemble methods that can provide high classification accuracy.