2PS: A novel swarm based selection algorithm for ensemble learning problem

Discovering the effective subset of models in a pool of classifiers is an important and remarkable topic in ensemble learning scope. Using meticulously selected subset instead of entire ensemble leads to more efficient and effective results. This paper introduces a novel hybrid ensemble selection method of firefly and forward search algorithms. Because of the two different selection phases in the proposed method, it is called 2PS (Two-Phase Selection) method. Empirical comparisons of the method 2PS and two similar methods are performed on ten standard machine learning problems. The results show that the method 2PS leads to 5.63% average accuracy improvement compared to rivals. This great success is due to diversity balancing and then error correcting capability of 2PS, which is due to the nature of its firefly algorithm. Moreover, 2PS achieves second great success in overhead reduction by excluding redundant and weaker models in prediction which is due to its forward search algorithm.