Study of selective ensemble learning method and its diversity based on decision tree and neural network

Diversity among base classifiers is known to be a necessary condition for improving ensemble learning performance. In this paper, methods of selective ensemble learning including hill-climbing selection, ensemble forward sequential selection, ensemble backward sequential selection and clustering selection are studied. To measure the diversity among base classifiers in ensemble learning, the entropy E is selected as measuring method of diversity. The results of experiment show that classifiers which have the highest diversity are obtained using selective methods, and the ensemble performance is superior to the best single classifier. In addition, the classifiers selected by clustering selective technology also have the above characteristics, and the changes of the diversity are smaller when the accuracy has smaller fluctuations. Meanwhile, the number of clusters also impacts on the ensemble performance.