A Multi-Tree Genetic Programming-Based Ensemble Approach to Image Classification With Limited Training Data [Research Frontier]

Large variations across images make image classification a challenging task; limited training data further increases its difficulty. Genetic programming (GP) has been considerably applied to image classification. However, most GP methods tend to directly evolve a single classifier or depend on a predefined classification algorithm, which typically does not lead to ideal generalization performance when only a few training instances are available. Applying ensemble learning to classification often outperforms employing a single classifier. However, single-tree representation (each individual contains a single tree) is widely employed in GP. Training multiple diverse and accurate base learners/classifiers based on single-tree GP is challenging. Therefore, this article proposes a new ensemble construction method based on multi-tree GP (each individual contains multiple trees) for image classification. A single individual forms an ensemble, and its multiple trees constitute base learners. To find the best individual in which multiple trees are diverse and effectively cooperate, i.e., the nth tree can correct the errors of the previous n-1 trees, the new method assigns different weights to multiple trees using the idea of AdaBoost and performs classification via weighted majority voting. Furthermore, a new tree representation is developed to evolve diverse and accurate base learners that extract useful features and conduct classification simultaneously. The new approach achieves significantly better performance than almost all benchmark methods on eight datasets. Additional analyses highlight the effectiveness of the new ensembles and tree representation, demonstrating the potential for providing valuable interpretability in ensemble trees.