ES-GP: An Ensemble Surrogate-Assisted Genetic Programming Approach to Image Classification

Genetic Programming (GP) is a promising evolutionary machine learning technique for image classification, known for its ability to evolve flexible, effective, and interpretable models. However, the high computational cost of fitness evaluations in evolutionary learning restricts its practical applications. While Surrogate models offer efficient approximations for costly fitness evaluations, their application in GP-based image classification remains in its early stages, facing challenges such as handing flexible tree-based representations with variable lengths, designing an effective surrogate, and the limited performance of a single surrogate across various image classification tasks. To address these issues, this paper proposes an ensemble surrogate-assisted GP approach to image classification. The new approach constructs one global surrogate model to explore broad areas and three local surrogate models within specific subspaces to exploit local regions, enabling more accurate predictions of GP individuals’ fitness. Moreover, a dynamic weighting strategy is developed to assign different weights to the base surrogate models in the ensemble, improving prediction accuracy. Additionally, the proposed approach refines the surrogate training set1 construction method, previously limited to single-tree GP, enabling it to accelerate both single-tree and multi-tree GP 2 for image classification. Experimental results on five datasets of varying difficulty demonstrate that the new ensemble surrogate method significantly reduces the number of expensive fitness evaluations of both single-tree and multi-tree GP-based image classification methods while achieving competitive performance. The comparisons with other state-of-the-art methods also confirm its effectiveness.