Evolutionary Sequential Transfer Learning for Multi-Objective Feature Selection in Classification

Over the past decades, evolutionary multi-objective algorithms have proven their efficacy in feature selection. Nevertheless, a prevalent approach involves addressing feature selection tasks in isolation, even when these tasks share common knowledge and interdependencies. In response to this, the emerging field of evolutionary sequential transfer learning is gaining attention for feature selection. This novel approach aims to transfer and leverage knowledge gleaned by evolutionary algorithms in a source domain, applying it intelligently to enhance feature selection outcomes in a target domain. Despite its promising potential to exploit shared insights, the adoption of this transfer learning paradigm for feature selection remains surprisingly limited due to the computational expense of existing methods, which learn a mapping between the source and target search spaces. This paper introduces an advanced multi-objective feature selection approach grounded in evolutionary sequential transfer learning, strategically crafted to tackle interconnected feature selection tasks with overlapping features. Our novel framework integrates probabilistic models to capture high-order information within feature selection solutions, successfully tackling the challenges of extracting and preserving knowledge from the source domain without an expensive cost. It also provides a better way to transfer the source knowledge when the feature spaces of the source and target domains diverge. We evaluate our proposed method against four prominent single-task feature selection approaches and a cutting-edge evolutionary transfer learning feature selection method. Through empirical evaluation, our proposed approach showcases superior performance across the majority of datasets, surpassing the effectiveness of the compared methods.

I. Introduction

Feature selection, which aims to find a subset of the most relevant and informative features for predicting the class labels from the original feature set, is a critical step in data preprocessing for many classification tasks [1], [2], [3], [4]. When these relevant and informative features are used as inputs for classification, the performance of the learned models is expected to be improved. Meanwhile, feature selection reduces the dimensionality of datasets by removing redundant and irrelevant features, which could speed up the learning process and avoid overfitting [5]. Feature selection presents a dual objective optimization problem, aiming to concurrently minimize the count of selected features and the classification error rate. Balancing these two objectives becomes a nuanced challenge in practice.