An Objective Space Constraint-Based Evolutionary Method for High-Dimensional Feature Selection [Research Frontier]

Evolutionary algorithms (EAs) have shown their competitiveness in solving the problem of feature selection. However, limited by their encoding scheme, most of them face the challenge of “curse of dimensionality”. To address the issue, in this paper, an objective space constraint-based evolutionary algorithm, named OSC-EA, is proposed for high-dimensional feature selection (HDFS). Although the decision space of EAs for HDFS is very huge, its objective space is the same as that of the low-dimensional feature selection. Based on this fact, in the proposed OSC-EA, the HDFS is firstly modeled as a constrained problem, where a constraint of the objective space is introduced and used to partition the whole objective space into the “feasible region” and the “infeasible region”. To handle the constrained problem, a two-stage $\varepsilon$ɛ constraint-based evolutionary scheme is designed. In the first stage, the value of $\varepsilon$ɛ is set to be very small, which ensures that the search concentrates on the “feasible region”, and the latent high-quality feature subsets can be found quickly. Then, in the second stage, the value of $\varepsilon$ɛ increases gradually, so that more solutions in the “infeasible region” are considered. Until the end of the scheme, $\varepsilon \rightarrow \infty$ɛ→∞; all the solutions in the objective space are considered. By using the search in the second stage, the quality of the obtained feature subsets is further improved. The empirical results on different high-dimensional datasets demonstrate the effectiveness and efficiency of the proposed OSC-EA.

I. Introduction

Classification learning, an important research field in data mining, has wide applications in areas such as computer vision, pattern recognition, and bioinformatics. However, in many applications, the data has high dimensions. It brings the classification algorithms with high computational complexity and poor learning performance. To tackle this problem, one effective way is to apply the feature selection (FS) technique, where a feature subset is chosen to achieve similar (or better) performance [1].