An Entropy Driven Multiobjective Particle Swarm Optimization Algorithm for Feature Selection

Feature selection is an important research field in machine learning since high-dimensionality is a common characteristic of real-world data. It has two main objectives, which are to maximize the classification accuracy while minimizing the number of selected features. As the two objectives are usually in conflict with each other, it makes feature selection a multi-objective problem. However, the large search space and discrete Pareto front makes it not easy for existing evolutionary multi-objective algorithms. In order to deal with the above mentioned difficulties in feature selection, an entropy driven multiobjective particle swarm optimization algorithm is proposed to remove redundant feature and decrease computational complexity. First, its basic idea is to model feature selection as a multiobjective optimization problem by optimizing the number of features and the classification accuracy in supervised condition simultaneously. Second, a particle initialization strategy based on information entropy is designed to improve the quality of initial solutions, and an adaptive velocity update rule is used to swap between local search and global search. Besides, a specified discrete nondominated sorting is designed. These strategies enable the proposed algorithm to gain better performance on both the quality and size of feature subset. The experimental results show that the proposed algorithm can maintain or improve the quality of Pareto fronts evolved by the state-of-the-art algorithms for feature selection.