Dimensionality Reduction for Classification Using Divide-and-Conquer Based Genetic Programming

Dimensionality reduction (DR) is to obtain meaningful low-dimensional representation concealed within high-dimensional data. Genetic programming (GP) has been used to achieve DR for classification because of the flexible representations and promising performance. However, one drawback of many current GP-based DR methods is the high computational demands, potentially limiting their utility in high-dimensional scenarios. To tackle this issue, this work develops a new GP algorithm to achieve DR for classification. A divide-and-conquer strategy based on feature clustering is proposed to split the original feature set into multiple small feature groups. Each tree in the multi-tree GP is used to learn one high-level feature from each feature group. The performance of the proposed method is examined on 10 classification datasets with varying difficulty. The results indicate that the new GP method achieves significantly better classification performance than commonly used feature construction and feature selection methods. More importantly, the proposed GP method uses fewer nodes but achieves better or comparable classification accuracy than the baseline multi-tree GP method.