Supervised Feature Selection With Orthogonal Regression and Mountain Range Area

In embedded feature selection methods, one criterion for evaluating features is the Feature Contribution to Each Class (FCEC). Unlike traditional feature contribution, it focuses more on the ability of features to represent each class, aiding in understanding the dependency of recognition tasks on different features. However, current feature selection methods often cannot fully utilize the FCEC when evaluating features, resulting in insufficient accuracy in feature evaluation. To address this problem, this paper proposes a novel supervised Feature Selection with Orthogonal Regression and Mountain Range Area (FORMRA). Specifically, orthogonal regression can retain more discriminative information, thereby obtaining more accurate FCEC. We employ a Projection-based Non-monotonic Line Search (PNMLS) method to solve the objective function of orthogonal regression and propose a multi-vector linear fusion search direction on this basis to improve convergence speed. After the objective function converges, we map the FCEC onto mountain ranges in a two-dimensional coordinate system, employing area calculation that is more sensitive to data distribution and shape for feature assessment, thus achieving the full utilization of FCEC. We conducted experiments on eight publicly available datasets, and the results indicate that compared to currently advanced feature selection methods, this approach can provide a more accurate evaluation of features, thereby selecting superior feature subsets and achieving higher model recognition rates.