Efficient Linear Feature Extraction Based on Large Margin Nearest Neighbor

Linear feature extraction methods have become indispensable tools in pattern recognition. The linear dimensionality reduction optimizes some objective to produce a linear transformation and derives the discriminative low-dimensional transformed data wherein the similarly labeled samples cluster tightly and the differently labeled samples keep away from one another. In the past, most of the methods achieve between-class distance by maximizing between-class-center-mean to make the differently labeled samples separated from each other. However, the samples in this study located in the class margin or within the other classes called hard samples often move slowly, which results in a small between-class distance and deteriorates the classification performance. Hence, we utilize the large margin nearest neighbor (LMNN) to quickly push only the hard samples toward the center of the class to get the large class margin instead of between-class, further improving the performance of classification. Combined with the linear discriminant analysis (LDA), a novel linear feature extraction method called LDA-LMNN is proposed. The method can address the limitations of LDA. Furthermore, the dropout is employed to the learning of linear transform matrix to improve the generalization ability and overcome the overfitting. The comparative experiments on various real-world datasets by using the state-of-the-art feature extraction methods demonstrate the effectiveness of the proposed method.