A novel feature extraction algorithm

Feature extraction or selection is one of the most important steps in pattern recognition or pattern classification, data mining, machine learning and so on. In this paper, we introduce the information theory, propose a new concept of probability information distance (PID) and prove that the PID satisfies four requests of axiomatization of the distance. So the PID is a kind of distance measure, which can be used to measure the degree of variation between two random variables. We make the PID be separability criterion of the classes for information feature extraction, and call it PID criterion (PIDC). Based on PIDC, we design a novel algorithm for information feature extraction. Compared with principal components analysis (PCA), correlation analysis etc., the algorithm put forward in this paper had regarded for the class information, and so it is a kind of supervised algorithm of feature extraction. The experimental results demonstrate that the algorithm is valid and reliable, and it provides a new research approach for feature extraction, data mining and pattern recognition.