The Reliability of Linear Feature Extractors

This paper introduces the concepts of capacity and reliability of a linear feature extractor. The relationship between the capacity introduced here and the channel capacity in information theory is discussed in some detail. The reliability is associated with the least favorable distributions and gives us a measure of the effectiveness for the worst possible case. It is of particular importance in pattern recognition problems because we have no control over the distributions of the patterns. It is shown that for the family of probability distributions with covariance matrix S, the most reliable (i.e., minimax) feature extractor is the Karhunan-Loève expansion. The concept of reliability is extended to the two-class pattern recognition problem and is discussed in terms of the Bhattacharyya distance.