On a relation between maximum likelihood classification and minimum relative-entropy classification (Corresp.)

Maximum likelihood (ML) and minimum relative-entropy (MRE) (minimum cross-entropy) classification of samples from an unknown probability density when the hypotheses comprise an exponential family are considered. It is shown that ML and MRE lead to the same classification nde, and the result is illustrated in terms of a method for estimating covariance matrices recently developed by Burg, Luenberger, and Wenger, MRE classification applies to the general case in which it cannot be assumed that the samples were generated by one of the hypothesis densities. The common use of ML in this case is technically incorrect, but the equivalence of MRE and ML provides a theoretical justification.