Conditions for Existence of Uniformly Consistent Classifiers

We consider the statistical problem of binary classification, which means attaching a random observation X from a separable metric space E to one of the two classes, 0 or 1. We prove that the consistent estimation of conditional probability p(X) = P(Y = 1 | X), where Y is the true class of X, is equivalent to the consistency of a class of empirical classifiers. We then investigate for what classes P there exist an estimate p̂ that is consistent uniformly in p ∈ P. We show that this holds if and only if P is a totally bounded subset of L
¹
(E, μ), where μ is the distribution of X. In the case, where E is countable, we give a complete characterization of classes Π, allowing consistent estimation of p, uniform in (μ, p) ∈ Π.