On the Inequality of Cover and Hart in Nearest Neighbor Discrimination

When (X
₁
, θ
₁
),..., (X
_n
, θ
_n
) are independent identically distributed random vectors from IR
^d
X {0, 1} distributed as (X, θ), and when θ is estimated by its nearest neighbor estimate θ
₍₁₎
, then Cover and Hart have shown that P{θ
₍₁₎
≠ θ}
^{n → ∞ →}
2E {η (X) (1 - η(X))} ⩽ 2R*(1 - R*) where R* is the Bayes probability of error and η(x) = P{θ = 1 | X = x}. They have conditions on the distribution of (X, θ). We give two proofs, one due to Stone and a short original one, of the same result for all distributions of (X, θ). If ties are carefully taken care of, we also show that P{θ
₍₁₎
ɠ θ|X
₁₁
, ..., X
_n
, θ
_n
} converges in probability to a constant for all distributions of (X, θ), thereby strengthening results of Wagner and Fritz.