On the Consistency of the Kozachenko-Leonenko Entropy Estimate

We revisit the problem of the estimation of the differential entropy
$H(f)$
of a random vector
$X$
in
$R^{d}$
with density
$f$
, assuming that
$H(f)$
exists and is finite. In this note, we study the consistency of the popular nearest neighbor estimate
$H_{n}$
of Kozachenko and Leonenko. Without any smoothness condition we show that the estimate is consistent (
$E\{|H_{n} - H(f)|\} \to 0$
as
$n \to \infty $
) if and only if
${\mathbb E}\{ \log (\| X \| + 1)\} < \infty $
. Furthermore, if
$X$
has compact support, then
$H_{n} \to H(f)$
almost surely.