On Pinsker's and Vajda's Type Inequalities for Csiszár's $f$ -Divergences

Let
D
and
V
denote respectively Information Divergence and Total Variation Distance. Pinsker's and Vajda's inequalities are respectively
D
≥ [ 1/ 2]
V
²
and
D
≥ log[( 2+
V
)/( 2-
V
)] - [( 2
V
)/( 2+
V
)]. In this paper, several generalizations and improvements of these inequalities are established for wide classes of <;i>f<;/i>-divergences. First, conditions on
f
are determined under which an
f
-divergence
Df
will satisfy
Df
≥
cf V
²
or
Df
≥
c
_2,
f V
²
+
c
_4,
f V
⁴
, where the constants
cf
,
c
_2,
f
and
c
_4,
f
are best possible. As a consequence, lower bounds in terms of
V
are obtained for many well known distance and divergence measures, including the χ
²
and Hellinger's discrimination and the families of Tsallis' and Rényi's divergences. For instance, if
D
_(α)
(
P
||
Q
) = [α(α-1)]
^-1
[∫
p
^α
q
^1-α
d
μ-1] and ℑ
_α
(
P
||
Q
) = (α-1)
^-1
log[∫
p
^α
q
^1-α
d
μ] are respectively the relative information of type α and the Rényi's information gain of order α, it is shown that
D
_(α)
≥ [ 1/ 2]
V
²
+ [ 1/ 72] (α+1)(2-α)
V
⁴
whenever -1 ≤ α ≤ 2, α ≠ 0,1 and that ℑ
_α
≥ [( α)/ 2]
V
²
+ [ 1/ 36] α(1 + 5 α- 5 α
²
)
V
⁴
for 0 <; α <; 1. In a somewhat different direction, and motivated by the fact that these Pinsker's type lower bounds are accurate only for small variation (
V
close to zero), lower bounds for
Df
which are accurate for both small and large variation (
V
close to two) are also obtained. In the special case of the information divergence they imply that
D
≥ log[ 2/( 2-
V
)] - [( 2-
V
)/2] log[( 2+
V
)/2], which uniformly improves Vajda's inequality.