Scheduled System Maintenance:
On May 6th, system maintenance will take place from 8:00 AM - 12:00 PM ET (12:00 - 16:00 UTC). During this time, there may be intermittent impact on performance. We apologize for the inconvenience.
By Topic

On Pinsker's and Vajda's Type Inequalities for Csiszár's f -Divergences

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

The purchase and pricing options are temporarily unavailable. Please try again later.
1 Author(s)
Gilardoni, G.L. ; Dept. of Stat., Univ. de Brasilia, Brasília, Brazil

Let D and V denote respectively Information Divergence and Total Variation Distance. Pinsker's and Vajda's inequalities are respectively D ≥ [ 1/ 2] V2 and D ≥ log[( 2+V)/( 2-V)] - [( 2V)/( 2+V)]. In this paper, several generalizations and improvements of these inequalities are established for wide classes of <;i>f<;/i>-divergences. First, conditions on f are determined under which an f-divergence Df will satisfy Dfcf V2 or Dfc2,f V2 + c4,f V4, where the constants cf, c2,f and c4,f are best possible. As a consequence, lower bounds in terms of V are obtained for many well known distance and divergence measures, including the χ2 and Hellinger's discrimination and the families of Tsallis' and Rényi's divergences. For instance, if D(α) (P||Q) = [α(α-1)]-1 [∫pαq1-αdμ-1] and ℑα (P||Q) = (α-1)-1 log[∫pαq1-αdμ] are respectively the relative information of type α and the Rényi's information gain of order α, it is shown that D(α) ≥ [ 1/ 2] V2 + [ 1/ 72] (α+1)(2-α) V4 whenever -1 ≤ α ≤ 2, α ≠ 0,1 and that ℑα ≥ [( α)/ 2] V2 + [ 1/ 36] α(1 + 5 α- 5 α2 ) V4 for 0 <; α <; 1. In a somewhat - - different direction, and motivated by the fact that these Pinsker's type lower bounds are accurate only for small variation (V close to zero), lower bounds for Df which are accurate for both small and large variation (V close to two) are also obtained. In the special case of the information divergence they imply that D ≥ log[ 2/( 2-V)] - [( 2-V)/2] log[( 2+V)/2], which uniformly improves Vajda's inequality.

Published in:

Information Theory, IEEE Transactions on  (Volume:56 ,  Issue: 11 )