On the Rényi divergence and the joint range of relative entropies

This paper starts with a study of the minimum of the Rényi divergence, of an arbitrary order α > 0, subject to a fixed (or minimal) value of the total variation distance. Relying on the solution of this minimization problem, we determine the exact region of the points (D(Q∥P1), D(Q∥P2)) where P1 and P2 are any probability distributions whose total variation distance is not below a fixed value, and the probability distribution Q is arbitrary (none of these three distributions is assumed to be fixed). It is further shown that all the points of this convex region are attained by a triple of 2-element probability distributions. As a byproduct of this characterization, we provide a geometric interpretation of the minimal Chernoff information subject to a minimal total variation distance. A full paper version, which includes more results and proofs, is available at http://arxiv.org/abs/1501.03616.