A variational characterization of Rényi divergences

We present a variational characterization of the Rényi divergences between any two probability distributions on an arbitrary measurable space, in terms of relative entropies. This yields as a corollary a recently developed variational formula, due to Atar, Chowdhary and Dupuis, for exponential integrals of bounded measurable functions in terms of Rényi divergences. We then develop a similar variational characterization of the Rényi divergence rates between two stationary finite state Markov chains in terms of relative entropy rates. This leads to an analog of the variational formula of Atar, Chowdhary and Dupuis in the framework of stationary finite state Markov chains.