Rocking in two by two: From Collatz-Wielandt to Donsker-Varadhan

We derive a variational formula for the optimal growth rate of reward in the infinite horizon risk-sensitive control problem for discrete time Markov decision processes with compact state and action spaces, extending a formula of Donsker and Varadhan for the Perron-Frobenius eigenvalue of a positive operator. This can be viewed as an abstract version of the Collatz-Wielandt formula for the Perron-Frobenius eigenvalue of a non-negative matrix. This leads to a concave maximization formulation of the problem of determining the optimal growth rate of risk-sensitive reward.