Maximum Rényi Entropy Rate

The supremum of the Rényi entropy rate over the class of discrete-time stationary stochastic processes, whose marginals are supported by some given set and satisfy some given cost constraint, is computed. Unlike the Shannon entropy, the Rényi entropy of a random vector can exceed the sum of the Rényi entropies of its components, and the supremum is, therefore, typically not achieved by memoryless processes. It is nonetheless related to Shannon's entropy: when the Rényi parameter exceeds one, the supremum is equal to the corresponding supremum of Shannon's entropy, and when it is smaller than one, the supremum equals the logarithm of the volume of the support set. A Burg-like supremum of the Rényi entropy rate over the class of stochastic processes, whose autocovariance function begins with some given values, is also solved. It is not achieved by Gauss-Markov processes, but it is nonetheless related to Burg's supremum: the two are equal when the Rényi parameter exceeds one, and the former is infinite otherwise.