Entropy Rate Bounds of Integer-Valued Processes via Second-Order Statistics

This work contains two single-letter upper bounds on the entropy rate of an integer-valued stationary stochastic process, which only depend on second-order statistics, and are primarily suitable for models which consist of relatively large alphabets. The first bound stems from Gaussian maximum-entropy considerations and depends on the power spectral density (PSD) function of the process. While the PSD function cannot always be calculated in a closed-form, we also propose a second bound, which merely relies on some finite collection of auto-covariance values of the process. Both of the bounds consist of a one-dimensional integral, while the second bound also consists of a minimization problem over a bounded region, hence they can be efficiently calculated numerically. In some models that appear naturally in various disciplines and that consist of infinitely large alphabets, assessing the related entropy rates may be a complicated numerical problem. In such cases, we argue that the new proposed bounds can still be efficiently calculated. Examples are also provided to show that the new bounds outperform the standard existing ones.