Approximating the Differential Entropy of Gaussian Mixtures

A Gaussian mixture is a weighted sum of several Gaussian densities. The probability density function (PDF) of the output of an additive white Gaussian noise (AWGN) channel with discrete input can be regarded as a Gaussian mixture. By constructing a sequence of functions to approach the Gaussian mixture, this paper presents a new approximation for the differential entropy of Gaussian mixtures. Comparing with the previous works, the proposed approximation has a much simpler form and lower computation complexity while tends to the real differential entropy. As an application, by using the approximation, we optimize the discrete input for some AWGN channels and obtain larger input-output mutual information than the previous works.