Optimal signaling for a given constellation under average symbol energy and bit rate constraints

Optimal signaling that minimizes the average symbol error rate for a given constellation is studied subject to constraints on average bit rate and average symbol energy. It is shown that the optimal distribution with which to select the symbols in the constellation is obtained as a fixed point of a function that depends parametrically on the symbol energies and the conditional correct decision probabilities corresponding to the maximum a posteriori decision rule. An optimization problem is specified to determine the optimal signaling distribution. Numerical results are provided to corroborate effectiveness of the proposed solution. Furthermore, it is observed that lower symbol error rate can be achieved with nonuniform signaling by increasing the constellation size while satisfying the same average bit rate and average symbol energy to those of the classical uniform signaling scheme where the symbols are chosen equally likely on an equidistant grid.