On Minimizing Symbol Error Rate Over Fading Channels With Low-Resolution Quantization

We analyze the symbol error probability (SEP) of
$M$
-ary pulse amplitude modulation (
$M$
-PAM) receivers equipped with optimal low-resolution quantizers. We first show that the optimum detector can be reduced to a simple decision rule. Using this simplification, an exact SEP expression for quantized
$M$
-PAM receivers is obtained when Nakagami-
$m$
fading channel is considered. The derived expression enables the optimization of the quantizer and/or constellation under the minimum SEP criterion. Our analysis of optimal quantization for equidistant
$M$
-PAM receiver reveals the existence of error floor which decays at a double exponential rate with increasing quantization bits,
$b$
. Moreover, by also allowing the transmitter to optimize the constellation based on the statistics of the fading channel, we prove that the error floor can be eliminated but at a lower decay exponent than the unquantized case. Characterization of this decay exponent is provided in this paper. We also expose the outage performance limitations of SEP-optimal uniform quantizers. To be more precise, its decay exponent does not improve with
$b$
. Lastly, we demonstrate that the decay exponent of a quantized receiver can be complemented by receive antenna diversity techniques.