Exact Error Probability Analysis of Orthogonal Space-Time Block Codes with Arbitrary Rectangular QAM over MIMO Nakagami-m Fading Channels

In this work, we derive exact closed-form expressions for the average symbol error probability (SEP) of arbitrary rectangular quadrature amplitude modulation (QAM) when used along with orthogonal space-time block coding (OSTBC) over independent but not necessarily identically distributed multiple-input multiple-output (MIMO) Nakagami-m fading channels. The derived SEP expressions account for diversity branches which may exhibit identical or distinctive power levels while their associated Nakagami indexes can take arbitrary real values not less than 1/2. Our work extends previous results pertaining to the performance analysis of OSTBC using M-ary rectangular QAM over Rayleigh fading channels and M-ary square QAM over Nakagami-m fading channels. For a MIMO diversity system with n_T transmit and n_R receive antennas and a corresponding set of arbitrary real-valued Nakagami indexes, our rectangular QAM SEP results are expressed in terms of Lauricella's multivariate hypergeometric functions F_D ^(L) and F_D ^{(L + 1)} where L = n_Tn_R is the total number of diversity branches.