Optimization of Hierarchical Modulated and LDPC-Coded BICM With Physical-Layer Network Coding

In this paper, we investigate the performance of physical-layer network coding (PNC) with hierarchical modulation (HM) over a two-way relay channel (TWRC). While Gray mapping is the optimal for bit-interleaved coded modulation (BICM) systems in point-to-point communications, the superimposed constellation at the relay in PNC transmission does not maintain the same characteristics as the original Gray mapping at the users. Thus, to address this issue and obtain the optimal mapping for PNC, we propose a Pseudo-Gray (Pe-Gray) constellation for user-end hierarchical quadrature amplitude modulation (H-QAM). In particular, we develop the design criterion for the user constellation to avoid demodulation ambiguity and more importantly, to ensure that the superimposed constellation can be Gray-mapped. The achievable rate analysis and simulation results show that the proposed Pe-Gray can achieve significant performance gains of up to 2 dB over the conventional Gray, M3, and MSED constellations in power-imbalanced HM-BICM-PNC. The performance superiority of the Pe-Gray is further demonstrated over the power-imbalanced channels, which also suggests its enhanced robustness in wireless fading channels.