Prototype Filter for QAM-FBMC Systems Based on Discrete Prolate Spheroidal Sequences (DPSS)

In conventional wireless communications, cyclic-prefix orthogonal frequency division multiplexing (CP-OFDM) has been adopted as the baseline multicarrier scheme. Despite their corroborated merits, they are explored only in the asynchronous and strictly orthogonal scenario. To overcome the limitations observed in CP-OFDM, to support the constraints imposed by different 5G scenarios, and also to improve robustness against channel impairments, several waveforms have been investigated. Quadrature amplitude modulation associated with filter-bank multicarrier (QAM-FBMC) has been an auspicious technology for 5G communication systems and beyond. The main feature of QAM-FBMC is its capacity for high spectral confinement, which is possible thanks to the per-subcarrier filtering. Aiming to improve the QAM-FBMC performance, in this paper, we propose a prototype filter design based on the discrete prolate spheroidal sequences (DPSS), also known as Slepian sequences. The presented filters were obtained by optimizing the weights that are used to compose the desired prototype filter to such an extent that they minimize the intrinsic interference of the system. At the same time, these weights keep spectral confinement of the filter for a previously determined bandwidth limitation. Simulation results show that the optimized filters achieve higher intrinsic interference attenuation than the competitors. Indeed, we can confirm the improvement in the system performance brought by the usage of the optimized filters through the bit error rate (BER) evaluation. Furthermore, the proposed method is flexible thanks to the suitability of its parameters.