BERT-Aided Sequence Detection With Natural Redundancy

In this letter, we consider exploiting the natural redundancy (NR) that widely exists in the transmitted sources to improve the performance of communication signal detection. To exploit the abundant NR in the sources, we introduce the pre-trained models (PTMs) with a powerful capability to capture knowledge from massive labeled and unlabeled data into sequence detection. Especially for uncompressed Chinese text sources, we propose an effective sequence detection scheme aided by the widely used Bidirectional Encoder Representation from Transformers (BERT) deep learning model and combined with the beam search (BS) algorithm, which can achieve approximate sequence MAP estimation with an acceptable complexity. Simulation results demonstrate that the bit error rate (BER) performance of our proposed BERT-aided BS scheme for sequence detection significantly outperforms the classical detector.