Enhancing Deep Semantic Communication Systems (Deep-SC) using the Pre-trained Models; BERT and BART

In recent years, emphasis has been placed on semantic communication as one of the proposals to reach 6 generation wireless communication due to its ability to preserve basic resources for a communications network. The reason for the development in this field is the remarkable development in deep learning, which has enabled us to enable these systems. One of these proposed systems is the deep learning based semantic communication system named Deep-SC which is a system based on transformer for text transmission, which shows a great potential in sent text over physical channel in low signal-to-noise (SNR) regime. In this paper, a new system was proposed using pre-trained models in the semantic encoder and decoder. That was done by utilizing BART and BERT models instead of the transformer model. The work contributes to the ongoing effort to bridge the gap between machine and human communication by demonstrating the transformative potential of leveraging pre-trained models like BERT and BART. BLEU Score and Sentence Similarity are used as performance metrics to assess the system performance. The results showed that the proposed system has significant enhanced the system performance against benchmark systems. It has shown more robustness to channel variation and achieved better performance, especially in low SNR conditions.