Using Large Language Models to Understand Telecom Standards

The Third Generation Partnership Project (3GPP) has successfully introduced standards for global mobility. However, the volume and complexity of these standards has increased over time, thus complicating access to relevant information for vendors and service providers. Use of Generative AI and in particular Large Language Models, may provide faster access to relevant information. In this paper, we evaluate the capability of state-of-art models to be used as Question-Answering assistants for technical standards document reference. Our contribution is threefold. First, we provide a benchmark and measuring method for evaluating performance of Large Language Models. Second, we do data preprocessing and finetuning for one of these models and provide guidelines to increase model performance. Third, we provide a model of our own, TeleRoBERTa, that performs on-par with foundation models but with an order of magnitude less number of parameters. Results show that Large Language Models can be used as a credible reference tool on telecom technical documents, and thus have potential for a number of different applications from troubleshooting and maintenance, to network operations and software product development.