Generating Commit Messages for Configuration Files in 5G Network Deployment Using LLMs

Network automation is crucial for improving network performance. Commit messages describes the different actions of the modification of network configuration files and deployments. This paper presents experiments and studies on automated commit message generation in the deployment of 5G networks. We extracted data from repositories of various projects engineered in Orange’s 5G network. We then developed five prompts for experiments to identify the most suitable methods for this task. To select large language models, we used an in-house GPT-4 interface provided by Orange, and locally deployed popular large models such as Llama3, Mistral. We used both automated and human evaluation methods, selecting BLEU, ROUGE, and METEOR as our metrics for automated assessment. Our experiments shows that commit messages for configuration files generated by Large Language Models (LLMs) have better scores when using one-shot and Retrieval-Augmented Generation (RAG) technologies, for messages generated both by humans and bots.