Enhancing Commit Message Generation in Software Repositories: A RAG-Based Approach

Creating clear and detailed commit messages manually is both time-consuming and prone to inconsistency. Existing automated methods, such as rule-based templates, retrieval-based systems, and neural sequence-to-sequence models, often fail to capture the full context of code changes, resulting in incomplete or inaccurate messages. To overcome these limitations, we introduce a Retrieval-Augmented Generation (RAG) framework that combines a pretrained retriever with transformer-based generators (BART and T5). Our approach builds a comprehensive knowledge base of code diffs and commit messages, incorporating relevant external context through joint end-to-end training. The dataset was sourced from the top GitHub repositories and curated using three key filtering and cleaning steps: (1) excluding non-target programming languages via regular expressions, (2) detecting the presence of “What” and “Why” information in commit messages with a Bidirectional LSTM model, and (3) manually evaluating high-quality messages with expert assistance. Experiments on curated datasets in Python, JavaScript, and Java demonstrate that our BART-based configuration significantly improves BLEU, METEOR, and ROUGE scores compared to existing methods, with the RAG-based models consistently outperforming other approaches across all evaluation metrics.