Enhancing Code Transformation in Large Language Models Through Retrieval-Augmented Fine-Tuning

Large language models (LLMs) have made substantial advancements in knowledge reasoning and are increasingly utilized in specialized domains such as code completion, legal analysis, and medical transcription, where accuracy is paramount. In such applications, document-specific precision is more critical than general reasoning capabilities. This paper proposes a novel approach based on Retrieval-Augmented Fine-Tuning (RAFT) to enhance model-generated outputs, particularly in code transformation tasks. RAFT integrates domain-specific knowledge, optimizing in-domain retrieval-augmented generation by training the model to discern the relationship between prompts, retrieved documents, and target outputs. This enables the model to extract relevant information while mitigating the impact of noise. Experimental results demonstrate that the proposed method improves accuracy of 2.4% and CodeBLEU of 1.3% for VB-to-C# code conversion, highlighting its effectiveness in domain-specific applications.