EvoAPR: Enhancing Large Language Models for Automatic Program Repair with Genetic Algorithm and Dynamic LoRA

Automated Program Repair (APR) aims to automate the patch generation for buggy code and is vital in software devel-opment and maintenance. While large language models (LLMs) excel in various tasks, our empirical study shows they still face challenges in APR. LLMs take infilling templates with different qualities as input, and low-quality templates may misguide LLMs in constantly generating incorrect patches. Additionally, LLMs lack project-specific knowledge and struggle to leverage bug context fully. Therefore, we propose EvoAPR, which integrates the genetic algorithm and dynamic LoRA technique to enhance LLMs for better APR. First, to generate high-quality infilling templates, buggy codes are encoded to genetic representations, and genetic operators and multi-angle evaluation are designed to produce better templates. Then, to effectively utilize bug context, a bug-context-aware LoRA fine-tuning method is proposed to fuse various bug contexts by dynamically activating certain blocks of the LoRA module according to the bug index. Finally, these high-quality infilling templates are fed the fine-tuned LLMs for patch generation. Experimental results demonstrate that EvoAPR significantly improves LLMs' performance, surpassing state-of-the-art methods.