Llm-Based Code Comment Summarization: Efficacy Evaluation and Challenges

Numerous methodologies have been introduced for code summarization and associated activities, including the utilization of large language model (LLM)-based code summarization, to aid software developers in the comprehension and maintenance of software systems. Code comment is one of the communication channels and practices that can assist developers in developing and maintaining software systems. The interpretation and comprehension of code comments can present significant challenges due to factors such as diverse writing styles and varying levels of background knowledge and experience among developers within a team. Self-admitted technical debt (SATD) is commonly indicated in code comments. This research investigates the efficacy of applying large language models to identify SATD and summarize code comments containing SATD to support code maintenance. We employed three widely used pre-trained LLMs, i.e., BART, Flan-T5, and T5 as summarizers. The experimental results indicated that 75 % of the natural language summaries generated by BART are readable with the appropriate structure and vocabulary used. However, almost all generated summaries are irrelevant to the original code comments while the T5 model can identify SATD and include it in the generated summaries. These findings highlight the potential and the limitations of using pre-trained LLMs for code comment summarization and SATD identification, paving the way for future improvements in this domain.