Automated Code Comments Generation Using Large Language Models: Empirical Evaluation of T5 and BART

Source code documentation plays a significant role in the software development lifecycle, substantially improving the comprehensibility and maintainability of software projects. Despite its importance, documentation is usually dismissed or fails to satisfy the envisaged standards. Recently, Large Language Models (LLMs) have demonstrated the ability to address these challenges by automating code comments generation. Nevertheless, the empirical evaluation of these models is essential to assess their capabilities to produce accurate, contextually relevant and coherent documentation. In this paper, we have conducted an empirical study to investigate the capabilities of two prominent open-source LLMs, such as T5 and BART, developed by Google AI and Facebook AI, respectively, in automating the code comments generation for Python and Java code snippets. We have rigorously evaluated the performance of these models against four key metrics, such as BLEU, ROUGE, METEOR, and Smoothed BLEU. The comprehensive analysis of the evaluation results clearly highlights BART as the superior model over T5 for single-intent code comment generation.