A Comparative Analysis between AI Generated Code and Human Written Code: A Preliminary Study

In today’s world where generative artificial intelligence has almost become an integral part of the coding, new challenges must be faced. Therefore, evaluating software bugs in both human written and AI generated code can be useful for the developers. A comparative analysis of these two coding practices is not only helpful for the decisions taken by the developers, it will also assist to give a direction on how to improve the quality of AI driven code. Currently, researchers have leveraged the role of software metrics to compare human written code and AI generated code as these metrics have long been utilized for software bug and vulnerability prediction by the researchers. They also analyzed the secure coding practices in terms of the number of bugs found in both AI and human written code. Our study is an extension of the current works as this study focuses on a set of metrics and a set of bugs as identified by some static analyzer tools. Investigating these two coding practices from different angles can help to reveal unknown relationships and factors that further can be analyzed to improve code quality of recent AI tools. Therefore, the main objective of our work is to identify the relationships between software metrics and bugs in AI generated code and human written code to compare and contrast the coding profiles of the two approaches. This will offer developers critical knowledge to enhance their strategies in mitigating potential bug risks across different coding methodologies. We have utilized top-rated Java solutions to 90 LeetCode problems, generated corresponding AI Java solutions to them, and utilized various static analysis tools to collect metrics and bugs. In this study, we found that two software metrics CountLineCodeDecl and CountLineCodeExe are positively correlated with the bug DLS_DEAD_LOCAL_STORE and the metric AvgCyclomatic is related to the bug AvoidLiteralsInIfCondition in both human written and AI generated code. These findings provide developers with critical insights into potential bug risks, enabling more effective mitigation strategies across different coding methodologies.