An Empirical Comparison on the Results of Different Clone Detection Setups for C-based Projects

Code clones have been used in many different software maintenance and evaluation tasks in practice (e.g., change proportion and evolution, refactoring, and vulnerability management). There are many clone detection techniques (e.g., text-based, token-based, and AST-based) which can detect code clones not only at the source code-level but also at the compiled artifacts (e.g., IR or binary) level. Unfortunately, there are few studies which thoroughly compare the results of various clone detection setups (a.k.a., different clone detection techniques applied at different artifacts), especially for C-based projects. Therefore, in this paper, we conduct a systematic study to compare the effectiveness of six different code clone detection setups. Each setup, which uses the representative one of the three clone detection techniques: token-based technique (SourcererCC), AST-based (NiCad) technique, and text-based (MSFinder) technique, is applied either at the source code-level or at the LLVM-based IR-level. We conduct our experiments on five C-based open-source systems, Apache, Python, PostgreSQL, FFmpeg, and Linux kernel. Experimental results show that the AST-based technique is better than the token-based and text-based techniques, and clone detection setups performed at the IR-level generally yield higher performance than those performed at the source code-level. The setup of AST-based technique applied at the IR-level, yields the highest performance overall, with an F-score of 84%. However, there is no one setup which can detect all the clones. Through manual qualitative analysis, we have identified ten reasons why certain clones cannot be detected at the IR level or at the source code-level, and two reasons why one of the techniques fails to detect clones. Our findings highlight the usefulness of conducting clone detection under different setups. Furthermore, this study also motivates the need for more application-oriented clone comparison studies.