BigCloneBench: A Retrospective and Roadmap

Since BigCloneBench was released in 2014, it has been widely used for evaluating the modern clone detection tools. In this position paper, we outline our observations over the years and our thoughts for next steps. We conclude that BigCloneBench has been an accurate and useful vehicle for evaluating the recall of contemporary clone detection tools and could be further improved for evaluating semantic clone detectors. We observe that its dataset has been used in training innovative machine-learning based approaches to Type-4 clone detection, although it was not specifically designed for this purpose and may not be optimal. We call for more research in task-specific clone benchmarks, including datasets built with ML training in mind.