Unadmitted Technical Debt: Dataset and Detection Approaches

In recent years, researchers have proposed various approaches to detect code comments that explicitly acknowledge Technical Debt (TD), which are referred to as Self-Admitted Technical Debt (SATD) comments. Previous studies have proven that hidden patterns can be learned from SATD code snippets to predict whether the code snippets hold TD without the aid of comments. In this study, we refer to such TD as unadmitted TD, i.e., TD whose code snippets exhibit patterns similar to those of SATD, but are not annotated with comments indicating the existence of TD. Given that current unadmitted TD datasets are limited to method-level and conditional-statement-level code snippets, we construct the world’s most comprehensive dataset of code snippets and their corresponding comments, which includes 18 popular Java open-source projects and covers code snippets at the file, class, method and block levels. Around this dataset, we have conducted four key research activities.

Firstly, we propose an automated framework for data collection and annotation, which extracts commented code snippets of varying granularity from projects and assigns SATD labels using three state-of-the-art SATD detection approaches. Secondly, we conduct a rigorous evaluation process, including the validity test, reliability test and manual verification, to ensure the accuracy and consistency of the dataset before further analysis and utilization. Additionally, we propose a metric-based detection approach named LiteM that detects unadmitted TD solely based on code metrics. As for the real-world scenarios where training data is scarce, we further introduce LiteMC, which generates pseudo-labels for commented code snippets and then employs LiteM to train a model on these pseudo-labeled data, enabling the detection of unadmitted TD in uncommented code snippets. The experimental results demonstrate the effectiveness and efficiency of both LiteM and LiteMC. The dataset and the code are available at https://github.com/HduDBSI/Dataset4TD.