Comparing Traceability through Information Retrieval, Commits, Interaction Logs, and Tags

Context and motivation: Traceability is used to follow and understand the relationships between various software engineering artifacts such as requirements and source code. Comprehensive traceability of software engineering artifacts is important to ensure that a software can be further developed or maintained. Traceability links are often created manually by using commit ids or retrospectively by using information retrieval (IR). Question/Problem: However, creating traceability links manually is costly and it is error-prone in retrospect. As part of our ongoing research on feature management where traceability is also of interest, we use a lightweight tagging approach to relate artifacts. We want to investigate how such a tag-based approach compares to approaches using commit ids, interaction logs (IL), and IR for creating traceability links. Principal ideas/results: We conducted a case study in which students applied the tag-based, the IL-based, and the commit-based approach. We transformed the tags to traceability links and compared them with the commit-based and IL-based approach as well as with IR-based approaches, using a gold standard. We applied different improvements. Contribution: This is the first study comparing four completely different traceability approaches in one project. The combination of IL and commit ids shows the best precision and recall but is intrusive. The other approaches differ less in precision and recall and both need improvement for practical application. We discuss the benefits and drawbacks of the different approaches and state implications for research and practice.