AI-driven methodology for refining and clustering Agile requirements

Refining the backlog is crucial in the Agile development process for prioritizing work, detecting and resolving problems quickly, and aligning development efforts with project objectives. With automated tools for refining and structuring the backlog, teams can operate more efficiently with a clear understanding of product requirements. In this paper, we present our automated approach for conducting requirements analysis within the Agile Methodology. This method includes the refinement of backlogs through the identification of duplicate user stories and their subsequent clustering. Our method is based on machine learning and natural language processing techniques. To implement our approach, we developed a web-based tool that selects the user stories file, groups them by actor, and applies the unsupervised k-means algorithm to create clusters. We then used word embedding with sentence transformers of BERT to evaluate the similarity rate between each pair of user stories in a cluster. The tool highlights the most similar user stories and facilitates the decision to delete or keep them. The proposed approach was evaluated on a set of case studies using different performance measures. The results demonstrated the effectiveness of our method in detecting duplicate user stories in the backlog.