A Data-Driven Recommendation System for Enhancing Non-Functional Requirements Elicitation in Scrum-Based Projects

Context: Agile software development, particularly Scrum, enables teams to manage evolving requirements by emphasizing face-to-face communication and incremental deliveries. Although effective in addressing functional requirements, agile methods often overlook non-functional requirements during the initial stages of software projects, potentially leading to cost overruns on software and hardware and project failures exceeding 60%. Objective: In this article, we introduce a data-driven recommendation system to assist Scrum teams in eliciting NFRs effectively and early in the development lifecycle. Method: Our proposed solution applies the k-nearest neighbors algorithm to recommend non-functional requirements by leveraging historical project data structured through a taxonomy of user stories. We evaluated the system through offline experiments under the cross-validation protocol, utilizing datasets from 13 real-world projects. Results: Our recommendation system achieved an F-measure of up to 79%, demonstrating its ability to provide accurate and context-aware non-functional requirements suggestions. Conclusion: These findings suggest that our solution supports agile teams by automating non-functional requirement elicitation and enhancing decision-making processes, thereby addressing critical gaps in non-functional requirement integration within Scrum-based projects.