The Smart Product Backlog: A Classification Model of User Stories

In agile software development processes, user stories (US) had been used to specify application functionalities from the users’ perspective. For intelligent applications leveraging artificial intelligence (AI), the Smart Product Backlog (SPB) has included both AI-implementable and non-AI functionalities. This paper had proposed a model employing supervised machine learning techniques to classify US descriptions based on their technical feasibility for AI implementation. This model had aimed to assist in constructing smart product backlogs (SPB). Classifying US in agile development, particularly with AI, had been a labor-intensive process demanding significant time and expert involvement. Additionally, the lack of a dedicated dataset for this task had limited the applicability of automated methods. This study addressed this challenge by having experts classify the Mendeley US dataset using binary classification (AI and non-AI). The proposal had involved developing an automatic classification model to process US descriptions and distinguish those suitable for AI implementation. This model had leveraged advanced text processing techniques to refine the textual features of the US. Additionally, it had employed three binary classification techniques: logistic regression, k-nearest neighbors (k-NN), and support vector machines (SVM). The model’s performance had been evaluated using metrics such as accuracy, loss (Log-Likelihood Loss), precision, recall, F1 score, area under the ROC curve, and specificity to identify the best-performing technique in binary classification. Logistic regression and SVM models had demonstrated high accuracy, with scores of 0.748 and 0.740, respectively. These results had highlighted the potential of an automated tool for recommending US feasible for AI development, thereby supporting decision-making in agile software projects.