Test Case Generation from Quality Attribute Scenarios Using Machine Learning Approach

Identifying testable quality attribute scenarios (QASs) and generating test cases is challenging due to subjectivity, complexity, interdependency, and lack of well-defined metrics in the quality attributes. In this study, we address the problems by developing a ML model that classifies QASs and generates test cases from testable QASs. The proposed model first classifies QASs into testable or non-testable and then it generates test cases from testable QASs. We collected 1967 QAS dataset from literature, textbooks, and publicly available software specification documents. The Machine Learning algorithms used to build the model are SVM, MNB and DTree for the classification of QASs and RF, AdaBoost and GBM for the generation of test cases. This study employed TF-IDF and word2vec techniques for feature extraction. Grid search techniques are used to tune the optimal value of hyperparameters from the predefined possible values. The QAS classification model prediction accuracies are 89%, 88%, and 82% DTree, SVM, and MNB with TF-IDF respectively and 70%, 79%, and 60% DTree, SVM, and MNB with word2vec respectively. For the generation of the test case, the proposed model performs 87 %, 78 %, and 86 % RF, AdaBoost, and GBM with TF-IDF respectively, and 54%, 56%, and 62% RF, AdaBoost, and GBM with word2vec respectively. DTree with TF-IDF outperforms the others for the QAS classification and RF with TF-IDF outperforms for the test case generation.