Toward a Multi-Criteria Framework for Selecting Software Testing Tools

Software testing is a vital part of software engineering process. Automated testing makes this process more accurate and more efficient. For automated testing, many different testing tools were introduced. Due to the large number and the variety of testing tools, selecting the appropriate tools became a difficult confusable task. This research aims at developing a comprehensive taxonomy for testing tools that cover a broad range of testing tools criteria. This comprehensive view would help software developers and software vendors to specify the testing tool/s they need/develop accurately. In details, the framework includes two main parts: (1) comprehensive taxonomy of testing tools; (2) multi-criteria selection method. The first part covers different criteria of testing tools. Because these criteria are large in numbers, wide and variant, a taxonomy of these criteria is needed. This taxonomy will help developers distinguish among testing tools based on a wide spectrum of different criteria. The second part of the framework is a multi-criteria selection method; that enables software developers to choose the appropriate testing tool using a systematic and adequate automated manner. The selection method employs scientific two well-known methods of multi-criteria decision-making techniques; Analytic Hierarchy Process (AHP) and Technique for Order Preference by Similarity to Ideal Solution (TOPSIS). The testing tools taxonomy is well validated by academic professionals in software engineering and achieved good scores in terms of significance, usefulness and comprehension. Academics reported that the taxonomy is slightly complex and needs to be simplified. The selection method was validated using different scenarios to prove the quality of selection even in complex cases with many criteria and many alternatives.