An Empirical Study to Compare the Accuracy of AHP and CBRanking Techniques for Requirements Prioritization

Requirements prioritization aims at identifying the most important requirements for a system (or a release). A large number of approaches have been proposed so far, to help decision makers in performing this activity. Some of them provide supporting tools. Questions on when a prioritization technique should be preferred to another one as well as on how to characterize and measure their properties arise. Several empirical studies have been conducted to analyze characteristics of the available approaches, but their results are often difficult to compare. In this paper we discuss an empirical study aiming at evaluating two state-of-the art, tool-supported requirements prioritization techniques, AHP and CBRanking. The experiment has been conducted with 18 experienced subjects on a set of 20 requirements from a real project. We focus on a crucial variable, namely the ranking accuracy. We discuss different ways to measure it and analyze the data collected in the experimental study with reference to this variable. Results indicate that AHP gives more accurate rankings than CBRanking, but the ranks produced by the two methods are similar for all the involved subjects.