OurRank: A Software Requirements Prioritization Method Based on Qualitative Assessment and Cost-Benefit Prediction

Requirements prioritization is an activity aimed at determining the essential requirements to include in a software release. Although there are several prioritization methods to systematize this task, there are still unresolved challenges. Existing methods do not guarantee that requirements prioritization meets stakeholder expectations and goals. This is because most prioritization methods operate by considering only quantitative information, making it difficult to formally capture stakeholder interests and perspectives that can rather be made explicit in qualitative terms. Likewise, methods including qualitative information only consider elements associated with benefit estimation, that is, positive aspects of the project, but neglect costs or negative aspects. As a result, the prioritization process is driven by a partial view of constraints. Such methods also fail at capturing and combining expert knowledge that decision-makers can bring into the decision-making process. In this research, we propose a novel method for software requirements prioritization, which facilitates incorporating experts’ qualitative assessment at the outset of the prioritization process and considers both benefit and cost constraints. Details of the method are presented, together with a case study describing a real application scenario. Recommendations and guidelines regarding the application of the method are proposed based on the results of the case study.


I. INTRODUCTION
Requirements prioritization is a fundamental activity in software development. It is aimed at determining requirements that are included in a software development plan considering various criteria, such as natural precedence, value generation, and risk minimization [1]. Depending on the software development process that is considered, requirements prioritization can be conducted by software engineers, requirements engineers and other specialized roles, such as product owners The associate editor coordinating the review of this manuscript and approving it for publication was Claudia Raibulet . and project managers. However, despite that a wide variety of software requirement prioritization methods exist [2], these activities are often carried out in an intuitive and/or informal fashion. Dismissing the use of formal methods to conduct requirements prioritization increases the risk of missing critical issues and hinders rigorousness in managing consistency among requirements.
Different authors [2], [3], [4] have identified challenges for existing requirement prioritization methods, regarding efficiency and accuracy criteria. Moreover, it has been often observed that results obtained through a given method do not match priorities that are expected by decision-makers [2].
Arguably, mismatch of expectations occurs due to methods being overreliant on quantitative information [4]. Methods that only capture and process quantitative information make it difficult to reify interests and perspectives of decisionmakers, which are of qualitative nature. Even though methods that include qualitative information to dynamically drive the prioritization process exist [5], [6], these only consider qualitative prioritization criteria associated with estimated benefits. Thus costs and other aspects with negative connotation fail to be considered as prioritization criteria [6]. Furthermore, cost prioritization criteria should be modelled after stakeholders' qualitative knowledge, and should be considered alongside benefit-related criteria in requirements prioritization [7]. It must be noted that other prioritization methods do not formally integrate nor combine decisionmakers' prior knowledge in the prioritization process.
In this research a novel software requirements prioritization method is proposed, which, being based on previous research, seeks to overcome the limitations of current methods discussed above [6], [8], and [9]. The main contributions of this work are the following: -A qualitative method for requirements prioritization called OurRank is presented. OurRank allows to formally capture, consider, and map benefit and cost prioritization criteria throughout the requirements prioritization process. In addition, OurRank integrates prior knowledge of decisionmakers in requirement prioritization.
-A case study encompassing application and qualitative evaluation of OurRank is analyzed. The case study reports on the participation of experts in the practice of requirements prioritization by means of OurRank in a corporate setting. The results obtained permit validating the applicability and utility of the method, in relation to its intended benefits.

A. METHODOLOGY AND STRUCTURE
This research is based on the methodology described by the Design Science framework [10], which comprises seven guidelines. Following guideline one, we have designed an innovative purposeful artifact established on a new algorithm for requirements prioritization in a specific problem domain. This, following guideline two, is the integration of qualitative prioritization criteria based on benefit and cost. Verifying guideline three, the solution proposed is suitable for the problem stated, as it is evident through the qualitative evaluation conducted, which proved to be efficient in dynamic work teams (achieving guideline four). Furthermore, as per guideline five, the proposal has been officially characterized using mathematical descriptions and internal properties, providing a search process that assembles the problem space and poses the prioritization mechanism to identify an effective solution alongside experienced decisionmakers via a case study (guideline six). Finally, acknowledging guideline seven, in this paper, both the solution and results have been demonstrated to a technical and managerial audience. This paper is organized as follows. Section II presents a revision of related work. Section III describes the formal details and operation of the proposed method. Section IV presents a case study to validate our proposal. Section V describes the results of the case study. Section VI presents a discussion of the qualitative analysis carried out on the case study. Section VII reviews the research limitations. Finally, Section VIII presents conclusions and future work.

II. RELATED WORK
There are various prioritization methods used in software development [3], [11], [12]. Analytics Hierarchy Process (AHP) [13] identifies the priority of requirements using a pairwise comparison matrix, being one of the most influential and used methods to date [14]. Different methods based on AHP have been proposed, such as Power Analytics Hierarchy Process (PAHPT) [15] that analyzes the power relations between stakeholders, and Cost-Value [16] that prioritizes requirements based on their perceived value and implementation costs. Additionally, CBRank [17] uses machine learning to reduce the input effort while DRank [18] uses AHP and machine learning to analyze stakeholder preferences and dependencies between requirements. Finally, the Interactive Genetic Algorithm (IGA) [19] uses a genetic algorithm for incremental knowledge acquisition.
One limitation of the abovementioned methods is that they do not formally capture project priorities. Furthermore, most are based on individual assessment of requirements through peer comparisons. This prevents a global understanding of the relevant aspects to guide the prioritization process, requiring qualitative elements to capture the interests and perspectives of decision-makers.
On the other hand, the Quality Functional Deployment (QFD) method [5] is based on a matrix to represent the needs and expectations of stakeholders. The Kano method [20] also allows requirements to be prioritized based on stakeholder preferences, but focuses on differences in product features. Similarly, the Wiegers method [21] estimates the relative priorities of the requirements through a scheme based on the QFD concept of customer value. Various methods have also been proposed in the field of agile methodologies. For instance, Planning Game [22], which is part of eXtreme Programming (XP), is based on user stories to classify requirements into three priority groups; $100 Allocation [23] or also known as Cumulative Voting, in which the stakeholders have one hundred fictitious dollars to ''spend'' between the requirements; Dot voting [24] that uses points assigned to each requirement; Multi Voting System [25] that allows a person to assign multiple votes on a single item; Roundthe-Group Prioritization [26], in which a consensus group ordering is performed; Weighted Criteria Analysis [27] in which criteria and weights are defined for each criterion; and Binary Search tree [28] that is based on the search algorithm of the same name to compare and establish priority relationships between requirements. The MoSCoW Method [29] uses an ordinal scale of four levels that are associated with each  VOLUME 10, 2022 requirement to prioritize those with the highest value. Similarly, the Value-Oriented method [30] focuses on elements of core value for the business. Additionally, two more recent methods include DLM-MLSRP [31] that considers aspects of cost and benefit, within a set of pre-fixed aspects, and uses the deep neural Lagrange multiplier to establish the ranking of requirements; and the SARiP method [32] that is based on an initial prioritization using the MosCoW method and the assessment of a set of predefined aspects. The latter method uses machine learning for the final estimation of prioritization around three groups (low, medium and high priority) and the corresponding ranking in each group.
These recent methods enable the capturing of various elements to obtain a global conception of the project (commercial objectives, customer expectations and stakeholder needs), allowing for better prioritization. However, they have limitations regarding the subjective use of ordinal scales and rankings [3]. This makes it difficult to provide objective qualitative elements to clarify requirements inconsistencies during the prioritization process.
Other methods mix different techniques in search of a better ranking. The PHandler method [33] generates an initial ranking of requirements by experts and then uses neural networks and the Fuzzy c-means clustering method to establish clusters of requirements according to priority, to generate a final ranking through AHP. The FBRS method [34] is based on fuzzy graph optimization and integer programming to identify value dependencies between requirements. Hybrid Ranking Model [35] contemplates a hybrid mathematical model that combines different strategies for prioritizing requirements. IQ-BR [36] is based on quantum optimization to perform prioritization, considering interdependencies between functional and non-functional requirements. Different approximations based on fuzzy logic have recently been proposed, taking advantage of its ability to represent degrees of uncertainty. For example, the Intuitionist Fuzzy Approach (IFA) [37] takes advantage of the Fuzzy method to represent the stakeholders' priorities, along with slicing and backtracking of requirements and page rank to analyze interdependencies. The LSDM-IFS method [38] uses ratings by criteria associated with requirements, and the use of intuitionist fuzzy sets to determine priority class membership. It also considers the use of a recommendation system to provide suggestions to stakeholders and guide them to reach a consensus on prioritization. However, despite the use of different techniques, these methods generally suffer from a lack of qualitative information in the prioritization process.
On the other hand, the Quantitative Requirements Prioritization method for HCS [9] uses different quantitative assessment matrices, the curriculum-based method and linear regression, to prioritize requirements in the field of Highly Configurable Systems. Similarly, the PAPS method [39] uses a fuzzy inference system to allow the partial satisfaction of requirements in the security field. These are examples of proposals focused on specific systems or types of requirements, so they are not of general applicability.
Recently proposed, the Qualitative Method for Prioritizing Software Requirements (QMPSR) [6] allows considering qualitative elements related to project priorities. However, this proposal only considers elements associated with the estimation of benefits (positive aspects of the project), without considering its costs or negative aspects, making it difficult to fully understand the priorities of the projects. Similarly, the Fuzzy method linguistics labels [8] is based on the use of fuzzy linguistic labels to establish prioritization of requirements individually by each stakeholder, considering multiple prioritization criteria that can be defined by the participants. The different preferences are then combined by a proposed new operator called MLIOWA. It also considers a multicriteria prioritization method to consider the perspectives of the different stakeholders and minimize collisions. However, this method does not consider a global assessment in conjunction with stakeholders, which could lead to problematic collision handling. Table 1 summarizes the related work.
Considering the above mentioned, it is possible to affirm that at least part of the project' priorities are overlooked or not fully captured by the requirements prioritization methods. Most of the existing approaches are based on quantitative measures that suffer from problems in formally capturing the interests and priorities of stakeholders in the prioritization process.

III. OURRANK: A PROPOSAL FOR SOFTWARE REQUIREMENTS PRIORITIZATION
OurRank aims to improve the prioritization process based on the incorporation of the relevant aspects of benefit (positive) and cost (negative) that define the requirements priority, via five steps, as described in Fig. 1 below. The project's relevant aspects are described and composed of a set of elements. The first step aims to formally capture decision-makers' prior knowledge through generation of an initial ranking requirements. This step comprises two activities. Firstly, define the initial ranking of requirements, and secondly, generate weights based on requirements' initial ranking. Details are as follows: VOLUME 10, 2022

1) DEFINE REQUIREMENTS' INITIAL RANKING
This activity involves generating a first ordering or ranking of requirements, in order to capture the prior knowledge of decision-makers.
Expert knowledge is necessary to conduct a decisionmaking process [40]. In general, attempts are made to link and integrate knowledge with decision-making [41] through various strategies and in different contexts [42], [43], [44], [45]. That is why the prioritization process begins with the generation of an initial ordering of the requirements, to capture the prior knowledge of the decision-makers. The objective is to integrate these results in the generation of the requirements final ranking and to support situations when more than one requirement has the same priority.

2) GENERATE WEIGHTS BASED ON REQUIREMENTS' INITIAL RANKING
This activity consists of weighting requirements in the initial ranking, so that these considerations can be then included in final requirements assessment. Weights applied to each requirement depend on their relative rank in prioritization. Let R = {r 1 , . . . ,r m ..,r n } be a set of ordered requirements (r m ∈ R), in which m represents the order of importance within the set and n r is the total number of requirements {n r ∈ N :n r ≥ 1}. The weighted value of r m , considering its initial ranking, is defined as:

100
(1) where V (r h ) > V (r m ) implies that r h (r h ∈ R) has a higher priority than r m .

B. STEP 2: DEFINE QUALITATIVE ASPECTS OF BENEFIT AND COST
The second step consists of identifying the prioritization criteria that drive the process. This second step consists of three activities:

1) IDENTIFY PROJECT ASPECTS
Aspects of a project correspond to important issues associated with conditions that add or subtract value, according to Riegel and Doerr [7]. Some of these types include requirement volatility or stability, performance, competitiveness, financial sanctions, and budget, among others. This method allows the definition of a non-limited set of aspects related to benefit and cost.

2) DEFINE THE PRIORITY ORDER OF THE ASPECTS DEFINED FOR THE PROJECT
In this activity, the priority ordering of the previously identified aspects is determined, according to their relevance. For a set of n aspects, the least important aspect is assigned the value 1, while the most important aspect is assigned the value n.

3) COMPUTE ASPECTS' NORMALIZED PRIORITY VALUES
In this step, the normalized priority value of the aspects (i.e., P) is determined, which represents the priority with respect to the total number of aspects. A normalized value is adopted to guarantee that P will take values between −1 and 1, identifying whether the aspect is one of benefit or cost. Let A = a 1 , . . . , a k , . . . , a n be a finite ordered set of aspects, where a k ∈ A, such that k represents the assigned order of importance, and n a is the total number of aspects defined n a ∈ N : n a ≥ 1}. The normalized priority value of an aspect a k is defined as: where BC (a k ) identifies whether a k is a benefit or cost aspect, defined in terms of the function BC, where BC: A → {1, −1}. Thus, P (a h ) < 0 identifies that aspect a h is a cost aspect and P(a h ) > P(a k ) means that aspect a h (a h ∈ A) has a higher normalized priority than a k .

C. STEP 3: IDENTIFY THE ASPECTS' ELEMENTS
The third step consists of identifying and prioritizing the elements of the previously identified aspects. This step consists of two activities:

1) IDENTIFY AND DEFINE PROJECT ELEMENTS
Each relevant aspect is made up of a set of common elements. Elements can be divided into subcategories to identify specific elements for the relevant benefit and cost aspects [7]. For example, for the cost aspect of Financial Sanctions, it is possible to define elements Costs for not implementing, Legal mandate, and Contractual commitment, among others. This method allows the definition of an unlimited set of elements for each aspect.

2) PRIORITIZE ELEMENTS ASSIGNED TO EACH ASPECT
This activity consists of assigning priorities to the identified elements, considering three levels: high, medium, or low. This defines a total order among the elements by their priority regardless of their quantity. Let E = e 11 , . . . , e 1z , . . . , e k1 , . . . , e kp , . . . , e n1 , . . . , e nq be a finite set of elements, where z, p and q correspond to the total of elements of aspects a 1 , a k and a n , respectively z, p, q ∈ N : z, p, q ≥ 1}. The priority of an element is defined in terms of function L(e kp ) where L : E → {1, 2, 3}; interpreting value 1 as low priority, 2 as medium priority and 3 as high priority.

D. STEP 4: PRIORITIZATION BASED ON THE ASPECTS' ELEMENTS
The fourth step aims to prioritize the requirements through the aspects' elements, by identifying relationships between elements and requirements. An element can be associated with one or many requirements, and a requirement can be associated with one, many, or no elements. Such relationships are specified and collectively negotiated between decisionmakers.
The relationship between an element e kv (e kv ∈ E) and a requirement r i ∈ R is defined by the function C (e kv , r i ), where C : E × R → {0, 1}; interpreting the value 0 as no link between element and requirement and value 1 as the existence of a link between e kv and r i .

E. STEP 5: CALCULATE FINAL RANKING OF REQUIREMENTS
The last step consists of generating the final ranking of the requirements based on the previously identified relationships and the initial ranking of the requirements. This last stage consists of two activities:

1) COMPUTE IMPORTANCE OF REQUIREMENTS PER ASPECT, CONSIDERING THE INITIAL REQUIREMENTS RANKING
In this step, the importance level of the requirements is determined. The relevance of a requirement is related to the priorities of the elements of the associated aspect plus an Association Factor (G) [6]. This G allows to prioritize elements that have more related requirements, and it can be used to compare and analyze elements with the same priority.
In this way, initially the number of requirements related to each of the elements of the project aspects is obtained. The total number of requirements related to the element e kv of the aspect a k is formally defined as: where C (e kv , r i ) is a function that identifies whether the element e kv relates to requirement r i . Knowing the value of TC it is possible to calculate the factor G of any element. Thus, association factor of element e kv is defined as: where TC(e kb ) corresponds to the maximum number of requirements to which an element of aspect a x is linked, such that e kv ∈ E, ∀e kx ∈E, TC(e kv ) ≥TC(e kx ). With these results, it is possible to calculate the priority in the existing relationships between elements and requirements of the project. Thus, the priority of an element e kv associated with requirement r i considering its respective factor G is defined by means of function I :E × R→ R, as follows: where L (e kv ) defines the element's priority function and G (e kv , e kb )corresponds to the element's association factor, for every case in which there is a link between element e kv and requirement r i , according to function C (e kv , r i ).
Once the priority of each requirement has been defined in the different elements of the project's relevant aspects, a complete assessment of the requirement is obtained, considering its initial ranking. Thus, let E k = {e k1 , . . . ,e kp be a finite subcollection of the elements related to aspect a k (E k ⊂ E). The total number of elements of aspect a k assigned to requirement r i considering its weighted value V in the initial ranking, is defined as: where V (r i ) identifies the weighted value of requirement r i in the initial ranking, computed with the formula (1). Furthermore, I (e kv , r i ) corresponds to the priority of element e kv linked to requirement r i considering its factor G, calculated according to (5). Finally, the relevance level of requirement r i with respect to aspect a k as a: where TI (a k , r i ) defines priority of requirement r i in all elements of aspect a k and considering its weighted value V , as defined in equation (6). Furthermore, TC (e kv ) identifies the total number of requirements r i linked to element e kv , calculated with (3). Hence, λ (a k , r i ) > λ a k , r j means that r i has greater priority than r j for aspect a k .

2) CALCULATE THE FINAL REQUIREMENTS RANKING CONSIDERING RELEVANCE BY ASPECT
Once the importance level of the requirements for each aspect has been obtained, considering its initial ranking, the final ranking of a requirement r i , considering its priority in all aspects, is formally defined as: where λ (a h , r i ) identifies the relevance level of requirement r i with respect to aspect a h , as defined in (7). In addition, P(a h ) defines the normalized value of cost or benefit aspect a h , calculated by means of (2). Lastly, RFR (r i ) > RFR r j means that requirement r i has a higher priority than r j . It must be noted that the final ranking of requirements considers all relationships amongst elements and requirements. These relationships are ought to be generated dynamically and collaboratively by stakeholders in the prioritization process. Furthermore, final priority of requirements also considers a weighted value related to the initial ranking of requirements; obtained from decision-makers' knowledge at the project outset.
In order to provide a better understanding of the phases described above, and qualitatively study how decision-makers reason in terms of the main characteristics of our proposal, a case study comprising implementation and evaluation of the OurRank method is presented in the following section.

IV. CASE-STUDY
The objective of this case study was to investigate how difficult it is for decision-makers to think in terms of the main features of our proposal. Specifically, it is investigated whether the initial ranking is a suitable mechanism to capture the initial expert knowledge of decision-makers and whether the qualitative aspects is a pertinent structure to drive a software requirements prioritization process. The application scenario for OurRank in this case study was in a leading services management company in the healthcare sector. The methodology was carried out by experts from the prioritization area of the company. Research questions driving this case study are as follows: RQ1: Is the initial ranking an adequate mechanism to capture the expert knowledge of decision-makers and appropriate to consider it with some weighting in the final ranking?
RQ2: Is the structure for defining aspects, considering benefits and costs, pertinent and consistent with needs of a dynamic organization?

A. PARTICIPANTS
The participants of the case study were employees of a health services management company, who usually work on tasks related to decision-making in project management. Written informed consent was obtained from all participants. A total of six engineers (M age = 30.8 years, SD age = 3.4, ages 26 to 36; 83% male) participated in the case study. Table 2 presents characteristics of participants. All participants had experience in the application of prioritization methods and had three to ten years' experience in prioritization tasks.

B. PROCEDURE
The participants applied the proposed method to a practical case (see Section IV.C). They were organized into two groups ('A' and 'B', as shown in Table 2 ) to independently deal with the proposed case by fully implementing the OurRank prioritization method. Subsequently, all participants met in a focus group. The focus group was organized into three stages.
1) A reflection on the activities conducted, and the perceived usefulness of the method. 2) Information was provided to participants on the results of the tasks carried out. Based on this, participants were invited to reflect on the differences found among the results of the two groups, and possible causes of these differences. 3) Participants were asked to give their opinion on the advantages and disadvantages of the method.
To analyze the focus group discussions, a method based on grounded theory was followed. In grounded theory [46], analyses is derived from systematically collected data and its evaluation. The process begins with description of phenomena, continues with conceptual ordering of collected data, and ends with a theorization. To analyze the content of focus group discussions, a set of theme categories was elaborated based on the interview protocol. Then this set was extended and revised as interview transcripts were coded. Based on the codes, patterns were sought to explain contexts, situations, facts and phenomena of interest.

C. GENERAL GUIDELINES
The application chosen to carry out the verification of our approach is an inventory, sales, and customer service system in pharmaceutical services. The end-users of this software are pharmacists, warehouser, and administrators of a certain branch of the pharmacy chain. In this way, software that has the following general guidelines is required: -Pharmacists can create and edit a customer's profile.
-Pharmacists can check customer information.
-Pharmacists and warehouser can create and edit medication items.
-Pharmacists and warehouser can consult the movements of medicine.
-Warehouser can record stock count of medicine.
-Administrators can create and edit pharmacy branches.
-Administrators can check the information of a branch.
-Administrators can manage the profiles of both pharmacists and warehouser.
In accordance with the general guidelines described above, the requirements for this information system are presented in Table 3. The general guidelines, requirements and prioritization criteria, proposed by [7], were the inputs for the participants to work collectively and collaboratively in the requirements prioritization; carrying out the different phases of the benefitcost qualitative method presented in Section III. Specifically, the tasks were directly related to the five phases of the proposed method (see Table 4).

V. RESULTS
In the following sections, the results of the implementation and complete application of the proposed method with experts from the prioritization area are presented. The results are presented according to the tasks performed by the participants (phases of the proposed method).

A. STAGE 1: GENERATE A PRELIMINARY RANKING OF REQUIREMENTS
This first task consisted of ordering the requirements hierarchically to identify an initial ranking. Each group carried out this task collaboratively, according to their previous knowledge and without going into details of other aspects related to the project. The results of this task are presented in Table 5. As can be seen, the requirements r 9 , r 11 and r 12 have the highest priority for group A. Accordingly, these requirements have a weighted value (V ) of 1, 0.99 and 0.98, respectively, as is defined in formula (1). As for group B, the requirements r 16 , r 1 and r 9 have the highest priority. These obtain a V of 1, 0.99 and 0.98, respectively. On the contrary, the requirements r 2 and r 15 correspond to the least important for group A and B, respectively, both with a V of 0.88. It is important to note that no coincidences are identified between the initial ranking that both groups assigned to the requirements.

B. STAGE 2: IDENTIFY PRIORITIZATION ASPECTS
This task consists of identifying and prioritizing the qualitative aspects (prioritization criteria) related to the priorities of the project. In addition, the standardized priority for each relevant aspect must be calculated (see Section III.B). This task is collectively negotiated among the decision-makers according to the main guidelines and requirements of the project.
The result of this task is presented in Table 6. As can be seen, in group A, a total of 5 benefit aspects have been defined (Benefits related to the strategy, Customer benefits, Financial benefits, Product or system quality, and Operational performance benefits) to formally consider the related customer, product and organizational (strategic, financial and operational) priorities during the prioritization process. In addition, this group has also identified cost aspects of Implementation risks and Development time costs related to project development priorities. As for group B, the benefit aspects of Operational performance benefits, Requirements dependency, Requirements quality, and Customer benefit have been identified, in order to take into account the priorities of organizational results and characteristics of the requirements. This group has also identified a cost aspect (Development time cost) related to project development.
On the other hand, the benefit and cost aspects in group A, of Benefits related to strategy, Customer benefits, Financial benefits, Product or system quality, Operational performance benefits, Implementation risks, and Development time costs have a priority of 7, 6, 5, 4, 3, 2, and 1, respectively, where 7 (Strategy related benefits) indicates the highest priority and 1 (Development time costs) the least important. Accordingly, group A aspects have a normalized priority (P) of 1, 0.86, 0.71, 0.57, 0.43, −0.29, and −0.14, respectively, as defined in formula (2). Note that cost aspects are identified with a negative value through the function BC. Finally, with respect to group B, the Operational performance benefits, Requirements dependency, Requirements quality, Development time cost and Customer benefit aspects have a normalized priority (P) 1, 0.80, 0.60, −0.40 and 0.20, respectively.

C. STAGE 3: IDENTIFY ELEMENTS WITHIN EACH OF THE PRIORITIZATION ASPECTS
Once the initial ranking and the relevant aspects of the project have been identified, the next task corresponds to the identification and prioritization of the aspects' elements defined for the project in the previous stage. Table 7 presents the results of the elements identified by the participants of both groups.
Elements have been identified and prioritized to formally capture the priorities of the pharmaceutical services system, in order to drive the software requirements prioritization process. For example, for the Operational performance benefits aspect, the element User task contribution has been defined to formally consider the importance of the requirements that provide added value to the proposed system's end-users. In this same aspect, the element Cost reduction has also been defined with a lower priority than Productivity improvement (that is, the elements have a priority L of 1 (low) and 3 (high), respectively), in order to give a higher priority to those requirements that improve efficiency of the results of the task, over those related to the effectiveness of its costs.
As shown in Table 7, the group A aspects of Benefits related to strategy, Customer benefits, Financial benefits, Product or system quality, Operational performance benefits, Implementation risks and Development time costs are made up of 3, 2, 2, 5, 5, 2, and 2 elements, respectively. Likewise, the group B aspects of Operational performance benefits, Requirements dependency, Requirements quality, Development time cost, and Customer benefit are made up of 4, 1, 3, 1, and 2 elements, respectively. In addition, each element is assigned a priority (L), using the High, Medium or Low scale. In this way, the result of this task is the set of elements according to the relevant aspects of the proposed pharmaceutical services system.

D. STAGE 4: PRIORITIZE REQUIREMENTS BY LINKING ASPECT ELEMENTS TO EACH OF THE REQUIREMENTS
Once the initial ranking and the relevant aspects' elements of the pharmaceutical services system have been identified, the second last stage of the prioritization method consists of materializing the requirements prioritization. This task consists of identifying the relationships between the relevant aspects' elements (see Table 7 ) and the project requirements (see Table 3 ). Like the previous tasks, these relationships are collectively and dynamically identified and agreed upon by decision-makers (group participants). The results of this task are presented in Table 8 and Table 9 for group A and B, respectively. As can be seen, while the first and second columns represent the aspects and elements, respectively, the following columns represent the project requirements. In this way, it is possible to identify the requirements (marked 1) related to each element, as defined in function C (described in Section III.D).

E. STAGE 5: CALCULATE THE FINAL RANKING
Once the relationships between the project requirements and relevant aspects' elements have been identified, the last stage of the proposed method consists of calculating the requirements final ranking (RFR). To carry out this task, the method considers the different existing relationships (that is, the results of Table 8 and Table 9 for group A and B, respectively) and the functions described in formulas 3 to 8 (Section III.E).
Specifically, to calculate the RFR, two subtasks are performed. The first subtask consists of calculating the relevance level that the requirements have in each of the defined aspects, considering at the same time their initial ranking. First, the total number of related requirements in each element is computed through the function TC, defined in formula (3). For example, in group A, the elements Availability and Accuracy of the Product or system quality aspect have seven related requirements (see Table 8 ). These elements have the maximum number of requirements to which an element of the Product or system quality aspect has been associated. Therefore, the Association Factor (G) for both elements is 1, according to formula (4). Once obtained from the value of G, it is possible to calculate the priority of a certain requirement in some of the related elements. Continuing with the same example, requirement r 5 has in the elements Scalability and Accuracy of the Product or system quality aspect a priority I , defined in formula (5), of 3.57 and 4, respectively. It should be noted that the priority I is different even when both elements have the same priority L (high) since their association factor G is different (that is, 0.57, and 1 for the elements Scalability and Accuracy, respectively). However, the priority above only VOLUME 10, 2022 considers the individual evaluations of each element in the requirements, without a total estimate. That is why formulas (6) and (7) are computed to generate a relevance level for each aspect, considering the initial ranking of the requirements. Table 10 shows the relevance level of each requirement (that is, from r 1 to r 16 ) regarding Product or system quality aspect, as a result of applying the relevance level of the requirements by aspect λ. In this way, the results for the requirements r 1 (4.66%) and r 2 (4.32%) indicate that r 1 has higher priority than r 2 for the Product or system quality aspect. Finally, in the second subtask, the requirements final ranking is generated considering the relevance level of each requirement per aspect and its normalized priority (P), calculated with formula (8).  Table 11 and Table 12 present the requirements final ranking of the proposed pharmaceutical services system for groups A and B, respectively. As can be seen, the requirements are ordered according to their final ranking, calculated with formula (8). Thus, in group A r 10 , r 7 and r 12 are the highest priority requirements, obtaining a final ranking of 75.92, 56.32 and 42.76, respectively. In group B, r 10 , r 2 and r 11 have the highest priority, that is, a final ranking of 24.66, 22.05 and 19.33, respectively. It should be noted that both groups have defined the requirements r 10 and r 16 with greater and lesser importance, respectively.
In short, the application of this case study using the prioritization method in both groups has allowed the participants to fully experience the different characteristics of our proposal. In the following section, a qualitative analysis is provided, obtained from the results of the focus group carried out with all the participants, to understand how the decision-makers reason with respect to the main characteristics of the method.

VI. DISCUSSION
The discussion is presented in the order of the research questions described above. As usual, in qualitative studies, we will complement our findings with quotes from the participants.
RQ1: Is the initial ranking an adequate mechanism to capture the expert knowledge of decision-makers and appropriate to consider it with some weighting in the final ranking?
To define the initial ordering, a preliminary analysis of the requirements by the team members is performed. Each participant provides both quantitative and qualitative arguments based on their own perceptions. To reach a group decision, every participant's opinion regarding their prioritization is highlighted. Therefore, their previous experiences and knowledge relate to the current decision process.  Next, the team must agree on the preliminary ordering that will be proposed as the first phase of the method. This process supports the development of an essential element: negotiation. This dynamic within the group allows the participants to establish their positions and interests. The decision process is a prioritization problem based on alternatives that are beneficial to all stakeholders. The preliminary ordering is, then, the product of an exchange of personal positions based on previous experiences. Therefore, this initial ordering becomes the base of the project's aspects and elements for the second phase of the prioritization method.
The prioritization is oriented towards an enriched negotiation of the participants through an analysis of relevant aspects and elements from professional experience. Meanwhile, the methods of the literature are aimed at reducing the complexity of the discussion by defining objectives or pre-established criteria [11], incorporating the power relationships of the stakeholders [13], or through the automatic discovery of contexts from the documentation [15], [16], [17]. Therefore, this new approach is aligned with the new team structure methods by a more horizontal and people-centered way, formally capturing project priorities and team characteristics. This expert knowledge base becomes explicit by means of the proposed method and evolves to be expressive and open. This, provides sufficient evidence to apply machine learning methods in order to discover inherent relationships between aspects, like the proposals of CBRank [15] and DRank [16].
The negotiation of the project's relevant aspects and elements is the key item for discovering the priorities. Thus, revealing the particularities of each project, by focusing on the benefits for the users and the business, or on the technical complexities of the product properties. This increases the qualitative expressiveness of the proposed method compared to those that focus on the needs and expectations of the stakeholders [5], [18], [19].
This qualitative expressiveness is reflected in the ability to explain aspects that influence the final ranking, which emerge from the expert knowledge of the members of the prioritization team.
RQ2: Is the structure for defining aspects, considering benefits and costs, pertinent and consistent with needs of a dynamic organization?
During the focus group session, developed with both groups, some criteria used during the preparation of the initial ranking emerged. One of the first criteria considered in Group B, among the requirements presented, is oriented to the development dependencies, where the participants mention that they have sought to give technical coherence to the requirements: ''. . . trying to give it an order, a logical order as to how, for example, if I want to manage something, first I have to create it. '' Meanwhile, within the same group, when assigning a priority to the requirements, criteria aimed at business management were considered in combination with dependence on development, as the participants indicated that: ''We didn't give dependency as much priority either, being that we look at the preliminary to order it, that's why there is a mismatch '' By contrary, group A proposes a strategy more focused on the business, stating that: ''Per se, prioritize one more business area or an area like this'' Therefore, business or development-oriented profiles are outlined in the selection of criteria for the justification and assignment of the initial ranking in this working group.
It is important to consider that the technical dependencies between the requirements are never set aside when evaluating coherence by group A, since: ''When one prioritizes activities as such of a project, one first has the technical restrictions of saying, I need this and this because. if not, it has no coherence'' Neither are the relationships with the business goals by group B dismissed, which is reflected in the prioritization through criteria associated with the observable benefit for the client: ''We gave a high score to the benefit to the client during the prioritization exercise that we did'' Subsequently, both groups considered business and technical elements when preparing the initial prioritization of requirements. However, the more relevant aspects in the prioritization are also evident; business focused by group A and development focused by group B. All this is consolidated in the initial prioritization of the requirements that is observed in Table 5. Group A gives the highest priorities to the requirements that deliver the greatest benefits to the business, that is, Create and edit medications, Enter medication registration of a branch, and Enter medication withdrawal. Meanwhile, group B gives a higher priority to the requirements based on their technical dependency, which are, Provide a file repository, Create and edit pharmacy branch, and Create and edit medicines.
The negotiation not only contributes to the explanation of the participants' personal positions, but also establishes the team profile. The ranking is the solution that satisfies most of the parties involved in the analysis; the requirements at the top positions outline the focus of the entire team. Group A, for example, has requirements oriented to the client and the impact they have on critical business processes. Meanwhile, group B oriented their decisions to technical aspects that would allow an incremental development of the software product. Therefore, the establishment of the project's aspects and elements in the prioritization method strongly depends on the negotiation of the preliminary ranking. This is clearly reflected in the aspects of both groups (Table 7), where group A opted for business benefits, such as Benefits related to the strategy, Customer benefits, and Financial benefits and group B preferred operational performance aspects and requirements dependencies, such as Operational performance benefits, Requirements dependency, and Requirements quality. Therefore, in both cases, the relationship between the prioritization positions and the prioritized aspects is direct and consistent with the profile of each group.
Consequently, expert knowledge is strongly represented during the analysis of the preliminary ranking. Therefore, such knowledge is integrated into the final ranking through aspects based on personal positions during negotiation.
The qualitative characteristics, based on the expert knowledge, are clear in the definition of cost and benefit aspects in a dynamic and agile way. Both the project and organizational level are sources of knowledge to analyze the aspects. One of the participants indicated that ''a project manager can actually prioritize their tasks internally, but for an organization it is much more valuable when you are already sorting by project level'', recognizing intrinsic business restrictions that influence ordering decisions and, consequently, the aspects and elements. On the other hand, the business strategy includes the software development process. This is reflected by the participants' discussions during the focus group session, as they gave ''a high score to the benefit to the client because of the prioritization exercise that we did [in the company]''. The participants also considered the impact that the requirements will have on the organizational needs, principally the priority they may have for the client's business as ''what we pay most attention to is the impact on the client''. These organizational needs are strongly consistent with the definition of the aspects, since the Customer benefit was recognized by both team A and B (6 (high) and 1 (low), respectively). Thus, the proposed method can define team profiles.
Consequently, the aspects definition is constrained by two sources: 1) the business, where the aspects associated with benefits and impact on the business are prominent; and 2) the project, where the technical tasks emerge to support the aspects. During the study, two analysis strategies were observed. While the first group prefers the benefits; strategy, clients, and finances; the second group favors operational and technical aspects associated with the requirements, as can be seen in Table 7. Therefore, since both teams belonged to the same organization, aspects that cover the business strategy develop. However, the professional profile of team members supports the generation of aspects and recognizes the participants' previous experience. Likewise, the teams manage to identify aspects not only associated with benefits, but also with project and business costs, which are relevant and pertinent for an appropriate adjustment to the requirements prioritization method.

VII. RESEARCH LIMITATIONS AND THREATS TO VALIDITY
The method proposed can be put into practice by teams in professional or research contexts, according to definitions in each of its stages. The results of the case study described, in which two different professional teams participated, show that the method by itself does not guarantee that its results can be consistently reproduced by different teams. That is to say, two different teams may prioritize requirements differently when applying the method. Moreover, the results obtained by either of the two teams analyzed in this study are not necessarily results that would satisfy expectations and preferences of every competent decision-maker in the organization.
The foregoing is not a limitation that exclusively affects the method presented, but it is a limitation that can potentially affect any method that includes decision-making based on evaluations or judgments by domain experts in an organization. According to the literature, we identify that this can be influenced by different factors. Firstly, it must be acknowledged that requirements engineering activities are knowledge-intensive [47], [48], and that characteristics of knowledge management processes in the organization will significantly influence results of requirements prioritization [49], [50]. Secondly, social interactions that occur among experts who enact the method, and the conditions that facilitate or hinder these interactions, such as cultural factors and organizational norms and structures, will shape the way prioritization decisions are made [51], [52]. Also, the organization's vision and the ability of its leaders to synchronize middle managers and executive roles at different levels with it can influence the requirements prioritization that is achieved through the proposed method [53], [54], [55]. Lastly, the proposed requirements prioritization method falls within the category of group decision making processes [56]. In these processes it is common that decision makers elicit assessments quantitatively. However, it is usually difficult for decision makers to elicit quantitative assessments on aspects that cannot be compared quantitatively and objectively in a straightforward manner. Thus, prioritization can be indeed influenced by domain experts' subjectivity, to the extent that it can be even influenced by their own cognitive distortions, such as the sunken costs fallacy, or confirmation bias. In the case study here presented, this can be considered a threat to validity of results.
Regarding knowledge management in the requirements prioritization process, it must be considered that expert knowledge in the organization takes two forms; tacit and/or explicit [54]. Tacit knowledge is difficult to communicate, transfer and share, as it is highly personal, whereas explicit knowledge can be better communicated, internalized, and synthesized by the parties that collaborate in a requirements prioritization process. Reliance on tacit knowledge, without sufficient explanation and systematization, that is, through adequate documentation and artifacts that allow collaborators to obtain all the relevant information for decision-making, is one of the reasons why different teams may prioritize requirements differently [47], [48]. Therefore, inconsistent results cannot be attributed to the requirements prioritization method itself, but rather to the knowledge management processes that occur in the context where the method is applied. Certainly, the method presented here can be accompanied by practices, tools, artifacts, and knowledge assets that facilitate the availability of complete and consistent explicit knowledge to collaborators. However, this is an aspect beyond the scope of the proposal hereby presented. In addition, the presented case study did not incorporate formal knowledge management methods to deal with these issues, which can be considered a threat to validity of results, due to omission of relevant information or project details. Thus further research is required to determine facilitators of the knowledge management process and the experts who determine the initial requirements prioritization.
In organizational cultures where there are highly hierarchical power relations, conformity with well-established social norms, and conflict avoidance by people, decision makers with higher professional status and greater experience will likely steer discussions and ultimately dominate the decision-making process in favor of their position [51], [52]. However, for the requirements prioritization methodology described here, and its success at meeting strategic objectives and stakeholder expectations, it is desirable that the prioritization process collects judgments of various collaborators, such as managers, executives, experts, and stakeholders from different functional areas, and even stakeholders beyond organizational boundaries to reconcile their interests. In organizations where rivalry or competitiveness exists among different departments or teams, or where a team enjoys greater social reputation or valuation over another, sources of bias may arise in the prioritization process due to power struggles or conflicting interests. This will also mean that the results will not be independently reproducible by different teams in the same organization. It must be noted that in the case study presented here there was no prior evaluation of existing corporate culture to control for these issues, thus this omission can be considered a threat to validity of results. The potential biases identified here could be moderated through the use of technologies for decision-making in a collaborative context, including Group Decision Support Systems (GDSS) [56] that, for example, support the prioritization process through anonymous voting by collaborators, and facilitate negotiation to reach consensual agreements [57].
Finally, an organization can be understood as a space in which knowledge is created [54]. Therefore, decisions made have an inextricable relationship with the knowledge that is created. In the organization, middle managers act as coordinators and executors of the knowledge generation process. Leaders, on the other hand, provide corporate the vision and synchronize the organization with it. Vision determines the identity of the organization, its purpose, its direction and how it achieves its objectives [55]. Also, vision determines the value system that evaluates, justifies and determines the quality of knowledge that informs decisions. For this reason, the requirements engineering processes, and in particular, the implementation of the method proposed in this research, must be carried out under the vision transmitted by the leadership in the organization. If in a software requirements prioritization process focus is on different types of users and their experiences and needs, on process innovation to achieve greater efficiencies, or on cost minimization for competitiveness, this must be understood by decision-makers from a shared vision in the requirements prioritization process. A disparity in relation to this understanding could result in disconnection with the objectives or the vision that should prevail in the organization [53].

VIII. CONCLUSION AND FUTURE WORK
This work presents a qualitative method to prioritize software requirements, considering aspects and elements of benefits and costs, which define the requirements' relevance. The method aims to lead the prioritization process by using qualitative elements to formally incorporate different perspectives based on the professional experiences of each of the team members, supporting the prioritization decisions. The formulation of the method has been presented, as well as the validation through a real use case of the method, allowing analyze of the main characteristics and highlighting the benefits of the qualitative prioritization process based on negotiation and dialogue, key principles in today's software development processes -mainly in people-focused agile developments.
The proposed method is aligned with agile development values that are commonly pursued nowadays in the software industry. In particular, the method provides a well-defined process to achieve effective prioritization. However, stakeholder's qualitative insights are taken into account at the outset of the process, thus it can be affirmed that in the context of the proposed method people are indeed valued over processes. This is an innovative feature of the methodology, when compared to other current prioritization methods found in the literature [6], [8], [9]. The case study presented validates that, by means of the method, a productive social context is elicited within the organization as stakeholders discuss and collaborate on relevant aspects and elements that underpin prioritization decisions.
This people-focused aspect of the proposed method provides a safe space for dialogue between team members, one of the main principles of modern software companies. It is this dialogue, present in the method and in current organizations, that enables negotiations to occur which highlight the professional experience of everyone, that is, expert knowledge that positively influences the considerations generated by the team. The prioritization dynamic contributes to outlining the final ranking based on the professional characteristics of the team. In other words, from the case study it is possible to identify the existence of prioritizations based on the product and the business, which depend on the expert knowledge of the team.
Likewise, the dynamic discussions, proposed by the method, support the definition of the aspects and elements that arise from the experience of the team's professionals. This consolidates a shared knowledge base among the members of each team in terms of evaluation criteria for project requirements. This is not only useful for the ongoing evaluation, but also consolidates and clarifies the knowledge as ranking, aspects or elements that has been generated by the team.
Consequently, the above facilitates nurturing a knowledge base of aspects that can benefit or limit software projects, allowing to appropriately adjust the methodological strategies of software development. This knowledge base allows to drive the change, or the probability of change, in the software businesses' industry, and can support developing automated text analysis methods, allowing implicit relationships to be extracted in the aspects present in the team's knowledge base and, with this, determine regularities specific to each team in the prioritization processes.
As for future work, we hope to extend this approach with methods to deal with multi-criteria group decision making. Linguistic decision-making models that apply computing with words present themselves as a promising approach for dealing with decision problems with high uncertainty and impreciseness. Also, the proposed approach can benefit from adopting a formal language to capture the description of the benefit and cost aspects, which allows consolidating an expert knowledge base shared by software development communities. Likewise, we hope to study the application of the proposed method in contexts of use other than software development, in order to know in detail, the specific objectives that decision makers want to achieve. Similarly, we will tackle the task of discovering new prioritization criteria, which are efficiently adapted to each specific use context. We also hope to carry out long-term comparative experiments, with synthetic data and real users, to evaluate specific characteristics of the proposal. In addition, we want to incorporate various deep learning techniques that allow generating automatic recommendations of relevant elements and aspects according to the main characteristics and guidelines of the projects. Also, we hope to study and identify the different facilitators of the expert knowledge management process, to support the tasks related to the initial requirements prioritization. Finally, we hope to build an easy-to-use CASE tool to apply the proposal and make its use more widespread in modern software companies.