Quality Requirement Documentation Guidelines for Agile Software Development

In agile software development (ASD), where minimal documentation and rapid delivery are the focus, Quality requirements (QRs) are often underspecified and not documented. Guidelines supporting QR documentation task are scarce. The study developed the Agile QR-Doc QR documentation guidelines, which aim to support QR documentation in ASD. We applied a design science research methodology (DSRM) to build the Agile QR-Doc. We used a survey questionnaire and open discussion with ten software practitioners, from two ASD companies to validate the Agile QR-Doc. The practitioners evaluated the guidelines in terms of usefulness, relevance, understandability, and coverage of important aspects for supporting QR documentation and their impact on the agility of the software development process. Agile QR-Doc list 12 recommendations that are grouped into two categories. The first category introduces three recommendations that focus on raising awareness about the significance of QRs, their documentation and related challenges. The second category lists nine recommendations that introduce artifacts, practices and important aspects for documenting QRs. The validation reveals the relevance, understandability and usefulness of the guidelines to support QR documentation in ASD. It also indicates that the guidelines consider important aspects for documenting QRs and that they do not negatively affect the agility of the software development process. Practitioners can utilize the practices, artifacts and knowledge from the guidelines to support QR documentation in ASD. Researchers can benefit from the knowledge on QR documentation in ASD, and application of DSRM in building artifacts.

2 VOLUME XX, 2017 instance, developers misinterpret minimal documentation as no need for documentation [19]. The difficulty in defining and measuring QRs also worsens the challenge of documenting them in ASD [5], [20]. To mitigate the aforementioned challenges and limitations, we believe that QR documentation requires a systematic approach. In this regard, guidelines are beneficial since they can help ASD teams to systematically approach QR documentation. However, such guidelines are scarce [10].
Recently, we conducted a systematic mapping study (SMS) to synthesize the state of the art on management of QRs in ASD (i.e., including elicitation, prioritization, documentation, testing and handling of QRs), by focusing on the strategies and challenges of management of QRs [10]. We found that there are only a few guidelines supporting QR documentation in ASD [13], [15], [22], [24], [26]. Additionally, these guidelines tend to address only specific types of QRs, such as usability or security [15], [22], [24], [26]. Based on our findings from the SMS and to the best of our knowledge, only one guidelines proposal focused on the documentation of QRs in general, in ASD [13]. We also noticed that some of the existing guidelines overlook a wider perspective of the causes of underspecifying and not documenting QRs in ASD and lack validations [13], [15], [26].
We believe that guidelines that take into account a relatively wider perspective of the causes of the lack of documentation, motivations for documenting QRs, and consider the perspectives of different stakeholders in ASD, can better support the documentation and management of QRs. Such guidelines need to be light-weight, empirically built and evaluated to have the potential of supporting practitioners in their QR documentation tasks in ASD. It is necessary that such guidelines do not negatively affect the agility, e.g., the speed and simplicity, of the software development process.
Considering the significance of QRs in software quality management, their economic implications (e.g., their impact on software quality and maintenance costs) [27], informal QR documentation practices and inadequate artifacts which raise the need for systematic approach of QR documentation, and the need for guidelines for documenting QRs in ASD [10], [11], [28], we developed the Agile QR-Doc guidelines. The Agile QR-Doc is developed under the Q-Rapids project 1 , which was a European Union Horizon 2020 project made up of industrial and academia consortium. The project was aimed at deriving a quality aware rapid software development framework, with a focus on improving software quality levels, shortening the time-to-market of software and increasing productivity by providing a tool support for quality aware rapid software development. The companies in the Q-Rapids consortium, provided a relevant and useful industrial context to explore the documentation and management of QRs in ASD and build our guidelines. 1 https://www.q-rapids.eu/about We applied a design science research methodology (DSRM) to develop the Agile QR-Doc guidelines [29]. In order to develop the guidelines, we explored the state of the practice of QR documentation in ASD, by conducting multiple case studies in companies applying ASD, within the Q-Rapids project consortium [8], [13], [30]. We also synthesized the state of the art of management of QRs in ASD through a SMS [10]. By using the findings of these studies ( [8], [10], [13], [30]), we analyzed the knowledge on existing QR documentation and management challenges and practices in ASD to build the recommendations of our guidelines. We validated the guidelines by evaluating them with ten software practitioners from two ASD companies of the Q-Rapids consortium, through survey questionnaires and open discussion. These practitioners have relevant expertise on the topic and included roles such as specification engineers, software project managers, and software architects. Agility was given attention during the development and validation.
The Agile QR-Doc guidelines have 12 recommendations which are organized into two categories. The first category focus on raising awareness on the significance of QRs, their documentation and related QR documentation challenges. The second category focus on introducing artifacts and practices, which are aimed at supporting and enhancing QR documentation tasks in ASD. Examples of recommendations in the guidelines include recognizing the significance of QRs, applying lightweight artifacts to document QRs, ensuring the clarity of QRs, and utilizing documentation artifacts built for specific QRs. The guidelines provide the rationale behind each of the recommendations and ways to realize the recommendations along with illustrative examples.
Software practitioners can utilize our work to systematically approach and support QR documentation in ASD. For instance, they can adapt light-weight artifact such as 'Given/When/Then' template, and system stories to document QRs, utilize UserXstories to document usability QR, and document security QRs using abuser stories and misuse cases. Product owners can get insight into actions that they can consider for supporting QR documentation. By using the guidelines, practitioners can spot the indicators of QR documentation challenges and take corrective actions on time.
Our work contributes to software engineering research by filling the gap in guidelines that support QR documentation in ASD. Researchers can utilize the knowledge created in building Agile QR-Doc, get a better understanding about the documentation and management of QRs in ASD. They can utilize the knowledge to develop solutions supporting QR documentation and management in ASD too.
Section II presents the related work. Section III presents the research methodology followed in the study. Section IV presents the Agile QR-Doc. Section V reports the validation of Agile QR-Doc. Section VI discusses the implications of the study and compares the Agile QR-Doc with related work. Section VII discusses the threats to the validity of the study. Finally, Section VIII concludes the paper.

II. Related work
This section presents the related work on QR documentation in ASD and the guidelines for documenting QRs in ASD.

A. Documentation of QRs in ASD
Requirements documentation aims at conveying and sharing requirements needs among development teams and other stakeholders. ASD promotes minimal documentation practices, the rapid delivery of working software and early return on investment to customers [32]. These, combined with its capability to respond to frequently changing requirements, have increased the popularity of ASD, and it is currently well established in the software industry [33]. In ASD, the continuous interaction with the customer is thought to lessen the need for the specification and documentation of QRs [34]. For instance, practitioners may communicate QRs in white board discussions, understand them implicitly, and not document them properly [6], [13], [35].
ASD favor 'Working software over comprehensive documentation' [36]. However, this value is often misunderstood as no requirement for documentation, leading to informal requirement specifications, which result in problems in the long run [12]. Ambiguity in what minimal documentation means, coupled with the elusive nature of QRs [6], [20] can render the documentation of QRs in ASD more challenging. Consequently, QRs are usually underspecified and undocumented and are handled improperly, resulting in accruing technical debt (TD), increased maintenance costs, and at times, project failures [5].
Requirement engineering artifacts and practices in ASD are often insufficient for specifying and documenting QRs too [10], [11], [17]. For instance, specifying QRs, such as security and safety, in user stories is more difficult than specifying FRs. Documenting such QRs require end-to-end documentation covering many aspects [37]. Although we can clearly specify and mark FRs specified in user stories as done, it is not always easy to specify and mark QRs, such as maintainability and security, in user stories as done [23]. Since these QRs may change repeatedly during development, ensuring that they are specified and managed properly requires continuously revising their documentation [30]. The literature also reveals challenges due to practitioners' lack of expertise in QRs affect QR documentation in ASD [21].

B. Guidelines for documenting QRs in ASD
The literature reveals guidelines proposals that support the documentation of QRs in ASD [13], [15], [22], [24], [26]. However, these guidelines, except [13], are focused on supporting the documentation of specific QRs, such as security and usability [22], [24], [26], and do not address the documentation of QRs in general.
Barbosa et al. [22] provide a guide for improving security measures in ASD based on an explorative literature review and interview study with practitioners. The guide recommends security backlogs to document security QRs and a corresponding security master who is responsible for defining and documenting security QRs in the security backlog. It also proposes evil user stories to document security requirements and recommends research vulnerabilities activity for the continuous identification and update of vulnerabilities and related security documents. The authors evaluated the guidelines by interviewing four practitioners.
Lee et al. [24] provide a usability pattern-based requirements analysis method and guidelines to improve usability specification in ASD. Their guidelines transform user tasks into features of a user interface specifying usability factors. They empirically evaluated the guidelines and methods and identified that these guidelines and methods supported usability requirements specification in ASD. Similarly, Cajander et al. [15] provide recommendations for addressing the documentation of usability and user experience in Scrum by interviewing practitioners. The recommendations included assigning clear responsibilities for handling usability and including clear and measurable usability goals in the specification.
There are also guidelines which lack empirical validation [13], [26]. For instance, our previous work [13] provides guidelines for documenting QRs in ASD. The guidelines consider classifying QRs by scope (i.e., system, group and local) and level of detail of the QRs (i.e., generic and detailed) and recommends using artifacts such as epics, user stories, and acceptance criteria. The system-wide scope applies to QRs covering the entire system and the group-wide scope applies to those QRs relating to more than one functionality. The local-scope applies to QRs relating to only one functionality or user story. For instance, a QR with a system-wide scope can be either generic or detailed. If it is generic, the recommendation is to use an epic to document the QR by clearly referring to the system. On the other hand, if it is detailed, the recommendation is to use a user story, clearly describing the functionalities to be covered by the QR. These guidelines lack empirical validation, thus, limiting their adoption to the industry.
Similarly, Wäyrynen et al. [26] propose recommendations to improve the documentation and management of security requirements in extreme programming (XP). The recommendation introduce a security engineer who will be responsible for documenting security-related user stories. This security engineer would be involved in pair programming to ensure the implementation of security requirements, documenting security architecture, and adding security review activity. These recommendations lack empirical validation, and this may limit their applicability.
Agile QR-Doc guidelines, which will be introduced shortly, complement existing guidelines merely focusing on specific types of QRs and those that lack empirical validation by 4 VOLUME XX, 2017 industrial practitioners. The gap in QR documentation guidelines, combined with challenges due to informal QR documentation practices (e.g., not strictly defining and documenting QRs, under-specification, outdated and missing documentation of QRs in ASD) [2], [15], which in turn have a negative impact on software projects [10], [16], motivated the need for developing Agile QR-Doc guidelines.

III. Research methodology
We adapted the DSRM by Peffers et al. [29] to build the Agile QR-Doc QR documentation guidelines. DSRM is a methodology used for conducting design science research, where artifacts that are aimed at solving a specific problem are developed and evaluated rigorously [29]. According to Peffers et al. [29], developing an artifact using DSRM involves the following activities: problem identification and motivation, defining the objectives of a solution, design and development, demonstration and evaluation, and communication. In what follows, we describe the DSRM activities applied to build the Agile QR-Doc set of guidelines.
Activity 1: Problem identification and motivation: the activity outlines a specific research problem and motivates the need for the design solution. QRs have great importance in determining the success of software projects [2]. However, in ASD, they are often neglected, under-specified and not documented resulting in poor quality software and increased maintenance costs [5], [14].
We conducted a preliminary work investigating the management of QRs in the Q-Rapids companies, and identified that ASD companies face QR documentation challenges such as the lack of traceability for QRs, difficulty in identifying dependencies between different QRs, and internally generated QRs being not documented [13]. We also conducted a SMS synthesizing the state of the art on challenges and strategies of managing QRs (i.e., including elicitation, prioritization, documentation, testing and handling of QRs), in ASD [10]. Our findings revealed that there are limitations with existing ASD artifacts in documenting QRs, challenges due to informal QR documentation practices and limited QR knowledge. There was a need for lightweight QR documentation and management strategies. We found that existing guidelines that support QR documentation and management are scarce. We observed that existing guidelines focused only on addressing specific QRs such as security and usability. Some of the guidelines lacked empirical validations ( [13], [15], [26]), limiting their adoption in ASD. These findings reveal the need for solutions, e.g., guidelines that support systematic approach of QR documentation in ASD. Therefore, we defined our research problem, a gap in guidelines supporting QR documentation in ASD.
Activity 2: Define the objectives for a solution: this activity defines the objective of the solution or artifact developed to address the research problem. This research builds the Agile 2 https://sites.google.com/view/qr-doct/home QR-Doc guidelines to solve the gap in guidelines that support QR documentation in ASD. The guidelines aim to support QR documentation in ASD, without compromising the software quality and agility of the process. Activity 3: Design and development: we used knowledge and experience that we obtained from researching documentation and management of QRs in ASD [8], [10], [13], [30], as inputs to design and develop the Agile QR-Doc guidelines. These knowledge were gained by studying the state of the practice of documentation and management of QRs in ASD, with companies involved in the Q-Rapids consortium through case studies [8], [13], [30] and by synthesizing the state of the art of the management of QRs in ASD through a SMS [10].
We first developed the Agile QR-Doc guidelines by synthesizing nine recommendations. To synthesize these recommendations, we analyzed knowledge on the existing challenges of QR documentation and management in ASD, and the drawbacks of existing QR documentation practices and artifacts in ASD, we found from our studies [8], [10], [13], [30]. Our analysis revealed that QR documentation challenges are associated with the limited QR documentation awareness and associated factors, and the limitations of QR documentation artifacts and practices. Therefore, we built recommendations by focusing on: I) aspects related to raising awareness on QRs, documentation needs and challenges, and II) introducing artifacts and practices to support QR documentation. When building our recommendations, we used knowledge about QR documentation in ASD (e.g., practices, factors affecting QR documentation) that we obtained from our studies [8], [10], [13], [30]. Agility was a focus when building the recommendations. Following the synthesis of the recommendations of the Agile QR-Doc guidelines, we developed a prototype of the guidelines in the form of a website 2 . Figure 1 shows screen capture of the prototype. Activity 4: Demonstration and evaluation: in our study, we applied one iteration of demonstration and evaluation. We demonstrated the use of Agile QR-Doc guidelines and evaluated them with ten practitioners of two ASD companies. The evaluation was used to determine how well the guidelines support QR documentation in ASD, and collect feedback to enhance them. It also served to validate the applicability of the guidelines for industry. Informed consents were used in this step. Details regarding the demonstration and evaluation of the guidelines and their findings are discussed in Section V.
Activity 5: Design and development to enhance the guidelines: following the demonstration and evaluation activity, a DSRM research can continue to the communication step. It is also possible that researchers go back and redefine the objectives of the solution. Another possible path is to proceed with the design and development and enhance the artifact based on the findings of the evaluation. In our case, we iterated back to the design and development. We used the knowledge obtained from evaluating the guidelines to improve them and include 3 new recommendations.
Activity 6: Communication: involves reporting the research problem, its significance, the design artifact and its utility, and the design process to the relevant audience. In our case, we report all the aforementioned aspects in this publication. Figure 2 shows the DSRM applied in building Agile QR-Doc.

IV. Agile QR-Doc: guidelines to support quality requirements documentation in agile software development
The Agile QR-Doc QR documentation guidelines provide 12 recommendations aimed at supporting documentation of QRs in ASD. They aim to do so without negatively affecting the software quality and agility of the software development process. The recommendations follow the format: recommendation title, description, justification, ways to realize the recommendation, and related work. The recommendation title identifies the recommendation, and the description explains details of the recommendation. Then, we justify the reason for the recommendation, list the means to implement the recommendation, and finally, present related work regarding the justifications and means of realizing the recommendation. The recommendations are organized into two categories. These are: A) Recommendations emphasizing awareness on QRs, their documentation and related issues and B) Recommendations that introduce artifacts and practices to support and enhance QR documentation.

A. RECOMMENDATIONS EMPHASIZING AWARENESS ON QRS, THEIR DOCUMENTATION AND RELATED ISSUES
In this category, the need for recognizing the importance of QRs, optimally documenting them and recognizing indicators of QR documentation challenges in ASD, are justified with the goal of supporting QR documentation. Strategies for realizing the recommendations are presented too.

1) RECOGNIZE THE SIGNIFICANCE OF QRS
Description: ASD teams and management need to acknowledge the importance of QRs in ASD and take actions that support raising awareness on QRs and their documentation. Justification: In cases where ASD teams, customers, and management have limited knowledge regarding QRs, they may not recognize, specify, and handle QRs properly. ASD teams may ignore and not document QRs until there is a specific request for handing them or until the impact of neglecting QRs becomes visible [5], [6], [10], [11], [38], [39]. Ways to realize recommendation a) ASD management and teams (e.g., developers, product owners, and testers) should develop an awareness on QRs and the skills for documenting and managing QRs.  Management and teams should invest in training on QRs in general, and on specific QRs such as security, and usability. b) ASD teams can utilize a company-specific agile playbook [8], for QRs, which is a lightweight guide describing QR classifications, their significance, and ways to document and manage them. c) ASD teams should adopt a 'quality mindset' by emphasizing QRs in the different steps of the software lifecycle and ensuring that corresponding QR needs are met. d) ASD teams can assist customers in recognizing the importance and value of QRs, since customers in some domains may overlook important QRs or may not have adequate knowledge about QRs [40]. For instance, customers in the energy domain may overlook important QRs such as security [41].

2) RECOGNIZE THE NEED FOR OPTIMAL DOCUMENTATION OF QRS
Description: Agile teams should acknowledge the need for optimal documentation of QRs (i.e., a satisfactory level of QR documentation that will not compromise agility and product quality). They should take actions to meet the need for optimal documentation of QRs and start documenting QRs at early stages. Justification: ASD's focus on minimal documentation does not translate to the avoidance or neglect of documentation. However, evidence shows that practitioners misinterpret the 'minimal documentation focus of ASD' as a call for no documentation [19]. Such misinterpretations lead to the lack of proper documentation and management of QRs [8], not strictly specifying QRs [15] and confusions regarding QR deliverables [42]. Therefore, practitioners should adopt practices that enable optimal QR documentation. Ways to realize the recommendation a) Educate ASD teams about the benefits of optimal QR documentation. b) Establish processes that help ASD teams adopt optimal QR documentation (e.g. applying agile playbooks, QR documentation templates). Related work: [5], [8], [10], [15], [19], [30], [42].

3) RECOGNIZE THE INDICATORS OF QR DOCUMENTATION CHALLENGES
Description: ASD teams and managers should be able to notice and identify different indicators of QR documentation challenges, the consequences of poor QR documentation practices and take actions to minimize or prevent them. Justification: ASD teams can proactively work on minimizing the challenges of QR documentation if they distinguish the indicators and consequences of challenges of documenting and managing QRs [9].
Way to realize the recommendation: ASD teams and managers should be aware of the indicators of QR documentation challenges, recognize the consequences of poor QR documentation practices, and utilize actions that help minimize or prevent the challenges (see Table 1). Related work: [8]- [10], [13], [30], [43].

B. RECOMMENDATIONS THAT INTRODUCE ARTIFACTS AND PRACTICES TO SUPPORT AND ENHANCE QR DOCUMENTATION
In this category, different artifacts for documenting QRs in ASD are provided with examples. Practices that support QR documentation and enhance aspects of documenting QRs in ASD are also suggested.

1) APPLY LIGHTWEIGHT ARTIFACTS TO DOCUMENT QRS
Description: ASD teams should be aware of lightweight artifacts that are available for specifying and documenting QRs and utilize them as needed. Justification: ASD teams face challenges from confusions on QR specification approaches [44]. In this regard, they should acquaint themselves with lightweight artifacts that are used for documenting QRs in ASD, and utilize them as needed. Ways to realize the recommendation a) User stories with acceptance criteria: ASD suggests user stories to specify and document requirements by focusing on users' goals. User stories can be used to document QRs by specifying them from users' perspective and explicitly attaching an acceptance criteria for the QRs, and a user story Id, which identifies the user story and its version. An example of a user story for documenting QRs is as follows:  User story Id: QRV1.0 User story: As an <end user>, I want to <the app to support multiple language > So that <I can access the app using Amharic, English, and Finnish language interfaces> Acceptance criteria: perform internationalization tests to ensure support for Amharic, English, and Finnish languages and to ensure that the corresponding layout formats function correctly. Note: in ASD, QRs may evolve and change. In such cases, the user story, and its acceptance criteria should be updated to reflect the corresponding change. b) Given/When/then templates: ASD teams utilize 'Given <>/When <>/Then <>' templates for specifying QRs [30]. For instance, a QR for battery consumption can be specified as:  Given <the battery level is less than 20%> When <the phone is up and running> Then <raise an alert for battery saver mode>. c) Apply a system story to specify QR: utilize system stories to specify features related to QRs that could not be allocated in user stories. System stories describe QRs which spread over multiple user stories [25], [47]. For instance, we can write a system story for a reliability QR verifying the presence of a SIM card interface in a mobile phone device by including a brief summary of the rationale behind the system story, as shown in Table 2. We need to verify the SIM card interfaces in the mobile phone are present and functioning properly. d) Specify QRs as part of the Definition of Done (DoD) of FRs: the DoD is a criterion for accepting tasks as done. QRs can be documented in the DoD of epic-, story-, and task-level requirements. The recommendation when using a DoD is to keep the DoD list as small as possible [30].  An example of a DoD that can be attached to a user story describing a login form can be written as 'Perform SSL (Secure Sockets Layer) certificate test for login form'. e) Apply iterative and incremental prototypes: companies that adopt model driven development in the context of ASD can apply iterative and incremental prototypes to document QRs. Here, the approach enables the discovery and documentation of QRs and the update of QRs as the system evolves [30]. It is also recommended that such prototypes are reviewed with a focus on QRs and the big picture of the system as the system evolves [45]. f) Specify QRs in the definition of ready: The definition of ready concept provides a means of checking for what work or task can enter a sprint [10], [46]. We specify QRs in the definition of ready for user stories that contain a definition of the user story, its acceptance criteria, dependencies with other user stories, a list of QRs, architecture criteria, a person responsible for accepting the user story, and reviewers of the user story. Related work: [10], [25], [30], [44]- [47].

2) UTILIZE DOCUMENTATION ARTIFACTS BUILT TO DOCUMENT SPECIFIC QRS, SUCH AS SECURITY AND USABILITY
Description: ASD teams may use documentation artifacts tailored to specify and document specific QRs, such as security and usability requirements. Justification: Security and usability are commonly discussed QRs in the literature on the management of QRs in ASD [10]. However, ASD user stories are seen as inadequate for specifying and documenting these QRs. In this regard, the recommendation is to use artifacts tailored to specify and document security and usability [10]. Ways to realize the recommendation a) Apply abuser stories to document security requirements: an abuser story describes the interaction of a malicious user posing a threat to a system and the potential threats and risks to the system that may occur from the malicious interaction [26]. Misuse stories and evil stories are other terminologies describing a similar concept. ASD teams should discuss abuser stories together with the customer team to ensure their relevance and importance. Abuser story examples for addressing security concerns that may arise from a potential malicious user with access to regular accounts in a web portal system of an institution can be written as follows:  As a malicious employee, I can change my account privileges, so that I can get access to admin rights and manipulate the system.  As a malicious employee, I want to brute force the login to steal business data. b) Apply misuse cases to document security requirements: misuse cases describe the sequence of events that may happen during security incidents, and they allow developers to visualize vulnerabilities and create countermeasures to resolve such vulnerabilities that may arise from the security incident [48]. Figure 3 shows a misuse case example of a malicious user of an online web store who may use the 'add review comment' feature on the website by adding malicious scripts in the review comments of the online web store. Here, we have three different actors: a regular user who adds review comments about a product with no malicious intentions, a malicious user whose purpose is to intrude the system by adding scripts in the review comments, and the admin, who moderates the review comments and detects malicious events. When a malicious user attempts to intrude the system by adding malicious comments, the admin can blacklist the ID and take further action.

3) ALLOCATE ROLES AND RESPONSIBILITIES FOR DOCUMENTING AND MANAGING QRS IN ASD
Description: Assign roles to individuals with expertise on QRs to assist in the documentation of QRs in ASD. Empirical evidence from the literature reveals that assigning roles to individuals with expertise in security, usability, and quality assurance have been helpful for documenting and managing QRs [17], [52], [53]. Justification: The lack of expertise in QRs is a challenge in documenting and managing QRs in ASD. Product owners and project managers may not always have the required skills for defining QRs, such as security and usability [10], [54]. They are also prone to forgetting QRs [55]. Some product domains may also have strict requirements and need separate organizations to handle QRs. Ways to realize the recommendation: a) Assign security masters and security experts for security QRs: the literature reports that such roles are beneficial in specifying and analyzing the security requirements and the involvement in related discussions with agile teams [10], [30], [52]. b) Assign usability experts and usability product owners for usability QRs: The literature reports that such roles have been beneficial in eliciting usability requirements, documenting, and facilitating usability discussions with agile teams. We can also have interaction designers that support the elicitation and documentation of QRs [10], [17], [53], [55]. c) Assign separate organizations or teams to document and handle QRs in the process: companies with product domains that require strict regulations (e.g., embedded systems, telecommunications) and ASD, may employ separate organizations to handle documentation and manage QRs (e.g., security organization or security team, performance teams) [10], [30]. Related work: [10], [17], [30], [52]- [55].

4) ENSURE CLARITY OF QRS WHEN DOCUMENTING THEM
Description: To ensure a clear specification and documentation of QRs. The specification and documentation of QRs need to be clear and unambiguous, precise, and measureable. Justification: The lack of clarity of QRs is a challenge reported in the ASD literature [54]. When product owners and project managers fail to specify QRs clearly, ASD teams may lack a common understanding of the QRs and consequently, adopt their own interpretations. These may lead to wrong implementations and at times, create friction among team members [8], [10], [11]. Ways to realize the recommendation: a) Use QR metrics, i.e., metrics for measuring QRs [56], to specify and document QRs in clear, precise, and measureable terms. For example, we can specify maintainability as 'complexity of files should be below 20%' by using the metric, complexity of files. b) Encourage open discussion to avoid ambiguous descriptions and definitions of QRs Related work: [8], [10], [11], [54], [56].

5) APPLY LEVEL OF ABSTRACTION WHEN SPECIFYING QRS
Description: The level of abstraction of a QR describes the granularity level of the QR (i.e., the scale or level of detail of the QR). The recommendation is to apply different levels of abstraction (i.e., splitting and refining QRs from higher to lower levels with corresponding optimal information) when specifying and documenting the QRs. Justification: Granular QR specifications are problematic in ASD [54]. An abstraction of requirements helps in communicating requirements better [57]. An appropriate level of abstraction can help in clarifying the QR task and achieving good effort estimation of the corresponding task [58].

Way to realize the recommendation:
Apply different levels of abstraction that entail optimal levels of detailed information appropriate for the corresponding granularity level of the QRs. One way to do this is to split and specify QRs as epics, stories, and tasks at decreasing levels of granularity. An epic describes a high-level requirement [30]. A story represents a refined, lower level requirement of an epic [13]. An epic is split and refined into many stories, and a task represents a QR item to implement. For instance, one can specify a testability QR at the epic level as 'The system shall be testable,' and this epic can be refined into multiple stories and tasks. Table 3 shows a test coverage story, which is under the testability QR epic. We specify the story by providing descriptions and DoDs and linking it with the epic.

DoD
With the additional test cases, the test coverage should reach 100%.

Epic link
The linked epic is the testability QR (The system shall be testable) Related work: [13], [30], [54], [57], [58] 6) ENSURE THE TRACEABILITY OF QRS Description: create and keep a linkage among different levels of abstraction of the QRs when documenting them. Document the dependencies among QRs and FRs when needed. Justification: ASD faces limitations in traceability mechanisms for QRs [11], [13]. Limited traceability to higherlevel requirements (both QRs and FRs) may slow down corrective actions taken to resolve errors during development [54], [59]. Practitioners can easily identify and correct QR issues when taking corrective actions based on documented QRs, which enable backward traceability to higher level requirements [28].

7) DOCUMENT DECISIONS RELATED TO QRS
Description: document important QR-related decisions, the rationale behind the decisions, and the affected components. QR-related decisions should be documented and saved in requirements management repositories that can later serve as knowledge repositories. Justification: Informal QR documentation practices lead to missing knowledge and rationale behind already made decisions regarding QR [2]. Documenting QR decisions will help minimize knowledge that may otherwise remain tacit to practitioners or be lost when these practitioners leave the company [21], [30], [55]. Ways to realize the recommendation: Small, collocated, and collaborative ASD teams may prefer to communicate QRs in whiteboard meetings and decide to document only important QR decisions [13]. In such scenarios, they should precisely document the decision regarding the respective QR along with a rationale for the This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and content may change prior to final publication. decision as shown in Table 5. Related work: [2], [13], [21], [30], [55].

8) APPLY SEPARATE OR SPECIFIC BACKLOGS TO DOCUMENT QRS
Description: utilizing separate or specific backlogs, which run in parallel to the product backlog, to document QRs identified during the development process and distinct QRs. When applying a specific or separate backlog, we should ensure that it does not affect the agility of the software development process. Justification: QRs which are discovered during the development process of ongoing sprint can be left undocumented [13]. At times QRs such as security may not receive adequate attention compared with other functional requirements in sprint backlogs [60]. Utilizing a distinct backlog to document specific QRs can help to give more focus to QRs [41]. Ways to realize the recommendation: a) Improvement backlog: a separate backlog that runs in parallel to sprint backlogs and is used to document QRs and improvement items which are identified by development team during the development process of an ongoing sprint [30]. b) Security backlog: separate backlogs allocated for specifying security requirements in ASD [41]. Items within the security backlog can be written in the form of abuser or misuse stories to describe how a person with malicious intent could do harm. Each item is prioritized according to risk and needs to be linked to product backlog items. Security backlogs can be better maintained by including a person who has expertise in security [10]. c) Shadow backlog: a separate backlog that contains issues such as inconsistencies and omitted usability tasks [61]. Related work: [10], [13], [30], [41], [60], [61].

9) WHEN SPECIFYING QRS ENSURE DOCUMENTING THE MEANS TO VERIFY AND VALIDATE THEM
Description: While specifying QRs at different levels of hierarchy, it is important to clearly document how to verify and validate the corresponding QRs. Justification: in ASD verification of QRs can be problematic in cases where QRs are defined without reaching agreement on the verification method [16]. Failing to specify the verification and validation aspects of QRs can lead to an underperforming product that does not meet the QR expectations of end users [16]. For instance, when the verification aspect of capacity of a login page is not considered and specified clearly, testing teams may not verify and validate the requirement, and the companies may deliver software that underperforms and does not meet end users' expectations [11]. Ways to realize the recommendation: Specify how the QR is to be verified and validated:  Peer reviews, design reviews, code reviews, application program interface (API) reviews [30].  Customer reviews (review sessions with customers and/or users to verify and validate QRs) [30].  Continuous integration [10].  DoDs and checking them against acceptance criteria [30]. Related work: [10], [11], [16], [30].

V. Validating Agile QR-Doc guidelines
This section presents validation of the Agile QR-Doc. Section A presents the validation design and Section B presents the findings from validation of the guidelines.

A. Agile QR-Doc guidelines validation design
In order to validate the guidelines and ensure their suitability for ASD, we considered: 1) Evaluating the usefulness, relevance, and the understandability of the guidelines, with respect to supporting QR documentation in ASD. The measurement of information quality aspects, i.e., of perceived usefulness, relevance, and understandability, provide an insight into users' satisfaction levels regarding the quality of information systems [62]. 2) Evaluating the impact on the agility of the software process, i.e., in terms of speed, and leanness or being simple enough to use in ASD. Speed and leanness or simplicity are relevant aspects which are used to measure agility of software process [71]. We believe that these provide an insight on the compatibility of the guidelines with ASD. It helps understand whether the guidelines are simple enough to use in supporting QR documentation tasks in ASD, and determine that their applications do not compromise the speed of development.

3) Evaluating the level that important aspects for
documenting QRs in ASD are covered in the guidelines. This helps determine whether the guidelines cover important aspects for documenting QRs in ASD. 4) Gathering suggestions for enhancing the guidelines.
Accordingly, we formulated the following research questions: RQ1. How do practitioners perceive the usefulness, relevance, and understandability of the guidelines in supporting QR documentation? RQ2. How do practitioners perceive the impact of the guidelines with respect to agility, i.e., the speed and leanness or simplicity, of the software development process? RQ3. Do practitioners perceive that the guidelines cover the important aspects to consider when documenting QRs in ASD? What aspects do they consider important when documenting QRs? RQ4. What do practitioners suggest for improving the guidelines?

PARTICIPANTS USED FOR VALIDATING THE GUIDELINES
Ten software practitioners, from two companies within the Q-Rapids consortium, participated in validating Agile QR-Doc (see Table 6). These two companies have rich experiences of ASD. QRs have great importance in their contexts. The first company (C1) operates in the embedded systems and telecommunications domain. At the time we run the validation, C1 has been applying ASD for 14 years and was using Scrum. It is medium-sized company with over 600 employees. C1 applies QR documentation practices and tools (e.g., applying agile playbook guidelines, and using JIRA to document QRs). The second company (C2) is a global, largescale telecommunications company with over 100,000 employees. It has been applying ASD for 12 years and uses large-scale distributed ASD.  [63] and consulted the representatives of the companies to recruit participants who have experience in requirements specification and management, to validate the guidelines. The key informant technique offers collecting feedback from relevant experts on a specific topic. In our study, we had participants, who have broad experience in software engineering and ASD.

GUIDELINES
Below, we describe the demonstration and evaluation activity, in the DSRM process, that we applied in each company to validate the guidelines.
Company 1: We organized the demonstration and evaluation session with five practitioners of the case, in October 2019. The duration of the demonstration and evaluation was 40 minutes. 1) First, a researcher presented the objective, scope, and the intended target of the guidelines. 2) Then the researcher demonstrated the use of Agile QR-Doc guidelines by going through the recommendations of the guidelines and briefly explaining how the recommendations can be used to support QR documentation, using the prototype. 3) We asked each participant to explore the guidelines in detail for ten minutes. Each participant explored the guidelines independently using the prototype for ten minutes. 4) Following the exploration, we asked the participants to provide their suggestions for improving the guidelines in open discussion. Their feedback was audio recorded and later transcribed for data analysis. 5) We asked each participant to answer a survey questionnaire. The survey questionnaire was structured in a way that it addressed RQ1-RQ4 (see Appendix). We used the Webropol online survey tool to collect the participants' feedback through an online questionnaire. Company 2: the demonstration and evaluation were done with five practitioners of the case separately, in October 2019. We performed the demonstration and evaluation with each of the practitioners separately. We did this because we were not able to organize the session by involving all participants at the same time. The demonstration and evaluation session with each of the practitioners lasted between 35 and 40 minutes. We followed similar steps as in company 1, i.e., the steps 1-5 described above.
In the survey questionnaire, we used a five-point Likert scale (1 = strong disagreement, 2= disagreement, 3 = neutral, 4 = agreement, and 5 = strong agreement) to collect feedback. We collected feedback regarding the usefulness, relevance, and the understandability of the guidelines and the impact of applying them in terms of agility. We also collected feedback using an open-ended question in the survey regarding the important aspects that the ASD practitioners consider when documenting QRs. We audio recorded the sessions of the demonstration and evaluation in both companies. The audio was professionally transcribed for analysis purpose. 12 VOLUME XX, 2017

DATA ANALYSIS OF COLLECTED FEEDBACK
In order to analyze data collected during the demonstration and evaluation phase, we applied quantitative and qualitative analyses. We used quantitative analysis for the results of the survey questionnaire. We report descriptive statistics, such as the sample size (N), minimum (Min), median (Mdn), and maximum (Max) values of the ratings. We also used a onesample Wilcoxon signed rank test [64], which is used to examine variations in the responses of the participants in cases of small sample sizes [64], [65]. We applied the one-sample Wilcoxon signed rank test to examine how the evaluation results differed from the median value, which was 3 or neutral in our scale. We report the results of the one-sample Wilcoxon signed rank observed value (V) and the significance level (p) by setting a 95% confidence interval for tests.
In order to analyze the qualitative data, we applied thematic analysis [66], to synthesize aspects that participants identify as important in documenting QRs and their suggestions for improving the guidelines. We first coded each of the responses as 'important aspects' and 'suggestions for improvement'. Then we compared the responses under each of these labels. We then grouped and labelled closely related concepts under themes. However, during this process there were also nonrecurring concepts that did not form a theme. These are reported as non-recurring themes.

B. Results from Agile QR-Doc validation
In this section, we report the findings from the validation of the Agile QR-Doc guidelines. Figure 4 plots summary of the findings of RQ1 and RQ2, regarding the usefulness, relevance, and the understandability of the guidelines and their impact on agility.

1) Usefulness, relevance, and the understandability of the guidelines in supporting QR documentation (RQ1)
We found that the practitioners perceived the guidelines to be useful to support the optimal documentation of QRs, as evidenced by the descriptive statistics and the one-sample Wilcoxon sign ranked test result (N = 10, Min = 3, Max = 5, Mdn = 4, V = 45, P = 0.006008). Similarly, nine respondents perceived the guidelines as useful for improving software product quality (N = 9, Min = 4, Max = 5, Mdn = 4, V = 45, P = 0.007412). However, one of the respondents did not respond to the question.
Regarding relevance, we asked whether they consider the guidelines helpful in identifying relevant actions for documenting QRs in ASD and whether the guidelines are appropriate for documenting QRs. Regarding understandability, the participants identified the guidelines as understandable enough to use for documenting QRs in ASD (N = 10, Min = 3, Max = 5, Mdn = 4, V = 45, P = 0.007412). Overall, the evaluation of the information quality aspects of the guidelines indicated that the guidelines were perceived as useful, relevant, and understandable to support the documentation of QRs in ASD.
2) The impact of the guidelines with respect to the agility, leanness or simplicity and speed, of the software development process (RQ2) The practitioners perceive the guidelines as simple enough to use in ASD (N = 10, Min = 3, Max = 5, Mdn = 4, V = 45, P = 0.006008). Regarding agility in terms of speed, we asked if they thought that the guidelines did not hinder ASD process. They perceived that the guidelines did not hinder ASD process (N = 10, Min = 2, Max = 5, Mdn = 4, V= 32, P = 0.04183). However, one respondent disagreed, whereas two others were neutral. The practitioners also perceived that the guidelines are helpful in the earlier consideration and documentation of QRs (N = 10, Min = 3, Max = 5, Mdn = 4, V = 45, P = 0.007412).

3) Coverage of important aspects for documenting QRs in ASD within the guidelines and aspects that the practitioners consider important when they document QRs (RQ3)
We asked the participants whether they thought the guidelines covered important aspects to be considered when documenting QRs in ASD. All the ten participants agreed that the guidelines covered important aspects that needed to be considered. Three participants further explained why they thought that the guidelines covered important aspects. The DevOps tech lead (P3) stated 'on a high level, this serves as a good reminder of things to consider when documenting QRs'. The specification engineer (P6) noted, 'with a short introduction, the guidelines seem to cover most important issues,' and another specification engineer (P5) commented, 'the decomposition from the top-level guidelines makes sense nicely. ' In Table 7, we summarized the themes and corresponding statements reported as important aspects to consider in documenting QRs by the participants. We found that the participants considered the clarity of QRs to be the most important aspect when documenting QRs. Clarity refers to ensuring a clear definition and quantification of the QR and minimizing the likelihood of misinterpretation when documenting QRs. Recognizing the importance of QRs and the need to document them was the second top recurring aspect. The traceability of QRs and the verification and validation of QRs were the third most recurring themes. Additionally, the participants reported five non-recurring aspects that are important to consider when documenting QRs. These include costs and related performance impacts, defining processes for handling QRs, the impact on product, dependency on other QRs, and expertise of practitioners defining QRs.

Figure 4 Plot and statistical summary of findings of RQ1 and RQ2
This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2022.3187106 Properly and clearly documenting the QR and its importance with respect to the project (quality). Clarity and quantification of QRs in order to make measurement, monitoring, and reporting easier. Clear terminology and discipline in requirement traceability, requirement categorization. No possibilities to interpret QRs wrongly. QRs are as important as FRs and need to be documented, It is important to have QRs. It is important to have QRs documented. Optimal documentation and significance of QRs. They are as important as FRs. They need to be documented as well so that what has been agreed on is known. Traceability of QRs , (2) Clear terminology and discipline in requirement traceability and requirement categorization. Transparency through higher-level requirements down to actual implementation. Verifying and validating QRs, Definition of ready and DOD and how to validate QRs. DoD and how to verify QRs. Costs and related performance impacts, (1) Performance effects, cost of getting required quality target.
Defining processes for handling QRs, Defining processes: how the common QRs are used in day-to-day ASD.

Impact on product
Impact on product Expertise of practitioners regarding QRs, Author expertise needs to be at a good level for related requirement.

Dependency on other QRs, (1)
Dependencies on other QRs.

4) Suggestions for improving the guidelines (RQ4)
The practitioners reported suggestions to improve the guidelines. Table 8 summarizes the participants' suggestions of improvement.
The top recurring suggestion was including aspects on how to verify and validate the QRs when documenting them. According to the participants, documenting QRs in ASD requires the specification of a means for verifying and validating them. The second top suggestion concerns backlogs suggested in the guidelines to document QRs. According to the feedback, including such backlogs may incur additional costs and be unfavorable in small-sized companies. There was a suggestion to clarify the details of documenting QRs in separate or specific backlogs. In addition, we found suggestions for including quality goals and process quality aspects when documenting QRs. One suggestion also emphasized the need for the earlier consideration of QRs (i.e., the definition and documentation of QRs should begin together with FRs).

VI. Discussion
This section discusses the implications of the study and compares Agile QR-Doc guidelines with related work.

A. IMPLICATIONS FROM THE EVALUATION FINDINGS
The results from the evaluation of the guidelines validated the suitability of Agile QR-Doc for the industry. The practitioners identified the guidelines to be understandable enough and useful in supporting QR documentation and in identifying relevant actions for documenting QRs in ASD. Although these findings are based on feedback from a small sample, they are remarkable and suggest the ease of understandability of the guidelines and their usefulness and compatibility with ASD.
It is crucial that strategies aimed at improving or supporting the documentation and management of QRs in ASD do not negatively affect the agility of software development processes [10]. Regarding the impact of applying the guidelines in terms of agility (i.e., how simple enough they are to use in ASD, and how they affect the speed of the development process in ASD), the practitioners perceive the guidelines as simple enough to use in ASD, not hindering the speed of ASD and helpful in the early consideration and documentation of QRs. However, we noticed that one respondent disagreed when asked if the guidelines did not hinder the speed of ASD. This could be because some recommendations were not described and justified adequately in the initial version of the guidelines. Therefore, taking the feedback into consideration we improved the guidelines by refining the descriptions, justifications, and providing illustrative examples. For instance, the recommendations of separate backlogs have been refined to clarify that their implementation should not compromise the agility of the process.
The findings of the evaluation revealed that the guidelines covered important aspects for documenting QRs in ASD. This shows the suitability and adequacy of the practices, artifacts and knowledge conveyed in the recommendations for supporting QR documentation in ASD.
The clarity of QRs was the most recurring important aspect that practitioners considered in documenting QRs. In ASD, conceptual challenges regarding QRs can lead to unclear QR specifications and confusions regarding QR specification approaches [11]. Agile QR-Doc include suggestions such as the agile playbook that can help practitioners gain knowledge on QRs and ways of implementing them. Such practices help minimize confusions and unclear QR specifications in ASD. In distributed ASD settings, where achieving shared understanding about QRs is challenging [9], [44], agile playbook is helpful. Additionally, light-weight artifacts that illustrate different means of specifying QRs (e.g., specifying QRs in system stories, documenting them as DOD of functional requirements) give practitioners insight into these approaches.
Traceability was another important aspect that the practitioners considered for documenting QRs in ASD. This is an important finding considering that traceability has been reported as QR documentation challenge in ASD. For instance, tracing security QRs with informal agile documentation practices is difficult [59]. Identifying dependencies between QRs is challenging in the presence of complex backlog structure [13]. Practitioners also identified the importance of QRs, and the need to document them early, and ensuring verification and validation, as important when documenting QRs in ASD. Considering the feedback, we included recommendations about ensuring traceability in QR documentation.
The practitioners evaluated the initial version of the guidelines that contained only nine recommendations, i.e., recommendations: A) 1-3, B) 1-4 and B) 7-8. The feedback from the evaluation helped us enrich the recommendations in our guidelines. For instance, there were concerns regarding applying separate or specific backlogs to document QRs (e.g., using security backlogs to document security requirements and using shadow backlogs to document usability requirements). The practitioners noted that implementing such backlogs may require additional resources and can be expensive in small-sized companies. Additionally, they noted the need for enhancing the descriptions in our guidelines. There were also suggestions regarding the inclusion of verification and validation aspects when documenting QRs. Following the evaluation, we updated the guidelines by refining the recommendations and including new ones, i.e., recommendations B) 5-6 and B) 9. We also included detailed rationale for the recommendations, and provided additional illustrative examples.

B. IMPLICATIONS FOR SOFTWARE ENGINEERING INDUSTRY, RESEARCH AND EDUCATION
The Agile QR-Doc guidelines contribute to the software engineering industry, research and education.
To the software engineering industry, our work contributes by providing guidelines for supporting QR documentation task in ASD. Practitioners can familiarize themselves with the QR documentation practices, artifacts and knowledge in the guidelines and apply them to proactively support QR documentation in ASD. For instance, requirements engineers and developers who may find documenting QRs, such as usability, difficult, can identify recommendations such as UserXstories artifacts, shadow backlogs, and using usability experts. These recommendations can help them better document and manage usability in ASD. Likewise, managers and product owners can learn how limited awareness on QRs and misconceptions on minimal documentation can negatively affect QR documentation. They can identify potential mitigation actions such as investing in QR trainings, adopting quality mindset development and agile playbook, which they can adapt to the betterment of the QR documentation in ASD.
To the software engineering research, our work contributes by filling the gap in guidelines that support the documentation of QRs in ASD. It also demonstrates the application of DSRM in building design science artifact. We reviewed the state of the art of management of QRs in ASD, and explored the documentation of QRs in ASD cases, when building the guidelines. Researchers can utilize the knowledge created in the process of building our guidelines to get better understanding about the documentation and management of QRs in ASD. The recommendations, their justifications, and realization techniques give insight into the research of QR documentation in ASD too. Researchers can also utilize our work to build new proposals for facilitating documentation and management of QRs in ASD.
We also believe that software engineering education can benefit from the guidelines. For instance, when these guidelines are incorporated in requirements engineering exercises, students can learn about the importance of QRs, and the need for optimally documenting them in ASD. They can be familiarized with the artifacts, practices and knowledge proposed in the recommendations too.

C. COMPARISON TO RELATED WORK
In the agile requirement engineering literature, we identified guidelines that support the documentation of QRs in ASD.
Barbosa et al. [22] proposed guidelines for integrating security QRs in ASD that were evaluated by three experts. Wäyrynen et al. [26] proposed recommendations to improve the documentation and management of security requirements in XP. Lee et al. [24] introduced guidelines to improve usability specification in ASD. Cajander et al. [15] provided recommendations for the documentation of usability and user experience. These guidelines focused only on security and usability. Our previous work provided guidelines for documenting QRs in ASD [13]. Agile QR-Doc guidelines address the documentation of QRs in general, within the context of ASD.
Recruiting large number of participants in empirical studies is a challenge in software engineering studies [70]. This is also witnessed in existing QR documentation guidelines proposals in ASD as shown here. We validated Agile QR-Doc by evaluating them with ten practitioners from two ASD companies. Barbosa and Sampaio [22] evaluated their guidelines with three experts, and Lee et al. [24] evaluated their guidelines with ten practitioners. However, the guidelines by Cajander et al. [15], Wäyrynen et al. [26], and Behutiye et al. [13] were not evaluated empirically. In this regard, we call for more industry-academia collaborations since they can offer opportunities for the development and validation of guidelines, tools and other proposals in software engineering.
Most studies on documenting and managing QRs in ASD do not examine the impact on agility [3]. Only the work by Lee et al. [24] and our work examined the perceived impact of applying the guidelines on agility. A common limitation among all guidelines including our own Agile QR-Doc is that they have not been applied in industrial projects. Extending the application of these guidelines beyond evaluations will be beneficial for the software industry. Additionally, it offers opportunity to enhance the guidelines and strengthen their adoption in the industry. In Table 9 , we present the comparison of these guidelines.

VII. Threats to validity
We use the validity aspect classification by Wohlin et al. [67] to discuss the validity threats in our study. We also discuss the measures we took to minimize the threats and improve the validity of our work. Construct validity: explains threats that may arise from differences in the interpretations of operational measures that are studied. We mitigated potential construct validity threats in the design of the guidelines by adapting a DSRM to systematically develop the guidelines. A grounded theory would have been alternative approach to adopt when developing the guidelines. However, we found DSRM models suitable and easier to adapt. We found DSRM model by Peffers et al. [29], relatively easier to adopt and guide the development of our guidelines, since they helped clearly distinguish the design artifact and knowledge created in the process of building the artifact. Additionally, when building the guidelines, we applied rigorous steps and relevant knowledge gained both from the state of the art, and the practice on documentation and management of QRs in ASD.
In the validation design, we employed the key informant technique and considered practitioners who have relevant background and experience. The authors of the paper reviewed the guidelines, and the survey questionnaire before the validation process, i.e., demonstration and evaluation activity. During the demonstration and evaluation, a researcher was present and provided clarifications regarding the guidelines and the survey questionnaire to the participants.
Internal validity: inspects the presence of factors that we do not have control over or factors we may not have measured, which may influence the outcome of our findings. We acknowledge that there might be factors that we did not consider, which could have influenced our findings. For instance, considering QR documentation practices from other ASD contexts, than those involved in our study can return practices and artifacts that were not included in our guidelines. It can also identify aspects that we have overlooked in our guidelines too. To minimize such threats, we included knowledge synthesized from studying the state of the art of management of QRs in ASD.
While there might have been factors, which we are unaware of, that influenced the outcome of the validation process, we took measures to minimize threats to internal validity. For instance, we triangulated data by transcribing the audio recorded during the demonstration and evaluation, listening to the audio recordings, and using the survey feedback.
External validity: explains the degree that one can generalize the findings of a study to wider scope. In order to improve the generalizability of the guidelines, we used knowledge gained both from studying the state of the practice of QR documentation in ASD cases, and synthesis of the state of the art on the topic to build the guidelines. Ten software practitioners validated the guidelines by assessing the usefulness, relevance, understandability and impact on agility of the guidelines for supporting QR documentation in ASD.
We acknowledge the threat to external validity due to the small number of participants used in validating the guidelines. However, to mitigate this threat, we took the following measures. Firstly, we recruited practitioners (e.g., including roles such as specification engineers, software project managers and software architects) who have rich experience in software engineering and ASD for validating the guidelines. Secondly, we applied established quantitative analysis technique to interpret our findings. We used, the one-sample Wilcoxon signed rank test, which is used to evaluate variations in the response of participants of small sample sized studies to make appropriate interpretations. The technique is advantageous for analyzing small sample sized data [68] and is used, for example, in medical studies with small number of participants. For example, Zhang et al. [69] utilize one-sample Wilcoxon signed rank for analyzing clinical data of 8 patients. Additionally, we incorporated the descriptive statistics when analyzing and reporting the quantitative data findings. Another measure we took to minimize the threat was collecting qualitative data. This helped us supplement the quantitative data. Thus the validation utilized both quantitative and qualitative data. We agree that it is important to have additional studies to extend the generalizability of the study. In this regard, we plan to conduct additional validation studies of the guidelines in varying ASD contexts (e.g., ASD projects with varying product domains, team sizes, ASD methods and).
Conclusion validity: explains the reliability regarding the data collection and analysis used in deriving the findings of a study. We used the Webropol tool to collect the survey feedback systematically. This helped us ensure proper data collection. We recorded audio of the demonstration and evaluation sessions and transcribed the data. We also applied data analysis techniques suitable for analyzing our data. For instance, we used a one-sample Wilcoxon ranked test for testing variations in the opinions of small-sized teams. Additionally, in order to enhance the reliability of our study, the second and third authors reviewed and discussed the findings from the data analysis with the first author.

VIII. Conclusion
This paper developed the Agile QR-Doc guidelines with the aim of addressing the gap in guidelines that support QR documentation in ASD. We adapted DSRM to build the guidelines. Therefore, a rigorous process was followed utilizing knowledge and experience gained from studying QR documentation and management in ASD cases, and reviewing the state of the art on the management of QRs in ASD.
The guidelines were validated by ASD practitioners. Practitioners identified them understandable and useful for identifying relevant actions for documenting QRs. They also identified Agile QR-Doc guidelines as simple enough to use in ASD, helpful in the early consideration and documentation of QRs, and not hindering the ASD process. According to them, the guidelines cover important aspects to consider when documenting QRs.
The guidelines presented recommendations focusing on raising awareness on the significance of QRs and their documentation and those proposing lightweight artifacts and practices for documenting QRs in ASD. Ensuring clarity of QRs, recognizing the significance of QRs and their documentation, ensuring the verification and validation aspects when documenting QRs, and ensuring traceability of QRs were some of the recommendations favored by the practitioners.
Our work contributes to the software engineering industry by providing guidelines for supporting QR documentation in ASD. For instance, requirements specification engineers can learn about the different light-weight artifacts used for documenting QRs in ASD. Junior developers can also benefit from the examples of artifacts and practices for documenting QRs presented in the guidelines.
Our work contributes to software engineering research since it addresses the gap in guidelines supporting QR documentation in ASD. Researchers can get a better insight into the documentation and management of QRs in ASD, by using the knowledge created in the design process of Agile QR-Doc. These include knowledge from the state of the art, the state of the practice of QR documentation and knowledge gained during the validation of the guidelines.
In the future, we consider extending the validation of the guidelines, with more cases of varying contexts (e.g., domain, product type, and ASD methodology) and participants of varying backgrounds. We also plan to utilize the guidelines with students in requirements engineering exercises.

ACKNOWLEDGMENT
This work is partially funded by the Q-Rapids project, European Union's Horizon 2020 research and innovation funded program and partially by the Nokia Foundation Scholarship.
WOUBSHET BEHUTIYE (M.sc) is a doctoral researcher in the M3S research unit, at the faculty of information technology and electrical engineering, University of Oulu. He received his MSc in Information processing science from the University of Oulu in 2015. His research interests include: Agile software development, continuous software engineering, requirements engineering, quality requirements, technical debt management and evidence based software engineering. He was actively involved in European Union's H2020 Q-Rapids project, 'Quality aware rapid software development' work package as a researcher. He has initiated and managed more than 100 national and international research projects. He has served as chair and committee member in organizing numerous international conferences and has been a reviewer for top scientific journals. He is a founding member and chair of the steering committee of ISERN.