Prioritization Based Taxonomy of DevOps Challenges Using Fuzzy AHP Analysis

The DevOps (development and operations) is a collaborative software development environment which offers the continues development and deployment of quality software project within short time. The DevOps practices are not yet mature enough, and the software organizations hesitate to adopt it. This study aims: 1) to explore the DevOps challenges by conducting systematic literature review (SLR) and to get the insight of industry experts via questionnaire survey study and 2) to prioritize the investigated challenges using fuzzy analytical hierarchy process (FAHP). The study findings provide the set of critical challenges faced by the software organizations while adopting DevOps and a prioritization-based taxonomy of the DevOps challenges. The application of FAHP is novel in this research area as it assists in addressing the vagueness of practitioners concerning the influencing factors of DevOps. We believe that the finding of this study will serve as a body of knowledge for real world practitioners and researchers to revise and develop the new strategies for the successful implementation of DevOps practices in the software industry.


I. INTRODUCTION
The software organization is continuously looking the better ways to develop good quality software with a significant return on investment. There is a dramatic change in the software development approaches 'from traditional waterfall to agile paradigm' over the years. Currently, the software organizations rapidly changing the development environment in terms of shortening and continuous development and release cycles, using the state-of-the-art development approach, namely as DevOps (development and operations) [1]- [3]. The DevOps environment has been adopted and accepted over a few years, and still, it is a lack of widely accepted definition. In this study, we have coated a recent and more comprehensive definition of DevOps defined by Leite et al. [1] ''DevOps is a collaborative and The associate editor coordinating the review of this manuscript and approving it for publication was Hailong Sun . multidisciplinary effort within an organization to automate continuous delivery of new software versions while guaranteeing their correctness and reliability. '' In order to shotren the development life cyle and to employ the continuous delivery process, the software development community increasingly adopting the practices of DevOps [4]- [6]. Bai et al. [7] stated that the production of quality projects and the in-time delivery is the critical aspect of software business organizations, and the continuous software engineering is essential to achieve such goals, which assist in adoptingthe DevOps paradigm. Balalaie et al. [8] and Sharma and Coyne [9] mention that the developers, operators, customers, and quality assurance teams continuously collaborated for delivery, reduce time, and attain market opportunities. They further indicated that the high-quality project production and delivery, rapid and timely entertainment of requirements changes, and reduced development time accelerated the acceptance of DevOps in the VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ software industry. The IBM elaborated that the DevOps is a business-oriented software development and delivery methodology as it consists of the lines of business, practitioners, managers, and suppliers [4], [9]. Various established digital giants like WebEx, McAfee, CISCO, Netflix, and Amazon are already using the DevOps practices to deliver the perfect fitting customer-centric software solution in the international market [10]. The software industry, especially the mediocre software organizations, faced various complexities while adopting DevOps practices. Aiming to implement the DevOps practices in software development organizations successfully, the mainstream research body has motivated to assists the practitionerin developing the new techniques and tools [11], [12]. Due to the increasing demand for DevOps in the software industry, currently, it becomes the hot research topic.
Besides the significance and criticality of DevOps in the software industry, little empirical research has been carried out to fix the complications faced by the practitioners. Hence, the importance of DevOps in software industry motivated us to conduct a compressive study to explore and analyze the challenges that are critical for the successful execution of DevOps paradigm. To address the study objective, firstly, the literature review was performed to explore the challenges reported by the academic researcher and were further validated them with real world practitioners via questionnaire survey. Secondly, the fuzzy analytical hierarchy process (FAHP) approach was applied to prioritize the investigated challenges concerning to their criticality for DevOps paradigm. Various researchers already used this technique in other software engineering domain. For example, Khan et al. [13] used the FAHP to rank the success factors of software process improvement. Yaghoobi [14] used the FAHP approach to rank the success factors of software project management. Bozbura et al. [15] prioritize the measurement indicators of human capital using FAHP. Shameem et al. [16] developed analyses the success factors of agile software development process. We believe that the in-depth review and analysis of the DevOps challenges will help the industry experts to revise their strategies and develop new roadmaps for the success and progression of DevOps execution in the software industry. The proposed research question of this study are: RQ1: What challenges of DevOps paradigm are reported in the literature? RQ2: Are the DevOps challenges reported in the literature related to real-world practices?
RQ3: How the investigated challenges be prioritized? RQ4: What would be the taxonomy of the investigated challenges?

II. BACKGROUND OF DevOps
The evolution of the rapid revolution in information technology causes the transformation of development tools and techniques. The business firms are highly motivated to transform their working environment from manual to digital form as the automation increases productivity and maintains the consistency of product quality, which significantly increases the demand for software systems. To meet the market demand, the software organization continually looking at the active development approaches to develop and deliver the quality software's orders within time and budget [17]. Dörnenburg [18] indicated that to meet the market demand and to address the technological transformation effectively, the software organization needs to adopt new and efficient software development approaches. By seeking this technological revolution and market trend, the traditional software development approaches (like Waterfall, Spiral, etc.) were replaced by the agile paradigm (i.e., Scrum and Kanban, etc.). The production and operational process are stressful, as manual processing is error-prone and causes a delay in feedback [19]. Therefore, to meet the current flows in the software industry, DevOps is the new and more efficient software development paradigm, which is based on the agile practices and operational aspects. The DevOps approach gave a complementary set of agile methods that assists to efficiently and continuously release the developed features in a shortened life cycle.
Initially, the term DevOps has ambiguity in its interoperation as part of the software community consider it a job opportunity that requires both skills, i.e., development and operation [20]. This ambiguity addressed by mainstream research by interpreting its actual meaning as DevOps is a development environment in which both development and operational teamwork with close collaboration [20]- [22]. In DevOps, the distinct silos for developers and operators still exist; the operational team is responsible for the management of modification during production and in-service levels [19], other-side the development staff are accountable for the continuous development of new features to attain the required business goals. Both teams have their independent tools, process, and knowledge bases. This mechanism allows the development staff to push new features into production continuously, and the operational teams attempt to operate the latest version and highlight the modification to maintain the consistency in project quality and other non-functional requirements [19]. To address the flows between development and operation teams, an automated pipeline is needed to be considered [23]. Humble and Farley [19] stated that ''the humble advocates for an automated deployment pipeline, in which any software version committed to the repository must be a production-candidate version.'' Humble [24] underlined that the automation process defines a path that allows the development and auto testing, and the tested feature of the software is sent to the production by pressing the button. Callanan and Spillane [11] emphasized the continuous delivery and stated the deployment pipeline as a DevOps platform.

III. RESEARCH DESIGN
In this paper, the research was design in three different steps: Step 1: Identified the DevOps challenges reported in the literature using a systematic literature review. Step 2: To get the insight of the industry practitioners concerning to the DevOps challenges, the questionnaire survey approach was used.
Step 3: Rank the identified list of challenges using the fuzzy AHP approach.

A. SYSTEMATIC LITERATURE REVIEW (SLR)
An SLR approach has been applied to collect and review the literature related to the study objectives. An SLR give the more comprehensive and valid results compare with informal literature review. The SLR guidelines proposed by Kitchenham and Charters [25] were considered to extract the potential literature related to the study objectives. According to Kitchenham and Charters [25], the SLR includes three core phases: ''planning the review,'' ''conducting the review,'' and ''reporting the review.'' The developed SLR protocols are explained in the subsequent section and diagrammatically indicated in Figure 1.

1) PLANNING THE REVIEW
To conduct the literature review, the following review protocols were developed:  [27]. To formulate the complete search string, we used the Boolean ''OR'' and ''AND'', as presented below: (''barriers'' OR ''obstacles'' OR ''hurdles'' OR ''difficulties'' OR ''impediments'' OR ''hindrance'' OR ''Concerns'' OR ''techniques'' OR ''tools,'' OR ''methods,'' OR ''process'' OR ''evaluation'') AND (''DevOps'' OR ''Development and Operation,'' OR ''Continues development and operation.'' The protocols were developed to decide the inclusion of literature collected from the selected databases. The inclusion protocols were designed by following the existing studies [29] and [30].  We have further developed the protocols to exclude the literature collected from databases initially. The exclusion criteria were developed by following the guidelines of [29]- [31]. (1) the most completed study from a similar research group VOLUME 8, 2020 The QA assessment process was performed to determine that how the selected literature effective to answer the research objective. The QA process is carried out by using the guidelines of [25]. For the QA process, the five-questions were developed (Table 1) and evaluated using the Likert scale given in Table 1. Similar criteria are adopted by various existing studies [29]- [33]. The detailed score of QA is given in Appendix-A.

2) CONDUCTING THE REVIEW a: FINAL STUDY SELECTION
The three different ways were used to collect the literature. Firstly, 6 studies were collected manually by exploring Research-Gate. Secondly, the selected databases were exploring by executing the search string and 688 studies were extracted. Therefore, for the final refinement of studies, the tollgate approach developed by Afzal [34] was adopted. By steps of tollgate approach (Figure 2), 54 studies were selected. Furthermore, we have performed the forward and backward snowballing on the reference list of selected studies, and 19 studies were selected. To conclude, 78 articles were considered for data extraction. All the studies were also assessed concerning the QA, and the results are given in Appendix-A. Each selected study is presented as ''PS'' to present its use as an SLR study.

b: DATA EXTRACTION AND SYNTHESIS
The selected studies ( Figure 2) were carefully reviewed for data extraction with correspondence to the study research objective. The first two authors of this study were continuously involved in the data extraction process, and author numbers three and four validate the extracted data. Initially, the statements, main themes, concepts, and DevOps challenging factors were obtained from the selected studies. We then synthesized the collected data into compact statements and formed the final 20 challenging factors of DevOps implementation in the software industry.
There may be a biasness between the study findings. Though, the ''inter-rater reliability test'' [34] was performed. We have requested the four external experts for the participation in validation process. They randomly selected the 12 studies and performed the data extraction process. Based on the findings of study authors and external experts, we have calculated the ''non-parametric Kendall's coefficient of concordance'' (W) [35]. The value of W=1 renders the complete agreement, and W=0 indicates the complete disagreement. The results of W=0.84 p=0.003 shows an agreement between the investigation of study authors and external experts. This indicated that the study findings are unbiased. The used code is given in this link: https://rdrr.io/cran/DescTools/man/KendallW.html.

3) REPORTING THE REVIEW a: QUALITY OF SELECTED STUDIES
The quality assessment of the selected studies shows how the selected literature is effective to answer the research question of this study. According to the accumulative results of the QA process shows that more than 70% of studies score ≥ 70%. The detail QA results are presented in Appendix-A. We have to use the 50% score as a threshold value.

b: PUBLICATION YEARS AND USED RESEARCH APPROACHES IN SELECTED STUDIES
We extract the publication years of the selected studies to determine the frequency of publication of DevOps related literature. The analysis indicated that the chosen set of studies was published from 2013 to 2019, and this shows a growing trend in the frequency of publication in recent years. Hence, this renders that DevOps is an important and attractive research area of mainstream research body. In addition, we also extracted the adopted research methodologies in the selected studies. The results show that the selected studies respectively adopted ''questionnaire survey'' (QS, 18%), ''case study'' (CS, 35%), ''grounded theory'' (GT, 17%), ''content analysis'' (CA, 5%), ''action research'' (AR, 9%) and ''mixed-method'' (MM, 16). Therefore, we observed that CS is the most commonly used research approach.

B. EMPIRICAL STUDY
The questionnaire survey is a suitable way to collect the most potential data from the dispersed and targeted population.
According to Kitchenham and Pfleeger [36] the assortment data collection method is based on ''available data collection resources,'' ''controlling mechanism of selected approach,'' and ''skill to operate the variable of interest.'' To collect the representative sample, the observation methods are hard [31], [37]. Hence, we have used the questionnaire method to answer the research question of this study, as it is an effective data collection way from dispersed population.

1) SURVEY INSTRUMENT DEVELOPMENT
We developed a questionnaire survey to get the feedback of real-world industry practitioners. To develop the questionnaire survey, the Google Form (i.e., ''docs.google.com/ forms'') is used. The questionnaire survey was consists of three main sections, which include; (1) the first section contains the queries related to the respondents bibliographic information.
(2) the second section includes the list of challenges extracted from literature (3) the third it contains the survey participants inputting the additional challenging factor which is not enlisted in the closed-ended section.

2) PILOT STUDY OF QUESTIONNAIRE SURVEY
After the development of the questionnaire, we have conducted a pilot assessment with the academic and industry experts to determine the understandability of the developed questionnaire [38]- [41]. To perform the pilot evaluation of survey instrument, three external experts were requested. The requested experts include one from academics (Chongqing University, Chain) and two from industry (Virtual force-Pakistan and QSoft-Vietnam). The experts suggest some modifications related to the questionnaire structure and the questions for the collection of bibliographic data of survey participants. They further suggest putting the questions in a tabular form. We update the questionnaire by carefully considering the recommendation of experts, and the final used survey instrument is given in Appendix-B.

3) ETHICS APPROVAL
Once the survey questionnaire was finalized, we have conducted ethical approval for data collection from the ''Research Ethics Board of College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing.'' After getting approval from the Research Ethical Board, we started the data collection process and made available the questionnaire for the targeted population.

4) DATA SOURCES
The data sources play a vital role in targeting the potential population. The potential population is important to target as it is necessary for the collection of pure data. As the objective of this survey study was to get the insight of experts concerning to DevOps challenging factors identified from literature via SLR study. Though, to target the population, we used both professional Email addresses, Research-Gate, and LinkedIn. The snowballing technique was used to spread the survey questionnaire to the target geographically dispersed population [39], [42], [43]. Snowball is an easy and cost-effective approach to collect the data from large and potential population [44].
The data was collectedfromDecember-2019 to March-2020. During the data collection process, a total of 102 responses were collected. All the received responses were checked, and nine responses were found uncompleted. By debating with the research team, we decided not to consider the uncompleted response for the data analysis process. Though, final 93 complete answers were considered for further analysis. The detail of respondents' bibliographic data is provided in section-4.2.

5) SURVEY DATA ANALYSIS
We have adopted the frequency analysis method as it is an effective technique to analyze the quantitative and qualitative data. It is an appropriate approach to compared the respondent's opinions among the variables and group of variables [45]. The same approach has been considered by several researcher of other software engineering domains [46]- [48].

C. PHASE 3: FUZZY SET THEORY AND AHP
The implementation process of fuzzy AHP steps is discussed in this section.

1) FUZZY SET THEORY
The fuzzy set theory is an extend version of classical set theory developed by [49]. That was considered to address the vagueness and uncertainties in the industry practices using multicriteria decision making problems. In the fuzzy set, a membership function µF(x) is characterized, which maps an object between 0 and 1. The definitions and preliminary of the fuzzy set theory are explained in subsequent sections: Definition: ''A triangular fuzzy number (TFN) F is denoted by a set (fl, fm, fu), as presented in Figure 4. The given equation'' (1) Defines the membership function µF(x) of F. The ''algebraic operational laws using two TFNs, namely (V 1 , V 2 )'' are given in Table 2.

2) FUZZY AHP
The fuzzy AHP is a useful approach for ''multicriteria decision making problems''. The key benefit of fuzzy AHP is that it is easy to apply and understandable; and it can manage both quantitative and qualitative data. Following are the main steps adopted to perform the fuzzy AHP:   Hence, the classical AHP has some limitations due to the implementing the AHP in the Crisp environment, ''Judgmental scale is unbalanced'', and the ''lack of ambiguity'', ''selection of judgment'' are subjective. Though, fuzzy AHP is an updated version of AHP and that was develop to fix the uncertainties more effectively [50], [51]. The fuzzy AHP is effective to address the uncertainty and imprecise judgment of experts by handling the linguistic variables. The fuzzy AHP has been applied in various other domain [14], [52]- [55]. In current study, we applied the fuzzy AHP that is introduced by [56], which offer more consistent and accurate results compared with other approaches.
In a ranking problem, let X = {x1, x2, . . . , xn} indicate the factors of main categories as an object set and U = {u1, u2, . . . , un} indicates the factors a particular category as a goal set. By [56] approach, every element is measured, and extent analysis for each goal (gi) is performed, respectively. Therefore, for each object, there are (m) extent analysis values that can be obtained with the following Equation (2) and (3): where, all F j gi , (j = 1, 2, . . . , m) are triangular fuzzy numbers (TFNs).
The main steps of Chang's extent analysis approach [56] are: Step 1: The value of fuzzy synthetic extent with respect to the ith object can be defined using Eq. (4): To achieve the expression and finally, calculate the inverse of the vector with the help of Eq. (7): Step 2: As F a and F b are two fuzzy triangular numbers, then the degree of possibility of is defined as follows and the Eq. 8 can also be similarly specified as below: Here, d indicates the ordinate of the highest intersection point between D, µ V a , and µ Vb (Figure 6). The values of T 1 (V a ≥ V b ) and T 2 (V a ≥ V b ) are required for determining the value of P 1 and P 2 .
Step 3: Calculate the overall degree of possibility of a convex fuzzy number, and the other convex fuzzy numbers V i (i = 1, 2, . . . , k) can be descripted as follow.
Step 4: The normalization, the normalized weight vectors are in equation 13, and the result will be a non-fuzzy number which renders the priority weight of the challenge: where W is a non-fuzzy number.
Step 5 (Checking Consistency Ratio): The developed pairwise comparison matrixes should be consistent in fuzzy AHP analysis [57], [58]. Hence, it is mandatory to measure the consistency ratio of the pairwise comparison matrixes. To address this, the ''graded mean integration'' method is used for defuzzifying the matrix. A triangular fuzzy number, denoted as P = (l, m, u), can be defuzzified to a crisp number as follows: Besides the defuzzification of every element of matrix, the consistency ration (CR) of each pairwise comparison matrix is easy to determine aiming to check as to determine the value of CR is less than 0.10 or not. For this, two primary where, I max : presents the largest eigenvalue of the pairwise matrix. n: presents the number of values being compared. RI: ''the random index and its value can opt from Table 3''. CR: If the value of CR is less than 0.1, then it denotes the consistent pairwise comparison matrix.

IV. RESULTS AND DISCUSSION
This section contains the results and analysis of this study.

A. FINDINGS OF SLR STUDY
The phases of the SLR approach were carefully executed to extract the challenging critical factors of DevOps practices. The list of investigated 22 challenges were enlisted in Table 4.
The identified challenges were further mapped in the core phases of the CAMS model, developed by Edwards and Willis [59]. The critical aspects of CAMS include ''Culture'', ''Automation'', ''Measurement'', and ''Sharing''. The CAMS model consists of a set of variables considered by various practitioners for the successful implementation of DevOps practices in the software industry. We develop a mapping team consists of three authors of this study (Author number 1, 3, 4). All the participants of the mapping team were continuously involved and using the critical steps of the coding scheme ((i.e., ''code,'' ''sub-categories,'' ''categories and theory\framework''), all the challenging factors were mapped and developed a framework as given in Figure 7. The principal objective of mapping is to perform the fuzzy AHP analysis.

1) CULTURE
''Culture is defined by the interaction of people and groups and is driven by behavior. Substantial communication improvement can result when there is a mutual understanding of others and their goals and responsibilities''.

2) AUTOMATION
''Automation can save time, effort, and money, just like culture, it truly focuses on people and processes and not just tools. The impact of implementing infrastructure as code as well as using continuous integration and continuous delivery pipelines can be magnified after understanding an VOLUME 8, 2020  organization's culture and goals. It helps to think of automation as an accelerator that enhances the benefits of DevOps as a whole''.

3) MEASUREMENT
''The measurement is helpful to determine the progresses and its intended direction. There are two main bumps that might be occur while using matrices i.e. (1) to make sure the parameters are correct ones and (2) to incentivize the right metrics. The DevOps encourage to see the forest from the trees by viewing the entire operation and evaluating it as a whole and not just focusing on small parts. Primary metrics include (but are certainly not limited to) income, costs, revenue, mean time to recovery, mean time between failures, and employee satisfaction''.

4) SHARING
''DevOps processes, similar to agile and scrum, place a very high premium on transparency and openness.
Spreading knowledge helps to tighten feedback loops and enables the organization to improve continuously. This collective intelligence makes the team a more efficient unit and allows it to become greater than just the sum of its parts.''

B. EMPIRICAL STUDY RESULTS
To verify the finding of the SLR study, the questionnaire survey study was conducted with experts and the analyzed responses of survey participates are given in subsequent sections.

1) DEMOGRAPHIC DATA ANALYSIS OF SURVEY PARTICIPANTS
The detail demographic information of the survey participants was collected during the data collection process. Patten [60] stated that ''the demographic data provides information about survey respondents and is essential for the determination of whether the participants in a particular study are a representative sample of the target population for results generalization purposes or not.'' Finstad [43] underlined that the bibliographic data of survey participants give the insight of survey respondents, which shows the maturity level of the collected data set. [61], underlined that the information of survey participants assists in determining ''what your target population is and what they are thinking about.'' Though, by seeking the importance of respondents' bibliographic data, we have analyses the respondent's data concerning to organization size, respondents designation and organization size. The brief analysis is discussed in the following sections.

a: RESPONDENT'S DESIGNATION
Finstad [43] underlined the implication and priority of the influencing factors that vary regarding the designation of the respondents. Furthermore, Niazi [29] defines that the impact of a factor depending upon the position of the practitioner, and they further stated that the influence of an element could be ranked exactly if the respondent frequently experiences to deal with that factor. The responder's designation-based analysis is presented in Figure 9 that shows the most of the survey respondents are project manager. According to the results the most common respondents' designations are: ''project manager'', ''software developer'', ''researcher'' ( Figure 9).

b: RESPONDENT'S EXPERIENCE
The experience of the survey participants reported in the questionnaire was also analyzed. The mean and medium were calculated, and the results show 7.5 and 5, respectively; this renders the young pool of survey participants. Besides, we also observed the significant variations in the experience of the survey participants. The detailed results of the survey participate are graphically shown in Figure 10. ≥200 employees)'' [62]. Akbar et al. [37] indicated that the organization size is also a critical entity to assess the  maturity level and explore of survey participants. The results presented in Figure 11 renders that 31(33%), 37 (40%), and 25 (27%) respondents are from a small, medium and large scale of firms, respectively. The detail of organization sizebased analysis is presented in Figure 11.

2) RESPONSES AGAINST DevOps CHALLENGES (RQ3)
The basic objective of empirical study was to get insight into the industry practitioners concerning the DevOps challenges identified via SLR study. The responses collected against the DevOps challenging factors were mainly categorized as: positive (''agree, strongly agree''), negative (''disagree, strongly disagree''), and ''neutral''. The positive category presents the frequency of those survey respondents who are considered the identified challenging factors that could negatively influence the DevOps practices. The negative group presents the frequency of those respondents who do not agree with the identification of SLR study. The neutral category shows the frequency of survey participates who are not sure about VOLUME 8, 2020 the effect of identified factors concerning DevOps activities. The detail results are given in Table 5.
The concluded results are given in Table 5, which renders that majority of the survey participant's agree with the identified challenges as they have a negative relation with DevOps related to real-world practices. The frequency analysis shows that all the challenging factors considered ≥70% of the survey participants, instead of two challenges, i.e., C7 (DevOps and regulatory compliance, 69%) and C8 (Lack of service virtualization, 68%). We further noted that C20 (''Lack of flexibility due to rigid industrial constraints'', 85%) was the highest reported challenging factors by the survey respondents.
We observed that P4 (Culture, 92%) was the highest considered category of the investigated challenging factors. P1 (Automation, 89%) and P3 (Sharing, 86%) were considered as the second and third most significant important categories of challenges.
The negative category shows C8 (Lack of service virtualization, 16%) is the highest-ranked challenge factors, this renders that 16% of the respondents are not agree with the C8 as a challenging factor for DevOps practices. C7 (DevOps and regulatory compliance, 14%), C18 (''Lack of feedback and bugs prioritization'', 14%), C19 (Resources accountability issues, 14%) and C21 (''Lack of strategic suggestions from leadership'', 14%) are mention as the second highest ranked challenging factors.
We further observed that C11 (Building and maintaining the deployment pipeline, 17%), C8 (Lack of service virtualization, 16%), C17 (Resistance to adopt DevOps, 16) are declared as the first and second highest ranked challenges for DevOps paradigm in software organizations, respectively.

C. APPLICATION OF FUZZY AHP
This section contains the fuzzy-AHP analysis of the explored challenges and their categories. The priority of the challenges was determined using the step by step protocols of fuzzy AHP, as presented in above (section 3.4).
Step 1 (Categorize the Complicated Problems Into Hierarchy Structure): To perform the fuzzy AHP analysis, the complicated problem is divided to an interconnected decision making elements. [57], [63]. The complicated problem is classified at minimum of 3 stages as presented in Figure 5, whereas the key aim of the problems is indicated at top level, the categories of challenge and their corresponding challenges are presented at stage 2 and 3, respectively. The proposed hierarchy structure is presented in Figure 12. Step 2 (Pairwise Comparison): The pairwise comparison was conducted based on the opinions of experts. To conduct the pairwise comparison, we have developed a questionnaire and contacted with the experts involved in first survey. The developed fuzzy-AHP questionnaire (Appendix-D) was sent to the experts and the 29 responses were collected. The collected complete responses were further manually analysed to find the incomplete entries. However, we did not find any incomplete entries and the collected 29 responses were considered for further analysis. In order to generalize results of fuzzy-AHP analysis the data of 29 respondents might not strong enough. As the fuzzy AHP is a subjective methodology and the small data size is acceptable [14], [52]- [54].
It is noted that the existing studies used the small of data size for fuzzy AHP analysis. In example, Akbar et al. [51] performed the fuzzy AHP analysis considering the data collected form 23 experts. [57] used the opinion of 5 experts for perfume AHP analysis. Shameem et al. [64] conducted pairwise comparison considering the opinions of 9 experts. Moreover, we found that Wong and Li [65] conducted the fuzzy AHP analysis using the data collected from 9 experts. Though, considering data size of existing studies, we are confident that the data collected from 29 experts is justified for generalizing the result of fuzzy AHP study. The collected opinions were converted into geometric mean aiming to develop the pairwise comparison matrixes.
The geometric mean is useful method to transform the judgement of survey respondents into TFN numbers. In this study, we have used the following formula of geometric mean: Geometric mean = n √ r1xr2 × r3 . . . . . . . . . .rn r = ''Weight of each response'' n = ''Number of responses''

Step 3 (Test the Consistency of the Pair-Wise Matrix):
The step by step procedure to calculate the consistency are presented in this section. The pairwise comparison matrixes should be consistent in fuzzy AHP. To do this, the Likert scale categories (Table 6) are considered. A triangular fuzzy number of the pair-wise comparison matrix of the main categories are defuzzified to crisp number using Equation 14 and obtained the corresponding Fuzzy Crisp Matrix (FCM) as shown in Table 7: The largest Eigenvector (I max ) value of the FCM matrix is determined by taking the sum of the elements of each column of FCM matrix ( Table 7). The determined sum is further divided with each element of FCM matrix and take the average of each element of each row, as presented in Table 8.
where, Cj = sum of the columns of Matrix [C] (Table 8), W= weight vector (Table 8), hence, I max = 2.55 * 0.38431 + 7.43 * 0.13617 + 3.55 * 0.28986 + 5.23 * 0.19077 = 4.0162 Thus, considering the value of I max (4.0162), the dimension of FCM is 4. Random Consistency Index (RI) for n=4 is 0.9 (Table 3). Using the equation 15 and 16, we calculated the consistency ration for each pairwise matrix as: The determined value of CR is 0.005985<0.10; hence, the pairwise comparison matrix of challenges categories is consistence and acceptable for fuzzy AHP. By considering the same steps, the consistency ratio for all the challenging factors categories were determined and the results are presented in Table 10, 11, 12, 13, and 14, respectively.   Step 4 (Determining the Local Priority Weight of Each Challenge): The local ranking presents the priority order of a challenge in their own category. The local ranking assists the practitioners to address the DevOps challenges mentioned in a particular category. To do this, the local weigh of each challenge was determined using the Equation 3.
Firstly, the synthetic extent values of categories (C1, C2, C3, C4) were calculated, and the priority weight of the attributes was mention using Equation 4. In this section, we exemplary present the weight calculation of main categories of the challenges.
In Table 9, the last column indicates the degree of possibility for each category that is determined using Equation 11 by considering the minimum value of each row. Hence, the calculated weight vector is W = (1, 0.028563, 0.76504, 0.28009).  Once, the value weight vector was normalized, the significance of attributes was determining i.e. W = (0.482233, 0.013764, 0.368925, 0.135066). The determined results show that 'Culture' is the most significant category or the challenging factors. The degree of possibility using Equation 9, as presented in Table 9, is determined.
Step 5 (Local and Global Weight Calculation): The local and global weigh of the challenges and their respective category were determined. The determined results are presented in Table 15, which shows significance of a challenge with in their respective category (local weight) and compared with all the investigated challenges (global weight).
The local weight was determined using the pairwise comparison conducted in step-4. For example, Table 15 shows that the local weight (LW) C3 (Resistance to change, W=0.382099) is the highest challenging factor in the 'Automation' category. It is also observed that C2 (Moving from legacy infrastructure to microservices, W=0.362363) and C5 (No DevOps center of excellence, W=0.170320) are standout as the second and third most significant challenging factors, respectively.
Moreover, the global weight of each challenge was calculated by multiplying its local weight with the weight of its VOLUME 8, 2020  corresponding category. For example, the global weight (GW) of challenge, C1 = 0.049895 × 0.482232 = 0.024061, where 0.482232 is the weight of its category (i.e., automation) and 482232 is its local weight (Table 15). By considering the same process, the global weight (GW) for all the enlisted challenges were determined (Table 15). The presented results (Table 15) show that C3 (Resistance to change, W=0.184261) is ranked as 1 st significant challenge for successful execution of DevOps practices in a software organization. The final ranking of the challenges was determined using the global weights presented in Table 15 Step 6 (Prioritizing of Challenges): The ultimate objective of fuzzy AHP analysis is to prioritize the investigated challenges concerning to their significance of DevOps paradigm. The determined final ranking for each challenge is given in Table 16. For determining the final rankings of challenges, the 202502 VOLUME 8, 2020 global weights are used. Considering the absolute rankings given in Table 15, C3 (Resistance to change) is the most significant challenge that needs to be addressed for the successful implementation of DevOps practices in software organizations. Furthermore, it is also observed that C2 (Moving from legacy infrastructure to microservices) and C16 (Communication and Collaboration issues) are declared as the 2nd and 3rd most priority challenges for the implementation of DevOps practices, respectively. We further noted that C22 (''Heterogeneity in development and operational structure'') ranked as least significant challenge for DevOps paradigm.

V. DISCUSSION AND SUMMARY
The ultimate aim of this study is to identify and rank the factors that could negatively affect DevOps practices. The address the study objectives, the systematic literature review study has been conducted to determine the DevOps challenging factors reported in the literature and were mapped in the core categories of CAMS model, i.e. (''Culture'', ''Automation'', ''Measurement'', and Sharing). The challenges and their classification were further verified with expert via questionnaire survey study. Finally, the fuzzy-AHP was performed to prioritize the reported challenges with respect to their significance for the success and progression of DevOps implementation in software development organizations.

A. RQ1 (INVESTIGATION OF CHALLENGES)
The systematic literature review was performed to investigate the DevOps challenges reported in the literature.
A total of 78 studies were selected by considering the step by step protocols of the SLR approach. The selected studies were explored, and a total of 22 challenges that are critical for the implementation of DevOps practices were identified. Moreover, the investigated challenges were further classified in the core categories of the CAMS model. The classification of investigated challenges is used for the application of the fuzzy-AHP process.

B. RQ2 (INVESTIGATION OF QUESTIONNAIRE SURVEY STUDY)
The questionnaire survey study was conducted to get the insight of the industry experts concerning the findings of the literature review. The results and analysis show that the investigated challenges of DevOps practices are related to industry practices.

C. RQ3 (PRIORITIZATION OF INVESTIGATED CHALLENGES)
To rank the investigated challenges and their categories, the fuzzy AHP analysis was performed. The pairwise comparison was conducted with the reported of the reported challenge and their categories. To determine the final ranking, the calculated global rank was used. The FAHP technique provides a complete understanding of decision-making problems that consider the DevOps challenges and their associated categories. Though we calculate the ranks of the identified challenges, and the results are presented in Tables 15 and 16. The results (Table 16) show that C3 (Resistance to change) is the most significant challenge that needs to be addressed for the successful implementation of DevOps practices in software organizations. Furthermore, it is also observed that C2 (Moving from legacy infrastructure to microservices) and C16 (Communication and Collaboration issues) are declared as the second and third most significant challenges for the implementation of DevOps practices, respectively.

D. RQ4 (PRIORITIZATION BASED TAXONOMY OF DevOps CHALLENGES)
The taxonomy of the investigated challenges was developed considering the local and global weights. The challenges were mapped in the core categories of CAMS model, i.e. (''Culture, Automation, Measurement, and Sharing'') [59].
The developed taxonomy (Figure 8), shows that Automation, CW=0.4822) is the top ranked category of reported challenges. This indicated that the experts consider the automation as key area that needs to be focused by the industry experts for the successful execution of DevOps practices.
Furthermore, it is noted that (Sharing, CW=0.368926), and (Measurement CW=0.135067) are declared as the 2 nd and 3red most important categories of the reported DevOps challenges.
The developed taxonomy ( Figure 13) presents the local and global weights of each challenge. This indicated that how a challenge affects DevOps activities. We observed that in the automation category, C3 (Resistance to change) is locally ranked as 1st ranked challenge. Consequently, C3 is also ranked as the highest priority challenge for the successful execution of DevOps practices. Besides, it is observed that C19 (Lack of feedback and bugs prioritization) is ranked as the highest priority challenge in the 'Culture' category, and it's standout the 19th priority challenging factor concerning the global ranking. This renders the importance of C19 within their category and for the overall project.
Similarly, both priorities rank of each challenging factors is presented in the developed taxonomy ( Figure 13), which assists the practitioners and researchers to consider the most critical challenges concerning their interest and requirements.

E. SUMMARY OF RESEARCH QUESTIONS
To address the aim of this study, the factors that could negatively affect the implementation of DevOps practices in the software development industry are identified and verified with experts via a questionnaire survey study. Further, the fuzzy-AHP technique was applied to determine the rank order of the investigated challenging factors concerning their significance of DevOps practices. The summary of the research questions findings is presented in Table 17.

VI. THREATS TO VALIDITY
Some potential risks need to be fixed for the generalization of study results. For example, there may be the researcher's biasness in the literature findings. We have conducted an inter-rater reliability test to check the researcher's biasness, and the results show that the findings are consistent and unbiased.
An external threat towards the generalization of study results is the small sample size of empirical study. The data set consists (n=93) might not be strong enough to generalize the results of this study. Though, by considering the existence of other software engineering domains, this sample size is representative of generalizing the study results [39], [66]- [68].
Most of the survey respondents were from developing countries (Asian countries); this may hinder to generalize the study results. Moreover, we also noted that a representative number of respondents are form developed continents (the USA or Australia), and this allows the generalization of results.

VII. STUDY IMPLICATION
The study sheds light on the challenging factors of DevOps implementation in the software industry, reported by the researchers and practitioners. The detailed overview of the DevOps existing literature and empirical investigations will provide the body of knowledge to researchers and practitioners to develop effective plans and strategies for the success and progression of the DevOps paradigm.
Moreover, the fuzzy AHP approach was performed to rank the reported challenges and their categories considering their significance for the successful implementation of DevOps activities. The calculated ranks orders serve as a knowledge base for practitioners and researchers to consider the most critical challenging factors or priority basis Besides, this study provides a taxonomy of the challenging factor by considering their global and local priorities. The identified challenging factors were classified into four key categories, and each category presents a particular key knowledge area of DevOps process improvement. The challenges of each category contain local and global weights that assist researchers and practitioners in choosing the most significant challenging factor concerning their interest and working area''.

VIII. CONCLUSION AND FUTURE DIRECTIONS
It is the priority of every business organization ''to get a good return on investment; therefore, the software development industry continuously looking the ways to develop effective development approaches. The DevOps is the latest and most significant approach, and it provides more satisfactory results. The significance of the DevOps process, motivate us to explore the challenging factors faced by the practitioners'' while adopting the DevOps process.
The systematic literature review approach has been adopted to identify the challenges of DevOps practices. The identified challenges were further mapped into core categories and verified with experts using a questionnaire survey approach. The empirical results show that the identified DevOps challenges are related to industry practices. This renders that it is critical to address the identified challenges for the successful implementation of DevOps practices.
Moreover, the fuzzy-AHP technique was applied to prioritize the investigated challenges and their categories with respect to their significance for the implementation of DevOps practices. The local and global ranks were determined using the fuzzy-AHP approach. The local ranks present the priority order of a challenge within their particular category. The global ranks show the significance of DevOps challenges for the overall study objective. By considering the final rankings, Automation is declared as the highest ranked category of DevOps challenges.
C3 (Resistance to change), C2 (Moving from legacy infrastructure to microservices), and C16 (Communication and Collaboration issues) are declared the most significant challenges for the DevOps paradigm in software industry. Study findings also provide the prioritization-based taxonomy of the investigated challenges, which assists the researchers and practitioners in developing the effective strategies for the success and progression of DevOps practices.
In the future, we will conduct the multivocal literature study to investigate the factors that have a negative and positive impact of DevOps practices. We also plan to perform empirical research to identify success factors and challenges. Besides, we also conducted a literature review and an empirical study to explore the best practices for the success and progression of DevOps practices.