Systems Thinking in Software Projects-an Artificial Neural Network Approach

The authors discuss why the current conceptual base of project management research and practice continues to attract criticism since it does not adequately address the complexity that leads to software-project failure. To do so, the study explores systems thinking and artificial neural networks to shed light on complexity in software-project behavior using nonlinear functional relationships between critical success factors and project success to utilize their connectedness as an approach in order to create project-outcome prediction models. The artificial neural networks were used to create two project-outcome prediction models: one for a binary classification task to discriminate failed from successful projects using a multi-input-single-output configuration and one for a multi-task binary classification to discriminate success from failure in multiple project-success dimensions using a multi-input multi-output configuration. The results yielded high-performance values for a binary classification task, performed to predict overall project success, and slightly lower performance values for the multi-task binary classification, which was also performed to predict success in project-success dimensions. It was found that the nonlinear behavior of critical success factors may be used to create prediction models, by embedding equifinality and connectedness constructs that prove to be useful to understand projects as complex, multi-loop, and nonlinear systems. Further research is needed to investigate the causality between critical success factors in order to explore the possible propagation of critical success factors within a project system network and its implications on project success.


I. INTRODUCTION
The software industry has a history of recording a high rate of failure in projects [1]. The CHAOS report indicates that only 32% of software projects in 2009 were successful; complexity was considered to be the main reason behind the failure [2]. A literature review shows that the overall challenge facing research in this field is rethinking project success and its critical factors through addressing complexity as well as broader project conceptualization and context (e.g. [3]- [6]).
Project-failure research has had its origins in the software industry since the latter half of the twentieth century when the term ''software crisis'' was first noted in 1968 [7]. Around the same time, the concept of critical success factors (CSFs) was created by Daniel [8] in his seminal work ''Management The associate editor coordinating the review of this manuscript and approving it for publication was Ahmed Mohamed Ahmed Almradi .
Information Crisis''. Popularized by Rockart [9] as an approach to improve information-system strategic planning, it was later implemented in project management practice as a means to improve project performances [10]. With the significant growth of project management research, project success and CSFs have earned their place in research traditions as one of the nine major schools of thought in this field [11]. The more recent focus of research in the success school has been placed onto identifying success as a multidimensional construct, including dimensions such as project management and product success [10], as well as business and future success [1]. Similarly, existing research on CSFs, in addition to their categorization for software projects, have focused on recognizing relationships between varied groups of factors and project success [10] as well as on creating contingency-fit models for traditional plan-based and agile methodologies [12]. However, in most cases, existing models are created using linear regression, neglecting non-linear relationships that exist between CSFs and project success. The very process of statistical analysis includes the identification of the factors most significant, disregarding ''less significant ones'' and their relationships with factors previously identified as significant. Doing so results in statistical models excluding factors that could indirectly affect project success. Thus, a major criticism of existing models is insufficient focus on complex functional relationships between individual factors and project success, in particular as concerns the nonlinear nature of such relationships. An additional issue is contingency-fit models failing to recognize that the same final state of project success is achievable from dissimilar initial conditions when using multiple combinations of CSFs.
A review of the literature shows that substantial progress has been made with the application of systems thinking as a holistic discipline that recognizes projects as interconnected technical and social factors producing behavior which otherwise cannot be predicted by simply aggregating the behavior of the projects' elements in isolation [13]. Over the last two decades, there has also been a considerable amount of research into intelligent systems and the application of machine learning (ML) as highly effective techniques to address problems of complexity (e.g. [14], [15]). In regards to prediction models, more recent research has included the application of artificial neural networks-ANNs [16], Bayesian classifiers (e.g. [17]- [19]), genetic algorithms (e.g. [19], [20]), support vector machine [20] and logistic regression models [21]. Nevertheless, ANNs are the most used techniques for software-project issues (e.g. [22], [23]). As may be anticipated, a number of studies have found that ANNs outperform traditional regression models (e.g. [24]- [26]). Furthermore, one of the major benefits of using ANNs is their ability to learn and communicate past knowledge [27] and even include valuable information from unsuccessful projects [28]. However, in the field of project management, most studies have only focused on predicting whether a project is risky or not [16], as well as on efforts estimation (e.g. [29]- [33]), duration estimation [33], and cost estimation (e.g. [34]- [36]). Despite the overwhelming interest, few researchers have yet to address the complex relationship between software-project CSFs and project success. Thus, the authors intended to conduct research that would provide MIMO (multi-input multi-output) and MISO (multi-input single-output) ANN models capturing complex relationships between CSFs and project success directly as based on real data from past software projects.
The main purpose of this study is to explore the nonlinear behavior of CSFs to predict the project success of software projects measured as a multidimensional construct, as well as to search for a management tool to balance success across dimensions of software projects. This study focuses on making ANN-based project outcome prediction models, taking into account that the same state of the project could be achieved using disparate sets of CSFs.
Specifically, the research aims to answer two questions: 1) ''How does the MISO model perform when identifying overall project success vs. failure?''-which will be addressed through a binary classification task to discriminate failed from successful projects; 2) ''How does the MIMO model perform when identifying success vs. failure of a project-success dimension?''-which will be answered through a multi-task binary classification to discriminate success from failure in multiple project-success dimensions. As an end effect, the authors hypothesize that using CSFs allows for creating prediction models to provide valuable data to project managers and improve software development projects and software project success. This study will use the nonlinear behavior of CSFs to bridge the gap of using only contingency and linearity constructs through a systems-thinking approach in order to attain better performance for complex software development projects.
Bearing in mind that it is one of the few papers to do so, this study will contribute to the theory of project management using the nonlinear behavior of CSFs to predict software-project success.
The paper is laid out in 6 sections. Section 2 reviews the key terms and theories utilized in the research. Section 3 covers a literature review of critical success factors. Section 4 details the method used, consisting of a data collection approach and ANN approach. In Section 5 the most significant results are presented and discussed. Section 6 concludes the paper and presents implications for project management theory and future research.

A. PROJECT SYSTEM AND PROJECT SUCCESS DEFINITION
This paper uses systems thinking to define projects as complex, multiple-loop, non-linear, and open systems (e.g. [37], [38]). Systems thinking allows for an understanding of the project as an open system of interconnected, technical, and social factors that produce a system's behavior [13]. Such a holistic viewpoint of a projects' definition addresses criticism of closed boundaries and the linear causality present in conventional project management theory.
In the context of project management, the boundary of a system is the scope of the stakeholders' interests and is mutable as the scope of interests changes [39]. Indeed, a systems-thinking approach uses the construct of causal connectedness to examine boundary management [38], which is based on the notion that any act of change could unpredictably influence the relationships between stakeholders and the project itself [40]. It is here argued that software development projects are characterized by high levels of boundary complexity since they tackle both internal and external stakeholders with their diverse characteristics and values; i.e., software-project success is achieved by balancing internal and external stakeholders' interests. There are, therefore, at least three separate dimensions of project success: 1) project management, 2) product and 3) strategic (e.g. [1], [10]).
The first dimension of project management success also referred to as the level of project success, is measured using internal efficiency measures of project management, such as budget, time, and similar performance measures for internally set constraints [1]. The second is product success, focused on delivering benefits to internal and external stakeholders (e.g. [10], [41], [42]). The third, strategic project success, concerns business efficacy and strategic effectiveness, as well as the ability to generate future success for the company (e.g. [1], [10], [42], [43]).

B. CRITICAL SUCCESS FACTORS
CSFs are several key areas which, if successfully managed, will lead to better organization performance (e.g. [9], [42], [44], [45]). In the context of project management, CSFs are characterized as those factors that directly influence and increase the probability of project success (e.g. [41], [46]- [49]. The discourse to prove this definition is composed of several distinct arguments.
In systems-thinking theory ''factors'' could be interpreted as supernatural, organizing principles [50]. In project systems, if values govern the actions of the project stakeholders, factors will govern the outcomes of those actions. The likelihood to achieve the desired project outcomes depends on the stakeholders' values and the factors linked to the desired behavior, thereby accounting for the difficulty in predicting behavior in any complex project. However, a project where every stakeholder understands and is committed to factor-based behavior will enable a system to achieve the desired project outcomes under a far higher likelihood. On the other hand, projects that are not grounded in success factors will result in a wide variety of project stakeholders' personal interpretations on how to behave.
Further, in systems thinking theory, CSFs enable the anticipation and rationalization of temporary collective behavior of a project system, by accounting for the interaction that exists between factors. Beyond simply identifying CSFs, there is also a need to incorporate their interaction in predicting project outcomes.
Finally, adopting an open-systems approach in project management includes generic constructs such as equifinality [38], which refers to a system being able to achieve the same final state from dissimilar initial conditions and do so in diverse manners (e.g. [51]- [53]). According to Child [54], equifinality implies that strategic choice is available to decision-makers when creating organizations to achieve high performance-particularly in complex organizations. Thus, this construct is necessary to effectively manage projects in non-linear unpredictable environments [38]. For the purposes of this study, the authors use equifinality to explore the phenomenon where the same level of performance is achievable from multiple contexts, using different sets of factors.

C. ARTIFICIAL NEURAL NETWORKS
ANNs are biologically inspired computational models mimicking the information-processing structure of the human brain through a similar network. In general, ANNs are considered to be powerful non-procedural and adaptable prediction tools, consisting of a number of interconnected neurons with linear and nonlinear activation functions [55]. In order to predict a project's system behavior, ANNs use an iterative training process to learn linear and nonlinear relationships between project system elements. One of the most important advantages of ANN applications is their ability to capture the underlying patterns of available datasets and model complex relationships between input and output variables, while not having a complete understanding of the complexity of functional relationships between variables. Even though prediction models are based on large amounts of data, the researchers often have to deal with much smaller data sets and create the most accurate model from the available data [56]. The generic nature of the methodology and lack of any limitation to the type of dataset or the number of input and output variables makes an ANN approach highly applicable in software projects.
The structure of an ANN consists of an input layer, an output layer, and one or more hidden layers. The input layer introduces the data to the network, while the output layer is the final layer that has a set of values to represent the network's output. The hidden layers are placed between both the input and the output layer to perform the calculations and produce internal representations of input patterns. ANNs are typically configured based on the choice of the number of the input and output nodes, the number of hidden layers, the choice of activation functions, and other design parameters. A trial and error approach is generally applied in order to achieve optimal architecture. Such parameters change throughout the experimental phase to select the optimum ANN architecture, taking into account an acceptable compromise between predictive accuracy and processing time. In nonlinear input-output mapping, the ability of hidden neurons is considered necessary to extract higher-order statistics. Using only a few neurons in the hidden layer might disable networks in order to capture the nonlinear trends in the dataset, thereby resulting in the lower predictive accuracy of the model. Nonetheless, too large a number of hidden neurons could result in the time-consuming process of training and ultimately over-fitting the training dataset.

III. LITERATURE REVIEW
Hitherto research reported in the literature involving the application of ANNs to project success concerns the objectives of project outcome prediction and CSF identification, as found in [57]- [65]. The main challenge is that an accurate prediction of project success has proven difficult due to the complex relationship between project success and CSFs, resulting in the application of multiple-input single-output models, where output variables are simple success measures. VOLUME 8, 2020 However, multiple studies have shown that success needs to be understood as a multidimensional construct in software projects to appreciate the complex dynamics and impacts within them [1]. In order to better comprehend the complexity that exists in software project systems and to address project success, there is a necessity to rise beyond a single output model and to bring forth multiple output models to predict success in multiple dimensions of software project success.
In the study of software projects, ANNs are generally recognized for their ability to produce reasonably accurate predictions in situations where there are nonlinear relationships between inputs and outputs [29]. As a non-parametric method, ANNs are easy to use because they extract implicit knowledge from past experience without involving project managers in complex subjective judgments [61]. However, software development is filled with such judgments. For example, the success of software projects clearly depends, to a significant extent, on the quality of the communication, yet communication quality is rarely measured and based on easily available data. While the assessment of such factors is the subject of CSF studies, the evaluation of CSFs on software projects with regard to ANNs can help to overcome the disadvantages of using only easily available data collected from past projects. In addition, the use of CSFs as input variables can add considerable value and insight into the modeling process of software project management. The key to modeling usable project outcome prediction models is to move beyond the limits of easily available data and to conceive of information as it relates to key areas of activity in which favorable results are absolutely necessary for project success.
Although a large number of studies categorize CSFs into key areas, suggesting alternative frameworks (e.g. [10], [12], [48], [66]), there is no general categorization of CFS for software projects [67]. To illustrate, many authors have categorized CSFs for either agile or traditional software projects. Authors such as Chow and Cao [66] focused their research on agile software projects and categorized 109 CSFs into five categories: organizational factors, people-related factors, process factors, technical factors, and project factors. In addition, Ahimbisibwe et al. [12] identified 37 CSFs for both agile and traditional software development projects and grouped them into organizational, team-related, customerrelated, and project factors. Sudhakar [10] proposed a conceptual model examining 80 CSFs, composed of seven categories of factors: communication, team, organization, environment, technical, project management, and product. Sudhakar [10] also recognized that CSFs of one category affects another, suggesting that, in addition to relationships between CSFs and project success, there are also relationships between the CSFs themselves. Based on the findings in the literature (e.g. [10], [12], [66]), the study presented here categorizes CSFs into five groups (Table 1): customer factors, technical factors, project management factors, organization factors, and team factors. Factors, such as customer involvement, customer experience, and customer support [12], are all variables related to project customer characteristics. Customer involvement has been reported by numerous empirical studies in software development projects to be one of the most significant factors that influence project success (e.g. [66], [68]- [71]). Furthermore, customer experience as a predictor of success refers to customer familiarity with this type of application and their understanding of the problem they wish to solve using a software development project (e.g. [12], [69], [70], [72], [73]). Studies have argued that customer leadership characteristics refer to the acceptable level of a business domain, conceptual skills, and personal characteristics that could increase the chances of project success ( [12], [74]).
Project management CSFs as predictor variables refer to adequacy or flexibility in planning and controlling practices used in a project. In general, planning positively influences the process' performance ( [68], [70]); however, innovative projects such as software projects, require flexibility, in which there are focal goals to decide on a course of action [38]-the same applies to task control and coordination. Hence, if there is low uncertainty in a project, traditional approaches planning, monitoring, and controlling the project's scope, resources, budget and milestones should work well. However, under circumstances of complexity and uncertainty, responding to change is one of the key adequacy principles of project management, which exclusively needs flexibility in planning and control practices. On the other hand, the predictability of project success is connected to the availability of data on a project's status that allows for project managers to track a project's progress and arrive at management decisions correctly [21]. In this context, a predictor of success is a variable that takes into account items such as adequacy of monitoring and reporting progress as well as the availability of data.
The literature considers adequate development methodology, documentation, and testing to be determiners of success in both plan-driven and agile methodologies as relates to the characteristics of the development environment [75]. In addition, technical factors as predictors of project success include variables that are related to requirements. Clear requirements and specifications are the most important predictor of project success [48]. Predictors of risk, on the other hand, may include such items as having clear and detailed requirements, both project members and customers equally understanding and sharing a commitment to the requirements, as well as appropriately managing specification changes [21].
Team factors contain variables pertaining to communication, competency level, training and education, turnover, and motivation. The success of software development projects is argued to depend greatly on effective communication and feedback (e.g. [12], [48], [66], [70]). Internal project communication is defined as a practice that increases information exchange and cohesion among development team members [12]. In general, internal project communication enhances information and knowledge sharing between the members of the project team thereby increasing team performance. Ali et al. [76] reported collaboration, coordination, and cooperation to be highly correlated with the strategic aspects of project success, such as new business opportunities. Together with maintaining commitment and motivation of project teams (e.g. [48], [66], [77], [78]), team capability, competences and skills are reported as the CSFs and backbone of the software industry (e.g. [66], [69], [73], [75], [78]). Finally, staff turnover is argued to influence project operations (e.g. [79], [80]), particularly its strategic aspects through the loss of future potential of already capable team members and the higher costs incurred when hiring replacements.
According to Howell et al. [81], since parent organizations play a dominant role in the project's environment, it is reasonable to view organizational variables as predictors indicating project behavior. The literature review shows that top management support is a crucial factor for the success of software development projects and, as such, is a significant predictor of project performance (e.g. [12], [82]- [85]). Further, multiple authors suggest that organizational culture has a positive effect on project success, especially in agile project management (e.g. [69], [71], [78]). Another essential factor is a clear vision and mission [78]. Kaufman et al. [86] assert as such, stating that an organization's objectives are crucial factors in strategic planning and thinking in project management.
Based on the literature, our study continues CSFs long place in software projects. The study aims to use CSFs as success predictors to create project outcome prediction models through the conjoint use of expert judgment and non-parametric methods such as ANNs.

IV. METHOD
The method applied in the study included conjoint use of expert judgment and non-parametric method to create a multi-organization data set and perform experiments on evaluating CSFs on software projects using a set of ANN models, MISO and MIMO. The study designed a method to answer its two research questions of how the MISO model performs when identifying overall project success vs. failure as well as how the MIMO model performs when identifying success vs. failure in multiple project-success dimensions.
As to create multi-organization derived data, the study has applied a survey-based empirical approach to collect data on past software projects, as well as a key-informant approach to identify the software-project management professionals to match the criteria of the study; i.e., experts who possess sufficient experience as project managers on software projects. As the most crucial element of the study was accessibility to software project professionals, convenience sampling as a type of non-probability sampling method was deemed best to apply.
Using project management practices from a country-bycountry perspective has been shown to be extremely valuable in the literature for creating project outcome prediction models [87]. Therefore, in order to access software project management professionals who possess sufficient experience in carrying out software projects, the study used a database limited to software project managers that carried out projects for software companies headquartered in Serbia, created by the Project Management Centre at the University of Belgrade. In addition, with more than 2,500 active software companies [88] and ranking as 12th out of the 131 world's economies in the export of ICT services (% of total trade) [89], Serbia is among the most globally relevant countries for knowledge sharing in ICT and makes it a fruitful source to carry out such research.
In order to address the research questions and perform experiments on evaluating CSFs on software projects, the study proposed an ANN-based approach to perform: 1) a binary classification task to discriminate successful projects from failed ones; 2) a multi-task binary classification to discriminate success from failure in multiple project-success dimensions. Accordingly, the study created a set of multilayer neural networks, MISO and MIMO, wherein input layers correspond to inputs of the project system-CSFs, and output layers representing the output of the project system -project success.
In order to establish a basis to draw a comparison between MISO and MIMO, the approach was designed to use the same CSFs as input variables and output constructs created from project success dimensions (PMS, PS, and SS). To use average values in creating output variables, the study proposed Cronbach's alpha coefficient to validate project success dimensions. For the binary classification tasks, output constructs of both MISO and MIMO were labeled as binary class outputs. The study employed k-means clustering, therefore, in order to ensure a balanced dataset in terms of distribution between the two classes, as well as to construct a single output VOLUME 8, 2020 model (MISO) based on all three dimensions of project success. As a result, MIMO output variables were labeled using success criteria, while the same for MISO used clustering results. Furthermore, the study used these binary-class output variables constructed for MISO, to perform the evaluation of input variables through a two-step evaluation process. First, by identifying whether input variables (CSFs) are nondiscriminatory towards successful and failed projects, followed by identifying whether there is a nonlinear relationship between inputs (CSFs) and outputs (project success) using threshold values for maximum correlation and minimum significance.
In order to create an ANN design for both MISO and MIMO, the study utilized a trial and error approach. Accordingly, a series of experiments needed to be performed in order to test the training algorithms, network architectures, preprocessing methods, transformation functions, as well as validation methods. The objective was to evaluate multiple ANN designs in terms of accuracy and processing time, then to decide on those most optimal for MISO and MIMO.
A. DATA COLLECTION APPROACH Some general information on the data collection approach used in the study is here provided.

1) QUESTIONNAIRE
The authors used a comprehensive survey questionnaire to collect information specific to CSFs and performance measures on past software projects. As suggested by Sekaran [90], the first section of the questionnaire covered question-items related to CSFs (Table 1), identified from the critical literature review of our study. The items were measured using a five-point Likert scale (1 -strongly disagree, to 5-strongly agree), as the authors used a measurement scale created by Ahimbisibwe et al. [12].
The second section of the questionnaire was used to measure performance for the three dimensions of project success: 1) project management success-PMS, 2) product success -PS, and 3) strategic success-SS. The list of the 27 performance measures grouped by project success dimensions is presented in Table 2. These items were also measured on a five-point Likert scale (1 -strongly disagree to 5-strongly agree).

2) PARTICIPANTS
The aim of the data collection was to accumulate information on how software projects were executed using the practitioners' experience as pertinent for the actuality of projects, as suggested by Cicmil et al. [91]. In order to obtain the needed information, the key informant approach was used and project managers were targeted as appropriate respondents. Such an approach is more time-intensive but allows for researchers to ask additional questions to clarify all aspects of the questionnaire.
Owing to the fact that the most critical element was accessibility, a non-probability sampling approach was applied. With the participating project managers being personally contacted, almost a 100% return was achieved. The average working experience of the project managers involved in this study was 14 years and almost half of the participants possessed more than 20 years of experience in the field.

3) SAMPLE
Data on 47 software projects were collected from 35 separate private firms by involving projects of a wide range of contextual and developmental characteristics. Consistent with prior findings on prediction models based on multi-organizational data sets [87], the sample organizations included in this study were both small and large in scale, ranging from 5 employees in the smallest to more than 5,000 in the largest. As concerns their organizational capacities, the targeted firms run both small and large-scale projects, as reflected in the projects' team size, duration, and budget. A breakdown of the companies and projects is provided in Table 3. As may be expected, smaller-sized organizations are more likely to operate smaller-scale projects (2-5 team members) under budgets not exceeding USD 100,000, whereas larger-sized organizations operate larger-scale projects that have over 50 team members, are implemented in a timeframe between 24 and 36 months (or exceeding this timeframe) and are worth at least one million USD but may also be more than 10 million USD.

B. ARTIFICIAL NEURAL NETWORK APPROACH 1) MIMO AND MISO MODELS
To create project outcome prediction models, the authors used two ANN models: 1) MISO is a multilayer neural network   architecture composed of a multiple-input and single-output configuration (Fig 1); and 2) MIMO is a multilayer architecture composed of multiple-input and multiple-output configurations (Fig 2). While, in the both models, each neuron in the input layer represents an input in the project system, such as a CSF, each neuron in the output layer corresponds to the output of the project system, such as the project success class. Further, for the MIMO model, it was decided to create output variables corresponding to each dimension of the success  construct, and for the MISO model, to use the overall success as a construct corresponding to a single output variable.

2) PROJECT SUCCESS
In order to validate all three dimensions of project success (PMS, PS, and SS), Cronbach's alpha value was calculated for each, whereby a set of 9 specific dimension item-questions were tested to find if they do measure the same construct ( Table 2). The PMS showed a satisfactory internal consistency of α = 0.936. PS did so also at α = 0.965 as well as the SS with α = 0.951. Considering that internal consistency is high, dimension scores are calculated as an arithmetic mean of question-items for dimensions of success (Fig 3).
Further, K-means clustering was applied in order to take into account the three project-success dimensions in order for every project to decide on overall project success. The results for initial and final centers are presented in Table 4.
While cluster one consists of projects that have the best scores for all three project-success dimensions (PMS, PS, and SS), cluster four consists of projects with scores higher than 4.00 for two out of three dimensions (PMS and PS); the projects in these two clusters could be considered successful. The other clusters two and three consists of projects of scores lower than 4.00 for all three project-success dimensions; accordingly, these two clusters consist of projects that could be considered to be ''failed'' projects.

3) LABELING OUTPUT VARIABLES -MIMO
In order to label output variables for MIMO, success criteria were applied based on scores for each success dimension, wherein scores greater than 4.00 were labeled as 1 (success), if-else 0 (failure).

4) LABELING OUTPUT VARIABLES -MISO
In order to label output variables for the MISO model, projects were assigned to clusters based on success scores in all three dimensions, wherein projects with scores greater than 4.00 in at least two dimensions out of three were labeled as 1 (successful projects), if-else 0 (failed projects).

5) INPUT VARIABLES
In order to determine the effectiveness of a selected set of input variables, we performed a two-step evaluation process as presented in  An independent sample t-test to each input variable was applied to test whether the data means are equal in the two classes of projects labeled as 1 (successful) and 0 (failed). The labels were obtained from the overall success classification of projects as either successful or failed, as presented above for the MISO model. A criterion of 0.05 p-values was used to rank input variables and identify non-discriminative ones.
A correlation matrix was applied to explore threshold values for maximum correlation and minimum significance. The correlation threshold included a higher bound that is the highest allowed correlation coefficient with respect to classes (0.6 in this study). The variables with the lowest p-value were considered relevant to be preserved among highly correlated variables.

6) ANN DESIGN
All models were back-propagation feedforward networks. The optimal ANN design for both MIMO and MISO models was determined through experiments to evaluate the training algorithms, network architectures, preprocessing methods, and transformation functions, as well as the Leave-p-out method (LPO) as related to their accuracy (the ratio of correct predictions) and processing time. While not all results of the ANN design task were significant, their overall direction showed trends that might be insightful regarding parameters  Conversely, RPA is significantly more efficient in comparison to SCGA in terms of processing time (Fig 6b).
For both MIMO and MISO models, the experiments were set to simulate gradual network-structure growth by adding new network elements and included models with one-, two-, three-and six-hidden-layer config-    (Fig 8), apart from the minimum values of the accuracy of the six-layer-networks that were considerably lower in comparison to others. Finally, for both the MIMO and MISO models, the number of neurons is indicated to influence processing time.
In addition to the experiments set using raw data, experiments were set with two types of normalization [0, 1] and [−1, 1] for the preprocessing methods of the MIMO model (Fig. 9). Concerning both accuracy and preprocessing time in comparison to the data of [0, 1] normalization and raw data, the preprocessing method of [−1, 1] normalization shows better performances for all three criteria. For MISO, the authors applied only [−1, 1] normalization.
Both the MIMO and MISO models used tansig, purelin, and satlin transformation functions to create a combination of hidden and output layers; for instance, tansig-satlin combines tansig for hidden layers and satlin for the output layer. For the MIMO model, in terms of both the maximum value of accuracy and processing time, satlin-satlin and tansig-satlin outperform tansig-purelin (Fig 10). For the MISO model, using multiple combinations of transformation functions yield intriguing results. Fig 11 shows that both satlin-satlin and tansig-satlin combinations outperform the combination of tansig-purelin in relation to their accuracy, particularly their processing time.
The LPO method was applied by removing seven samples (p=7) sequentially or randomly from the complete set as the test set and using the remaining as the training set. For the MIMO model, in terms of the maximum value of accuracy, the random method was better than the sequential (Fig 12), even if slightly less time-efficient. For the MISO model, only the sequential LPO method was applied.

A. PERFORMANCE EVALUATION
In order to report results on MISO and MIMO performances, the study uses the following metrics: 1) Accuracy (ACC); 2) False-positive ratio (FPR); 3) Precision (PREC); 4) Recall (REC); 5) F-Score; and 6) Area under the ROC curve (AUC). While accuracy is a better performance measure to reflect on positive and negative classes correctly predicted, it is still unable to provide information on how the model performs on negative classes incorrectly predicted as positive classes (as provided by FPR). Further, precision and recall are preferable metrics when predicting positive classes is important, yet the F-Score is a better metric when incorrectly predicted classes VOLUME 8, 2020  are too costly; particularly when the benefit of correctly predicted classes is less important than the cost of incorrectly predicted classes. To complete a thorough performance evaluation, the study uses AUC to provide information on the quality of the model's predictions. All these metrics are equally important to indicate how well models perform.
Research question 1: ''How does the MISO model perform when identifying overall project success vs. failure?'' In order to provide an answer to this question, a binary classification task was performed and the results are presented in Table 5. With an accuracy of 0.83, the results show that MISO is highly capable of predicting overall success vs. failure in software projects. With a precision of 0.87, the probabilities that the model correctly predicts positive classes (success) are more than acceptable, whose sensitivity shows a value of recall at 0.88. The model's satisfactory performances are confirmed by the F-Score (0.88) and FPR (0.13), demonstrating there to be a low probability of false alerts to be raised by failed projects that are incorrectly identified as successful. Certainly, the high performance of tradeoffs between sensitivity and specificity in a binary classifier is confirmed by the AUC of 0.84, which demonstrates the model's overall high predictive quality.
Research question 2: ''How does the MIMO model perform when identifying success vs. failure of a project-success dimension?'' In order to provide an answer to this question a multi-task binary classification was performed, the results of which are presented in Table 5. With an acceptable average accuracy of 0.72, the result showed that MIMO performs well when predicting success vs. failure in multiple project success dimensions. However, with an accuracy of 0.73 in PMS, 0.80 in PS, and 0.62 in SS, the results also do indicate there to be a significant difference between performances in predicting success vs. failure in multiple project success dimensions. Its probability to correctly predict success in PMS and PS is fairly high, with a precision of 0.76 for PMS and an excellent precision of 0.87 for PS, but unacceptably low for predicting outcome in the SS dimension. The sensitivity of the MIMO showed reasonably good performances at an average recall of 0.79, as well as the overall quality of the models' predictive capability with an average AUC of 0.75. However, the results for the FPR show a high probability of incorrectly classifying failure as success with a fall-out of 0.42 in the SS dimension, which is better, but still a high fall-out of 0.26 in the PMS dimension in comparison to the 0.13 in the PS dimension. Finally, with a reasonable F-Score of 0.76, the results show that MIMO does well on average in terms of the test's accuracy.

B. COMPARISON WITH PREVIOUS METHODS
One of the main challenges of project managers is the early prediction of project outcomes. If project managers are able to anticipate the success or failure in disparate project success dimensions, they would be able to better decide on key areas of action and better prepare for the future. Accordingly, this study used ANNs to improve project success and project success prediction with regard to complex relationships existing between input and output variables of the software project system. As a result, ANN models showed satisfactory performance when modeling nonlinear relationships existing between CSFs and an overall project success based on the multidimensional construct (MISO) or multiple project success dimensions in software projects (MIMO).
In comparison to the traditional parametric method using historical project data to create linear models based on regression analysis, ANNs do not depend on data distribution or assume a fixed structure of a model [62]. These models, owing to their robust nature, are more accurate than linear models are (e.g. [61], [65]). Furthermore, where there are nonlinear relationships between input and output variables, as well as where there is less information available on the relationships between variables, parametric methods fail to provide an accurate prediction.
In addition to parametric methods, expert judgment is another traditional method used to predict project outcomes. In this paper it is found that expert judgment could be used for variable evaluation, adding value to the quality of data collected in key project management areas. Although the complexity of software projects makes using expert judgment difficult to accurately predict the future, the expert judgment itself may be used in conjunction with other methods, such as has been used as an adjustment factor in parametric models [65] or to create Bayesian models [92]. Further, metrics selection based on the expert judgment was a method used in one of the first applications of artificial intelligence to project success, the model of which was created by applying a Bayesian classifier to estimate the project outcome (e.g. [17], [93]). Therefore, the conjoint use of expert judgment and non-parametric methods is already recognized in the literature as it brings value to software project management through interaction between project managers' complex subjective judgments and ANNs as non-procedural and highly adaptable prediction tools capable to learn linear and nonlinear relationships between project system elements.

C. ADVANTAGES OF THE METHOD
The application of the MISO and MIMO models allowed the authors to address the criticism of closed boundaries through the multidimensional construct of project success. The authors argue that software development projects are characterized by high levels of boundary complexity and that project success is determined by the diverse interests of both the internal and external stakeholders [38]. Hence, the output variables for both the MISO and MIMO models were based on a project-success multidimensional construct that included the three project-success dimensions of PMS, PS, and SS (e.g. [1], [10]). Further, the application of the MISO and MIMO models allowed the authors to use equifinality instead of contingency theory and to address the criticism of the linear causality present in conventional project management theory through nonlinear relationships between CSFs and project success. Equifinality implies the availability of strategic choice to project managers to effectively manage projects in complex and unpredictable environments [38]. The diversity of CSFs used as input variables proves that a project system is able to achieve the same final state, success or failure, from distinct initial conditions and using multiple combinations of CSFs.
The majority of research on project-outcome prediction models thus far has focused on creating appropriate methods to identify the characteristics or similarities of software projects used as predictors of project success (e.g. [17], [19], [94]- [96]). This study selected CSFs, as predictors of project success, from organizational, customer, team, technical, as well as project management perspectives and used arguments provided by Sudhakar [10] that CSFs are interconnected, both directly and indirectly affecting project success. However, due to the complexity of project systems, existing models of CSFs that have been researched as linear models are unable to address factor interactions, owing to complex functional relationships tending to invalidate the assumptions of most statistical methods [38]. Therefore, rather than a few measurable variables in linear relationships, a network of CSFs is a more appropriate design to comprehend the behavior of the project system owing to the numerous interactions and dependencies among system elements in complex project systems. In such networks, CSF interaction is determined by the causality between multiple factors and as interaction is oriented from one factor to another, the precedence of the relationship between factors is crucial when defining the nature of CSF interactions, as well as the dynamism of CSFs [64]. The literature review shows that companies use AI to deliver value in various business domains (e.g. [97]- [99]). However, due to the complexity of the interaction between humans and AI (e.g. [100]- [102]), some aspects of AI application still need a human expert to reflect on the problem domain [103]. Therefore, the study finds that the conjoint use of ANNs and CSFs obtained through the expert judgment method contribute to performance improvements in terms of decision-making in software projects.

D. LIMITATIONS
The output variables for both the MISO and MIMO models were based on a project-success multidimensional construct that included three project-success dimensions PMS, PS, and SS (e.g. [1], [10]). However, one challenge facing this study is that obtained datasets in practice are often imbalanced, which results in lower performance of classification tasks for underrepresented classes in the training data (e.g. [104], [105]). For this reason, k-means clustering was applied to create a more balanced distribution between two classes for the MISO model, labeled as successful or failed projects. In contrast, the authors decided to apply success-score criteria to the MIMO model in order to label project-success dimensions (scores > 4.00 = 1 -success, if-else 0 -failure). Consequently, one limitation for this model was the SS dimension's imbalance in the distribution between the two classes, yielding a worse performance of classification tasks than for the PMS and PS. A solution to this problem could be advanced sampling techniques that generate new data in under-represented classes based on current data [105].
In order to create a prediction model by capturing the nonlinear relationships between variables, input-variable analysis for the MISO model was achieved through the application of an independent sample t-test and correlation matrix to select input variables by exploring threshold values for maximum correlation and minimum significance. By doing so, the study overcomes the linearity assumption of statistical methods based on the selection of the most significant factors, while disregarding ''less significant ones'' and their relationships with factors previously identified as significant, as well as includes factors that could both, directly and indirectly, affect project success. However, the same input-variable analysis process for the MIMO model was not applied, as it was based on input variables previously selected for the MISO model. A separate input-variable analysis for the MIMO model would require further exploring relationships between CSFs and project-success dimensions, possibly resulting in the necessity to utilize different sets of input variables for every project-success dimension and to create three separate MISO models.

E. THREATS TO VALIDITY
This study tests the variables that contribute to the project success proposed in the literature to construct a theory related to predicting the project outcome of software projects. Hence, this subsection includes a construct, conclusion, as well as internal and external validity analysis to assess how well a research method identifies concepts of interest and what the implications are for the measures used, its findings, and conclusions. In order to make sure that there is a relationship between theory and investigation, the authors performed a comprehensive review of the literature on CSFs for software projects, finding that the majority of the identified CSFs were covered by the survey's questionnaire. In addition, in order to validate all three dimensions of project success (PMS, PS & SS), the Cronbachs' alpha value was calculated for each.
All participants were asked to respond to at least one questionnaire on finished projects in which they had been project managers. In order to prevent hypothesis guessing, those surveyed remained uninformed about the main objectives of the study. Since the models in this study are tested using data on already finished projects, the relevance of such models could be identified as a threat to its validity. Therefore, the authors plan to test the models using on-going projects at the time of research in future studies.
In order to make sure that there is a relationship between the method and the outcome, all participants were selected due to their sufficient experience in software projects. Basic instructions to answer the questions were provided to ensure reliability and consistency in the responses.
A threat to internal validity is that participants are prone to positivity bias thereby being less critical when reporting any negatives in successful projects or being too critical when reporting any negatives in failed projects. The respondents were therefore asked to provide information on projects under no specific instructions to choose a successful or failed project.
Regarding external validity, this study sample was gathered through a convenience sample; all the participants from Serbia were software project management professionals holding positions in the industry. Particular considerations of external validity were a reflection of the practitioners sampled; as such, the authors strove to best guarantee that participants were involved in professional software development across several industries, project sizes, and project types, including in-house and outsourced development projects. Apart from access to the final study results, there were no other incentives offered. Finally, this study shows that the models are general enough to provide acceptable predictive results even though they were built with project data from multiple companies and projects that are both in-house and outsourced.

VI. CONCLUSION
In conventional project management theories, when faced with practical problems, the focus tends to be placed on statistical models based on few measurable variables in a linear relationship, which thereby fails to take into account that projects are complex, multi-loop, and nonlinear systems. However, the consistently high rate of failure in software development projects leaves one to question these statistical methods.
This study, therefore, advocates that software development projects are open systems of interconnected CSFs which produce a project system's behavior. Results from this study confirm that CSFs are capable of doing so. Furthermore, it is found that ANN-based project-outcome prediction models (MISO and MIMO) may be created by using nonlinear functional relationships between CSFs and project success. The same results of the study indicate that equifinality should be embedded in project management theories to help understand that projects are capable of achieving the same final state from disparate initial conditions. A systems-thinking approach, therefore, is able to contribute to balancing the interests of project-success dimensions from multiple stakeholders by implementing causality and equifinality in contemporary project management methods.
Implications for project management theory and future research could be summarized in conclusions drawn from the results. Firstly, in complex systems, such as software projects, results show that equifinality (flexible project management strategy based on multiple combinations of CSFs) and boundary management (causal relationships between CSFs and project-success dimensions) may be highly significant to a successful practice and more significant than contingency fit models. Secondly, even though the emphasis on nonlinear relationships between CSFs and project success leads to a better understanding of the behavior of the software project system, these arguments should not be perceived as the exclusion of linear models or conventional project management methods. This study challenges only the emphasis placed on linear models and other conventional project management theories such as contingency theory, proposing that these should be improved by introducing constructs such as equifinality and causality. Research from other project management fields, such as risk project management, has recognized systems thinking to be an approach that allows projects to be more successful by better understanding the complexity and causal connectedness between risks [37]. Further research is needed to quantify the interaction of CSFs and explore possible propagation of specific factors within a project success network in order to identify the most influential factors in the project system as a whole.
Therefore, this study challenges the theory of conventional project management, unable to appreciate the complex nature of software development [106]. The findings of this study provide evidence running counter to contingency fit models of CSFs for software development projects [12]. The authors make distinct contributions to the theory of software project management (e.g. [1], [10]) by providing systems thinking approach, which may prove to lead to a more successful practice.
Overall, this study encourages further research into the field of software projects to discover how systems thinking could be better applied by software project managers. There is quite strong support for the idea that a combination of CSFs and artificial intelligence tools, such as ANNs could add more value in software project management, especially through better understanding complex relationships existing between CSFs and multiple dimensions of project success. Since this study does confirm the value of such an approach, it will deliver greater confidence to software project managers in their continuing to investigate project outcome prediction models through the interaction of humans and AI. JOVANA Ð. RAKIĆEVIĆ received the B.Sc. degree in information systems and technologies and the M.S. degree in management of development from the Faculty of Organizational Sciences, University of Belgrade, Belgrade, Serbia, in 2011 and 2013, respectively, where she is currently pursuing the Ph.D. degree in information systems and quantitative management.
Since 2011, she has been a Teaching Assistant with the Faculty of Organizational Sciences, Department for Technology, Innovation and Development Management, University of Belgrade. After about ten years of teaching in the field of technology and innovation management, two years ago she started intensive extracurricular work with undergraduate students on building their knowledge and skills in technology entrepreneurship. Her research and teaching interest are in the fields of technology and innovation management and technology entrepreneurship, technology commercialization and transfer, and entrepreneurial ecosystem support. She is an author and co-author of five chapters in international monographs, 10 scientific papers in domestic and international journals, and over 40 papers in proceedings of domestic and international conferences. She is a member of the Triple Helix Association and the European Society for Fuzzy Logic and Technology.
ELENA T. JELISIĆ received the B.S. degree in information systems and technologies and the M.S. degree in information systems from the University of Belgrade, in 2011 and 2013, respectively, where she is currently pursuing the Ph.D. degree in information systems. Since 2011, she has been engaged as a Teaching Assistant at the Faculty of Organizational Sciences, Department of information systems, University of Belgrade. Her research interests include software development and database design. Recently, her research is significantly focused on integration and interoperability of enterprise services and applications, specifically on investigating mechanisms that could ensure complete semantic interoperability. VOLUME 8, 2020