Blockchain-Based Software Effort Estimation: An Empirical Study

Context: The success or failure of any software development project significantly depends on the accuracy of its effort estimates. Software development effort estimation is the foundation for project bidding, budgeting, planning, and cost control. Problem: The literature shows that a lot of work has been done on software effort estimation. But still, there is a need for improvement in effort estimation by introducing new methodologies. The structured group-based and analogy-based effort estimations are the widely used estimation methods. Nevertheless, there are several shortcomings of using these methods such as lack of experts, lack of historical data, and biasness in expert opinion, which negatively affect the estimation results. Motivation: With the advancement of technologies, such limitations could be overcome. Such as leveraging the applicability of blockchain in several domains such as improvement in the software development process and network security. Method: In this article, we have proposed a Blockchain-Based Software Effort Estimation (BBSEE) methodology to improve the software effort estimation. We employ the proposed method using Web and blockchain technologies. Moreover, we also proposed evaluation criteria to assess the efficacy of the proposed method in terms of Mean Magnitude of Relative Error (MMRE), Mean Absolute Error (MAE) and percentage of successful predictions falling (PRED (25)). Result: We performed several case studies and analyses of expert opinions of 52 organizations to present the efficacy of the proposed method. Conclusion: We observe that the BBSEE method outperforms expert judgment and analogy-based effort estimation methodologies in terms of software effort estimation.


I. INTRODUCTION
The software development effort estimation is the measure of most likely effort [1]. Effort estimation is a crucial software management activity and provides a foundation for project planning, budgeting, bidding, and cost control [2]. Although, a lot of work is done on software effort The associate editor coordinating the review of this manuscript and approving it for publication was Porfirio Tramontana . estimation [3], however, literature shows a gap of improvement in the effort estimation methodologies, especially in the context of agile software development where requirement changes frequently [4], [5]. Moreover, the success or failure of any software project significantly depends on the accuracy of its effort estimation as indicated by the analysis of the Standish group [6]. In the era 2011-2015, the analysts have determined the success and the failure rate of more than 25000 projects in the era of 2011-2015. According to their analysis, 27-31% of the projects were successful (on budget, on time, acceptable), 17-22% of the projects failed (either canceled or rejected on delivery), and 4956% of the projects were highly challenged (over budget and schedule but rated satisfactory). Moreover, the annual average budget overrun was 68-72% and the annual average schedule overrun was 66-81% [6]. The level of overrun in budget and schedule arises a need for improvement in existing software effort estimation methodologies. The common estimation methods used can be seen in the decision tree shown in Figure 1, amongst the methods Expert judgment is the most used approach [1]. The layout of existing methodologies is shown in Figure 1, which indicates that structured group-based effort estimation methods are widely used [7]. Although, there are several benefits of using group-based effort estimation techniques reported. However, there are numerous shortcomings of their use also reported such as lack of experts and biasness in expert opinion [8], [9]. Moreover, analogy-based methods are the other most common effort estimation tool used by industry experts [7]. However, it also has some limitations which include a lack of historical data, especially in small organizations [8]. Even though the research community is very active and has proposed many estimation methods, the results are not substantial as observed in the analysis of the Standish group [6]. Moreover, finding domain-specific expert opinions and project-relevant data for precise estimation remains always a dream of a project manager. The availability of experts and reliable historical data always lead to the success of a project estimate. Subsequently, the current situation encouraged us to propose a method that would overcome the above-discussed challenges and ultimately improve the effort estimation results. The revolution in the advancement of technology is taking place very rapidly and opening new directions to solve different problems. Besides, the researchers are strongly motivated to adopt advanced technologies to solve the trending effort estimation problem, such VOLUME 10, 2022 as Machine Learning (ML) based estimation methods introduced to improve the accuracy of software effort estimation to encounter the shortcomings of existing techniques [10].
From the list of recent technological techniques, Software Engineering (SE) and Computer Network (CN) community has started to leverage the capabilities of blockchain to improve its performance. Blockchain is a distributed and decentralized ledger of transactions, it has also been acknowledged as a digital currency platform after introducing bitcoin the largest cryptocurrency [11]. The applications of blockchain are gradually increasing and a lot of research has been carried out. In the past few years blockchain was introduced to the domain of computer science to solve several problems [12], such as the implications of blockchain to improve the software process [36]. Recently, the blockchain was also introduced to the domain of the SE research community and software engineers are focusing on blockchain implications to resolve certain software development issues [13], [14]. This gives us a new direction to solve the software effort estimation inaccuracy problem using blockchain. In our opinion, the use of a blockchain-based method, that integrates the benefits of Group and analogybased estimation can overcome their limitations that are why we proposed a blockchain-based solution. The main contributions of the proposed BBSEE can be summarized as follows.
• The proposed BBSEE overcomes the problem of a lack of experts by introducing a remote network that encapsulates a verifiable community of estimation experts that are always available over the network.
• The proposed BBSEE overcomes the limitation of biasness in the physical existing group by introducing a remote network that encapsulates a verifiable community of estimation experts, whose decision is not affected by any type of biasness.
• The proposed BBSEE overcomes the problem of lack of historical data by introducing a blockchain-based repository, where historical data is securely stored and publicly available to be used over the network.
The rest of this article is structured as follows. Section II defines the related studies and a summary of findings. The commonly used estimation methods were described in Section III. In Section IV, the proposed methodology is presented. In Section V, the research design is defined. Section VI presents the results and discussion of this study. Furthermore, the threats to validity are presented in Section VII. In Section VIII, the conclusion and future work of the proposed research are defined.

II. RELATED WORK
The success rate of software projects is challenging, from the analysis of the Standish group, we came across the fact that the success of any software project is significantly dependent on the accuracy of its effort estimates. Several studies have been examined and contributed to the improvement of effort estimation, but the level of improvement was not satisfactory [6]. Most of the work has been done on effort estimation which focused on proposing and improving the traditional models. One of these models is the COCOMO model which is introduced by Barry Boehm. The COCOMO model is continuously improved with time and gained more attention from the SE research community [15], [16], Similarly, Function Pointbased analysis was introduced by Albrecht to measure the size of the functionality of systems [17]. Similarly, an analogybased generic method is introduced for effort estimation, whose main idea was that whenever a new project is started, the historical data of an existing similar project is viewed and used while making estimation decisions [18]. Similarly, many other traditional methods were proposed and improved continuously, but there was no better effort estimation method [9], [19].
As agile software development is achieving popularity gradually and replacing traditional methods of software development [20]. Different studies have acknowledged that the accuracy of software effort estimation is more crucial in the context of agile software development [4], [5]. For instance, technology is evolving with each elapsing day and exploring new dimensions to solve different problems, due to which the research community is keenly interested to adopt advanced technologies to solve the problem of inaccuracy in effort estimation, Furthermore, the literature review indicates that several methods have been proposed [19], such as Ontology Oriented Software Effort Estimation (OOSEE) was introduced for E-commerce applications based on Extreme programming, and Scrum methodologies and an Ontology-based system for the agile method was also introduced. The common aim of these techniques was to improve the accuracy of effort estimation [21], [22]. Similarly, certain machine learning-based effort estimation methods are also proposed. The study of Polkowski et al. [23] states that machine learning-based effort estimation increases estimation accuracy [23]. Systematic Literature Review (SLR) compared different machine learning-based effort estimation methods so that one might select appropriate and effective effort estimation methods for the desired context [24]. Although, Machine Learning (ML) based effort estimation methods are comprehensive and acknowledged extensively. However, there are some limitations to using them, as indicated by Shukla et al. The ML-based methods do not work well on all datasets. Moreover, this paper recommends exploring the Neural Networks-based estimation method to improve estimation accuracy [10].
The use of Neural Networks in estimation is a wellpracticed approach as indicated by Nassif et al [25]. In this article, the authors have compared different Neural Networks-based methods [25] and reported the outperformed method. Similarly, in research, it is specified that the creative and abstract nature of software makes it difficult to measure its estimation accuracy and the use of neural networks can help in the improvement of estimation accuracy [26].
Similarly, the use of Neural Networks also encounters some shortcomings [19]. The emerging situation needs a perfect matching technology that can improve the estimation results. Finally, SE researchers have proposed several approaches which are exceptionally capable to manage the uncertainty as effort, duration, and velocity prediction [3].
Blockchain applications are increasing gradually which is absorbed in a paper by Porru et al. [13]. In this article, the authors have acknowledged that there is a need for blockchain-based software development [13]. Similarly, in a concept paper by Beller et al. [14], the authors published and formulated the term ''Blockchain-based Software Engineering'' in which they introduced the concept of solving software engineering problems using Blockchain.
The divergence of the research community toward the use of advanced technologies has motivated us to solve the software effort estimation inaccuracy problem using Blockchain. To the best of our knowledge, no work is previously done on Blockchain-based software effort estimation.
According to a survey [7], the most frequently used estimation methods such as structured group and analogy-based effort estimation are reported. But these methods do not give significant results due to their above-discussed limitations. The shortcomings of group-based effort estimation include lack of experts and biasness in expert opinion [8], [9], Furthermore, in analogy-based estimation, the accuracy of estimates significantly depends on the availability of historical data [8]. This study proposed a blockchain-based solution that will overcome the above-mentioned limitations and improve the estimation results.

III. ESTIMATION BY ANALOGY AND EXPERTS GROUP
There are different estimation methods available (as shown in Figure 1), from which you can select and use the most suitable one, to get better results. From Figure 1, we can observe that there are three options available to perform effort estimation, such as Expert estimation, Formal models, or a combination of both. Most of the time, combined approaches are used to get better results [27]. We have further selected two methods based on their popularity, which are Expert group and Analogy-based estimation, and preferred to use them in combination.

A. GROUP ESTIMATION
The usage of combining effort estimates on average leads to more accurate estimates, especially in the case where it is not clear which method is better or who has the relevant experience, the essential technique is that estimation should be done independently from each other and takes benefit from their combine usage [9]. There are two ways to arrange a group, one of them is known as a structured group and the other is known as an unstructured group. In effort estimation, the use of an unstructured group is not appreciated by the experts. The experts recommend using a structured group arrangement while estimating, there are different techniques based on the structured group. The most common techniques include Planning poker and Wide-band Delphi, the high-level working of wide-band Delphi is given as [28].

1) GETTING READY FOR THE ESTIMATION PROCESS
• Develop estimation-related supporting material • Finalize estimation team and their moderator

2) START THE ESTIMATION MEETING
• Moderator briefs the team about the estimation problem, estimation stuff, estimation process, etc.
• Team discussion of the members of the team and supporting material

3) ESTIMATION BY EACH TEAM MEMBER INDIVIDUALLY
• Identification of activities and estimates • Experts that are not part of the estimating team can be consulted

4) ESTIMATION MEETING
• Moderator briefs the team about the estimates received • All Experts discuss the estimates

5) SUM UP THE ESTIMATION PHASE
• Done by moderator and project leader

B. ANALOGOUS ESTIMATION
The most common example of analogy-based project estimation is case-based reasoning, where identical projects from the lesson learned are identified and used for estimation. They can be used in combination with collective expert opinion and formal models [29], Software effort estimation by using an analogy-based tech commonly involves the following steps. 1) Measuring the values of identified metrics of the software project for which estimation is being performed (target project). 2) Finding a similar project from the repository. 3) Using the estimated effort values of the selected projects to use as an initial estimate for the target project. 4) Comparison of metric's value for the target project and selected project. 5) Adjustment of effort estimates in view of the comparison performed in the previous step.

IV. PROPOSED BBSEE METHOD A. BBSEE OVERVIEW
The proposed method BBSEE is a concept of the freelance effort estimation market. It overcomes the limitations of group-based estimation and analogy-based estimation by using the blockchain concept. The BBSEE helps organizations lacking the estimation abilities to estimate and improve their estimated results.

B. BBSEE PROCESS FLOW
Our proposed Blockchain-Based Software Effort Estimation (BBSEE) method comprises four layers, namely the VOLUME 10, 2022 Estimation seeker layer, Graphical User Interface layer (GUI), Blockchain layer, and Experts network layer. The proposed BBSEE method is shown in Figure 2, which contains a network that encapsulates a verifiable community of estimation experts that carry out effort estimation for organizations and individuals. The effort estimation with the proposed BBSEE requires the estimation seeker from the estimation seeker layer to send an estimation request to the GUI layer, which will send back the required template for the requirement and specifications of the system. The seeker is supposed to write requirements and specifications in the defined template and send it along with the transaction request to the GUI layer. Since the fixed number of activities are performed in both layers. So, the runtime complexity of both layers could be computed as O(c). The template of the block is used to group the information into two sections, namely header and transaction sections. The header section of the block contains information about hash reference with another block and timestamp. While the transaction section contains information related to requirements and specifications of cost estimations retrieved from the experts. Afterward, the transaction is forwarded to the blockchain layer and added to the mempool (waiting queue for transactions), and upon its turn estimation request broadcasts to the network of available experts on the expert's network layer, from which the interested experts accept the request. Due to the involvement of n transactions, the runtime complexity of the blockchain layer will remain O(n). Subsequently, it is added to a network of selected experts that will carry out estimation decisions for that project, from the list of selected experts, one of the experts is selected as a facilitator on the base of their experience. Due to the involvement of the expert network, the complexity of the ''Expert Network'' layer is O(V * E), where V referred to the list of experts while E referred to the values to describe the relationship between experts.
Afterward according to requirements and specifications each expert individually divides the software system into modules and estimates the effort required for each module using historical data available over the chain of the Blockchain layer. Each expert reveals its estimates, a joint consensus is done by all the experts, where the facilitator helps them in reaching the final estimate. The facilitator forwards the final estimates to the blockchain layer. Subsequently, the final estimates are then added to the chain for future use. Moreover, the estimates are sent back to the GUI layer which passes them to the Estimation seeker layer where the seeker receives the estimates. The final estimates are used by the estimation seeker, who continuously checks for estimation accuracy on the base of which decision is made, whether to use the estimates or re-estimate again for improvement in estimates accuracy. The detailed architecture of the proposed BBSEE method is shown in Figure 2. The overall complexity of BBSEE is computed as follows.
The combination of different effort estimation methods helps to achieve the benefits of multiple estimation methodologies. Our proposed system overcomes the challenges such as lack of experts, biasness in expert opinion, and lack of historical data by combining the benefits of group-based and analogy-based effort estimation using blockchain. As a result, this approach would help to improve the estimation accuracy.

V. RESEARCH DESIGN
We have selected the software effort estimation domain and performed an exploratory study to identify the most used practices and their limitations. The results of the exploratory study identified that Analogy and Group-based estimation methods are the most used estimation practices. We have explored a solution that can overcome their limitations, namely Blockchain-Based Software Engineering (BBSE), which gives us an idea to propose a new method. We have proposed the Blockchain-Based Software Effort Estimation (BBSEE) methodology, which overcomes the limitations of group and analogy-based effort estimation. For the empirical evaluation of the BBSEE, we have simulated all its possible cases and developed a working BBSEE system. Afterward, two case studies were performed to evaluate current practices and collect feedback about the BBSEE. Similarly, the performance of the proposed BBSEE is also evaluated using two experiments in which the performance measures MMRE, MAE, and PRED (25) are used. The results of the case studies and experiments were collected and analyzed to prove our claims; the overview of the research can also be seen in Figure 3.

A. BBSEE SIMULATION
All the evaluations required in this research (Case study I, Case study II and performance evaluation) need the availability of a working BBSEE system, this brings us to develop a Decentralized Application (dApp), which covers all the workflows of the proposed methodology as shown in Figure 2. The proposed system was developed using solidity, JavaScript, PHP, MySQL, and HTML. and CSS. The system was deployed using 000webhost, and the firebase and Rinkebey testnet were used to automate the file storage, retrieval, and execution of smart contracts, and to make sure that the file is secure, decentralized, and publically accessible to make the system work as intended.

B. CASE STUDY I 1) PARTICIPANTS
We invited over 120 software organizations to participate in the activity from which only 76 organizations express interest to participate, amongst which only 52 organization representatives participated in the activity. All the representatives have their academic background in computer science and have professional experience in software development, some of the participants also had blockchain backgrounds and most of the representatives were serving their organizations in senior positions.

2) PROCEDURE
This study aims to analyze the current situation of organizations. Our main aim was to evaluate the ratio of the organizations performing effort estimation activity. The ratio of the organizations conforming to the standard effort estimation methodology and identifying the most frequently used estimation methods and their major barriers that prevent organizations from performing estimation activities. We evaluated this study by performing a short presentation session. We have presented a brief overview of the existing effort estimation methodology and asked the participants to answer a set of questions regarding the current practices of their organizations.

C. CASE STUDY II 1) PARTICIPANTS
The participants of case study II were the same 52 representatives of the organization who participated in case study I.

2) PROCEDURE
The second case study is performed to evaluate and validate the proposed BBSEE. The major goals of the proposed system are to overcome the above-identified limitations and to make the estimation activities efficient and convenient for all organizations. We perform this study by providing a brief overview of the traditional methods (along with their shortcomings), and the benefits of the proposed BBSEE method were presented using a storyboard. To make the audience familiar with our proposed system, we have presented a demonstration of the use of the proposed system. We elaborated on the above-mentioned limitations and requested the audience to use the proposed BBSEE method and give their feedback about it.

D. PERFORMANCE EVALUATION
Prediction accuracy is an important part of the estimation method; we have performed two experiments, in which we have selected 6 experts who were currently working in the software industry with 3-6 years of professional experience. In the first experiment, the experts use structured group-based estimation (Wide-band Delphi) for performing estimation activities. In the second experiment, the experts use the proposed BBSEE method for performing estimation activities.

1) EXPERIMENT
From the list of 81 projects, we have selected 15 projects for estimation using Wide-band Delphi, selected projects can be viewed in Table 11.
We have selected 15 projects from the list of 81 projects and instructed our experts to use the Wide-band Delphi technique to estimate all the 15 selected projects one by one and monitored them while they were performing estimation activities. The results of the estimation were used for evaluating the performance of the Wide-band Delphi.

2) EXPERIMENT II
We have selected other 15 projects from the dataset of 81 projects and used the remaining 66 projects as historical data and introduced the selected experts to our proposed BBSEE method. Afterward, we performed a controlled experiment in which the selected experts were asked to estimate the target 15 projects using the proposed BBSEE method. The results of the experiment were used to evaluate the performance of BBSEE.

3) EVALUATION MEASURES
The commonly used evaluation criteria for effort estimation methods are the Magnitude of Relative Error (MRE) [30], which is represented in equation 1.

MRE = Actual Effort − Estimated Effort Actual Effort
The MRE is calculated for all selected N projects, and the Mean Magnitude of Relative Error (MMRE) is the average of N projects [31], as defined in equation 2.
The success percentage of predictions laying within 25% of the actual values is also represented as PRED (25) and defined in equation 3, where N is the number of selected projects [32].
The Mean Absolute Error (MAE) is the average magnitude of error in a set of predictions as defined in equation 4, where y i is prediction, x i is the actual value and n is the total number of points.

4) DATASET
The dataset used in experiments I & II comes from the Desharnais promise repository dataset [33]. We have collected data on 81 projects. The detailed statistics for the dataset can be viewed using the promise repository.

VI. RESULTS AND DISCUSSION
This section reports and evaluates the results of two case studies in which the representatives of 52 organizations participated. This section also reports the results of two experiments in which effort estimation for the selected project was performed using BBSEE and Wide-band Delphi to evaluate their performance.

A. CASE STUDY I
The first case study aimed to analyze the current practices of organizations. Our main aims of this case study include evaluating the current practices, extracting demographic information, and identifying the barriers preventing organizations from performing estimation activities. We evaluated the study by performing a short presentation session. In this section, we have considered a few hypotheses to understand the developer's opinion about cost estimation practices in terms of their self and their organization perspectives.

1) HYPOTHESIS-1
Software firms are performing some sort of software effort estimation practices. According to the response of the experts' 63.5% of the organizations perform estimation and somehow 36.5% of the organization doesn't perform the estimation. The results imply that the majority of the organizations perform effort estimation as shown in Table 1.

2) HYPOTHESIS-2
Software firms are using well-known methodologies for software effort estimation. The results of Table 2 indicate that 81.8% of organizations were conforming to standard estimation methodologies, whereas 18.2% of organizations don't conform to any effort estimation methodology..

3) HYPOTHESIS-3
Software firms are more inclined toward the Group-based estimation process.
The results of the Table 3 indicate that most organization is following a group-based estimation rather than following some standards such as COCOMO Model.
Hence, it can be concluded that the most frequently used effort estimation methods are group-based and analogy-based effort estimation methods. Software firms are facing barriers to performing effort estimation activities. The results of Table 4 indicate that there are several barriers for the organization to perform precise effort estimation such as Lack of Experts, Time, group biasness, and lack of historical data. Moreover, we can observe that lack of experts and historical data are the major barriers to an organization. In the case study, I was aiming to extract the current practices of participating organizations, the results of the study favor our claims that the most frequently used estimation methods were group-based and analogy-based estimation. The results of the study also favor our claim that lack of experts, lack of historical data, and biasness in the group were the major barriers preventing organizations from performing estimation activities. The results of the study also identified that the majority of organizations were performing effort estimation and following different techniques for estimation.

B. CASE STUDY II
The second case study is performed to assess the efficacy of the proposed BBSEE. We performed this case study with experts who have experience with BBSEE and collected their feedback to assess our hypotheses, such as.

1) HYPOTHESIS-5
BBSEE could aid organizations to overcome the problem of lack of experts.
The result of Table 5 indicates that 77% of expert agrees with to use of the BBSEE method to improve the effort estimation in case of a lack of experts in the organization. Moreover, only 5% are not agreeing to the use of BBSEE when organizations don't have experts in effort estimations. 2) HYPOTHESIS-6 BBSEE could aid organizations to overcome the problem of the expert's biasness.
The result of Table 6 indicates that 73.1% of experts agree with to use of the BBSEE method to improve the effort estimation in terms of overcoming the expert biasness. Moreover, only 1% is not agreeing with the use of BBSEE to overcome the expert biasness.

3) HYPOTHESIS-7
BBSEE could aid organizations to overcome the problem of lack of experts.
The result of Table 7 indicates that 70.1% of expert agrees with to use of the BBSEE method to improve the effort estimation in case of a lack of historical data. Moreover, only 9.6% are not agreeing with the use of BBSEE when an organization doesn't have historical data for effort estimations.

4) HYPOTHESIS-8
BBSEE freelance market could affect the overall cost of effort estimation.
The result of Table 8 indicates that 52.9% of experts agree with the significant decrease in the cost of effort estimation by incorporating the use of the BBSEE freelance market. Moreover, only 28.9% do not agree with the decrease in the cost of effort estimation using the BBSEE freelance market.

5) HYPOTHESIS-9
BBSEE freelance market could affect the overall duration of effort estimation.
The result of Table 9 indicates that 51.9% of expert agrees with the significant decrease in duration of effort estimation by incorporating the use of the BBSEE freelance market. Moreover, only 28.8% do not agree with the decrease in duration of effort estimation using the BBSEE freelance market.

6) HYPOTHESIS-10
BBSEE method can perform effort estimation activities more efficiently as compared to traditional methods.
The result of Table 10 indicates that 80.8% of expert agrees that the BBSEE method can perform effort estimation activities more effectively as compared to traditional methods. Moreover, only 19.2% do not agree that effort estimation activities are not performed efficiently as compared to the existing methods. Case study II aimed to assess the proposed BBSEE and validate it from the industry, and the results of the study are in favor of our assumptions about the proposed BBSEE. Moreover, the results support our claim that the use of BBSEE can overcome the problems of lack of expertise, lack of historical data, and biasness in group decisions. We can also imply from the results that the use of the proposed BBSEE will decrease the cost and time required for estimation activities. The results also imply that the proposed BBSEE method could efficiently perform estimation activities as compared to existing traditional models.

C. PERFORMANCE EVALUATION 1) EXPERIMENT I
The expert performed the estimation of the selected 15 projects using the Wide-band Delphi technique, the estimation and performance evaluation results can be viewed in Table 11.
The performance evaluation results of the experiment I report that MMRE for Wide-band Delphi is 0.256, MAE is 1049, and PRED (25) is 60%. This implies that the performance of the Wide-band Delphi technique is not satisfactory. The software estimation method is considered satisfactory if its MMRE is less than 0.25 and PRED (25) greater than 75% [34].
We implied that the performance of the Wide-band Delphi is not satisfactory due to the shortcoming of group-based estimation. As discussed earlier the lack of relevant experts and biasness in the group affect the estimation results negatively, additionally the availability of historical data and lack of historical data while making estimation decisions can also improve the estimation results.

2) EXPERIMENT II
The experts performed the estimation of the other 15 selected projects out of 81 by using the proposed BBSEE. where 66 remaining projects were used as historical data, the estimation and the performance evaluation results can be viewed in Table 12.

3) COMPARISON
The difference between the actual and estimated effort for experiment 1 by using wide-band Delphi is shown in Figure 4. The difference between the actual and estimated effort of VOLUME 10, 2022  experiment 2, where the estimation is done by using the BBSEE method is shown in Figure 5.
The main consequences of the results of Figure 4 and Figure 5 are as follows.
• The difference between actual and estimated values of BBSEE is lower than the difference estimated through wide-band Delphi. We observed the lowest value of MMRE (i.e., 0.17) for BBSEE while we observe the value of MMRE as 0.25 for wide-band Delphi. The detailed result is shown in Figure 6.
• We observed the highest value of PRED(25) (i.e. 0.80) for BBSEE while we observe the value of MMRE as 60^for wide-band Delphi. The detailed result is shown in Figure 7.
• We observed the highest value of MAE (i.e., 1049) for BBSEE while we observe the value of MMRE   as 596^for wide-band Delphi. The detailed result is shown in Figure 8.

4) HOW BBSEE OUTPERFORM
The Delphi technique has some limitations which negatively affect the estimated results such as the inclusion of expert opinion during the estimation process. This limitation is overcome by using the proposed BBSEE method, where the availability of domain experts is certain using the blockchain, and by introducing a freelance effort estimation market where experts are always available. Similarly, the biasness in a physically existing group also negatively affects the results of the Delphi technique. The problem of biasness in the group is overcome by BBSEE by introducing remote groups. Finally, the problem of lack of historical data in analogy-based estimation is overcome by BBSEE by introducing a blockchain-based repository, where historical data of all the previous estimations is maintained on the chain for future use. The combined benefits of group and analogybased estimation improve the estimation results due to which BBSEE outperforms.

5) CONCLUSIVE DISCUSSION
The experimental validation results implied that the proposed BBSEE has overcome the limitation of lack of experts by introducing a freelance estimation market concept, where relevant experts are always available. Similarly, the limitations such as biasness in a group and lack of historical data have been overcome by maintaining a repository on the chain. Finally, the results also imply that the BBSEE has a positive impact on the time and cost required for estimation. The performance evaluation results implied that the proposed BBSEE outperformed as compared to the Delphi technique.

VII. THREATS TO VALIDITY
Empirical approaches are practiced commonly in software engineering, but there is no agreement on how to practice them properly [35]. In this research, the performance of the proposed BBSEE method may vary due to different factors, as the results of this study were dependent on the participating organizations, the estimation experts, evaluation measures, and the selected dataset, some of the threats are described below. The experts and facilitator perform a very crucial role in the proposed BBSEE Similarly, its performance may vary depending on the level of experience of the experts and facilitator. It also depends on the number of experts involved in the estimation process. The applied dataset also performs a very central role in the performance assessment of the proposed system. The results may also vary with the change of dataset or change in the size of the dataset. The participating organizations also have a significant impact on the results of this research, which is another threat to our study. As we have involved 52 organizations in the case studies, the presented results may vary with an increase or decrease in the number of participants or an increase or decrease in the size of participating organizations. The performance measures used to perform a very central role in estimation results. Therefore, by changing the performance measure the efficiency of the proposed system may vary.
The most difficult decision was about the validation of the proposed model for which there was no better choice than developing a working model of the proposed method. The second difficult decision is of validating the proposed method, so we have selected those organizations which are considered as actively working software industries at the time when the proposed study is conducted.

VIII. CONCLUSION AND FUTURE WORK
The software development effort estimation is one of the crucial software management activities. Due to the importance of software effort estimation, a lot of methods and models are proposed, but there is no best effort estimation method or model is reported. The commonly used estimation methods are classified into a group and analogy-based estimation approaches. However, both have some shortcomings such as historical data, lack of experts, and biasness in a group, which could affect the estimated results. Currently, the research communities are appreciating the use of advanced technologies to improve the estimation results. Due to the emerging situation, this research introduces a Blockchain-Based Software Effort Estimation (BBSEE) method to overcome the shortcomings of the group and analogy-based estimation, and ultimately improve the estimation results.
We have simulated the proposed BBSEE and conducted two case studies using the experts of 52 organizations. The results of the first case study support our assumption that lack of historical data, lack of experts, and biasness in a group are the major barriers and prevent the organizations to perform estimation activities more effectively. The results of the second case study support our assumption that the use of BBSEE could help the project manager overcomes the reported problems such as lack of expertise, biasness in the group members, lack of historical data in the group, and analogy-based estimation. The findings also support the assumption that the proposed BBSEE performs estimation activities more efficiently as compared to traditional methods. The findings of the study also reported that most of the participants voted in favor. Moreover, the proposed BBSEE will decrease the time and cost required for estimation activities.
The Performance of the proposed BBSEE is evaluated by using the Mean Magnitude of Relative Error (MMRE), Mean Absolute Error (MAE), and percentage of successful predictions falling (PRED (25)), we have performed two experiments using the same six experts and using a dataset of 81 projects. In the first experiment, we selected 15 projects and asked experts to use the wide-band Delphi technique to estimate and evaluated the performance. The experimental results in terms of MMRE, MAE, and PRED (25) indicate the efficacy of BBSEE as compared to wide-band Delphi.
In future work, we will enhance our proposed BBSEE methodology by integrating ML-based effort estimation approaches. These future enhancements will run ML-based effort estimation in parallel with the proposed BBSEE method, and the results of ML-based estimation will be used by the experts while making the decisions. Future enhancements are expected to improve the estimation results.
NAEEM IQBAL (Member, IEEE) received the B.S. degree in computer science from COMSATS University Islamabad, Attock Campus, Punjab, Pakistan, and the M.S. degree in computer science from COMSATS University Islamabad, Attock Campus, in 2019. He is currently pursuing the Ph.D. degree with the Department of Computer Engineering, Jeju National University, Republic of Korea. He has professional experience in the software development industry and academics as well. He has published more than 25 papers in peer-reviewed international journals and conferences. He is serving as a professional reviewer for various well-reputed journals and conferences. His research interests include AI-based intelligent systems, data science, big data analytics, machine learning, deep learning, analysis of optimization algorithms, the IoT, and blockchain-based secured applications.
FARAZ HUSSAIN received the B.S. degree in software engineering from International Islamic University, Islamabad, Pakistan, in 2017. He is currently pursuing the M.S. degree in software engineering with COMSATS University, Islamabad. His research interests include solving trending software engineering problems using advanced technologies, such as blockchain and machine learning.
MURAD-ALI KHAN received the B.S. degree in computer science from COMSATS University Islamabad, Attock Campus, Punjab, Pakistan, in 2020. He is currently pursuing the integrated Ph.D. degree with the Department of Computer Engineering, Jeju National University, Republic of Korea. He has professional experience in the software development industry. His research interests include ML and data mining related applications.
MARKUS HELFERT is currently the Director of the Innovation Value Institute Research Center and a Professor in digital service innovation at Maynooth University. His research interests include PayTech, information management, innovation, cloud computing, service science, enterprise architecture and smart cities, and smart buildings. He collaborates with a range of enterprises in relation to innovation and IT management. He is also the Project Coordinator of the H2020 Projects: PERFORM on Digital Retail and is the Chair of Steering Committee Elite-S H2020 Marie Curie Project on 'Future Leaders in ICT Standardization for Europe. He has authored more than 200 academic articles, journals, and book contributions, and has presented his work at international conferences. He is a member of the International Program Committees and a Reviewer and an Associate Editor of IS Journals. He is currently an Associate Professor with the Department of Biomedical Engineering, Gachon University, South Korea. His research interests include integrated circuits and system designs for nanopore sequencing technology and other biomedical applications. VOLUME 10, 2022