A Novel Weapon System Effectiveness Assessment Method Based on the Interval-Valued Evidential Reasoning Algorithm and the Analytical Hierarchy Process

As one of the most essential parts of the military operation, the ability of weapon systems to achieve the operation mission is vital to the success of a military operation. Focusing on the problem of weapon system effectiveness assessment (WSEA) under interval uncertainty, this paper proposed WSEA method that utilized the interval-valued evidential reasoning (ER) algorithm, the analytical hierarchy process (AHP), and the two-grade interval ranking method to properly deal with interval uncertainty in the assessment as well as provide reliable assessment results. Firstly, the AHP is used to determine the weight of different attributes of the weapon system based on experts’ knowledge, which could enhance the subjectiveness of the assessment result. Then, the interval-valued ER algorithm is applied to combine the assessment of different attributes under interval uncertainty and provide the overall assessment result. Finally, the two-grade interval ranking method is utilized to compare and rank the assessment results of different weapons. Case study shows that the proposed method could provide reliable and accurate results for weapon effectiveness assessment. Comparison results and sensitivity analysis results further confirm that the proposed method is not only as effective as existing methods with precise data, but also has the ability to provide reliable results under uncertainty. In conclusion, by utilizing the interval-valued ER algorithm, the AHP and the two-grade interval ranking method, the proposed method provides a novel and effective way for weapon effectiveness assessment under uncertainty, especially interval uncertainty.


I. INTRODUCTION
Weapon system is one of the most essential parts of military operations, and its ability to achieve the operation mission is decisive to the success of a military operation [1]- [5]. However, how to effectively and properly evaluate the ability of weapon systems remains an open and urgent issue. Hence, weapon system effectiveness assessment (WSEA), which is to sum the various elements of the weapon system against the operation goal, is of significant importance to weapon system development, design, and military planning.
The associate editor coordinating the review of this manuscript and approving it for publication was Long Wang . Generally, the WSEA problem has been regarded as a multi-attribute decision making (MADM) problem [6]- [10]. Among the popular MADM methods such as analytic hierarchy process (AHP) [11]- [16], technique for order performance by similarity to ideal solution (TOPSIS) [17]- [23] and evidential reasoning (ER) approach [24]- [31], many have been applied to this problem. Mon et al. [11] firstly adopted the FAHP based on entropy weight to weapon evaluation and conducted missile systems selection. Cheng et al. [32] developed a weapon system evaluation method based on AHP using linguistic variable weight, and used the proposed method to evaluate attack helicopters. Cheng et al. [33] used the parallel experiments approach for weapon system of systems effectiveness analysis, and applied the proposed method for evaluating missile defense system. Ding et al. [34] developed a method of machine learning and visualization to analyze the capabilities of different weapon system of systems in a two-dimensional plane.
However, despite these advances, there still exists some challenges in the WSEA problem, and the most significant one is the handling of uncertainty [35]- [38]. Since the WSEA may be conducted not only when the weapon system already being deployed, but also when it is still under development, it is generally difficult to obtain detailed and precise information of the weapon system. In fact, in many cases, the information of weapon system is either based on the judgments of experts, or based on documents, which could lead to different kinds of uncertain information, such as qualitative judgments and interval data. However, this kind of uncertainty has been largely ignored by current studies, and how to properly model these uncertain information in the WSEA problem remains an open issue.
To this end, focusing on the problem of WSEA under uncertainty, this paper proposed a novel weapon system effectiveness assessment method to deal with uncertain information in weapon systems, where the interval-valued evidential reasoning (ER) algorithm, the analytical hierarchical process (AHP) and the two-grade interval ranking method are utilized to deal with different kinds of uncertainties, especially interval uncertainty. Firstly, the weights of different attributes of the weapon system are determined using AHP. Then, the information of weapon system under uncertainty is combined using the interval-valued ER algorithm while taking the their weights into account. Then, the two-grade interval ranking method is applied to rank the assessment results and provide the rank of different weapon systems. Finally, a case study on missile system effectiveness assessment is conducted, which shows the effectiveness of the proposed method. Sensitivity analysis further shows that the proposed method could provide reliable assessment result with or without uncertain information. It can be concluded that the proposed method could provide reliable and accurate result for weapon system effectiveness assessment, even under uncertainty.
The remainder of this paper is organized as follows: Section 2 provides some basic information for the proposed method, and the proposed method is detailed in Section 3. In Section 4, a case study on missile system effectiveness assessment is conducted to demonstrate the feasibility of the proposed method. Section 5 concludes the paper.

II. PRELIMINARIES
Evidence theory is one of the most effective tool to deal with uncertainty, and it has been widely used in fields such as classification, clustering, and many others. One of the most basic and important concepts in evidence theory is the frame of discernment, which is a set of mutually exclusive and collective exhaustive elements, denoted as Θ = {θ 1 , θ 2 , . . . , θ N }. The power set defined on Θ, denoted as 2 Θ , consists of 2 N of subsets, and is represented as follows: In 2 , a basic belief assignment (BBA), denoted by m : 2 Θ → [0, 1], is defined a mapping of the power set to a number between 0 and 1, and it satisfies [39]: where the BBA assigned to 2 Θ is defined as the degree of global ignorance, and the BBA assigned to the smaller subset of Θ except for any singleton proposition or Θ is referred to as the degree of local ignorance. When m(A) > 0, A is called a focal element of Θ, and the set of all focal elements is called the core of a BBA.
In evidence theory, m(A) measures how strongly the evidence supports A, and the belief measure Bel and plausibility measure Pl represent the lower and upper bounds of the degree of support for each proposition in the BBA, and Bel and Pl are defined as: [Bel(A), Pl(A)] is defined as the belief interval of A. Two independent BBAs can be combined by using the Dempster's rule of combination. Suppose the two BBAs are denoted as m 1 and m 2 , they can be combined as [40]: where K is the conflicting factor.

A. EVIDENTIAL REASONING ALGORITHM
Proposed by Yang and Singh [40], the evidential reasoning (ER) algorithm has been widely used in the field of decisionmaking and evaluation for its advantages in modeling both quantitative and qualitative information under uncertainty using the same framework. Based on evidence theory, the ER algorithm uses a modified evidence combination rule to deal with highly conflicted evidences, and is shown to be effective in many cases.
In the ER algorithm, the hypotheses are normally singletons, and the belief distribution is used to measure the extents to which the evidence supports each hypothesis, which is also known as a piece of evidence. Hence, a piece of evidence e p can be represented by a belief distribution on the power set of the frame of discernment Θ as follows: e p = {(θ n , β n,p ), n = 1, . . . , N }, N n=1 β n,p ≤ 1 (6) where θ n denotes the nth singleton of Θ, and β n,p is the corresponding belief degree. (θ n , β n,p ) is called a focal element of e p and represents that the evidence supports proposition θ i to a degree of β n,p . In the definition of evidence, both global ignorance and local ignorance are taken into account. The belief distribution is called complete when N n=1 β n,p = 1, and is incomplete when N n=1 β n,p < 1. The ER algorithm also extends evidence theory by introducing evidence weight to reflect the relative importance of different evidences. Hence, a piece of evidence e p is characterized by two elements including the belief distribution (θ n , β n,p ) and weight ω p in the framework of the ER algorithm, and a weighted belief distribution is defined as follows [41]: where m i,p represents the degree of support for θ i from evidence e p while taking the weight ω p into consideration, i.e. the BBA, and is defined as: It should be noted that m Θ,p is the degree of residual support that reflects the uncertainty of evidence e p , and it satisfies m Θ,p = 1 − N n=1 m n,p . It can be divided into two parts: m Θ,p andm Θ,p , wherem Θ,p = 1 − ω p is caused by the relative importance of the evidence e p andm Θ,p = ω p (1 − θ ⊆Θ m θ,p ) is caused by the cognitive uncertainty in the information on e p , i.e. global ignorance and local ignorance.
Once the belief distributions are obtained, then can be combined to calculate the final results. Suppose there are M evidences to be combined, and the pth belief distribution is defined as: where β n,p is the belief degree to which the belief distribution m p is assigned to θ i . Suppose ω p is the weight of m p , then the weighted BBA is calculated as [42]: where m Θ,p is the probability mass unassigned to any individual consequent, and m Θ,p = m Θ,p + m Θ,p .
Then, the combined belief degree can be obtained as [41]: with where m n,I (p+1) is the combined belief degree after aggregating the first p belief distributions, and m Θ is the remaining unassigned belief degree. It is supposed that m n,I (1) = m n,1 and m Θ, Then the belief degree β n for the nth singleton θ n is: For ranking purposes, the expected utility can be calculated. Suppose the utility of the ith singleton θ n is denoted by u(θ n ), without loss of generality, there is u(θ n+1 ) > u(θ n ), then the expected utility can be obtained as follows:

B. ANALYTICAL HIERARCHY PROCESS
In this paper, the Analytical Hierarchy Process (AHP) is applied to determine the weight of the attributes of the weapon system to obtain more reliable and comprehensive results [43]. First, assume N decision elements are (E 1 , . . . , E N ), then pairwise comparison judgment matrix is defined as where each element in the matrix m ij represents the judgment concerning the relative importance of decision element E i over E j . Then, in order to varify the consistency of the matrix, the consistency ratio (CR) is defined as: where CI = λ max −N N −1 is the consistency index, RI is the random consistency index, and its value is related to the dimension of the matrix, shown in Table 1.
Normally, if CR < 0.1, the consistency of the pairwise matrix M is said to be acceptable and the eigenvector corresponding to λ max can be calculated. The eigenvector is calculated as follows [44]: where λ max is the largest eigenvalue of M N ×N . To obtain the final weights of the decision elements, the eigenvector can be normalized.

III. PROPOSED METHOD
In this section, a weapon system effectiveness assessment method based on the interval-valued ER algorithm, the AHP and the two-grade interval ranking method is developed, and the proposed method consists of three parts: (1) transform different kinds of information into belief distribution; (2) combine the interval-valued belief distribution for calculating the evaluation result; (3)

A. INFORMATION TRANSFORMATION
For the WSEA problem, it often involves different kinds of information under uncertainty, namely qualitative judgments, precise data and interval data, hence, it is necessary to collect these different information and transform them into belief distributions. For qualitative judgments, they can be easily transformed into belief distributions since themselves are expressed in the forms of grades, and thus can be simply viewed as belief distributions with total belief assigned to a certain grade. As for quantitative information, however, the utility-based transformation technique [45] should be applied. Generally, there are two different kinds of quantitative information in this case, namely, precise data and interval data.

1) PRECISE DATA TRANSFORMATION
In the WSEA problem, many characteristic data would be in the form of precise data, however, they may be of different metric, thus, it is necessary to transform these data under the same framework for evaluation. Suppose the utility value u n is equivalent to the grade θ n , without loss of generality, suppose a large utility value u n+1 is said to be preferred to a smaller value u n . Let u N be the largest utility value and u 1 be the smallest, then an input x can be transformed into the following belief distribution: where

2) INTERVAL DATA TRANSFORMATION
As shown in Fig 2, there are two different cases of interval data, the range of the interval data may be included within two neighboring grades, or it may span several grades [41]. For the first case where the interval data is totally included by two neighboring grades, suppose x ∈ [x − , x + ] is the assessed value included by θ n and θ n+1 . Then the belief degree to which x is assessed to θ n and θ n+1 can be obtained as: It can be noted that the obtained belief degrees β n ∈ [β − n , β + n ] and β n+1 ∈ [β − n+1 , β + n+1 ] are not independent, in fact, they should satisfy β n +β n+1 = 1. Hence, the interval data x ∈ [x − , x + ] can be expressed as using the following belief distribution: For the second case where the value x ∈ [x − , x + ] spans three or more neighboring grades. Suppose there are k grades involved in this case, denoted as θ m = {θ n , . . . , θ n+k }, with m representing the subscript of the involved grades. Then the belief distribution of x can be expressed as: where β m ∈ [β − m , β + m ] for m = n, . . . , n + k, and n+k m=n β m = 1. It is clear that though each grade in θ m = {θ n , . . . , θ n+k } has the same probability of being assigned, only one or two grades would actually be used. To demonstrate this phenomenon, Wang et al. [41] introduced a set of 0-1 integer variables as: with n+k−1 y=n I y,y+1 = 1 and n+k−1 y=n I y,y+1 = 0. Hence, the interval belief degrees can be expressed as follows: with n+k y=n β y = 1 and n+k−1 y=n I y,y+1 = 1.

B. ASSESSMENT RESULT AGGREGATION USING THE INTERVAL-VALUED ER ALGORITHM
Once the assessments on the attributes of weapon system are transformed into belief distributions, the aggregation algorithm should be applied to combine these belief distribution to obtain the final evaluation. However, the ER algorithm cannot effectively deal with interval uncertainty as the belief degrees are all precise values instead of interval values, hence, the interval-valued ER algorithm [41], which itself is an extension of the ER algorithm, is applied in this problem to aggregate different belief distributions and obtain the final evaluation result.

1) STEP 1: ATTRIBUTE WEIGHT DETERMINATION
Firstly, the weight of the attributes should be determined. The weight of an attribute represents the degree to which it influences the effectiveness of the weapon system, and an attribute would have a larger weight if it has a larger influence on the weapon system effectiveness. In the proposed method, the analytical hierarchy process (AHP) introduced in Section II-B is used to determine the weight of the attributes.

2) STEP 2: BELIEF DISTRIBUTION TRANSFORMATION
Then, by using the information transformation method introduced in Section III-A, the belief distributions of different attributes can be obtained. Suppose the assessment of the ith attribute x i is transformed into the following belief distribution: where θ i,n is the nth grade of the ith attribute, β i,n ∈ [β − i,n , β + i,n ] is the belief degree to which the assessment is assigned to θ i,h .

3) STEP 3: INTERVAL-VALUED ER-BASED AGGREGATION OF BELIEF DISTRIBUTIONS
Once both the weight and belief distribution of each attribute are obtained, all the belief distributions should be combined with regard to their weights. In this case, a nonlinear optimization model for combining interval-valued belief distributions is used, and the combined belief degree β n of the nth grade is calculated as follows: max/min β n (26) s.t. β − n,k ≤ β n,k ≤ β + n,k sum N n=1 β n,k = 1 I n,n+1 = 0 or 1 for n = 1, . . . , N − 1 (27) where β n denotes the belief degree assigned to the nth grade, and is calculated by using Eqs. (10)- (13). β n,k denotes the belief degree of the kth attribute on the nth grade.

4) STEP 4: ASSESSMENT RESULT CALCULATION
Finally, in order to draw a final conclusion on the weapon effectiveness, the expected utility value should be calculated for analysis and comparison. Suppose u n is the expected utility of the nth grade θ n , since interval data is involved in this problem, the expected utility value would also be in the form of interval data, and the maximum and minimum expected utility values can be calculated as follows: Therefore, the evaluation result of weapon effectiveness can be obtained as u = [u max , u min ].

C. WEAPON SYSTEM RANKING BASED ON THE TWO-GRADE RANKING METHOD
Once the evaluation results are obtained in the forms of interval data, it is necessary to compare and rank the effectiveness of different weapon system to draw a more comprehensive conclusion. In this case, in order to provide a more reliable and accurate result on the ranking of interval data, a two-grade approach proposed by Song et al. [37] is used.
Firstly, suppose there are M weapon systems to be ranked, and is the assessment result of the ith weapon system, then the dominance degree D(u i , u j ) between two evaluation results u i and u j is defined as: where | · | denotes the cardinality of a set, and [ Based on the dominance degree, the entire dominance degree D(u i ) is defined as: Then, the entire dominance degree can be used to determine the ranking place of the evaluation results. The bigger the value of D(u i ), the better this weapon system. However, after this process, some evaluation results cannot be ranked completely, instead, they are put into the same place.
Therefore, after obtaining a ranking result by the dominance degree and the entire dominance degree, the second grade is conducted, where a directional distance index is used to measure the preferability degree of two assessment results with the same rank. For two evaluation results with the same rank u i and u j , the directional distance index is defined as: where max(u + ) = max{u + 1 , . . . , u + M }, min(u − ) = min{u − 1 , . . . , u − M }. Then the entire directional distance index can be obtained as: Finally, the ranking of different assessment results can be obtained based on the directional distance index and entire directional distance index. The process of the weapon system ranking method can be summarized as follows: Let u i and u j be two assessment results, 1) If D(u i ) > D(u j ), then u i is superior to x j , denoted as x i x j ; 2) If D(u i ) < D(u j ), then u i is inferior to x j , denoted as denoted as x i = x j ; In order to illustrate the effectiveness and efficiency of the proposed method, a case study on missile system effectiveness assessment with interval data derived from [35] is studied.

D. PROBLEM DESCRIPTION
Missile system effectiveness assessment is an open issue in missile system development and design, since it is normally expensive and impossible to conduct the actual attack for evaluation, missile system effectiveness assessment using MADM methods becomes a feasible way to properly determine the performance of the missile system. Normally, for missile system effectiveness assessment problem, directly assessing the overall system effectiveness would be relatively difficult. Therefore, in this case, the effectiveness of missile system is represented using a hierarchical structure of three levels, where the first is the goal of the problem, i.e., missile system effectiveness, and the second level includes four ability attributes, namely, mobility ability, attack ability, operation ability, and defense ability. Each ability attribute is divided into several sub-attributes, and the hierarchical structure is shown in Fig 3.  In this case, six different missile systems are chosen, with both quantitative and qualitative data obtained from documents and knowledge of experts. Interval information in encountered for several sub-attribute, and the collected characteristic data of six missile systems are shown in Table 2.

E. ATTRIBUTE WEIGHT CALCULATION
In order to conduct the evaluation, the weight of the attributes should be determined using the AHP, as introduced in Section III-B.
Take the assessment of the overall effectiveness as an example, there are four attributes involved, therefore, the pairwise judgment matrix would be a 4×4 matrix, and the matrix is obtained based on experts' judgments as follows: Then the CR can be obtained as: where λ max = 4.1568.
Since CR < 0.1, it can be said that the pairwise judgment matrix is consistent, hence, the weights of these four attributes can be obtained using (17) as follows: ω Mobility = 0.1885, ω Attack = 0.5485, Similarly, the weights of all the sub-attributes can also be calculated, and the weights of all the attributes are shown in Table 3.

F. AGGREGATION OF BELIEF DISTRIBUTIONS
In order to conduct the effectiveness assessment, all the characteristic data should be transformed into belief distributions,  Table 4.
For precise data, the belief degree can be easily determined by using (21)- (22). For example, the reaction time of Missile 1 is 12s, and the referential value of reaction time shown in Table 4  Therefore, all the collected data can be transformed into belief distributions, as shown in Table 5.
Once all the characteristic data are transformed into belief distributions, the interval-valued ER algorithm can be applied to combine these belief distributions and obtain the final assessment result. Since the assessment structure is hierarchical, all sub-attributes should be combined to obtained the assessment results of four abilities at first. In order to illustrate the process of interval-valued ER-based evaluation method, the assessment of mobility ability of Missile 1 is taken as an example.
Firstly, the weights of all three sub-attributes, i.e., reaction time, transportation ability and maximum velocity should be determined. As shown in Table 3, the weight of these subattributes can be obtained as:  Then, the assessment of mobility ability can be obtained by aggregating the belief distributions of these three subattributes while considering their weights, and the result can be obtained as: Similarly, the assessment of four ability attributes of six missile systems can be obtained using the same process, and the aggregated belief distributions of these four attributes of six missile systems are shown in Table 6. Then the assessment results of all six missile systems can be obtained by aggregating the belief distributions of four attributes using (28)- (30), and the maximum and minimum utility values are shown in Table 7. In order to make full use of the assessment result, the results of six missile systems are used to compare and rank all six missile systems, as introduced in Section III-C.
Firstly, the dominance degree between each two assessment results is calculated, and the dominance degree relation matrix can be obtained as: Hence, for the first grade, the ranking can be obtained as Clearly, though most assessment results can be ranked, it is not satisfactory because some results cannot be ranked completely in present rank, i.e., u 1 and u 6 . That is because the entire dominance degree D(u i ) mainly investigates the relative ranking position of assessment results in entire universe from the viewpoint of rank, while ignoring more detailed difference of two results. Hence, the second grade of the two-grade ranking method is conducted to obtain a complete ranking result.
Since there are only two assessment results cannot be ranked by the first grade, the directional distance index relation matrix does not need to be calculated. Thus, the entire directional distance index of u 1 and u 6 can be calculated as: Hence, it can be concluded that u 1 u 6 . Finally, the ranking of the assessment results of six missile systems can be obtained as:

H. COMPARATIVE ANALYSIS
In order to further demonstrate the effectiveness of the proposed method, the assessment results of the proposed method are compared to the results using TOPSIS method [19], the AHP-TOPSIS method [17], and the fuzzy weighted average (FWA) method [46] in terms of the ranking result, and the comparison results are shown in Table 7.
As shown in Table 7, it can be seen that the ranking result of the proposed method is the same as that of the AHP-TOPSIS method and the FWA method, which confirms its validity. Though the ranking result of the TOPSIS method is slightly different from the result of the proposed method, i.e., the fourth and fifth missile systems have swapped rankings, the overall ranking result is consistent to the result of the proposed method, which further confirms the reliability of the proposed method. Furthermore, it should be noted that only the proposed method could provide interval data of the assessment results, which enhances its usability. Therefore, it can be concluded that by integrating interval-valued ER algorithm, AHP and the two-grade interval ranking method, the proposed method could provide accurate and reliable assessment and ranking results of missile systems.

I. SENSITIVITY ANALYSIS
In order to illustrate the feasibility of the proposed method, we further analyze the sensitivity of the change in the value of the attribute to the assessment result using the proposed method. The characteristic data of the analyzed missile system is shown in Table 8. For simplicity, the attribute Trainability is used to analyze the impact of changing its value of the final assessment result. Suppose the uncertainty degree of its value r raise from 0 to 1, in other words, its value changes from 0.6 to [0.4, 0.8], i.e., completely uncertain. The results of the sensitivity analysis are shown in Fig 4. For the sake of comparison, the assessment result using the method from Jiang et al. [24] is used, it should be noted that it is only applicable for the case of r = 0.  [24], which shows the effectiveness of the proposed method.
Moreover, when the value of r rises, the upper bound of the assessment result increases, and the lower bound of the assessment result first decreases, then increases, and finally decreases again. With the increase of the value of r, the uncertainty degree in the assessment is generally increasing, as shown in Fig 5. However, it should be noted that in the extreme case of r = 1, i.e., the value of Trainability is total uncertain, the uncertainty degree of the assessment remains relatively low, and that is because by utilizing AHP and interval-valued ER algorithm, the proposed method is able to deal with uncertainty in the input while properly combining 53488 VOLUME 9, 2021 the accurate values of other attributes, which ensures that the impact of the uncertain information on the assessment result is limited. Therefore, it can be concluded that the sensitivity analysis shows that the proposed method is as effective as existing method when uncertainty is not considered, and it also has the ability to properly deal with uncertainty in the inputs and limit the impact of uncertainty on the assessment result.

IV. CONCLUSION
Weapon system effectiveness assessment is an open and urgent issue in weapon system development, design and military planning, and it often involves different kinds of information such as qualitative judgments, precise data, and interval data under uncertainty. However, current studies often fails to properly deal with different kinds of information under uncertainty, especially interval data. To this end, this paper utilized the interval-valued ER algorithm, the AHP and the two-grade interval ranking method, and proposed a novel weapon system effectiveness assessment method, where AHP is used to determine the weight of different attributes and the interval-valued ER algorithm is used to combine the evaluation of different attributes under uncertainty. Furthermore, the two-grade interval ranking method is used to provide the ranking result of different weapon systems. A case study on missile system effectiveness assessment is conducted to demonstrate the effectiveness and efficiency of the proposed method. The results show that the proposed method could provide reliable and accurate result for missile system effectiveness assessment, moreover, comparison with other methods and sensitivity analysis further confirms the effectiveness and validity of the proposed method. In conclusion, by utilizing the interval-valued ER algorithm, the AHP and the two-grade interval ranking method, the proposed method provides a novel way for weapon system effectiveness evaluation under uncertainty, especially with interval data.
Moreover, the proposed method mainly focuses on the interval uncertainty, however, due to the complexity of the real world, different kinds of uncertainties, such as Pythagorean fuzzy uncertainty and even incompleteness could exist, how to expand the proposed method to those environments will be further studied in the future. Furthermore, the AHP is used to determine the weights of different attributes, as many other methods such as the entropy-based method and the DEMATEL have shown to be effective in weight calculation, we will also further investigate the application of other weight calculation methods.