Similarity and Agreement Measures and Their Application in Medical Diagnostic Prediction System

Due to importance of measuring the degree of resemblance, the similarity measure is widely adopted in various areas of the information systems (e.g., medical informatics and information retrieval) and in several applications like medical diagnostic, image processing, and pattern recognition. However, most of the existing similarity measures focus mainly on the degree of similarity without consulting expert(s) about the results. In this paper, an efficient tool for measuring similarity and agreement of objects that embeds experts’ opinions is proposed to assess similarity among features and agreement of opinions among experts. To obtain such robust measuring tool, three construction steps were followed. Firstly, adapting soft expert set as a general structure that consists of four components: objects, attributes, experts, and experts’ opinions. Secondly, representing the soft expert set, without losing stored information, in such a way as to fit the proposed similarity-agreement measure and make it simpler and more meaningful than the similar existing measures. Thirdly, axiomatizing the similarity-agreement measure for the case of two experts to simplify the model. Further, a diagnostic prediction application and its algorithm is discussed in this context, along with analysis of the experimental results. Analysis of performance of the proposed similarity-agreement measure revealed that it has high accuracy, sensitivity, and value of the F-measure and that it has better performance than existing state-of-the-art tools.


I. INTRODUCTION
The human being counts on her/his own experiences as well as on those of trustworthy people crossing her/his path in life. This usually happens at the early age via direct exposure to the world and language and indirect exposure through contact with the people whom we trust to be having more and better knowledge than ourselves. In effect, reliance on experts' opinions extends throughout the rest of one's life. For example, one consults a physician about diagnosis and treatment, a lawyer about legal troubles, a doctor about the way how to do research, and so on.
In general, one may seek an advice in routine problems for one or more of the following reasons: time constraints, where the decision must be taken within a deadline; riskiness, where the decision maker has no clear idea about the consequences The associate editor coordinating the review of this manuscript and approving it for publication was Yongming Li . of her/his decision; solvability, where the decision maker has the intuition of the existence of some keys for making a confidant decision; and availability of expert [1].
Appealing to expert's opinion is encouraged, especially when it comes to cost-sensitive situations [2]. For instance, diagnostic errors can lead to patient harm. Nonetheless, they received inadequate exploration in comparison with other patient safety concerns [3]. Consequently, and from a theoretical point of view, one needs to evaluate the agreement among experts' opinions, which prompts further study of the agreement measure.
The agreement measure is a measuring tool that fits when it comes to experts' or observers' opinions. In other respects, the similarity measures are a math-based matching tool that plays an important role in many applications of information systems such as natural language processing, image processing, machine learning, and pattern recognition problems. VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ Validity of the similarity measure is usually the bottleneck for many information system applications [4]. Thus, combination of a similarity measure and agreement measure is needed for assessment of extent of match between two things in general setting, where these two things are embedded opinions of experts. A theoretical motivation for such combination is the nature of the soft expert set [5], which usually comprises four components: objects, attributes, experts, and experts' opinions. In the case of the objects, attributes, and experts, matching is handled by the similarity measure while in the case of the opinions it is handled by the agreement measure, which is the common tool in use for matching.
As far as we know, there are no published studies of measurement of distance, similarity, or agreement in soft expert sets. In this article, a combined measure is proposed to determine how close, similar, or so assumable are two soft expert sets to each other. Applicability of this proposed measure to prediction of medical diagnosis is also discussed.
In sum, this study aimed at achieving five objectives ( Figure 1). First, to modify similarity measure of soft sets. Second, to propose an axiomatic definition of agreement measure. Third, to represent soft expert set. Fourth, to propose an axiomatic definition of similarity-agreement measure. Fifth, to present an application of the proposed similarity-agreement measure, along with its algorithm, to a medical diagnosis prediction problem.

II. PRELIMINARIES
The purpose of this section is threefold. Firstly, it provides a briefing on some measurement tools, namely, distance, similarity, and agreement. Secondly, it introduces soft sets and highlights their distance and similarity measures. Thirdly, it presents soft expert sets and discusses new representations of them.

A. MEASUREMENT TOOLS
Measuring distance between two numbers is common in mathematics. The absolute difference between the numbers is usually the target, which is conceived as a distance measure. However, such tool does not work when it comes to measuring distances among other mathematical objects like functions, sequences, and matrices [6].
The 20th century was a generalization and axiomatization era for mathematics. Axiomatizing concepts based on their nature makes them clear and overcomes any associated discrepancy. Fréchet [7] axiomatized the concept of distance and called it metric. His definition is applicable to any mathematical object in a geometrical sense.
Definition 1 ( [6]): Let X be a nonempty set and x, y, z ∈ X . A mapping d : X ×X → R that satisfies the following Example 1: Let R be the set of all real numbers and d be a metric on R. Then the distance between any two numbers that belong to R must fulfill the four axioms of the Definition 1. For example, for x and y that belong to R, the mapping d(x, y) = |x − y| is a metric.
Normalizing the first axiom in Definition 1 for it to be less than one or equal to it rather than to be only greater than zero or equal to it, that is, will be more convenient, especially when the counter part of distance, i.e., the similarity, is involved. One can say that distance and similarity are dual concepts, 1 that is, they are inversely related. Accordingly, similarity can be defined as follows: However, a definition for the concept of similarity by its own right is still needed. The following axiomatic definition is an attempt.
Definition 2: Let X be a nonempty set and x, y, z ∈ X . A mapping s : X ×X → R that satisfies the following axioms 1) 0 ≤ s(x, y) ≤ 1, 2) s(x, y) = s(y, x), 3) s(x, y) = 1 if and only if x = y, and 4) s(x, y) ≥ s(x, z) + s(z, y) is called a similarity measure.
In other words, the similarity measure should satisfy the following five conditions [8]- [11] 1) It should be a bounded real-valued function whose value ranges from zero to one. 2) It should be a symmetric function.
3) If the compared two sets coincide, then the value of the similarity measure must be one. 4) If the compared two sets are the universal set and the empty set, then the value of the similarity measure must be zero. 5) The more similar the compared two sets, the closer to one is the value of the similarity measure.
Also, we refer to the following recent results of distance and similarity measures for better understanding of the fuzzy counter part of the work [13]- [15]. The agreement measure is a well known and widely used statistic. Nevertheless, it did not receive much attention, especially as regards establishment of rigorous mathematical foundation for it. Consequently, it will be much helpful to provide or suggest axioms that all agreement measures should obey. This issue is discussed in Section III.

B. SIMILARITY MEASURES OF SOFT SETS
Measuring distance and similarity in classical sets is well known and it has been developed over time [16]. However, for the soft sets, the similarity measures had their own development journey. The soft sets were introduced by [17] as a parameterized family of attributes that assigns labels to objects having these attributes. In mathematical sense, the soft set can be written as a map: where E is a subset of a finite set of attributes , U is a finite universe, and P(U) is the set of all subsets of U, also known as the power set of U. For convenience, let U = {u 1 , u 2 , . . . , u i } be a finite universe, E = {e 1 , e 2 , . . . , e j } be a finite set of attributes, and S(U) be the set of all soft sets over U.
In their article about the operations of soft sets, [18] suggested that F should map attribute e ∈ E to a nonempty subset of U. This is justified by the fact that the soft set stores no direct information by assigning attribute e to no-object in the universe of discourse. If there is a case, considering F(e) = ∅ with e not belonging to E, will be helpful [19]. However, we will not consider such case in this article, and, hence, the soft set will be characterized by where P * (U) donates the power set of U except the empty set ∅.
The similarity measure for the soft sets was introduced in [20] as follows. 2 Definition 3 ( [20]): A similarity measure of soft sets (F, A) and (G, B) over the same universe U and same attribute set, that is A = B, is a mapping . (1) If the attribute sets of (F, A) and (G, B) are not coincident with some common elements, then the similarity measure s 1 is defined by the Formula (1) over the same universe U and on the set of common attributes, that is A ∩ B.
Remark 1: Note here, Definition 3 is not axiom-based definition.
The following example was given in [20]. Example 2: Let a universe of discourse U be {x 1 , x 2 , x 3 } and a set of attributes E be {e 1 , e 2 , e 3 }. Then, the measure of similarity between (F, A) and (G, B) is s 1 : However, a counter example of the Formula (1) was given in [21]. It used the fact that the similarity measure s 1 ((F, A), (G, B)) must be a real number whose value ranges from zero to one.
Then by using Formula (1), similarity measure s 1 between (F ∅ , E) and itself s 1 ((F ∅ , E), (F ∅ , E)) provides the indeterminate value 0 0 instead of one. From the viewpoint of [18], the work of [20] on similarity of soft sets will be less imprecise, where the counter example pointed out by [21] will fade since there are no empty sets in the image of F. Nevertheless, the uniqueness of representation in [20] needs to be further rectified as mentioned by [21]. Though, [21] provided another tool to measure the similarity between soft sets that is based on the axiomatic set theory. Still, his proposed tool is deficient too when it comes to equal soft sets [22]. Definition 4 ( [21]): A similarity measure of soft sets (F, A) and (G, B) over the same universe U is a mapping satisfying the following axioms. A)), and 4) s 2 is monotonic. Further, Kharal (2010) provided a formula for s 2 that seems to be complying with the foregoing four axioms: . (2) The following counter example was given by [22]. Then This shows that Formula (2) violates the condition that the similarity measure should be less than, or equal to, one. Yang (2013) corrected for the defect in [21] and provided another measuring tool for the soft sets that could successfully address the soft set attributes and objects separately.
Definition 5 ( [22]): A similarity measure of soft sets (F, A) and (G, B) over the same universe U is a mapping The following mappings satisfy the aforementioned four axioms.
It is worth mentioning that Yang's [22] definition of similarity can be re-written in more standard notation as follows. Let and be two interval-valued functions. Then, the similarity measure is a mapping with w λ + w µ = 1 of linear combination of µ and ν with the desired axioms. Seeking simplicity, we modify Yang's [22] similarity definition by assigning a similarity with single number instead of pair of numbers.

C. REPRESENTATION OF SOFT EXPERT SETS
A generalized form of soft set where experts' opinions about the information stored in the set are present was given by [5]. However, experts' opinions and time-dependence of their opinions should exist whenever an investigation of operations or applications of soft expert sets is involved. This argument was emphasized by [25]  One can write soft expert set as the set of all ordered pair (ζ, η) where ζ belongs to A and η is a subset of P(U), i.e., (F, A) = {(ζ, η) : ζ ∈ A and η ⊆ P(U)}.
In our effort to propose a similarity-agreement measure of soft expert sets, we represent the soft expert set, without losing stored information, as follows: A hypothetical element in (F, A) will be as ((e, j, 1), , that is, objects u 1 and u 2 have the attribute e with full agreement of expert j. On the other hand, an element in (F,Ã) will store the same information and can be written as (({u 1 , u 2 }, e, j), 1). We conclude this section by emphasizing that soft expert sets lack measuring, both for distance for similarity.

III. AGREEMENT MEASURE
Following the mathematical philosophy of axiomatizing concepts and tools, we axiomatize the concept of agreement measure. Doing so provides a rigorous mathematical foundation for the concept and makes it clearer and more applicable than before and precludes commitment of errors that may be made under the condition of lack of rigorous foundation.
The agreement measure is simply conceived as an opinion-matching tool, both when experts agree to agree and when they agree to disagree. The description of the experts' opinions can be represented as confusion matrix. Consider, for instance, the following table.
Let M n×n be the set of all matrices of order n × n and let A be one of those matrices, symbolically A ∈ M n×n .
Obviously, the confusion matrix stores experts' opinions. So for an axiomatic definition of an agreement measure we define A 1 to represent the first expert's opinion and A 2 to represent the second expert's opinions with A 2 = A T 1 , where A T 1 is the transpose matrix of A 1 . In other words, the agreement measure should have the following properties: 1) It is a bounded real-valued function whose value falls in the range of -1 to 1.
It satisfies Definition 8, where P 0 is the relative identical to accuracy observed agreement among experts, and P c is the probability that experts agreed on some observation by chance. If the experts are in perfect agreement then κ = 1. However, if there is no agreement exists among the experts other than what is expected by chance, then κ = 0. Lastly, Cohen's Kappa function can be negative, that is, the agreement among experts is less then what may occur by chance [2], [24].

IV. SIMILARITY-AGREEMENT MEASURE
To provide an axiomatic definition of a similarity-agreement measure, formulation of axiomatic definitions for similarity and agreement measures is unavoidable or at least is recommended. For the agreement measure, Definition 8 addresses the concept without any need for modification. However, in the case of the similarity measure, Definition 6 needs to be generalized. The reason behind this is that Definition 6 was originally proposed for the soft set, not for the soft expert set. Therefore, the researchers provide the following two definitions in order to generalize Definition 6.
Definition 9: Let = (F, A, J ) be a mathematical structure, where F(e) is the set of objects in the structure that are assigned to the attribute e ∈ A, and J is the set of experts. A structure is called a non-opinion part of the soft expert set and is donated as N O(U).
Before introducing the second definition, let us define the following functions: with w λ + w µ + w ν = 1 is called a similarity measure of non-opinion parts if the following axioms hold By now, the way is paved for proposing an axiom-based definition of a similarity-agreement measure.
Definition 11: A similarity-agreement measure of soft expert sets (F,Ã) and (G,B) is a mapping defined as where the definition s and κ are given in Definition 10 and Definition 8 respectively. Definition 11 can be conceived as a general definition of similarity-agreement measure, since the soft expert set can be thought of a structure of the desired components.
Let us consider an explicit example of similarityagreement measure that satisfies Definition 11. Consider two VOLUME 8, 2020 mappings treating the four components of soft expert sets as follows: Then a mapping κ s ((F,Ã), (G,B)) = (s(ξ p , ξ q ), κ(o r , o s )) is a similarity-agreement measure. Remark 2: The mapping s(ξ p , ξ q ) can be weighted as follows with w 1 + w 2 + w 3 = 1.

Remark 3:
Although the mapping κ s can be unified in a way similar to that in Definition 6, to assign the similarity-agreement measure a single value that ranges from 0 to 1, Definition 11 has its own advantage in that it provides separate information about the degree of similarity and the degree of agreement, which is very much helpful when this measure is used.

V. APPLICATION: DIAGNOSTIC PREDICTION
A bulky volume of research hightailed the importance of consulting experts about certain medical diagnosis situations [26]- [28]. In a major medical procedure, for instance, seeking advice from several specialists at different medical centers is a common practice. Even for internists and physicians, predicting diagnosis is still challenging [29] because of the availability of a vast volume of clinical data that exceeds the ability of the human brain to assimilate and analyze [30]. This issue raises the need for soliciting an expert and, thus, using agreement measure. However, the clinical data sources may appear as structured, unstructured, or semi-structured and incomplete. Hence, ability of the similarity-agreement measure to predict a medical diagnosis can be limited. Within this context, this section presents a medical diagnostic prediction system framework for prediction of a diagnosis of the health condition of a current patient by means of a similarity-agreement measure (see Figure 2).
For any given pathology, let E = {e 1 , e 2 , . . . , e m } be a set of symptoms, P = {p 1 , p 2 , . . . , p n } be a set of previous patients (objects), p current be a current patient, D = {d 1 , d 2 , . . . , d k } be a set of diagnoses, and J be a set of physicians. Moreover, let (F,Ã) be a soft expert set with a single patient and (G,B) be a soft expert set with all previous patients. Now, to achieve the goal of predicting a diagnosis of the current patient, we provide a medical diagnostic prediction algorithm based on similarity-agreement measure.
The algorithm takes as inputs (i) the previous patients and their symptoms and diagnosis, and (ii) the current patient and her/his symptoms (Lines 1-4). Therefore, the physicians, as experts, give their opinions on symptoms of each J ← add j p i to list 9: End for 10: j p current ← Enter Doctor Opinion for Patient Symptoms 11: For each p i in P do 12: (G,B) ← Transfer (P, E, J ) to construct soft expert set 13: End for 14: (F,Ã) ← Transfer (p current , E, j p current ) to construct soft expert set 15: For each p i in (G,B) do 16: (κ s ) i ← Compute κ s ((F,Ã), (G,B)) 17: SA ← Add (κ s ) i to list 18: End for 19: d i ← max SA 20: return D 21: End procedure and every previous patient on a two-point scale of agree, corresponding to a score of 1, or disagree, corresponding to a score of 0 (Lines 6-9). Likewise, the physicians give their opinions on symptoms of every current patient on the same two-point scale (Lines 6-9). Afterwards, the algorithm transforms the membership value of every symptom, as well as the associated opinion of each physician, to a soft expert set for the previous patients (Lines [11][12][13] and the current patient (Lines 1-4). Then, the algorithm computes the similarity-agreement score for the pervious and current patients (Lines [15][16][17][18]. Consequently, the diagnostic prediction for the current patient will be the diagnosis of the pre-vious patient that achieved the highest similarity-agreement score (Line 19).

VI. EXPERIMENTAL ANALYSIS
Effectiveness of the proposed similarity-agreement measure was verified in the present study by using seven different medical datasets drawn from the UCI Machine Learning Repository database [31], which are the Hepatitis Domain dataset (D1), Heart Diseases dataset (D2), Pima Indians Diabetes Database (D3), Wisconsin Diagnostic Breast Cancer dataset (D4), Diabetic Retinopathy Debrecen dataset (D5), BUPA Liver Disorders dataset (D6), and Chronic Kidney Disease dataset (D7). Since the database is heterogeneous, instances of missing data are expected. Such data points were excluded from analysis. So, the researchers excluded missing data from analysis and selected the relevant items randomly to get suitable data about 45 patients from each dataset for the analysis.
The datasets employed in this study were transformed into soft expert datasets by using Python, which is available in open source. Then, the similarity-agreement measures were computed and the results were compared with the results generated by existing similarity measures; s 1 , s 2 and s 3 that were calculated using formulas (1), (2), and (4) respectively. Performance comparisons were based on three performance evaluation criteria: sensitivity, accuracy, and the F-measure [32]. Assessments were made for 500 adult patients who previously visited internal medicine clinic at Al-karak Public Hospital in Jordan during February 2020. The performance evaluation outcomes and a comparison of level of performance of the proposed measure with those of the aforementioned three measures are presented in Table 2 and illustrated in Figure 3.  It can be seen in Table 2 and Figure 3 that the herein proposed similarity-agreement measure has higher sensitivity, accuracy, and value of the F-measure than the other investigated measures.

VII. CONCLUSION AND LIMITATIONS
The similarity-agreement measure is a useful solution for decision making in cost-sensitive situations. In this paper, a new general framework for the similarity-agreement measure is proposed for dealing with uncertainty in costsensitive cases. A relevant algorithm was constructed and, then, applied for medical diagnosis prediction. The proposed similarity-agreement tool is competent with state-of-the-art similarity measures for sensitive cases like s 1 , s 2 , and s 3 . Accordingly, this study adds to the literature a matching tool of soft expert sets. Theoretically, it developed a similarityagreement tool for measuring the degree of resemblance. Practically, this tool is very important. It is quite useful for handling several cost-sensitive and uncertainty problems such as the problems associated with of the medical diagnostic systems, fraud detection systems, and image processing systems. As such, this study has several contributions. First, it modified the similarity measure of soft sets. Second, it proposed an axiomatic definition of the agreement measure. Third, it represented a soft expert set. Fourth, it proposed an axiomatic definition for the similarity-agreement measure. Finally, it presented an application of the proposed similarityagreement measure, along with its algorithm, to a medical diagnosis prediction problem.
A limitation of this study is that the proposed similarity-agreement measure focuses only on the common features amongst objects. A different similarity measure [12] has room for distinctive features as well. Another limitation to mention is that the herein proposed similarity-agreement measure took into account opinions of two experts only even though the soft expert set allows for many experts to contribute. Future works are advised to take these two limitations into consideration.