Unsupervised EA-Based Fuzzy Clustering for Image Segmentation

This paper presents an unsupervised fuzzy clustering based on evolutionary algorithm for image segmentation. It needs no prior information about exact numbers of segments. Local and nonlocal spatial information derived from observed images are incorporated into fuzzy clustering process. It consists of three major procedures. First, a multi-objective evolutionary sampling is proposed to locate image pixels with a variety of image information. Secondly, optimizing fuzzy compactness and fuzzy separation, a multi-objective evolutionary fuzzy clustering with spatial information is performed on sampling pixels. The particular numbers of segments and balances of spatial information can be obtained. Then fuzzy clustering segmentation on whole image is carried out by two fuzzy clustering approaches, which are depended on fuzzy c-means and evolutionary algorithm respectively. To enhance qualities of final segmentation results, a label correction based on entropy and local spatial information is introduced. Experiments on different types of images demonstrate the effectiveness of our approach for image segmentation.


I. INTRODUCTION
Image segmentation, the extraction of important objects from observed image, has received extensive attention since past few decades [1]. It is an essential step in many image applications. And thus a number of approaches [2], [3] have been proposed to utilize effective techniques to deal with image segmentation. Among the widely applied techniques for image segmentation, clustering [4]- [6] aims to divide a dataset into a finite set of clusters. It partitions image pixels into several clusters according to some characteristics, such as The associate editor coordinating the review of this manuscript and approving it for publication was Hisao Ishibuchi . grey level, texture or intensity. Incorporating fuzzy set [7] into clustering [8], fuzzy clustering has been studied and utilized in many substantive applications [9]- [12].

A. CONVENTIONAL FUZZY CLUSTERING ALGORITHMS FOR IMAGE SEGMENTATION
Among fuzzy clustering algorithms, fuzzy c-means algorithm (FCM) [4], [5] is one of the most popular and effective techniques for image segmentation [13]. Despite FCM performs well, it is sensitive to noise. To improve performance on images corrupted by noise, a variety of approaches [14], [15] have been proposed to incorporate local spatial information derived from images into energy function of VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see http://creativecommons.org/licenses/by/4.0/ FCM [16]- [18]. For example, Ahmed et al. [19] incorporated a neighbor information term into objective function of FCM. Chen and Zhang [20] proposed two variants (FCM_S1 and FCM_S2) to define the local spatial information term on an extra mean-filtered image and a median-filtered image, respectively. Except above algorithms with local spatial information, several approaches [21], [22] have been presented by incorporating nonlocal spatial information [23] into FCM. Compared with local information, nonlocal information is derived from a large domain of pixels with a similar configuration of center pixels. When the observed images are corrupted by noise seriously, FCM variants with nonlocal information most likely perform better than FCM variants with local information. Although above FCM variants achieve well segmentation performance on noisy images, most of them need a parameter to control the influence of spatial information in the fuzzy clustering process. This parameter, which is usually set experimentally, has a significant impact on fuzzy clustering. Moreover, the classical FCM energy function may be suppressed by the spatial information term in the fuzzy clustering process [24], [25]. Additionally, in the fuzzy clustering process, only the total distances within clusters are considered. It is helpful to deal with wellseparated or spherical clusters. But it may be not applicable for detection of clusters with more complex structures. Also, the number of clusters needs to be set before in most of FCM-based approaches. FCM-based approaches are sensitive to initialization and may get stuck at the local optimum [26]. Addressing above issues, fuzzy clustering algorithms based on global optimization for image segmentation have been investigated in recent years. Among global optimization techniques, evolutionary algorithm (EA) is widely applied to deal with fuzzy clustering problems [27], [28]. EA is a population-based metaheuristic optimization algorithm [29], and the main idea of EA is the mechanism inspired by biological evolution [30]. Each individual in population represents a candidate solution in the search space, and the fitness function determines the qualities of the individuals. EA starts from the initial individuals and gradually improves current individuals through the iterative evolution process, until the optimal solution or satisfactory solution is obtained [31]. Instead of a single solution, EA starts searching from multiple solutions. It is helpful to increase probability of searching overall optimal solution. Evolution of the population takes place after the repeated application of evolution operators, such as reproduction, mutation, recombination and selection [32]. EA ideally does not make any assumption about the underlying fitness landscape, which is helpful to approximate solutions to all types of problems. These characteristics mentioned above make EA perform well for searching potential regions of search space.
Compared with FCM-based approaches, above EA-based methods require no prior information about the numbers of clusters and consider distances between clusters. However, it is not applicable to deal with complex real world situations by optimizing single clustering validity measure. In this regard, several multi-objective evolutionary fuzzy clustering algorithms [41], [42] have been proposed. The fuzzy clustering problem is converted into multi-objective problem (MOP) [43] and optimized by multi-objective evolutionary algorithm (MOEA) [44], [45]. In a variety of multiobjective evolutionary fuzzy clustering algorithms [46]- [50] for image segmentation, different cluster validity measures have been considered simultaneously in the fuzzy clustering process. For example, the classical FCM energy function and XB index were minimized by MOEA in [41] and [47]. It is helpful to deal with different characteristics of partitioning and search potential solution towards different data properties. However, above two fuzzy clustering validity measures are not independent to each other completely. It may lead to not good enough solutions. Thus, MOVGA [42] was proposed by optimizing the global fuzzy compactness and fuzzy separation simultaneously. In order to improve performance of image segmentation, MSFCA [48] and MOEFC [51] were presented to incorporate nonlocal spatial information and local spatial information into multi-objective evolutionary fuzzy clustering, respectively. Compared with MOVGA, MSFCA incorporated nonlocal spatial information into global fuzzy compactness. Unlike MOVGA and MSFCA, MOEFC concerned a trade-off between preserving image details and restraining noise. But the decomposition strategy of MOEFC based on weighted summation approach [43] was not applicable for non-convex (or nonconcave) problems, which was not conducive to complex real world situations.

B. MAIN IDEA OF OUR WORK
This paper presents an unsupervised EA-based fuzzy clustering (EAFC) for image segmentation. It needs no prior information about exact number of segments. Both local and nonlocal spatial information derived from images are considered in the fuzzy clustering process. It is helpful to improve the quality of the segmentation performance. The general framework of EAFC shown in Fig. 1 can be described as follows: (1) Firstly, a multi-objective evolutionary sampling is proposed by optimizing the function to preserve image information and the function to calculate the number of sampling pixels simultaneously. It can preserve image information while reducing number of sampling pixels.
(2) Then multi-objective evolutionary fuzzy clustering with spatial information is performed on sampling pixels, and the particular number of segments can be obtained. Besides local spatial information, nonlocal spatial information derived from observed images are incorporated into multi-objective evolutionary fuzzy clustering to restrain noise effectively. Nonlocal spatial information is derived from a larger domain of pixels with a similar configuration of center pixels, and most likely performs better when the observed images are corrupted by noise seriously. In multi-objective evolutionary fuzzy clustering, local and nonlocal spatial information are controlled adaptively, and the balances of spatial information can be obtained.
(3) With obtained balances of spatial information and particular numbers of segments, the fuzzy clustering segmentation on whole image is performed. Two fuzzy clustering approaches, which are depended on FCM and EA respectively, are proposed to deal with the fuzzy clustering segmentation of whole image. A label correction based on entropy and local spatial information is introduced to enhance qualities of final segmentation results.
With above brief descriptions of EAFC, the main contributions and advantages can be seen as follows: (1) The multi-objective evolutionary sampling can obtain the trade-off between preserving image information and reducing number of sampling pixels. It can be incorporated into the algorithms which need to sample pixels. Moreover, the function to preserve image information can be replaced by the function based on measures defined by decision makers. The minimum number of sampling pixels, which can preserve important image information, can be obtained under the conditions of decision maker.
(2) It needs no prior information about exact number of segments. The number of the clusters is not required to be set before. A particular number of segments can be obtained by multi-objective evolutionary fuzzy clustering with local and nonlocal spatial information. The performance of fuzzy clustering for image segmentation can be improved by incorporating local and nonlocal spatial information simultaneously. The relationships between local and nonlocal spatial information can be balanced adaptively in the evolutionary fuzzy clustering process. Then the suitable solution can be located by the improved cluster validate index with local and nonlocal spatial information.
(3) Two fuzzy clustering approaches via FCM and EA respectively are proposed to segment whole observed image. Fuzzy clustering segmentation via FCM aims to obtain particular segmentation results in less time. Compared with fuzzy clustering segmentation via FCM, fuzzy clustering segmentation via EA trades relatively high time cost for better segmentation results. Decision makers can choose one of them according to needs. Furthermore, a label correction based on entropy and local spatial information is introduced to improve the quality of segmentation result by correcting outliers. It can be incorporated into any segmentation algorithms directly to improve the performance of final segmentation results.
In the remainder of this paper, the detailed procedures of EAFC are described in the Section II. Then Section III presents experimental results to show how EAFC works and its performances compared with other widely used algorithms for image segmentation. Finally, concluding remarks are drawn in Section IV.

A. DESCRIPTION OF MULTI-OBJECTIVE PROBLEMS
A k objectives minimization problem can be defined by where x represents a decision variable vector and n is the dimension number of x [52], [53]. If and only if the decision variable vector x A is said to dominate x B .
A vector x * ∈ is Pareto optimal if no vector x ∈ can dominate it. It also can be said that x * is a non-dominated solution [54]. The Pareto optimal set (PS) is composed of all non-dominated solutions. The potential solutions of MOP can be found in the PS [55], [56].
The Pareto front (PF) consists of all the Pareto optimal objective vectors. It can be defined by Then MOP can be converted into searching non-dominated vectors which can approximate the true PF.

B. MULTI-OBJECTIVE EVOLUTIONARY SAMPLING
Multi-objective evolutionary sampling approach aims to preserve image information with partial image pixels. The partial image pixels are the pixels which can preserve most of image information in the observed image. Based on the partial image VOLUME 8, 2020  pixels, the suitable number of clusters can be estimated by the subsequent multi-objective evolutionary fuzzy clustering, and the cost is less than the one of the same operation on whole image pixels. Generally speaking, the more pixels are sampled, the more image information are preserved. However, to reduce cost, the number of sampling pixels should not be too large. Hence, the sampling problem is converted into a MOP to search a trade-off between preserving image information and reducing the number of sampling pixels. The function to preserve image information is defined as a sum of variations of sampling pixels, where the variations can measure the homogeneities of pixels. Fig. 2 shows the flowchart of multi-objective evolutionary sampling. Let an image X = {x 1 , x 2 , . . . , x N } be observed, where x i represents the ith pixel and its value equals to the gray level value of the ith pixel. N is the total number of pixels. The detailed computation of variation I i for pixel x i is shown in Fig. 3.
In Fig. 3a, C i is the variation of pixel x i transformed from the observed image. x is a pixel set which consists of pixel x i and its neighbor pixels in the 3×3 square window N i . Pixel x i is the center of N i , and n i is the cardinality of N i . var (x) and mean (x) are the variance and mean of the pixels in x. The definition of variation C i is similar to the work in [18] and [51]. It can be seen that the value of C i indicates the differences among pixels in x. C is the average of the variations in N i . Here ξ i represents the coefficient of pixel x i , which is calculated by projecting the variation C i into kernel space. Then variation I i can be computed by normalizing the coefficient ξ i and protecting the normalized coefficient over interval [1,3]. When the value of C i is smaller than the one of C in N i , the value of variation I i is within interval [2,3] and increases with the decrease of C i . Moreover, when C i is not smaller than C, variation I i is within interval [1,2] and increases with the increase of C i . In the real world situations, the homogenous regions of images are composed of pixels with small differences. The inhomogeneous regions include edges and noises. Fig. 3b shows two examples of computation of variation C. It can be seen that C of the window at the edge is larger than the one of the window corrupted by noise. Then the pixels of the homogenous regions and edges have larger variations than noises. It is helpful to preserve detailed image information while restraining noise by sampling pixels with large variations. Then the MOP for sampling pixels is defined on the threshold value of sampling, and can be described as follows: where T is the threshold value of sampling. P i indicates whether pixel x i is sampled or not. If the value of variation I i of pixel x i is not less than threshold T , pixel x i is sampled. Then function f 2 is defined as the number of sampling pixels. Function f 1 is computed by the reciprocal of the sum of variations of the sampling pixels. eps is a constant with small value, and is set to 10 −5 here. Because of the small value of number of sampling pixels, f 2 decreases with the increase of threshold T . But f 1 increases with the small value of number of sampling pixels. In the multi-objective optimization, an adaptive differential evolution (DE) [57] is selected as the crossover operator. Then two parameters of DE, F and CR are determined by the adaptive strategy [58]. An example of individual encoding is shown in the Fig. 4. The details procedures of multi-objective optimization are shown in the Algorithm 1.
In the Algorithm 1, the selection of multi-objective optimization is defined on non-dominated sorting and crowd binary tournament selection method in [44] and [59]. Furthermore, in the reproduction procedure, the candidates are generated by the adaptive DE operator or the polynomial mutation operator. The two adaptive parameters of DE, F i and CR i , are updated by [58]: where rand r , r ∈ {1, 2, 3, 4}, are uniform random values over interval [0, 1]. When the candidates need to be generated by the polynomial mutation operator, the parameter γ of the polynomial mutation operator is computed by: where η = 20 and ρ is a rand value falling into interval [0, 1]. LU and LP are maximum and minimum values of individual T i . In multi-objective evolutionary sampling, they equal to I max and I min respectively. With above multi-objective optimization, a set of nondominated thresholds for sampling pixels can be searched. In step 3 of Algorithm 1, to preserve more image information while reducing number of sampling pixels, the nondominated solutions, which correspond to the number of sampling pixels falling into interval [0.2 × N , 0.6 × N ], are preferred as the feasible thresholds. Here N is the number of image pixels. Then locate the sampling pixels by the feasible thresholds. For the ith pixel x i , if it is located by more than half of the feasible solutions, it will be regarded as a sampling pixel. In this way, the final set of sampling pixels, which is the trade-off between preserving image information and reducing number of sampling pixels, can be obtained.

C. MULTI-OBJECTIVE EVOLUTIONARY FUZZY CLUSTERING WITH SPATIAL INFORMATION 1) FITNESS FUNCTIONS OF MULTI-OBJECTIVE EVOLUTIONARY FUZZY CLUSTERING
In order to search the particular number of clusters, multiobjective evolutionary fuzzy clustering is applied on the sampling pixels. Global fuzzy compactness and fuzzy separations are defined as the fitness functions. To improve performance of fuzzy clustering, nonlocal and local spatial Algorithm 1 Procedures of Multi-Objective Evolutionary Sampling Parameters: Population size N M 1 and maximum generation number G 1 . Input: Variations I = {I 1 , I 2 , . . . , I N } of image pixels. I max and I min represents the maximum and minimum value of I, respectively. Output: Sampling pixels.
Step 1) Initialization: Generate individuals randomly as the initial population. Step 2) Cycle: Perform non-dominated sorting on the population and give a rank to each individual. Then adopt binary tournament selection based on crowded comparison to generate the mating pool.
Step 2.1) Reproduction: For each individual in the mating pool, generate a candidate by the adaptive DE operator or the polynomial mutation operator. 4: if rand ≤ CR i then Select three individuals T r 1 , T r 2 and T r 3 from population randomly. Generate the candidate byT Then setT i = max min(T i , I max ), I min to ensure the effectiveness of the candidate. 10: Step 2.2) Selection: To generate the new population, the elitism is performed by combining the original population and candidates. Then adopt non-dominated sorting and select top N M 1 individuals by crowded comparison from combined population.
Step 2.3) Stop condition: If the value of the current generation number t equals to G 1 , then stop cycle and output final non-dominated solutions. Otherwise, set t = t + 1 and go to step 2. 12: Step 3) Sample pixels based on the final non-dominated solutions and output the sampling pixels.
information derived from observed images are incorporated into multi-objective evolutionary fuzzy clustering. Weight vectors to control relationships between nonlocal and local spatial information are incorporated into evolutionary fuzzy clustering process. Then balances of spatial information can be obtained by the multi-objective optimization. It is helpful to improve performance of subsequent segmentation on the whole observed image.
To improve performance on restraining noise, Gaussian Radial basis function is adopted as the distance measure, and is written by The bandwidth σ is defined as the distance standard deviation [18], [51] of the observed image X, and reflects the distribution of image pixels. The calculation can be writ- Let Y = {y 1 , y 2 , . . . , y NI } be the set of sampling pixels, where NI is the number of sampling pixels. z = z p cz p=1 is the set of candidate cluster centers, where z p is the pth cluster center and cz is the current number of clusters. If y i represents the ith pixel in Y, the fitness functions can be written by: where FC is global fuzzy compactness and FS is fuzzy separations. m is the weighting parameter on each fuzzy membership, and is set to 2 here.ỹ i andȳ i represent nonlocal and local spatial information of pixel y i , respectively. The nonlocal spatial information is gray level spatial information derived from a 21 × 21 search window around the center pixel y i in the observed image. The local spatial information derived from a 3 × 3 square window around y i consists of gray level spatial information and space spatial information.
The detailed computations of nonlocal and local spatial information are described in the first section of Supplementary Materials. The nonlocal and local spatial information are controlled by a weight vector [λ, 1 − λ] T . µ ip represents the fuzzy membership of pixel y i to z p . β pq represents the membership degree of z q to z p . The computations of µ ip and β pq can be defined as follows: With the fitness functions defined in Eqs. (6) and (7), the flowchart of multi-objective evolutionary fuzzy clustering can be categorized as Fig. 5. At the beginning, the initial population is generated randomly. Fig. 6 shows an example of individual encoding. An individual includes cluster centers, activation indexes, weight vector and parameters. The numbers of clusters and activation indexes equal to c max , which is the maximum value of numbers of clusters. The values  In the multi-objective evolutionary fuzzy clustering process, the mating pool is generated by the same strategy of the multi-objective evolutionary sampling. Non-dominated sorting is performed on the current population, and a rank is given to each individual. Then binary tournament selection based on crowded comparison is adopted to generate the mating pool. For each individual in the mating pool, the reproduction procedure includes adaptive DE crossover operator and polynomial mutation operator. In the crossover step, the adaptive DE operator is performed on the individual except the part of parameters. Then the candidateĝ i of individual g i is generated in terms of: where g r 1 and g r 2 are two individuals selected randomly from other individuals in the mating pool. Here k is not larger than 2 × c max + 1. In other words, the first value of weight vector is reproduced by the crossover operator. Two parameters, F i and CR i , are updated by Eq. (4). To keep diversity of the population in the evolutionary process, the polynomial mutation operator is performed on the candidateĝ i as Algorithm 2.
If a cluster center is mutated, the corresponding active index needs mutation. The mutation operator is also performed on the weight vector.
In the evolutionary fuzzy clustering process, it may happen that very few pixels belong to some clusters of the candidates sometimes. If no more than two pixels belong to one cluster, the cluster is regarded as an empty cluster. To enhance effectiveness of solutions, the candidates with empty clusters need repairing to improve their qualities. Let set the numbers of valid clusters and empty clusters of a candidate as n 1 and n 2 respectively. If the value of n 1 −n 2 is larger than one, it means that the candidate has no less than two nonempty clusters. It only needs to set the activation indexes of empty clusters as random values less than 0.5. If the value of n 1 − n 2 equals to one, there is only one nonempty cluster in this candidate. Both the valid cluster centers and activation indexes need repairing. Let set V represent the nonempty valid cluster center, a new cluster center vector can be generated by where rand 1 and rand 2 are two random values falling into [0, 1]. MAX Y and MI N Y represents the maximum and minimum gray level value of sampling pixels Y. If the number of valid clusters n 1 equals to 2, these two valid cluster centers can be set with above new cluster center vector. If n 1 is larger than 2, select two valid clusters randomly and set them with above new cluster center vector. Except the activation indexes corresponding to the selected two valid cluster centers, other activation indexes of valid cluster centers are set as random values less than 0.5. With the repair strategy, the effectiveness of candidates in the reproduction procedure can be ensured. In the selection procedure of the evolutionary process, the elitism is performed by combining the original population and candidates. Then adopt non-dominated sorting and select top N M 2 individuals by crowded comparison from above combined population, where N M 2 is the population size. When the value of the current generation number equals to the maximum generation number, the evolutionary cycle is stopped. The final non-dominated individuals are obtained.

3) SOLUTION SELECTION OF MULTI-OBJECTIVE EVOLUTIONARY FUZZY CLUSTERING
Decoding the non-dominated individuals, several clustering results are obtained. All the non-dominated clustering results are equally important. However, it is necessary to select a particular solution for users. In [42] and [48], a cluster validity Step 1) Mutation for cluster centers: For each cluster center do if rand ≤ pm then 3: Generate the cluster center of candidate byĝ Set the cluster center of candidate with the one of current candidateĝ k i =ĝ k i . 6: end if Step 2) Mutation for activation indexes: For each activation index do if the corresponding cluster center is mutated then 9: Generate the activation index of candidate bŷ g c max +k The PBM is composed of three items: c, E c and D c , where E c is the cohesion measures of the dataset and D c is the maximum separation between cluster centers. E 1 is a constant for given dataset. L is used to control the contrast between different cluster configuration, and is always set to 2 [60]. In Eq. (10), E c decreases with the increase of K , and D c increases with the increase of K . These three items are VOLUME 8, 2020 designed to compete with and balance each other critically to achieve proper partitioning. Hence, the maximization of PBM can lead to a partition with the least number of compact clusters and large separation between at least two clusters. In this paper, to locate a suitable solution, local and nonlocal spatial information are incorporated into PBM index. The kernel distance measure is also utilized. In order to measure all the non-dominated solutions for the whole observed image Herex i andx i represent nonlocal and local spatial information of pixel x i , respectively. [λ, 1 − λ] T is the weight vector of each non-dominated individuals. u ip , the fuzzy membership of pixel x i to cluster center v p , is calculated by: With the PBMI index, the final solution can be selected from obtained PF. Then the particular number of clusters, valid cluster centers and the balance between local and nonlocal spatial information can be obtained.

D. FUZZY CLUSTERING SEGMENTATION VIA FCM WITH SPATIAL INFORMATION
With the obtained number of clusters, fuzzy clustering segmentation with spatial information is performed on the whole observed image. In this paper, two approaches, one based on FCM (EAFCM) and another based on EA (EAFCA), are introduced to deal with fuzzy clustering segmentation. In EAFCM, the balance between nonlocal and local spatial information is controlled by weight vector of the selected individual output by the multi-objective evolutionary fuzzy clustering. Also, the initial cluster centers are the valid cluster centers of the selected individual. Let c be the number of clusters, the energy function to segment the whole observed image X = {x 1 , x 2 , . . . , x N } can be written as where v p is the pth cluster center and u ip represents the fuzzy membership of pixel x i to v p . m is the weighting parameter on each fuzzy membership, and is set to 2.x i andx i represent nonlocal and local spatial information of pixel x i , respectively. The nonlocal and local spatial information are controlled by weight vector [λ, 1 − λ] T . The detailed iterations of EAFCM can be described as follows: Step (1) Set the stop condition ε and the loop counter b = 0. Initialize cluster centers v (b) with the valid cluster centers of the selected individual of multi-objective evolutionary fuzzy clustering.
Step (2) The fuzzy membership matrix u (b) can be computed by Step (3) The cluster centers v (b+1) can be computed by Step (4) If max v (b) − v (b+1) < ε, then stop. Otherwise, set b = b + 1 and go to step (2). When the iteration stops, the label matrix of pixels based on fuzzy membership matrix can be obtained. To improve performance of segmentation, a method for correcting labels of outliers is proposed on the basis of entropy [61] and local spatial information in this paper. The detailed procedures are written in the Algorithm 3. The entropy of a pixel has a large value when the pixel is uncertainty [34], [61], and the pixels with large entropies are seen as outliers. When the entropy of a pixel is larger than mean entropy of its neighbor pixels, the pixel needs correcting labels on the basis of its neighbor pixels.

E. FUZZY CLUSTERING SEGMENTATION VIA EA WITH SPATIAL INFORMATION
Except fuzzy clustering segmentation based on FCM, another fuzzy clustering segmentation via EA (EAFCA) is proposed in this paper. In EAFCA, besides the selected individual, other individuals which have the same number of clusters obtained by the multi-objective evolutionary fuzzy clustering are utilized in the initialization. If c is the number of clusters, the fitness function of EAFCA on the whole observed image X = {x 1 , x 2 , . . . , x N } can be written as: where v p is the pth cluster center and v q is the qth cluster center. u ip represents the fuzzy membership of pixel x i to v p . m is the weighting parameter on each fuzzy membership, and is set to 2.x i andx i represent nonlocal and local spatial information of pixel x i , respectively. The nonlocal and local spatial information are controlled by weight vector [λ, 1 − λ] T . Unlike EAFCM, the weight vector [λ, 1 − λ] T is incorporated into evolutionary optimization. The example of an individual encoding is shown as Fig. 7. It can

Algorithm 3 Label Correction for Outliers
Input: The fuzzy membership matrix u and label matrix L1. The local spatial information weights w computed as Fig. 1 in Supplementary Materials. Ouput: The corrected label matrix L2.
Step 1) Calculate the entropy of each pixel by E (i) = c p=1 −u ip · log u ip , where u ip represents fuzzy membership of the ith pixel belong to the pth cluster. Then normalize the entropies by where E min and E max represent the maximum and minimum entropy respectively.
Step 2) For each pixel, based on its neighbor pixels in the 3 × 3 square window, do be seen that an individual includes cluster centers, weight vector and parameters. The parameters are utilized to control crossover operator in EAFCA, and they are updated by Eq.(4). The detailed procedures of EAFCA are described in Algorithm 4.
In EAFCA, the initial population includes individuals obtained by multi-objective evolutionary fuzzy clustering and random ones. The cluster centers and weight vector are reproduced by crossover and mutation operators. The crossover operator is defined on adaptive DE strategy. The mutation operator is defined on polynomial mutation. The repair strategy is adopted to deal with the individuals including empty cluster centers. The label correction is utilized to improve final segmentation performance. Step 1) Initialization: Step 1.1) Initialization based on outputs: If the number of non-dominated solutions with c valid clusters output by multi-objective evolutionary fuzzy clustering is not less than 0.5 × N M 3 , select 0.5 × N M 3 individuals from above non-dominated solutions as initial individuals randomly. Otherwise, all above non-dominated solutions are selected as initial individuals.

Step 1.2) Initialization based on random generation:
Generate other initial individuals randomly. Then calculate the fitness functions of all the initial individuals as Eq. (16). Step . Generate the weight of the candidate byλ i = λ i + F i · (λ r1 − λ r2 ). Then the parameters F i and CR i are updated by Eq. (4). else 10: Step 2.2) Repair: If the candidate includes empty cluster, the candidate is generated randomly. Then calculate the fitness function of the candidate.
Step 2.3) Selection: If the fitness function of candidate is better than the one of original individual, the candidate is added into the current population instead of original individual.
Step 2.4) Stop condition: If the value of the current generation number t equals to G 3 , then stop cycle and select the individual with the best fitness found so far. Otherwise, set t = t + 1 and go to step 2.
Step 3) Label correction: Obtain the label matrix of pixels based on the output individual. Then perform label correction (shown in Algorithm 3) on the label matrix and output the corrected label matrix of pixels. image pixels, the time cost by fitness computation step is O (N × N M 1 ). The time cost by the selection step is O 2 × N M 1 2 . Let G 1 be the maximum generation number. When the value of N is larger than 2 × N M 1 , the computational complexity of multi-objective evolutionary sampling is O (N × N M 1 × G 1 ). In multi-objective evolutionary fuzzy clustering, let NI be the number of sampling pixels. When the value of NI is larger than 2 × N M 2 , the computational com- where G 2 is the maximum generation number and c max is the maximum number of clusters. In fuzzy clustering segmentation based on EA with spatial information, let c represent the number of clusters. The computational complexity is O (N × N M where N M 3 is the population size and G 3 is the maximum generation number.

III. EXPERIMENTS STUDY A. EXPERIMENTS SETTING
To study the performance of EAFC, our experiments include two parts. First, the detailed analyses of each procedure in EAFC are shown. Then a comparison between EAFC and other widely used algorithms for image segmentation selected from literature is presented. Table 1 contains the names of observed images along with their types, sizes, numbers of segments and figure numbers. The synthetic images, synthetic aperture radar (SAR) images, medical images and natural images are observed in our experiments. The synthetic images SI 1 and SI 2 are corrupted by 30% Gaussian noise and 30% Salt & Pepper noise, respectively. The validity indexes to evaluate the performances of contestant algorithms quantitatively are also listed in Table 1. The accurate rate (AR) and the adjusted rand index (ARI) [62] are utilized to validate segmentation results on synthetic images. In the experiments on SAR images, SAR1, SAR2 and SAR3 are three SAR image datasets, where each dataset consists of two SAR images acquired in the same geographical area at different time. In order to measure the difference between the two SAR images, the difference image (DI), which is generated by the log-ratio operator [17] on the basis of the two SAR images, needs to be partitioned into two segments, changed and unchanged. Then the percentage correct classification (PCC) [63] and the Kappa coefficient (KC) are used to evaluate the performances of algorithms on SAR images. The values of PCC and KC fall into interval [0, 1], and large value means the segmentation result is similar to groundtruth. SAR4 is a section of a SAR image over the Aegean Sea of oil pollution near La Coruna in Northern Spain and provided by ERS-1 on 3 December 1992. It consists of two types of land cover: water (dark) and bare land (bright). To evaluate segmentation on SAR4 without groundtruth quantitatively, Vpc [64] and Vpe [65] are utilized. The maximal value of Vpc is and the minimal value of Vpe indicate the well clustering performance. In the experiments on medical images, two simulated Magnetic Resonance (SMR) images [66] are observed, SMR1 and SMR2, which are selected from the BrainWeb [67]. As shown in Fig. 16, these two simulated MR images with 5% noise and 20% intensity non-uniformity are the 89th and 92th brain region slice in the axial plane with high-resolution T1 weighted phantom, 1 mm slice thickness, respectively. The observed images include four clusters: background, cerebral spinal fluid (CSF), gray matter (GM) and white matter (WM). To calculate the statistical accuracies of the segmentation results with ground truths, the Jaccard similarity (JS) [68], [69] and Dice coefficient (DC) [70]- [72] are utilized. The larger values of JS and DC indicate that the segmentation result is more similar to ground truths. Unlike synthetic images, SAR images and SMR images, four natural images selected from Berkeley database [73] are observed without exact numbers of segments. The natural images NI 1(14037) and NI 4(124084) are corrupted by 15% and 25% Gaussian noise, respectively. NI 2(24063) and NI 3(86016) are corrupted by 10% and 15% Salt & Pepper noise, respectively. Then the probabilistic rand index (PR) [74] and variation of information (VoI) [75]

B. ANALYSES OF EAFC
This subsection gives detailed analyses on the effectiveness of multi-objective evolutionary sampling and multi-objective evolutionary fuzzy clustering in EAFC. Additionally, to save economy, some parameter settings of EAFC are discussed in details in the third section of Supplementary Materials.

1) ANALYSES OF MULTI-OBJECTIVE EVOLUTIONARY SAMPLING
In multi-objective evolutionary sampling, both the population size and maximum generation number are set to 100. The detailed analyses of setting these two parameters are shown in Fig. 2 of Supplementary Materials. Fig. 8 presents the normalized PFs obtained by multi-objective evolutionary sampling on three different observed images. The individuals, which are seen as feasible thresholds, are marked as red points. Furthermore, Fig. 8 shows the sampling results corresponding to some feasible individuals on SI 1, SAR1, SMR1 and NI 2. The sampling pixels are marked as bright pixels. It can be seen that the preserved image information is increased with the increase of number of sampling pixels. The final sampling images can preserve most regions with image details while reducing number of sampling pixels. Moreover, the normalized PFs obtained by multi-objective evolutionary sampling of other observed images are shown in Fig. 3 of Supplementary Materials. To illustrate the stability of multiobjective evolutionary sampling, box-plots of numbers of sampling pixels over 20 independent runs on the observed images are given in Fig. 3 of Supplementary Materials. The observation can be obtained that the variabilities of numbers of sampling pixels on most of observed images are relatively small.

2) ANALYSES OF MULTI-OBJECTIVE EVOLUTIONARY FUZZY CLUSTERING
In the following discussion, multi-objective evolutionary fuzzy clustering is performed on the sampling pixels of VOLUME 8, 2020 observed images. In this paper, the maximum number of clusters c max of multi-objective evolutionary fuzzy clustering is set to 10. The parameters need to be set before in multi-objective evolutionary fuzzy clustering include the mutation possibility, the population size and maximum generation number. Fig. 9 gives detailed analyses of the setting of these three parameters. The population size is set to 5 × c max , 10 × c max , 15 × c max and 20 × c max , respectively. The mutation possibility is set to 0, 1/c max , 2/c max , 3/c max , 4/c max and 5/c max , respectively. Then multi-objective evolutionary fuzzy clustering are carried out on two synthetic images over 20 independent runs. The average numbers of individuals with correct cluster number over 20 independent runs are observed. Here, the individuals with correct cluster number are named as feasible individuals, and the average numbers of feasible individuals are normalized by the population size. In Fig. 9, it can be observed that most of the independent runs achieve large numbers of feasible individuals when the generation number equals to 75. Moreover, in Fig. 9a, the independent runs on SI 1 outperform others when the population size is set to 100 and 200. Also, when the population size equals to 100, the independent runs on SI 2 achieve large numbers of feasible individuals. Taking stable performance and complexity into consideration, the population size is set to 10 × c max . In Fig. 9b, considering the performances on SI 1 and SI 2, the independent runs achieve stable performance when the mutation possibility is set to 0.1. Hence, in multi-objective evolutionary fuzzy clustering, the mutation possibility and the maximum generation number is set to 1/c max and 75, respectively.
In order to evaluate the effectiveness of multi-objective evolutionary fuzzy clustering, the PFs on the observed images are shown in Figs. 10 and 11. Fig. 10 lists the normalized PFs on SI 1, SI 2, SAR1 and SMR1. The values of PBMI index corresponding to the non-dominated solutions are also shown. The solution selection based on PBMI index can locate the individuals with correct number of clusters on both synthetic images and SAR images. The normalized PFs on other observed images are shown in Fig. 4 of Supplementary Materials. Moreover, Fig. 11 gives the normalized PFs on natural images. For each number of clusters, the individuals with largest PBMI index are located, and EAFCM is performed on the located individuals. Then the corresponding segmentation results and validity indexes (PR and VoI) are shown. It can be seen that a set of well segmentation results are obtained. In these segmentation results, image details are preserved while most of noise is removed. Furthermore, the individuals selected by PBMI index from total populations can make EAFCM acquire particular segmentation results.
Additionally, in the fuzzy clustering segmentation of EAFC, two approaches, EAFCM and EAFCA, are proposed. In EAFCA, the mutation possibility, the population size and maximum generation number need to be set. Fig. 12 presents analyses on setting of these parameters. It is performed on two synthetic images over 20 independent runs. The initializations are the results output by multi-objective evolutionary fuzzy clustering over 20 independent runs in before subsection. Let c represent the correct number of segments, the population size is set to 5×c, 10×c, 15×c and 20×c, respectively. The mutation possibility falls into [0, 1], and the value interval is set to 1/c. It can be seen that EAFCA achieves well performance when the population size and mutation possibility equals to 30 and 1/3 respectively. Thus, in EAFCA, the value of population size and mutation possibility is set to 10 × c and 1/c, where c is the cluster number. In Fig. 12e, the mean values of accuracies and fitness functions obtained by EAFCA on synthetic images at each generation are shown. Taking stable performance and time complexity into consideration, the maximum generation number of EAFCA is set to 45.

C. COMPARISONS BETWEEN EAFC AND OTHER TECHNIQUES
In the comparison of EAFC against other widely used techniques, five classical clustering algorithms, K-means [76] and four fuzzy clustering algorithms for image segmentation, are selected from literature as comparison algorithms. The comparison algorithms based on fuzzy clustering algorithms include KFCM_S1, KFCM_S2 [20], KFNDE [38] and MSFCA [48]. KFCM_S1 and KFCM_S2 are two FCM-based algorithms. To save economy, the better one of the results obtained by KFCM_S1 and KFCM_S2 on each observed image is shown. KFNDE is a single objective fuzzy clustering via DE, and MSFCA is a multi-objective fuzzy clustering via MOEA. In these four comparison algorithms, kernel distance measure is introduced in KFCM_S1, KFCM_S2 and KFNDE. Spatial information derived from images are incorporated into fuzzy clustering of KFCM_S1, KFCM_S2 and MSFCA. Additionally, KFNDE and MSFCA are two unsupervised evolutionary fuzzy clustering algorithms. Hence, these four techniques are adopted as contestant approaches in our comparison experiments. In comparisons on each observed image, KFCM_S1 and KFCM_S2 are performed with different numbers of clusters ranging from 2 to 10. The solution with highest value of PBM index is selected as the final segmentation result. The maximum iteration number of KFCM_S1 and KFCM_S2 is set equal to 7500, which is the product of the population size and maximum generation number of multi-objective evolutionary fuzzy clustering in EAFC. Then KFCM_S1 and KFCM_S2 need to be executed for 7500 iterations unless they have converged. To reduce the instability of initializations, each contestant algorithm is carried out 20 independent runs. The final results of each algorithm are calculated by the statistical mean and standard deviation of the numerical results obtained from VOLUME 8, 2020  20 independent runs. The standard deviations are reported in brackets, and the better numerical results are marked in bold. Moreover, in order to show the effectiveness of label correction in EAFC, the segmentation results obtained by EAFC without label correction are also observed. EAFC without label correction are named as EAFCMo and EAFCAo in the following experiments.

1) EXPERIMENTS ON SYNTHETIC IMAGES
In this experiment, synthetic images corrupted by different levels of Gaussian noise and Salt & Pepper noise are observed. Fig. 13 shows the segmentation results obtained by contestant algorithms on synthetic images with Gaussian noise (0,0.06) and Salt & Pepper noise (0.25). Table 2 lists the numerical results produced by contestant algorithms on synthetic images corrupted by different level of noise over 20 independent runs. The possibilities of independent runs which can obtain correct number of clusters are reported as CP. It can be seen that EAFC can locate correct number of segments in all 20 independent runs. MSFCA can locate correct number of clusters for synthetic images with Gaussian noise (0,0.03) and Salt & Pepper noise (0.25). KFNDE achieves 7 runs with correct number of clusters for SI1 with Gaussian noise (0,0.03). Then the statistical means and standard deviations of the numerical results obtained by KFNDE over these 7 runs are reported as its final results. KFCM_S1 and KFCM_S2 do not locate correct number of segments, and their cluster number are set to 3 over 20 independent runs. In Table 2, KFCM_S achieves largest values of validity indexes on SI 2 among contestant algorithms. Compared with other algorithms, the boundary of the center region obtained by KFCM_S in Fig. 13 is smoother. But it does not achieve well performance on SI 1. In Fig. 13, the segmentation result of KFCM_S on SI 1 includes much noise. The segmentation results of EAFCM and EAFCA can obtain clear edges and smooth regions while removing all most of noise. VOLUME 8, 2020  Compared with other contestant algorithms, the boundaries between two different regions obtained by EAFCA in Fig. 13 are clearer. Furthermore, in Table 2, EAFCM and EAFCA achieve larger values of validity indexes than EAFC without label correction. Fig. 14 gives the observed SAR images. Table 3 lists the numerical results produced by contestant algorithms on SAR images over 20 independent runs. Fig. 15 shows the segmentation results obtained by contestant algorithms on SAR images. In numerical results on SAR images, FP represents the area labeled with changed in segmentation result but unchanged in groundtruth, and FN represents the area labeled with unchanged in segmentation result but changed in groundtruth. It can be seen that KFCM_S1 and KFCM_S2 do not locate correct number of segments over 20 independent runs on SAR2 and SAR3. KFNDE achieves 6 runs with correct number of clusters on SAR1. MSFCA and EAFC can search correct numbers of clusters on three SAR image datasets. For SAR1 and SAR2, the segmentation results of EAFCAo and EAFCA are more similar to groundtruth. Furthermore, in Table 3, EAFCA and EAFCAo achieve better PCC and KC among contestant algorithms on three SAR image datasets. EAFCM and EAFCMo also achieve better validity indexes than other three comparison algorithms on SAR2. However, EAFCMo and EAFCM perform terribly on SAR3. In the experiments of KFCM_S1 and KFCM_S2 on SAR3, we obtain two completely different segmentation results, and both of these two segmentation results are shown in Fig. 15. The validity indexes of the better segmentation result are also given in Fig. 15. The number of better results obtained by KFCM_S over 20 independent runs is four. It can be seen that the FCM-based comparison approaches do not well in SAR3. For SAR4, KFNDE and MSFCA fail to obtain correct number of segments over 20 independent runs. The segmentation result of K-means on SAR4 is corrupted by noise in Fig. 15. In Table 3, KFCM_S1 and KFCM_S2 achieve the best performance of fuzzy clustering among contestant algorithms. In Fig. 15, the segmentation results obtained by four approaches of EAFC have clear edges and smooth regions.

3) EXPERIMENTS ON MEDICAL IMAGES
In this experiment, two SMR images, SMR1 and SMR2, are observed. Table 4 gives the statistical means and standard deviations of the metrics of CSF, GM and WM produced by the algorithms. Among 20 independent runs, EAFC can  locate correct number of segments in 18 and 14 runs for SMR1 and SMR2, repectively. Fig. 16 show the segmentation results on SMR1 and SMR2 respectively. In Table 4, KFCM_S achieves largest accuracies on SMR1 and SMR1. But compare with EAFC, the segmentation results on SMR1 and SMR2 obtained by K-means in Fig. 16 still include some noise. In Table 4, EAFCMo and EAFCAo perform best among contestant algorithms based on fuzzy clustering. Visually, the results of KFNDE are corrupted by noise seriously in Fig. 16. The segmentation results of MSFCA lose image details.
Compared with EFCA with label correction, EAECMo and EAFCAo preserve more image details. Fig. 17 shows the segmentation results obtained by contestant algorithms on natural images. Table 5 gives the numbers of segments obtained by contestant algorithms on natural images over 20 independent runs. Table 6 lists the numerical results produced by contestant algorithms on natural images over 20 independent runs. It also shows different VOLUME 8, 2020     with PBMI index. Moreover, EAFCA achieves better validity indexes than other contestant algorithms on four natural images. In Fig. 17, the segmentation results obtained by EAFC preserve clear edges and smooth regions. For NI 1, EAFCAo and EAFCA obtain uniform segment of mountain. For NI 2, except K-means and KFNDE, other contestant algorithms can remove most of noise. For NI 3 and NI 4, the segmentation results of K-means, KFNDE and KFCM_S are corrupted by noise. Compared with other algorithms, EAFCM and EAFCA obtain uniform segments and smooth boundaries. For NI 4, compared with MSFCA, EAFC obtains uniform segments of flowers and smooth boundary between petal and center. Compared with other contestant algorithms, the segmentation results obtained by EAFC with label correction can remove more noise.

5) DISCUSSION OF COMPARISONS
To illustrate the capability of EAFC, comparisons between EAFC and other widely used techniques have been performed on different types of images. The segmentation results and numerical results obtained by contestant algorithms are observed. The possibilities of numbers of segments obtained by contestant algorithms on all observed images over 20 independent runs are shown in Fig. 7 of Supplementary Materials. The t-tests between EAFC and the best method among comparison algorithms for each observed image are given in Tables 1-4 of Supplementary Materials. By above comparisons, it indicates that EAFC can obtain clear edges and smooth regions while removing noise for image segmentation without prior information about exact number of segments.

D. SUMMARY OF OBSERVATIONS
With above experiments and discussions, the observation can be obtained that: (a) Multi-objective evolutionary sampling of EAFC can preserve image information while reducing numbers of sampling pixels. It is helpful to improve the qualities and reduce the complexity of the subsequent multi-objective evolutionary fuzzy clustering with spatial information.
(b) Multi-objective evolutionary fuzzy clustering can obtain a set of feasible solutions with different numbers of clusters and balances of spatial information. The solution selection based on PBMI index can locate a suitable solution. It is helpful to improve the performances of the subsequent segmentations on whole observed images.
(c) With obtained particular number of clusters and balance of spatial information, fuzzy clustering segmentation via FCM and EA can achieve well segmentation performance. Moreover, the label correction based on entropy and local spatial information can further improve the performance of final segmentation.

IV. CONCLUSION
This paper presents an unsupervised EA-based fuzzy clustering to deal with image segmentation without prior information about exact number of segments. It contains several contributions. Firstly, a multi-objective evolutionary sampling is proposed to sample pixels adaptively. It can preserve image information while reducing numbers of sampling pixels. Secondly, a multi-objective evolutionary fuzzy clustering with local and nonlocal spatial information is proposed to perform on sampling pixels. It can search a set of feasible solutions, which include different numbers of clusters and balances of spatial information. The selection of solution based on PBMI index is introduced to locate a suitable solution. Thirdly, a FCM-based approach and an EA-based fuzzy clustering approach are introduced to deal with final image segmentation on whole observed image. Decision maker can choose one of them according to the needs. Fourthly, a label correction based on entropy and local spatial information is proposed to improve the performance of final segmentation. Additionally, this paper has given a variety of experimental results on different types of images, which can show the advantages of EAFC over other widely used approaches.
It is noteworthy that a set of feasible solutions are obtained by multi-objective evolutionary fuzzy clustering with spatial information in EAFC. Although selection of solution based on PBMI index can locate a suitable solution with particular number of clusters and balance of spatial information, there still exist several other feasible solutions. This phenomenon particularly appears in the segmentation on images with a wealth of information. In this regard, we will investigate to obtain solutions from PFs with a robust way, and incorporate useful information of Pareto solutions into the subsequent fuzzy clustering segmentation. Moreover, only the gray level information is considered in EAFC. In our future work, other useful information, such as color information and texture information, will be taken into consideration to apply EAFC to solve complex problems in image understanding fields. VOLUME 8, 2020