Multi-Exemplar Particle Swarm Optimization

PSO and its variants have proven to be useful algorithms for tackling a wide range of optimization problems in recent decades. However, PSO and most of its variants only consider the influences caused by global best position and personal historical best position. Such a single way of influencing often leads to an issue of insufficient diversity of population, and further makes the algorithms be prone to falling into local optimum. In this paper, we propose a multi-exemplar particle swarm optimization (MEPSO) to deal with this issue. Specifically, each particle will choose the global best particle and the best companion particle as its exemplars, which brings more useful knowledge for particle update. To further describe the influences with respect to different exemplars, we define two influence coefficients inspired by mechanics. Such influence coefficients ensure that the best current experience is shared while enrich the diversity of population. Moreover, in the light of the distance between each particle and its best companion particle on each dimension, a variable-scale search is given in this paper to enhance the overall convergence ability. To verify the effectiveness of our algorithm, we conduct abundant experiments on all functions of CEC2013 test suite. The experimental results show that MEPSO performs better than 14 competitors in terms of comprehensive performance and achieves state-of-the-art results.


I. INTRODUCTION
Inspired by swarm behaviors of animals in nature, Particle Swarm Optimization (PSO), as one of representative swarm intelligence algorithms, was proposed by Kennedy and Eberhart in 1995 [1]. Due to its strong universality, PSO has attracted extensive attention in recent decades. As PSO is easy to put into practice, it has been successfully applied to various fields such as machine design [2], [3], neural networks [4], [5], path planning [6], [7] and etc.
In original PSO, two acceleration coefficients with respect to the influences of global best position and personal historical best position play a crucial role to determine the search trajectory of particle and control the search ability. To further balance the global and local search, Shi and Eberhart proposed a canonical PSO with a new parameter named inertia weight [8]. Then they conducted an empirical study on the linearly decreasing inertia weight from 0.9 to 0.4, considering the extent and accuracy of search in some extent [9]. Currently, the common methods for updating the inertia weight The associate editor coordinating the review of this manuscript and approving it for publication was Xiangtao Li . still follow the linearly decreasing rule, aiming to meet search requirements in different evolutionary stages [10], [11]. Benefited from the idea of time-varying inertia weight, a series of PSO variants with time-varying acceleration coefficients were set forth [12], [13]. In addition, the topological adjustments based on iteration number enhance the performance of PSO in some extent [14], [15]. However, these improvements just considering iteration number to adjust PSO may result in the lack of wisdom for particles, especially in the face of complex tasks. Moreover, at the end of search, the global search ability will be lost. Differing from the improvements just by using iteration number, some PSO variants employ fitness to adjust parameters [16], [17] and learning models [18], [19]. These fitness-based improvements assign effective controls to each particle, forcing the particles in swarm to play different roles in dealing with complex tasks.
In this paper, we propose a novel multi-exemplar particle swarm optimization to expand the diversity of the population and enhance the convergence ability. Specifically, for each particle we first select the global best particle and the best companion particle as its exemplars, which brings more useful knowledge for particle update. To describe the influences with respect to different exemplars, we define two influence coefficients inspired by mechanics. Such influence coefficients ensure that the best current experience is shared while enrich the diversity of population. Moreover, in terms of the distance between each particle and its best companion particle on each dimension, the variable-scale search is given in this paper to enhance the overall convergence ability.
The rest of this paper is organized as follows. Section II introduces the canonical PSO and its variants. The details of MEPSO are described in Section III. In Section IV, we conduct abundant experiments by comparing MEPSO with the canonical PSO, 10 state-of-art PSO variants and 3 representative non-PSO optimization algorithms. The experimental results show that MEPSO achieves better comprehensive performance. Finally, we conclude the study in Section V.

II. RELATED WORK A. CANONICAL PSO
In the canonical PSO, the position of each particle represents a possible solution in the solution space while the velocity of each particle represents the motive of movement. In a D dimensional space, suppose the velocity vector and position vector of the ith particle are represented by . , x D i ) respectively, they will be updated in each iteration according to Eqs. (1) and ( where t and t + 1 represent the tth and the (t + 1)th generations respectively, d refers to the dth dimension of each particle. r 1 and r 2 are two independent random variables following uniform distribution in range [0, 1]. c 1 and c 2 are two acceleration coefficients, and w is an inertia weight. pbest i = (pbest 1 i , pbest 2 i , . . . , pbest D i ) represents the personal best position achieved so far for the ith particle, and gbest = (gbest 1 , gbest 2 , . . . , gbest D ) is the global best position found in the whole swarm.
In canonical PSO, particles are first initialized with a population of random positions in the solution space. Then, each particle updates its velocity and position based on Eqs. (1) and (2) in the way of iteration. Note that, during the search process, the personal best fitness and personal best position of each particle are maintained. Once a particle finds a better solution, the corresponding fitness and position are utilized to update the personal best fitness and the personal best position respectively. Similarly, the swarm will update global best position and global best fitness when a better global best solution is found.

B. PSO VARIANTS BASED ON VARIOUS LEARING MODELS
To enhance the learning ability of swarm, a series of PSO variants were developed based on different topology spaces and learning approaches. [20] that a smaller neighborhood is proper for handling multimodal problems while a larger neighborhood can perform much better on unimodal problems. Thus, to overcome the shortcoming of fixed neighborhoods, some dynamic neighborhood update strategies were studied for finding the suitable topology of neighborhood. For instance, Liang et al. [21] proposed a Comprehensive Learning Particles Swarm Optimization (CLPSO) to find high quality solutions for multimodal functions. In CLPSO, each particle is allowed to learn from the personal best position of a randomly selected particle. In comparison with the canonical PSO, the variants based on the given topology spaces maintain the swarm diversity and avoid premature convergence. However, the individuals in the same neighborhood usually have similar information, which narrows the information diffusion within the population. In addition, frequent topology adjustment will inevitably increase extra computational cost.

It has been verified by Kennedy and Mendes
On the other hand, Xu et al. [22] try to overcome the phenomena of ''two steps forward, one step back''. They focused on the dimensional information of global best particle and proposed a dimensional learning approach. In such an approach, particles learn effective information from each dimension of the global best particle, aiming to improve the convergence ability. However, all particles in swarm overly depend on the experience of the global best particle, causing an issue of insufficient diversity in population. To deal with this issue, Xia et al. proposed a XPSO with a forgetting ability [23]. It reduces the amount of information obtained from the global best particle, aiming to enrich the population diversity. However, the convergence ability of XPSO will be reduced just by enriching the diversity of population.
In contrast, MEPSO is proposed in this paper to expand the diversity of the population and enhance the convergence ability. Specifically, the multi-exemplar learning based on influence coefficient is first given mainly for enriching the diversity of population. Moreover, a variable-scale search is proposed based on the distance between each particle and its best companion particle on each dimension to improve the overall convergence ability of MEPSO.

III. MULTI-EXEMPLAR PARTICLE SWARM OPTIMIZATION (MEPSO) A. MULTI-EXEMPLAR MODEL
In the canonical PSO, the global best particle is regarded as the unique elite of ''social learning'' part. However, just learning from the global best particle will force the whole population to approach that particle, which may reduce the diversity of the swarm. In this study, we propose a multi-exemplar model, in which the global best particle is considered as one of the exemplars for each particle. In fact, the particle swarm can be seen as the human society, in which learning from multiple exemplars is more effective, and such a learning mode has become a common phenomenon in human society. It has been verified by many studies that individual with multi-exemplars can obtain a better learning capability [24], [25]. Whereas, learning from too many exemplars will bring learning into chaos, influencing the efficiency of learning. Thus, besides the global best particle (gbest), best companion particle (cbest) is also considered as one of the exemplars in multi-exemplar model. Specifically, the companion particles are randomly selected from the current swarm. Differing from selecting the leaning object in the same neighborhood topology, in this paper each particle selects its companions randomly, which can bring manifold information.

B. MULTI-EXEMPLAR LEARNING BASED ON INFLUENCE COEFFICIENT
In recent years, researchers have attempted to utilize gravity model to explain various sociology phenomena from the perspective of mechanics [26]. In this paper, to further describe the influences with respect to different exemplars, we define two influence coefficients inspired by mechanics. Firstly, to clearly reflect the impact of gbest and pbest, we remove the independent random variables r 1 and r 2 , the acceleration coefficients c 1 and c 2 , and inertia weight w in Eq. (1). The simplified velocity update formula is given as As seen from Eq. (3), it is obvious that the motion of each particle will be affected by gbest and pbest. According to Newton's first law of motion, force is the unique factor to change the motion of object. Thus, the two vectors, i.e., can be seen as two forces. That is because they affect the motion of each particle. Since pbest reflects the learning experience of each individual, and a series of pbests in population can be regarded as the swarm diversity. Differing from pbest, gbest reflects the best learning experience in population, and it affects all individuals as a primary factor. Based on Eq. (3), the farther an individual is from gbest, the greater the force will be, causing a decline of swarm diversity. To address this issue, we define a global influence coefficient ggr i and a companion influence coefficient cgr i to describe the influences of the gbest and cbest on the ith particle, respectively, from the perspective of mechanics where dist g max is the distance between the global best particle and its farthest particle. dist c max,i represents the distance between the best companion and its farthest companion. Subsequently, dist g max and dist c max,i are respectively calculated by In Eqs. (4) and (5), dist_g i and dist_c i represent the distances between the ith particle and its two exemplars. Subsequently, dist_g i and dist_c i are respectively calculated by Because dist g max is the upper limit of dist_g i , based on Eq. (4), it is obvious that ggr i belongs to the range of [0.5,1]. In addition, as the ith particle and its best companion particle choose their companions separately, There is no direct relation between dist_c i and dist c max,i . Thus, based on Eq. (5) cgr i is in the interval of (0,1). In MEPSO, the global best particle is used to share the best experience of the swarm to each particle. Thus, we set ggr i with a relatively large value (0.5 ≤ ggr i ≤ 1) to ensure that the best experience can be learned. Differing from the global best particle, the best companion particle is utilized to expand the diversity of swarm. Thus, we set the range of cgr i as 0 < cgr i < 1. Based on Eq. (5), the smaller the distance between the ith particle and its best companion particle, the greater the cgr i . Moreover, in contrast to the canonical PSO, ggr i is utilized reduce the influence of the global best particle to some extent and makes the best experience of swarm can be shared. More importantly, we use cgr i to enrich the diversity of swarm. After defining ggr i and cgr i , the velocity of each particle is updated as (10) where cbest i represents the personal best position of the best companion particle. Similar to Eq. (1), r 1 , r 2 and r 3 are three independent random variables in range [0, 1]. c 1 c 2 and c 3 are three acceleration coefficients.

C. VARIABLE-SCALE SEARCH BASED ON BEST COMPANION
In this paper, the multi-exemplar learning based on influent coefficient is firstly proposed mainly for expanding the diversity of swarm, which increases the possibility of MEPSO converging to global optimum. To make the algorithm converge to the global optimum, based on the distance between each particle and its best companion particle on each dimension, a variable-scale search is further proposed to improve the overall convergence ability of MEPSO.
In the research field of swarm intelligence, crisscross optimization algorithm (CSO) is regarded as a ''catalytic agent'' that has the powerful ability to search for desired solution with high quality. In CSO, each pair of randomly selected particles will implement crisscross search on every dimension. It accelerates the global convergence of PSO without sacrificing swarm diversity [27], [28]. Inspired by CSO, a variable-scale search is proposed in this section to enhance the overall convergence ability of MEPSO. Differing from CSO, the variable-scale search only considers the impact of the best companion particle for each particle, and it does not modify the position of the best companion. Upon updating the velocity and position of each particle in the light of Eqs. (10) and (2), the variable-scale search is implemented based on distance between particle and its best companion on each dimension. The variable-scale search is defined by where r stands for a random variable following the uniform distribution of [−1, 1]. In addition, a competitive mechanism is adopted to guarantee a better position. Namely, x_tmp can survive only when it obtains a better fitness. As seen from Eq. (11), the variable-scale search is proposed to control search scale of each particle with iteration. Specifically, at the beginning of search, because the population has a rich diversity, the distance between each particle and its best companion is relatively large, and each particle will perform extensive search to obtain potential solutions with high quality. Then, with the increase of iteration, the population tends to converge, and the distance becomes relatively small. In contrast, each particle will implement local search to refine the high quality solution. Thus, each particle performs the variable-scale search from extensive range to the range around itself, which can enhance the overall convergence of MEPSO.

D. ANALYSIS OF MEPSO 1) COMPLEXITY ANALYSIS
The computational cost of the canonical PSO algorithm includes initialization, evaluation, and velocity and position In this paper, each particle will update its companions and calculate the influence coefficients only when the personal best position is not updated within consecutive T generations. In the worst-case, all particles in population reselect their companions, which causes the time complexity of O(m). Meanwhile, in such a case, we calculate the distance between each particle and its exemplars, and this step is performed for whole swarm resulting in the time complexity of O(mn). Based on the distances obtained, the time complexity for calculating ggr i of whole swarm is O(m), because the time In addition, since each particle employs variable-scale search on every dimension, the time complexity of whole swarm to perform this operation is O(mn). To summarize, on basis of above analysis, the final time complexity of MEPSO is also O(mn), which maintains the same order of magnitude to that of the canonical PSO.

2) MOVEMENT ANALYSIS
Based on the velocity and position update equations of PSO, i.e., Eqs. (1) and (2), it is obvious that each particle moves towards the global best position independently as shown in Figure. 1 (a). In MEPSO, best companion particle as well as global best particle determines the new position of each particle. As shown in Figure. 1 (b), with the multi-exemplar learning based on influent coefficient, the lagged particle, which is far away from the global best particle, receives relatively less effect from the global best particle in comparison with the effect it received in Figure. 1 (a). Namely, such a lagged particle prefers to carry local search, contributing to improve population diversity. In addition, each particle will perform the variable-scale search in terms of distance between particle and its best companion on each dimension, resulting in different search scales in different dimension, which is shown in Figure. 1 (c).

E. FLOW-PATH OF MEPSO
To describe MEPSO clearly, the pseudo-code of MEPSO is shown in Algorithm 1, and the flow chart of MEPSO is shown in Figure. 2. (1 ≤ i ≤ N ); 02: Evaluate fitness i of each particle; 03: Initialize personal best position pbest i and personal best fitness pbestfit i ; 04: Initialize global best position gbest and global best fitness gbestfit; 05: Initialize companion particles for each particle; 06: Calculate distance between each particle and its exemplars; 07: Calculate global influence coefficientggr i and companion influence coefficients cgr i ; 08: for FEs = 1 : MaxFEs 09: for i = 1 : N 10:

Algorithm 1 Pseudo-Code of MEPSO
if flag i == T then 11: Reselect companion particles; 12: Calculate distance between ith particle and its exemplars; 13: Calculate ggr i and cgr i by Eqs. (4)  In this section, to demonstrate the effectiveness of MEPSO, the benchmark CEC2013 test suite is adopted in our    Table 1.

A. PARAMETER SETTING
In our experiments, the dimension D of solution space and the maximum number of function evaluation MaxFEs are set as 30 and 300000 respectively in the light of the stipulation of 2013 IEEE Congress on Evolutionary Computation (CEC) [29]. The population size and the maximum number of generations for reselecting companion particles T are set as 50 and 5 empirically. Additionally, to determine the three acceleration coefficients, i.e., c 1 , c 2 and c 3 in Eq.(10), the self-adapting acceleration coefficient adjustment method proposed by Xia et al. [23] is adopted in this paper to control these three coefficients. Specifically, we first initialize these three coefficients of each particle using normal distribution N (µ i , 0.1 2 ), where µ i represents the expectation of N (µ i , 0.1 2 ), and 0.1 stands for the standard deviation. According to the characteristic of the normal distribution, the acceleration coefficients of each particle are perturbed around µ i . Moreover, during the search process, we adjust µ i of each particle in terms of the experience of its current exemplars W. Song, Z. Hua: MEPSO where η represents the learning rate, and η = 0.1 has been proven to be effective in [23]. c exemplar i is the ith acceleration coefficient of the exemplars, and avg(< c exemplar i >) represents the mean acceleration coefficient of exemplars. In each generation, c i of each particle obeys N (µ i , 0.1 2 ). Namely, c i perturbs around the current µ i . Thus, once the initialized µ i is given in prior, the current µ i can be updated based on Eq. (12), and c i can be further generated with N (µ i , 0.1 2 ). In our experiment, we initialize the same µ i for all particles in swarm, i.e., µ 1 = µ 2 = µ 3 = µ, based on the method given by Xia et al. [23], and test different µ on all 28 functions of CEC2013 test suite. Figure 3 shows the fitness of six functions with different µ.
As seen from Figure 3, on f 6 , f 10 , f 12 and f 28 , MEPSO obtains its best performance with the lowest fitness when µ is 1.0. In addition, on f 1 and f 22, MEPSO obtains the best performance when µ is 1.4 or 1.  MEPSO-IC. In addition, on f 5 and f 9 , the final fitness of MEPSO is the same to that of MEPSO-IC. On f 26 , the final fitness of MEPSO is the same to that of MEPSO-VS. Namely, for most functions MEPSO performs better than MEPSO-VS and MEPSO-IC while MEPSO performs as same as MEPSO-VS or MEPSO-IC for the other functions. That is because at the beginning of search, an extensive search is performed for MEPSO, contributing to finding several solutions with high quality. With the increase of iteration, population tends to converge. In contrast, each particle will implement local search to refine the high quality solution. Moreover, MEPSO can find a better solution even at the end of the search, e.g., on f 13 and f 16 , because IC expands the swarm diversity of MEPSO.

C. COMPARISONS OF MEPSO WITH PSO-BASED ALGORITHMS
In this section, the canonical PSO and 10 state-of-the-art PSO variants, i.e., Frankenstein's PSO (F-PSO) [30], OLPSO [31], DEPSO [32], PSODDS [33], CCPSO-ISM [34], SRPSO [17], HCLPSO [35], GLPSO [36], EPSO [37], and XPSO [23], are selected to compare with MEPSO in terms of the mean of fitness. The parameter settings of these algorithms are listed in Table 2, and they are set based on the values given in the original papers. To obtain the mean of fitness, each algorithm is performed 30 independent runs on each function. Table 3 illustrates the mean of fitness with respect to these 12 algorithms. The best results among these 12 algorithms on each function are marked in bold and underline type. To further illustrate the average rank of these 12 algorithms and the performance differences between MEPSO and the other 11 PSO-based algorithms, the Friedman test with significance level α = 0.05 is adopted in this section, and the comparing results are listed in Table 4 and Table 5.
As seen from Table 3, for the 5 unimodal functions (f 1 -f 5 ), MEPSO achieves the best performance with the lowest fitness on f 4 . OLPSO, PSODDS, XPSO and GLPSO obtain the best performance on f 1 , f 2 , f 3 and f 5 respectively. Meanwhile, in terms of the Friedman test results in Table 4, we can see that MEPSO obtains the best performance with rank of 3.40 on these 5 unimodal functions, which is better than XPSO and EPSO with ranks of 4.20 and 4.60 respectively.
In Table 3, f 6 -f 20 belong to the multimodal functions of CEC2013 test suite. As shown in Table 3, MEPSO achieves the best performance with the lowest fitness on f 8 , f 10 , f 13 and f 18 . In addition, we can see from Table 4 that GLPSO obtains the best performance with rank of 3.73 on these 15 multimodal functions. The rank of MEPSO is just not as good as that of GLPSO, but it is better than those of the other 10 algorithms.
In addition, for the 8 composition functions (f 21 -f 28 ), we can also see from Table 3 that MEPSO exhibits the best performance on f 24 and f 28 . Similarly, as seen from Table 4, MEPSO performs better than the other 11 PSO-based VOLUME 8, 2020 algorithms with rank of 3.13 on these 8 composition functions. In summary, MEPSO achieves the best comprehensive performance with the rank of 3.70 on all CEC2013 functions.
As seen from Table 5, we use P-value and chi-square statistic on basis of Friedman test (significance level α = 0.05) to indicate the performance differences between MEPSO and the other 11 PSO-based algorithms. On basis of the definition of P-value, if P-value is less than or equals to 0.05, such case means that it has a significant difference for each pair of compared algorithms. we can see from Table 5 that, MEPSO is better than PSO, F-PSO, DEPSO, OLPSO, PSODDS, CCPSO-ISM and SRPSO significantly on the 28 functions based on P-value. In addition, the chisquare statistic is utilized to evaluate the difference between each pair of compared algorithms. Specifically, the larger the chi-square statistic is, the greater the difference between the compared algorithms. On the 28 functions, in terms of chi-square statistic, MEPSO is much different from PSO, DEPSO, SRPSO, PSODDS, F-PSO, OLPSO and CCPSO-ISM when comparing it with the other 4 algorithms.
Moreover, in Table 6 we compare the real time of these 12 algorithms. As seen from Table 6, on f 1 -f 6 and f 10 , the real time of MEPSO is just longer than those of PSO and OLPSO. However, it is faster than other 9 algorithms. For other functions, MEPSO is relatively longer. That is because the extra time consumption mainly occurs for IC calculation, and some complex functions, e.g., f 18 and f 25 , need frequent IC calculation resulting extra time consumptions in comparison with simple functions. Fortunately, MEPSO achieves the best comprehensive performance when comparing the other 11 algorithms in Table 4. In addition, MEPSO almost maintains the same order of magnitude as real time of the other 11 algorithms from Table 6. Furthermore, the time complexity analysis in Section IV can also verify such a result.

D. COMPARISONS OF MEPSO WITH NON-PSO OPTIMIZATION ALGORITHMS
As discussed in table IV, MEPSO performs better than the other 11 PSO-based algorithms in terms of comprehensive performance. In this section MEPSO is further compared with 3 non-PSO optimization algorithms, i.e., modified artificial bee colony algorithm with neighborhood search (MABC-NS) [38], evolution strategy with covariance matrix adaptation (CMAES) [39], and population-based incremental learning (PBILc) [40]. Each algorithm is also performed 30 independent runs on every CEC2013 test suite function. The mean fitness and rank for each of these 4 algorithms are shown in Table 7. Note that, the rank is also calculated based on Friedman test. We can see from Table 7 that, in the light of the performance of MEPSO on the 28 functions, it achieves the best rank with value 2.05, which is better than the ranks of PBILc, MABC-NS and CMAES with values 2.41, 2.48 and 3.05 respectively.
Moreover, in Table 8 we adopt P-value and chi-square statistic on basis of Friedman test (significance level α = 0.05) to indicate the performance differences between MEPSO and the 3 non-PSO optimization algorithms. As seen from Table 8, MEPSO is better than CMAES significantly on the 28 functions based on P-value. In addition, in terms of chi-square statistic, MEPSO is much different from CMAES, MABC-NS when comparing it with PBILc.

V. CONCLUSION AND FUTURE WORK
In this paper, we propose a multi-exemplar particle swarm optimization, named MEPSO, to improve the diversity and convergence. Specifically, the global best particle and best companion of each particle are both considered as its exemplars, which brings more useful knowledge for particle update. To further describe the influences with respect to different exemplars, we define two influence coefficients inspired by mechanics. Such influence coefficients ensure that the best current experience is shared while enrich the diversity of population. Moreover, the variable-scale search is given based on the distance between particle and its best companion particles to enhance the overall convergence ability. To verify the effectiveness of our algorithm, the canonical PSO, 10 state-of-art PSO variants and 3 representative non-PSO optimization algorithms are compared with MEPSO on CEC2013 test suite. Experimental results demonstrate that MEPSO performs better than the 14 competitors in terms of comprehensive performance.
Currently, we are focusing on the following extensions to the proposed algorithm. First, to further enhance the performance of MEPSO, we try to combine MEPSO with other well performed optimization algorithms, considering their advantages and disadvantages for combination. Moreover, to address better applications of our algorithm and realize it as an effective optimizer in depth, we are now trying to develop its applications to discrete optimization and multi-objective optimization.