Landscape-Based Similarity Check Strategy for Dynamic Optimization Problems

In many real-world decision problems, there are specific objective, decision variables, conditions, data and/or parameters that may vary over time. When these problems are solved through an optimization process, they are generally known as dynamic optimization. Dynamic optimization is a challenging research topic as the optimal solution usually moves with any change in the problem environment. In locating the optimal point in any optimization problems, the effectiveness of the search process is influenced by the nature of the problem landscape. In order to improve the effectiveness of the search process, in this article, a new approach is developed by integrating a landscape-based strategy with appropriately designed evolutionary algorithms for solving dynamic problems. The proposed approach is named as Landscape Influenced Dynamic Optimization Algorithm (LIDOA). LIDOA checks the similarity before and after the changed environment which is then used as an input to guide the evolutionary search process. The dynamic benchmark functions from IEEE-CEC2009 are solved using LIDOA. LIDOA was able to enhance the performance of the evolutionary algorithms in their adaptation to dynamic changes, which in turn enhanced their ability to attain better results.


I. INTRODUCTION
The interest in optimization under dynamic environments have been growing over the years due to the fact that many real-world decisions and design problems belong to this category, such as trade market prediction, meteorological forecast, robotics motion control, computational protein design [1], intelligent watermarking in document images [2], and vehicle routing problems [3]. Common applications of dynamic optimization problems include: power system scheduling, production scheduling, and moving objects scheduling. In power system scheduling, the load on the electricity network and the generation from renewable resources changes with time. With the arrival of new jobs and experiencing defective materials and machine breakdowns, a scheduling problem changes with time. Supplying fuel to patrolling boats by a ship or aircraft, or delivering hazardous materials to multiple moving units by a robot are also typical real-world dynamic problems [4].
In general terms, optimization problems that change over time are called dynamic optimization problems (DOPs), The associate editor coordinating the review of this manuscript and approving it for publication was Xiangtao Li . which can be mathematically represented as [5]: where x is decision variable(s), φ is parameter(s), and t is time. In DOPs, the objective function, constraints, data, and/or environmental conditions may change over time due to many different reasons. Therefore, the goal of DOPs is not only to find an optimal solution, but also to closely track the change of solutions over time [6]. In a dynamic environment, the key points for solving such problems are adaptability and efficiency. In fact, one of the main challenges in dealing with DOPs lies in reducing the computational cost in locating the new solution when there is a change in environment. In other words, a quick response to the changes and search for the optima in diverse environments are the basic requirements for methods toward these problems.
Over the last two decades, a good number of different evolutionary algorithms (EAs) have been proposed to solve DOPs. Compared with the conventional optimization methods, EAs are mostly simple to conduct the search operations and impose no requirement of specific mathematical properties, thus being more robust and flexible. For employment of EAs in DOPs, two key points are considered in this article: the optimization problems to be solved and the class of algorithms to be designed, where the key element of the former is landscape analysis and the latter frequently leads to type of the algorithm, such as framework imitating natural processes, infinite population models, ideas based on thermodynamics and statistical mechanics [7].
Landscapes analysis is one of the fundamental approaches to understand the geometry of the search space that can provide important information for the development of a search algorithm [1]. The simple landscape measures such as the ruggedness, peak number, height, separation, and clustering in the solution space, can be used to reflect the changes in the landscapes for dynamic problems [8]. These landscape indicators can be a useful component in designing the algorithm for DOPs. However, most existing landscape measures are based on static optimization problems, and have limited effects on improving the algorithm's performance.
In this article, the Landscape Influenced Dynamic Optimization Algorithm (LIDOA) is proposed, where a landscape-based strategy is integrated with appropriately designed EAs for solving DOPs. First, some points are selected from the search space based on the sequence-based deterministic initialization technique [9] and their fitness values are calculated. These points are then used to determine the landscape-based similarity indicators for comparing similarities in different environments. In this case, it is assumed that the search boundary will not change but the landscape will vary over time. For the convenience of comparisons, the landscape measures are preserved for different environments. When the fitness landscape changes, the features of the current landscape are compared with those of the previous ones based on the landscape measures. Depending on the level of similarities, the individuals from the past environments will be selected for the current one and this information would help to guide the search process in the current environment. Several classical EAs, including genetic algorithm (GA), self-adaptive differential evolution algorithm (jDE) [10] and covariance matrix adaptation evolution strategy (CMA-ES) [11], are employed to examine the efficiency of LIDOA.
The remainder of this article is organized as follows. An overview of EAs for DOPs and the basic theory of the methods employed in this article are presented in Section II and section III, respectively. The proposed LIDOA is detailed in Section IV. The analysis of the experimental results and the conclusion are provided in Sections V and VI, respectively.

II. BACKGROUND
This section reviews the popular evolutionary algorithm (EA) schemes that have been designed for DOPs. Compared to the stationary optimization problems, where the goal is to find the global optimum as fast as possible, DOPs require EAs to track the trajectory of changing optima in the search space. One of the biggest challenges that DOPs have in comparison to traditional EAs is the convergence issue: once converged, it's hard to escape from an old optimum [7]. Therefore, researchers have developed several approaches into EAs to enhance their performance for DOPs, including diversity, memory, prediction and multi-population schemes.
Diversity schemes refer to introducing diversity in population after a change happens, or maintaining diversity through the search process without explicitly detecting changes. A diversity scheme can be achieved by adapting search operators [3], introducing randomised individuals, distributing sentinel individuals [12], as well as explicitly keeping individuals from getting too close to one another [7].
Memory schemes preserve past solutions. In many cases, the best solutions from the previous landscapes are kept in memory for possible population initialization when a landscape changes [13]. The intermediate solutions and solutions from the latest problem landscapes can also comprise the memory [13], [14].
Prediction schemes can be used to track the moving optima [15], [16], locations that individuals should be re-initialized to [17], as well as the time when the next change will occur and which possible environments will appear in the next change [18].
Multi-population schemes concurrently maintain multiple sub-populations where each of them handles a different task. In this case, the sub-populations can use a single algorithm or multiple algorithms [19]. In order to improve the performance of multi-population schemes, additional strategies such as the quantum strategy [20] and scheduling approaches were employed [21].
Each of the above-mentioned schemes has its own advantages and disadvantages. Therefore, combinations of the above-mentioned schemes have been explored. The most popular method consists of a memory scheme and a multipopulation scheme, where various strategies have been proposed [10], [22], [23]. Another commonly used method is the combination of a memory scheme and a diversity scheme, as was proposed in [24], [25]. In addition, prediction based on memory and/or multi-population approaches have been studied on multiple EAs [26], [27].
Despite that many methods have been proposed to deal with DOPs, very few of them consider using landscape information to direct the search process, and even fewer attempts have been made on what landscape information can be used for and how to use it. Among the works that employ landscape knowledge, they either learn from only the latest problem landscape, or seldom use the landscape information to direct the current search.
Vellasques et al. [2] proposed a series of memory-based dynamic PSO to deal with recurrent optimization problems. The basic idea is to build a comprehensive model of the stream of optimization problems by using representative problem instances [31]. In that, the optimal swarm obtained in each problem landscape is archived in two memories and re-evaluated when landscape changes. The re-evaluated solutions are then compared with the memory using a Kolmogorov-Smirnov statistical test to find a historical landscape that is similar to the current one. If a similar historical landscape is found, the optimal swarm from that landscape is used directly to save computational time. Otherwise, L-best dynamic PSO is employed with the optimal swarm of the latest landscape as an initial population. In [31], they modified the algorithm by introducing a Gaussian mixture model built on all of the solutions to represent the problem landscape. To be specific, at the end of each environment, a short term memory stores the best individual in the last environment and the density estimation generated by the Gaussian mixture model. Then at the beginning of the changed environment, a population is re-sampled based on the density estimation and is re-evaluated. The distribution similarity between the re-evaluated population and that in the last environment is compared by using a Kolmogorov-Smirnov test, which will determine if the best solution from the last environment will be re-used. In addition, a long term memory stores the information of multiple historical environments is kept for the same purpose. Then in [32], they further modified the algorithm by using a surrogated-based PSO as an optimization method, with a portion of one population using exact fitness evaluations and the other one using a regression model for fitness.
It is worth noting that although this algorithm employs a similarity check based on the problem landscape, it is different from LIDOA in several aspects: 1) the information used to represent the problem landscape and the way to compare similarity are quite different, where the sequence-based deterministic initialization [9] is employed to generate the problem landscape in this article; 2) the optimal swarm from the similar historical landscape is used directly without being further optimized; 3) the algorithm mainly focuses on the reduction of computational cost instead of improving the fitness error, which is the other way around in this article; 4) only recurrent optimization problems, especially the document images and watermarking systems, are examined, while different functions and change types are considered in this article. These aspects are also significant components that make the proposed similarity check strategy efficient.
In order to quickly find or closely track the optima in a changing environment, it is important to efficiently use information from the current problem environment and previous environments. Although EAs are reasonably efficient for solving DOPs in each environment separately, it is still an open research issue for developing a framework that can fully use the environment information and generally benefit EAs in dealing with DOPs.

A. FITNESS LANDSCAPE MEASURES
The concept of fitness landscapes originated from theoretical biology in 1932 [33]. In evolutionary computation, fitness landscapes are used to draw an analogy with real landscapes to gain a better understanding of how and where algorithms operate [34], [35]. Within this generic model, a location in the landscape is a solution to a given problem, the elevation captures how good that solution is, and solutions that are similar in nature are typically placed close to each other [36].
Fitness landscapes play a vital role in understanding the search space of optimization problems and in guiding the search process to reach an optimum solution. Despite the attempts that have been made in using landscape information to solve DOPs, there are still many issues to be addressed in terms of what characteristics of problem landscapes are useful and how to use them. In a continuously changing environment, the similarities between two problem landscapes indicate certain resemblances between fitness distributions. In this article, fitness landscape measures are used to check the similarities between different problem environments. The purpose of this similarity check is to identify the extent of changes between environments which can then be used to decide the search strategy in solving the problem in subsequent environments. Any similarity found will provide an opportunity to re-use some of the preserved solutions and any dissimilarity will provide a clue of how to generate the population and conduct the search process more effectively. Therefore, such an action will help the algorithm to find a solution with lower computational time, hence improving the performance of the algorithm.
As previously mentioned, the main goal of the similarity check strategy is to find the most similar historical landscape to the current one, so that historical information can be re-used to guide the search process. This strategy is conducted under the assumption that overlapping areas exist among different landscapes. To do so, in this article, several landscape measures are employed, including the Euclidean distance (1-norm) between two landscapes, statistic kurtosis, statistic skewness, as well as the domination landscape matrix [37].
In the Euclidean-based landscape measure, the Euclidean distance between the landscape fitness of the current landscape and that of previous ones are calculated. The historical landscape with the shortest distance is taken as the most similar landscape. Kurtosis and skewness are measures to the symmetry and tailedness of distribution of a landscape fitness, respectively. There are different methods to calculate the two measures, with the following equations used in this article: where Y i indicates the fitness of each solution in the landscape population, Y is the mean value of all the Y i , σ is the corresponding standard deviation, and N is the population size. The domination landscape (DL) describes the domination relationship among solutions, which can be used in determining the parameters for fitness scaling, the EA's convergence robustness against fitness noise, as well as classifying the optimization problems [37]. In this article, the domination landscape matrix is established based on the performance of each solution. The domination relationship between n solutions can be set as a n-by-n matrix: where the element in the ith row and jth column is DL i,j = 1 when the ith solution is better than the jth solution, i.e., the ith solution dominates the jth solution, otherwise DL i,j = 0. The Hamming distance between the DL matrix in the current landscape and that in each of the previous landscapes are then calculated, and the most similar historical landscape is accordingly selected.

B. CLASSICAL EVOLUTIONARY ALGORITHMS
EAs have been proved to be effective in dealing with DOPs. This article uses three classical EAs, i.e., GA, jDE and CMA-ES, to examine the performance of the landscape-based strategy. The working mechanisms of the three algorithms are introduced in this section.

1) GA
GA is a randomized optimization algorithm inspired by natural selection and natural genetics [38]. It starts with initializing a population of individuals, where each individual represents a solution within the search range. Then the fitness of the initialized population is evaluated, according to which certain individuals are selected as parent individuals to generate new individuals for the next generation. In most cases, an individual with better fitness is more likely to be selected. After selection, crossover and mutation are conducted on the parent individuals to produce new individuals. Crossover is a procedure of swapping randomly selected information between two individuals. This is exemplified in Table 2, where the first two elements of two individuals are swapped. Mutation alters one or more elements in an individual to modify a solution. For example, in Table 2, the fourth elements of individual 1 and the last element of individual 2 are changed to be different values. The routine of selection, crossover and mutation is repeated in each generation until the termination is satisfied. Many other methods have been proposed to implement crossover and mutation, but the basic idea is to diversify the individuals.

2) jDE
DE is a powerful stochastic optimization algorithm [39], and its computational steps include initialization, mutation, crossover and selection. Based on DE, jDE employs self-adaptive control parameters to perform better in numerical optimization problems [40]. jDE was further modified to adapt to dynamic environments [10], which is how it is used in this article. The procedure of jDE, as used by this article, is as follows.
At the beginning, a population consisting of PS vectors is randomly initialized, where each vector represents a solution to the optimization problem. The mutation strategy is then used to create one donor vector v for each vector x, also known as target vector. A basic form of mutation strategy on the kthe donor vector is as follows: where randomly selected integers r 1 , r 2 , r 3 ∈ [1, PS], r 1 = r 2 = r 3 = k, v k is the donor vector corresponding to the kth target vector x k , k = 1, 2, . . . , PS, and G is the number of generations. In jDE, the control parameter F is self-adapted: where F l and F u are fixed values, rand 1 and rand 2 are uniform random values in [0, 1], and τ 1 , τ 2 represent probabilities to adjust F [41]. After the donor vector v k is obtained, it's components are recombined with the corresponding target vector x k to generate the trail vector u k . This procedure is called crossover, which can enhance the potential diversity of the population [42]. The crossover rate CR is also a self-adapted parameter: where rand 3 and rand 4 are uniform random values in [0, 1]. Afterwards, the performance of x k and u k is compared and the better one is selected to survive to the next iteration. The mutation, crossover and selection strategies are repeated until the terminal criterion is satisfied.
On top of the aforementioned procedures, a multipopulation strategy without information sharing, an aging strategy to deal with local optima, and an archive to keep the best individuals were introduced to adapt to dynamic environments. More details of jDE can be found in [10].

3) CMA-ES
CMA-ES is an evolutionary algorithm specifically designed for black-box continuous optimisation problems [43]. It has similar procedures as GA and DE, but with different operations.
In CMA-ES, a new population is initialized by sampling candidate solutions based on a multivariate normal distribution. Then new solutions are generated by adding a normally distributed random vector, which amounts to mutation. The generated solutions are evaluated and the ones with better fitness are selected as the parent solutions in the next generation. In this process, correlations between the variables are defined by a covariance matrix, which is updated by using covariance matrix adaptation (CMA) in CMA-ES. In addition, the mean of the distribution is updated on the basis of maximizing the likelihood of previously successful solutions. More details of CMA-ES can be found in [44].

C. SEQUENCE-BASED DETERMINISTIC INITIALIZATION
Sequence-based deterministic initialization is a heuristic space-filling approach to improve the performance of EAs [9]. This initialization technique takes into account both the function to be optimized and the search space, and shows promising performance in providing useful information about the problem's behavior as well as producing high-quality solutions.
To generate the initial population, the search domain [ x, x] is evenly divided into q + 1 segment vectors. The interval I is determined accordingly: The sequence of segment vectors is started with the lower bound vector of all the decision variables, and ended with the upper bound vector of all the variables. The kth segment vector S k is generated based on the following equation: Therefore, the maximum number of individuals that can be generated is: The details in terms of parameter settings and further analysis can be found in [9].

IV. DESIGN OF ALGORITHM
In many real-world applications, an environmental change happens gradually instead of abruptly; therefore, overlapping areas generally exist among changed landscapes [22]. Accordingly, the knowledge learnt in historical landscapes can be re-used to guide the search process in the current landscape under limited computational resources and running time. This article proposes an algorithm based on a similarity check strategy to efficiently deal with DOPs with limited resources, which is called Landscape Influenced Dynamic Optimization Algorithm (LIDOA) in this article.

A. LANDSCAPE INFLUENCED DYNAMIC OPTIMIZATION ALGORITHM (LIDOA)
The framework of LIDOA has three main components. They are initialization, similarity check, and search process. In the first component, the sequence-based deterministic initialization [9] method is used to generate a population of a fixed number of individuals which are used to determine the fitness landscape measure in different environments. The similarity check strategy is then conducted to judge how similar the previous landscapes are to the current one, with the most similar one then selected to utilize the learnt knowledge from it in the optimization process. Finally, the evolutionary process is applied to generate solutions. Considering the general nature of the framework, any evolutionary algorithm can be applied as the search process, within this framework. Each of these components are discussed in more details below.
The steps of LIDOA are presented in Algorithm 1. It is noted that the similarity check strategy is not employed on the first several landscapes, as there would not be sufficient information to use it.
Firstly, EA parameters are initialized and an archive Hist to store the best found solutions in each problem environment is set. By using the sequence-based deterministic initialization approach, an initial population X land is generated, and is recognized as the 'landscape population', which remains the same for calculation of landscape measures in all environments.
As shown in line 5 in Algorithm 1, at the beginning of each new environment, the landscape population X land is evaluated and the corresponding fitness landscape measure FL(l) is calculated and archived for the lth environment.
The search process in any environment starts with an initial random population but one can also use the landscape population as well. In this article, for the first few environments, the landscape similarity strategy is not applied as the comparison will not be useful until a reasonable amount of historical information is collected. The number of problem Algorithm 1 Landscape Influenced Dynamic Optimization Algorithm (LIDOA) 1: Initialize EA parameters; set the archive Hist for historical solutions 2: Generate landscape population X land using sequence-based deterministic initialization 3: Set landscape index l = 0 4: for l = 1 : l max do 5: Evaluate X land to obtain the fitness landscape information FL(l) of the lth problem 6: Randomly initialize population X 7: if l > 6 then 8: Compare FL(l) with FL(i), i = 1, 2, · · · , l − 1 9: Select the idxth landscape that is most similar to the lth landscape 10: Use Hist(idx) to replace the same number of randomly selected solutions in X 11: end if 12: EA to search for the optimal solution 13: Archive the optimal solution(s) in Hist(l) 14: end for environments for this purpose can be set based on the variability of the environments, but this article employs 1/10 of the total number of environments, i.e. 6 when there are 60 environments in total.
When the proposed similarity check strategy is applied, the fitness landscape measure of the current problem FL(l) is compared with each of the previous problems FL(i), i = 1, 2, · · · , l − 1. Out of l − 1 problem landscapes, the one with index idx that is most similar to the current problem can be selected. Afterwards, the best solutions archived from the selected problem Hist(idx) are used to replace randomly selected initial individuals (usually one or a few only) in the current problem. In this way, the knowledge learnt from previous landscapes is conveyed and that benefits the current search process.
The above strategy is a general procedure that can be integrated with any population based algorithms to enhance their performance when solving dynamic optimization problems.

B. COMPLEXITY ANALYSIS
In this section, the computational complexity of LIDOA is discussed.
The major components of LIDOA are the initialization of the landscape population, the evaluation of landscape population, the similarity check between the current fitness landscape with the previous landscapes, and the re-use of the learnt knowledge in the search process.
According to Algorithm 1, the time complexity for the initialization of landscape population is O(S), where S is the size of the landscape population (X land ). Note that this is done only once at the beginning of the process. At the beginning of each new problem environment, the landscape population is evaluated which requires O(S). It should be noted that the fitness landscape is a vector with length L, so the time complexity of each similarity check is O(L) for each problem environment, which applies to each of the four landscape measures presented in subsection III-A. In terms of re-using knowledge, a certain number of current initial solutions are replaced by the archived best ones in O(n) time, where n is the number of replaced solutions, that can be one or more. In summary, the computational complexity of LIDOA in each generation where it is applied is O(S), assuming S is bigger than L, which is fairly insignificant compared to the overall computational complexity required by the other processes of EAs.
The space required by LIDOA includes one fixed landscape population for all problem environments, one fitness landscape measure and one solution for each problem environment. Considering that the fitness landscape is a single row vector, the space taken by adding the proposed method to EAs is minor.

V. EXPERIMENTS
This section examines the performance of LIDOA. The employed benchmark problems are introduced. Then four landscape measures are evaluated, and the best one is selected to show the efficiency of LIDOA on several classical EAs. To conclude this section, the influence of two parameters on LIDOA is analysed.
The algorithms are implemented with C++. The processor is i7-8700, the processor speed is 3.20 GHz and the installed random-access memory is 64.0 GB.

A. BENCHMARK PROBLEMS
The generalized dynamic benchmark generator (GDBG) from the CEC'2009 competition on dynamic environments [45] were employed, including seven test functions. The first two functions are rotation peak functions with 10 peaks and 50 peaks, respectively, while the other five functions are composition of Sphere's function, composition of Rastrigin's function, composition of Griewank's function, composition of Ackley's function and a hybrid composition function. GDBG tunes the system control parameters, such as peak heights, peak widths and peak positions, so that deviations are introduced on the solution distribution. It can be represented as follows [45]: where t is the real-world time, φ is the system control parameter, and φ is a deviation from the current system control parameters. Six change types are covered for the dynamism of system control parameters. The framework of the dynamic changes are described as follows [45]: 1. small step: 2. large step: (13) 3. random: 4. chaotic: 5. recurrent: 6. recurrent with noise: (17) where φ , φ severity and φ min are the change range, the change severity and the minimum value of φ, respectively. In addition, α and α max are constant values, r is a random number between (−1, 1), and noisy severity is the noisy severity. sign(x) is a sign function of x, and N (0, 1) denotes a random number which obeys to the standard normal distribution. For chaotic change, A is a positive constant value between (1.0, 4.0). For the recurrent change and recurrent with noise change, P is the period, ϕ is the initial phase. The parameter settings are presented in Table 3. Each experiment was run for 20 times. The mean value of the best fitness error obtained over 60 landscapes were calculated, then the average of the mean value over 20 runs were employed as fitness error for further analyses.

B. LANDSCAPE MEASURES ANALYSIS
Before analysing the performance of LIDOA, the four landscape measures (Euclidean distance, domination landscape matrix, skewness and kurtosis) are examined. In order to check whether the landscape measures select the similar historical landscape, the landscape fitness values of the current landscape and selected historical landscapes are given. Six benchmark instances are exemplified in Figure 1, including function 1 (number of peaks = 50) with small changes, function 2 with large changes, function 3 with random changes, function 4 with chaotic changes, function 5 with recurrent changes, and function 6 with recurrent with noise changes.    Due to limited space, only the results from the 40th landscape as the current landscape, are presented in the paper.
For functions 1 and 4, historical landscapes selected by the four measures are all very similar to the current landscape. In terms of the other four functions, the landscapes selected     show unstable performances. With regard to change types, the Euclidean-based measure is able to find similar historical landscapes for all the six changes. On the other hand, the domination-based measure is less competent on both random changes and recurrent with noise changes. The kurtosis-based and skewness-based measures have difficulty in handling recurrent with noisy changes and random changes, respectively. It can be concluded that the Euclidean-based measure characterizes problem landscapes in a more comprehensive way in comparison with the other three landscape measures.
In addition, the overall performance of the four landscape measures were examined and compared on all benchmark instances based on GA. The relative fitness error of each landscape measure on the benchmark instances was calculated by the following formula: (18) where f i is the fitness error obtained by the ith method, ∀i = 1, 2, 3, 4, and f min the best fitness error obtained by f i .
The smaller the sum of relative fitness error, the better the corresponding algorithm performs. GA with Euclidean distance as its landscape measure obtained the best fitness on 30 out of 42 instances with the sum of relative fitness error 0.05. GA with domination matrix outperforms other methods on 7 out of 42 instances with the sum of relative fitness error being 0.53, which is slightly better than GA with kurtosis and GA with skewness as their landscape measure, respectively. The relative fitness error obtained by GA with each landscape measure on different instances can be found in Appendix.
The relative fitness error regarding change types is presented in Figure 2. The Euclidean-based measure was the best on all change types except for recurrent with noise changes, on which the domination-based measure outperforms the other three landscape measures. In addition, the domination-based measure ranked second on small changes, large changes, random changes and recurrent changes. Therefore, both Euclidean distance and domination landscape matrix will be employed to review the efficiency of the proposed LIDOA.

C. EFFECT OF LIDOA ON DIFFERENT EAs
To assess the performance of LIDOA, the similarity check strategy is adopted with three different algorithms (GA, jDE and CMA-ES) and compared with those without the proposed approach. The versions with LIDOA are named with '+' at the end, i.e., GA+, jDE+ and CMA-ES+. Note that, due to its good results obtained above, only the landscape measures based on Euclidean distance and domination matrix are used.
In order to analyse the overall performance, the fitness improvement achieved in each problem is calculated using Eq (19), that is, the normalised difference in both fitness errors, where a positive value of FI means the EA with LIDOA is better.
where X is the fitness error achieved by the original algorithm, and X + the corresponding fitness error obtained by EA with LIDOA. FI is calculated on each of the 42 benchmark instances. The greater the fitness improvement FI , the better the corresponding method performs. The comparison between EAs and EAs with a landscapebased strategy is presented in Figures 3 and 4. LIDOA with the Euclidean-based measure improves the performance of GA, jDE and CMA-ES on 42, 24 and 23 instances out of 42 instances, respectively. In other words, LIDOA enhances all of the three examined EAs on more than half the cases. In addition, the sum of FI indicates that the similarity check strategy has overall advantages in dealing with dynamic optimization problems. The detailed fitness improvement on each benchmark instance can be found in Appendix.
The performance of LIDOA is further analysed by using a Wilcoxon test. The Wilcoxon test was proposed by Frank Wilcoxon [46] to evaluate the difference between two sets of data. For two data sets X and Y with size n, the procedures to implement the Wilcoxon signed-ranks test are as follows: VOLUME 8, 2020

1) Calculate the difference score D between X and Y
where the size of D should also be n. 2) Omit the element in D where the value is zero, and the reduced difference score is D with size n , n ≤ n. 3) Rank D according to its absolute value in ascending order. 4) Assign the corresponding symbol '+' or '−' to the rank of D . 5) The Wilcoxon test statistic is subsequently obtained by the sum of the ranks. A Wilcoxon test was conducted to examine if there is a significant difference between EAs (GA, jDE and CMA-ES) and EAs that use Euclidean as the similarity check measure. The results are presented in Table 4. At the 5% significant level, a significant difference can be observed between GA and GA with Euclidean-based similarity check. However, there is no significant difference on jDE and CMA-ES. One of the reasons is that the Euclidean-based strategy improves only a limited number of instances on the two EAs.
The comparison of the original EAs and those with domination landscape matrix were conducted to further analyse LIDOA. Figure 4 shows that the modified EA with domination landscape matrix obtains better results for 42, 26 and 23 out of 42 instances, respectively. As mentioned earlier, positive FI values also show the advantage of LIDOA with domination matrix measure. Similar to the Euclidean-based measure, the domination matrix based similarity check shows a significant difference when employed on GA, but not on jDE and CMA-ES, as presented in Table 4. Though the Wilcoxon test on GA shows the capability of the similarity check strategy, the results on jDE and CMA-ES indicate that more effort should be made to make the method more efficient.
The performance of the similarity strategy regarding the change types is presented in Table 5. The fitness improvement rates of different functions from the same change type are added, so as to compare the effect of Euclidean distance and domination matrix landscape measures on various change types. As can be seen from the table, GA with Euclidean distance shows overall advantage on all change types. The same conclusion can be made on GA with domination matrix. jDE with Euclidean distance outperforms jDE on random and chaotic changes, while jDE with domination matrix obtains better results on the other four change types. In terms of CMA-ES, the Euclidean distance landscape measure improves the performance on most change types except for chaotic and recurrent with noise changes, while the domination matrix landscape measure is overall superior except for on the large changes.
In Table 6, the '+' sign indicates that the Euclidean-based measure performs better than the domination-based measure on the corresponding instance, while the '−' sign indicates the opposite result. Although the Euclidean-based measure outperforms the domination-based measure on most change types, the latter is more reliable on recurrent changes and recurrent with noise changes. The comparison between the Euclidean-based measure and the domination-based measure on change types further indicates the further improvement that may be found from the integration of different landscape measures.

D. EFFECT OF THE LANDSCAPE POPULATION SIZE
The landscape population size is essential in LIDOA as it is the basis of the similarity check strategy. A larger landscape population can better reflect the characteristics of the problem, but can be time consuming as well as it can mislead the search with too many details, while a smaller population may be efficient but suffer from insufficient information. In this section, four landscape population sizes, i.e. population with 124 (q = 3), 427 (q = 6), 1111 (q = 10) and 4221(q = 20) individuals, respectively, were examined to investigate the effect of this parameter. The experiments were conducted on GA because the landscape-based strategy showed greater impact on it compared to jDE and CMA-ES, as can be seen in Tables 3 and 4. Therefore, the effect imposed by different parameter settings can be more easily observed on GA. The experiments were conducted on GA with Euclidean distance as the landscape measure, which is referred to as GA+ in Figure 5. The relative fitness errors regarding change types show that the landscape population size of 427 performed best on small changes and recurrent changes, while the landscape population size of 1111 was the best on the other four change types. Similarly, the fitness error obtained on each benchmark instance indicates that the landscape population sizes of 427 and 1111 show advantages over the other two population sizes on most of the benchmark instances. It can be concluded that the selection of population size should be considered based on the optimization objectives. Additional experimental results on jDE can be found in Appendix.

E. EFFECT OF THE NUMBER OF HISTORICAL LANDSCAPES
In the real-world dynamic problems, the current problem landscape can be similar to multiple historical landscapes, as the current one can share different characteristics with each of them. Therefore, the knowledge from various historical landscapes can provide more information as well as diversify the population. However, the re-use of too many historical landscapes will not only slow down the optimization but may also distract the search. In this section, the effect of the number of historical landscapes is examined. The best solution from the 1, 3 and 5 most similar landscapes are re-used in the new problem, and the corresponding experimental results are shown in Figure 6. The experiments were conducted on GA due to the same reasons as stated in section V-D and Euclidean distance was employed as the landscape measure, referring to as GA+ in the figure.
It can be clearly seen that GA with 5 historical landscapes outperformed other settings on all change types, and GA with 3 historical landscapes is better than re-using only one landscape. This indicates that more information can generally produce better results when a landscape-based strategy is used with GA. Additional experiments on jDE that generate different results can be seen in Appendix.

VI. CONCLUSION
In this research, Landscape Influenced Dynamic Optimization Algorithm (LIDOA) was proposed to improve the efficiency of EAs in solving DOPs. The knowledge learned in each landscape was archived and re-utilized in the new environment. On the basis of the sequence-based deterministic initialization technique, landscape measures were employed to select the historical landscape that was most similar to the current one. The optimal solution from the chosen landscape was then re-used to enhance the optimization ability of EAs.
The commonly used benchmark from CEC2009 was employed, which consists of seven functions and six change types. Four landscape measures were examined, and it can be concluded that the Euclidean distance and domination matrix were better measures, compared to skewness and kurtosis, yet none of them could clearly dominate the other. Subsequently, for further analysis, those two measures were then adopted with GA, jDE and CMA-ES. Experimental results show the overall advantage of LIDOA.
The analysis regarding change types indicate that the Euclidean distance and domination matrix landscape measures have advantages over different instances. Accordingly, better performance can be expected with appropriate integration of multiple landscape measures. The effect of different parameters, including landscape population size and re-used landscapes, on the optimization was also analysed.
The general advantages that LIDOA shows come from multiple resources. First of all, the landscape-based similarity check strategy is independent of the problem types. Therefore, it is applicable to a large range of problems. Similarly, LIDOA can improve the performance of various EAs. In addition, the re-used global optimum helps to escape from the local optima. However, there are also shortcomings of the proposed approach. The computational complexity of EAs is not reduced after using LIDOA, as it is added to EAs without changing them. Apart from this, only limited information has been re-used in this article. The performance can be expected to be further enhanced if more knowledge is re-utilized. Although this article is a step-forward to integrate landscape measures with EAs to solve DOPs, further improvements are still needed. A possible direction may be the use of multi-algorithm and multi-operator frameworks. Better approaches to share information among environments are also required. DARYL ESSAM (Member, IEEE) received the B.Sc. degree in computer science from the University of New England, Australia, in 1990, and the Ph.D. degree from the University of New South Wales (UNSW) at Canberra, Australia, in 2000. Since 1994, he has been with UNSW Canberra, where he is currently a Senior Lecturer. His research interest includes genetic algorithms, with a focus on both evolutionary optimization and large scale problems.