Golden Search Optimization Algorithm

This study introduces an effective population-based optimization algorithm, namely the Golden Search Optimization Algorithm (GSO), for numerical function optimization. The new algorithm has a simple but effective strategy for solving complex problems. GSO starts with random possible solutions called objects, which interact with each other based on a simple mathematical model to reach the global optimum. To provide a fine balance between the explorative and exploitative behavior of a search, the proposed method utilizes a transfer operator in the adaptive step size adjustment scheme. The proposed algorithm is benchmarked with 23 unimodal, multimodal, and fixed dimensional functions and the results are verified by a comparative study with the well-known Gravitational Search Algorithm (GSA), Sine-Cosine Algorithm (SCA), Grey Wolf Optimization (GWO), and Tunicate Swarm Algorithm (TSA). In addition, the nonparametric Wilcoxon’s rank sum test is performed to measure the pair-wise statistical performance of the GSO and provide a valid judgment about the performance of the algorithm. The simulation results demonstrate that GSO is superior and could generate better optimal solutions when compared with other competitive algorithms.


I. INTRODUCTION
Many real world design problems can be considered as optimization problems, and an appropriate optimization method is required for the solution. On the other hand, design problems have become harder when discontinuities, incomplete information, dynamicity, and uncertainties are involved. In such a case, classical optimization algorithms based on mathematical principles demand exponential time or may not find the optimal solution at all. One of these classical algorithms is the gradient-based methods, which utilize the gradient of the objective function for the configuration of the optimization problem. To overcome the mentioned problem, during the last few decades, introducing new efficient metaheuristic optimization algorithms to deal with the drawbacks of classical techniques has been of great concern. The privileges of these algorithms include derivation-free mechanisms, simple concepts and structure, local optima avoidance, and effectiveness for discrete and The associate editor coordinating the review of this manuscript and approving it for publication was Zhenzhou Tang . continuous functions. Accordingly, there is an increasing interest in presenting new metaheuristic algorithms that have high efficiency, great accuracy, and an increased speed rate in dealing with difficult optimization problems. Generally, metaheuristic algorithms have two types: single-solution based methods (also known as trajectory methods) and population-based algorithms.
As the name indicates, in the former type, only one solution is generated (usually at random) and processed during the optimization phase until a stopping criterion is satisfied.
Some of these methods are Simulated Annealing (SA) [1], Tabu Search (TS) [2], Iterated Local Search (ILS) [3], and Vortex Search Algorithm (VSA) [4]. In the latter type, a set of solutions (i.e., a population) is generated randomly and updated iteratively in each iteration of the optimization process until satisfying a stopping criteria. Some well-known examples of these algorithms are the Genetic Algorithm [5], Ant Colony Optimization [6], Particle Swarm Optimization [7], Firefly Algorithm [8] and Harris hawks optimization [9]. Regardless of the wide variety of recently developed metaheuristic algorithms, they have two main VOLUME 10, 2022 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ phases include investigating the various promising regions in a search space (exploration) and the local search around the obtained promising regions (exploitation). The basic differences between metaheuristic methods is the special procedure that they applied for well balance among these two phases. Based on the nature of the metaheuristics, populationbased algorithms are more exploration-oriented, while singlesolution algorithms are more exploitation-oriented [10]. Although all population-based metaheuristics could obtain relatively acceptable results, there is no metaheuristic method that provides superior performance in solving all optimizing problems [11]. Hence, proposing new high performance metaheuristic algorithms is welcome. In addition, as shown by Mirjalali [12], a meta-heuristic does not necessarily need actual inspiration, and simple mathematical functions can also be used to design optimization algorithms.
Therefore, the motivation of this study is to propose a simple and effective global optimization technique for complex problems. The proposed algorithm utilizes a new adaptive step size adjustment scheme, which has a simple concept and significantly improves the performance of the search process. It is worth mentioning that the presented method could provide a well balance between exploration and exploitation. According to the capability of the proposed algorithm in finding a (near) global solution in a reasonable time, it is called a ''Golden Search Optimizer'' (GSO).
The remainder of this paper is organized as follows: In section II, a survey on metaheuristic algorithms is presented. Section III presents a description and step-by-step framework of the proposed GSO algorithm. In section IV, the empirical evaluation of GSO in an extensive test environment is carried out, and the simulation results are compared with the selected metaheuristic algorithms. Finally, the summary and conclusions of the study are presented in Section V.

II. RELATED WORKS
In recent years, optimization has become a popular research field and an economical way to find an optimal solution to complex problems. According to Fig. 1, we have divided the optimization algorithms into four categories based on the type of inspiration. The first group is swarmbased algorithms, which consist of a population of simple agents interacting locally with one another and with their environment. These algorithms employ artificial swarms of autonomous agents that follow simple rules (relative to the system's complexity) to update their state through time. In the optimization framework, a swarm consists of a number of search agents whose state represents a candidate solution to the optimization problem at hand. The agents adhere to basic rules that promote their cooperation during the search for better solutions. The search is typically defined as an iterative, highly distributed procedure where the agents explore the given search space for the problem while communicating their findings to their mates. Information sharing is a key issue for the efficiency of such computational schemes [13]. Some examples of swarm based algorithms include Particle Swarm Optimization (PSO) [14], Firefly Algorithm (FFA) [15], Ant Colony Optimization (ACO) [16], Artificial Bee Colony (ABC) [17], Krill Herd (KH) [18], Whale Optimization Algorithm (WOA), Crow Search Algorithm (CSA) [19], Rat Swarm Optimizer [20], Sperm Swarm Optimization [21], Chameleon Swarm Algorithm [22]. The second group is evolutionary algorithms, which are efficient heuristic search methods based on Darwinian evolution with powerful characteristics of robustness and flexibility to capture global solutions to complex optimization problems. Genetic Algorithm (GA), Evolution Strategy (ES), Genetic Programming (GP), Biogeography Based Optimizer (BBO), Evolutionary Programming (EP) [23], Differential Evolution (DE) [24], Virulence Optimization Algorithm (VOA) [25] are some of these algorithms. The third category is physics-based algorithms inspired by natural phenomena and imitating physical and biological processes of nature, such as Simulated Annealing (SA) [1], Black Hole Algorithm (BH) [26], Curved Space Optimization Algorithm (CSO) [27], Ray Optimization (RO) [28], Gravitational Search Algorithm (GSA) [29]. The fourth category is human-based algorithms that allow humans to contribute solution suggestions to the evolutionary process, such as Harmony Search (HS) [30], Teaching Learning Based Optimization (TLBO) [31], Imperialist Competitive Algorithm (ICA) [32], Exchanged Market Algorithm (EMA) [33], Thermal Exchange Optimization (TEO) [34] and Tabu Search (TS) [35]. There are other algorithms that are a combination or modification of algorithms in these four categories. Some of them are: modified particle swarm optimization [36], [37], modified harmony search algorithm [38], modified gravitational search algorithm [39], [40], modified ant colony optimization [41], modified sine cosine algorithm [42], [43], modified wild horse optimization [44], modified slime mould algorithm [45], hybrid genetic algorithm and particle swarm optimization [46], hybrid firefly algorithm [47], hybrid sperm swarm optimization and gravitational search algorithm [48], hybrid tunicate swarm algorithm and pattern search [49], hybrid arithmetic optimization algorithm and sine cosine algorithm [50].
Some of the most famous of these optimization algorithms are described below. The Genetic Algorithm (GA) is the most popular evolutionary inspiration technique that mimics the principles of Charles Darwin's Compatibility Survival Theory. This method involves the selection process, the crossover, and the mutation process to replace the worst solution in each generation. In this algorithm, solutions are improved according to the best solutions obtained by each particle so far and the best solution found for the overall swarm. The ACO algorithm imitates the collective behavior of ants in finding the shortest path from the nest to the food source. One of the most important and most interesting behaviors of ants is their behavior when finding food, and in particular, how to find the shortest path between food and nest. This kind of behavior by the ants has a kind of swarm intelligence that has recently attracted the attention of scientists. In the natural world, ants of some species (initially) wander randomly, and upon finding food, return to their colony while laying down pheromone trails. If other ants find such a path, they are likely not to keep travelling at random, but instead to follow the trail, returning and reinforcing it if they eventually find food. The DE is presented to overcome the main defect of the genetic algorithm, namely the lack of local search in this algorithm.
The main difference between GA and DE is in the selection operators. In the ABC model, the colony consists of three groups of bees: employed bees, onlookers, and scouts. It is assumed that there is only one artificially employed bee for each food source. In other words, the number of employed bees in the colony is equal to the number of food sources around the hive. Employed bees go to their food source and come back to the hive to dance in this area. The employed bee whose food source has been abandoned becomes a scout and starts to search for a new food source. Onlookers watch the dances of employed bees and choose food sources depending on the dances. The FA is a meta-heuristic algorithm, inspired by the flashing behavior of fireflies. The primary purpose of a firefly's flash is to act as a signal to attract other fireflies. The Firefly Algorithm is based on the life cycle of firefly worms. In the firefly algorithm, worms tend to be attracted to highattraction lights. The worm with less light can be absorbed by the worm with more light. In this situation, the swarm moves like a PSO. Of course, the degree of motion is also taken randomly to prevent the early convergence of the FA.
Optimization algorithms have advantages and disadvantages. For example, the PSO algorithm solves problems with small and simple dimensions well, but has poor performance in high-dimensional, complex, and hybrid problems and suffers from premature convergence [51].
The Gravitational search algorithm (GSA) is a novel meta-heuristic stochastic optimization algorithm inspired by the law of gravity and mass interactions [29]. In GSA, individuals, called agents, are a collection of masses that interact with each other based on Newtonian gravity and the laws of motion. The agents share information using the gravitational force to guide the search towards the best location in the search space. The high performance and the global search ability of GSA in solving various nonlinear functions are inferred from the results of experiments undertaken previously [29], [52]. In GSA, all agents move to a new place, the direction and distance are determined by their velocities. By changing the velocities over time, the agents are likely to move towards the global optima.
The sine cosine algorithm (SCA) is a relatively new meta-heuristic optimization approach developed by Mirjalili in 2016 [12]. Compared with other meta-heuristics, the SCA has a simple concept and structure and does not have complicated mathematical functions. In the SCA, the formulas for updating the population rely solely on sine and cosine functions. SCA is better than other competitive methods at finding optimal solutions and is suitable for tackling real-world optimization problems [53].
The Tunicate Swarm Algorithm (TSA) is a recently developed swarm-based algorithm [54]. Tunicates use swarm intelligence and jet propulsion at sea to choose the optimal state for seeking food in their surroundings. TSA outperforms other competitor approaches when it comes to identifying optimal solutions and is well-suited to real-world optimization challenges.   [55]. The wolf, as a top predator in the food chain, has a strong ability to capture prey. Wolves generally like social life, and in the interior of the wolves, there exists a rigid social hierarchy. The GWO algorithm mimics the leadership hierarchy and hunting mechanism of grey wolves in nature. Four types of grey wolves, such as alpha, beta, delta, and omega, are employed in simulating the leadership hierarchy [55]. In addition, the three main steps of hunting: searching for prey, encircling prey, and attacking prey are implemented. The GWO is able to provide very competitive results compared to other well-known metaheuristics [55].

III. GOLDEN SEARCH OPTIMIZATION (GSO) ALGORITHM
Despite the wide variety of population-based algorithms in the field of stochastic optimization, they are almost all the same in the process of finding the optimum. These algorithms commonly start the search process with a randomly generated initial population (potential solutions). These random solutions are evaluated during iterations using a fitness function and improved by a set of formulas, which is the core of an optimization technique, until satisfying some termination condition.
In this study, according to the principles and commonly used procedures of metaheuristic algorithms, a very simple but effective optimization method called the Golden Search Optimizer (GSO) is proposed. In fact, the new method combines some major advantages of previously presented algorithms like particle swarm optimization (PSO) and sine cosine algorithms (SCA) to provide a fine balance between global exploration and local exploitation and to avoid premature convergence. In the GSO algorithm, the objects update their position using a step size parameter, which is almost the same as the velocity in the PSO algorithm [7]. However, instead of simple random values, the GSO utilizes sine and cosine functions. The oscillation behavior of sine and cosine functions allows one object to be re-positioned around another one, and it can guarantee exploitation of the space defined between two solutions. In addition, the exploration capability of the algorithm will be improved by increasing the range of sine and cosine functions, which allow a solution to update its position outside the space between itself and another solution [12]. Apart from its simple structure, the algorithm outperforms other metaheuristics in terms of convergence to the global best solution.
As a population-based metaheuristic optimization technique, GSO also commences the search process with an initial random population of objects (candidate solutions). In every iteration, the algorithm updates the position of the objects using a step size parameter until satisfying some termination criteria. The following is the detailed mathematical expression of the GSO steps.

A. ALGORITHMIC STEPS
Theoretically, as a global optimization algorithm, GSO encompasses both exploration and exploitation phases and could provide a well balance between these two contradictory abilities.
The algorithm has three main parts; including population initialization, population evaluation, and updating the current population. The step-by-step procedure of the proposed GSO is detailed as follows.
Step 1-population initialization GSO starts the search process with a set of randomly generated objects (possible solutions) in the search space according to the following equation: where O i presents the location of ith objects in the search space. Moreover, ub i and lb i are the lower and upper bounds of the object, respectively.
Step 2-population evaluation In this step, initial population will be evaluated based on objective function and the object with the best fitness value selected as Ogbest i Step3-golden change In the third step, the objects will be sorted according to their fitness and the object with the worst fitness will be changed by a random solution.
Step4-step size evaluation In each iteration of the optimization process, the objects are moved toward the best solution using the step size operator (St i ). St equation consists of three parts. The first part represents the previous value of the step size that is multiple by the transform operator (T ) which decreases iteratively to balance global and local search of the algorithm. The second part presents the distance between current position of the ith object and its personal best position obtained so far by the cosine of a random value in the range of 0 and 1.
The final part represents the distance between the ith object's current position and the best position obtained thus far among all objects, multiplied by the sine of a random value between 0 and 1. In the first iteration of optimization process, St i will be generated randomly and updated using the following equation during the iterations: where C 1 and C 2 are a random numbers between 0 and 2, r 1 and r 2 are random numbers in the range of (0,1), Obest i is the best previous position obtained by the ith object so far and T is transfer operator which transforms search from exploration to exploitation to improve the search performance and controlling the balance between global search in early iterations and local search in late iterations. Actually, T is a decreasing function and evaluated using Eq. (3) where. t max is the maximum number of iterations.
Step5-step size limitation At each iteration, the algorithm proceeds by adjusting the distance that each object moves in every dimension of the problem hyperspace. Equation (2) shows that the step size is a stochastic variable and may allow the objects follows wider cycles in the problem space. In order to control these oscillations and to avoid explosion and divergence, a reasonable interval is introduced to clamp the object's movement according to: where St imax is a designated maximum movement allowed, which defines the maximum change one object can undergo in its positional coordinates during an iteration based on the following equation: Step 6-update position (generate new population) In this stage, the objects move toward the global optimum in the search space according to the following equation: The flowchart of the proposed GSO algorithm is presented in Fig. 2.

B. COMPARATIVE TIME COMPLEXITY ANALYSIS
Computational time complexity analysis can be conducted in order to evaluate the overall performance of a new optimization algorithm from different points of view. In computer sciences, the ''Big O notation'' is a mathematical notation that represents the required running time of an algorithm by considering the growth rate when dealing with different inputs [13].
The time complexity analysis of most algorithms involves analyses of three components. Likewise, the time complexity analysis of the proposed GSA also requires analyses of these three components: 1. Time complexity of initialization of the population, generally calculated by O (N × D) where N denotes the population size and D denotes the dimensions of the problem.
2. Time complexity of initial fitness evaluation, generally evaluated by O (N × F(X)), where F(X) represents the objective function.
3. Time complexity of the main loop, generally calculated by O (t max × (N ×D+N ×F(X))), where t max is the maximum number of iterations.
Hence, the total time complexity of GSO algorithm is O(t max (N × D + N × F(X))).

IV. COMPARATIVE ANALYSIS OF THE GSO
Because of the stochastic nature of metaheuristic algorithms, several test cases should be employed to confirm the effectiveness of an algorithm. In this study, the performance of GSO is evaluated on a well-studied and diverse set of benchmark functions from the literature [56], [57] against a good combination of some well-known state-of-the-art algorithms. All of these functions are minimization problems, which are useful for evaluating the search efficiency and convergence rate of optimization algorithms. The mathematical formulation and characteristics of these test functions are available in Tables 1, 2 and 3. Figures 3, 4 and 5 show three-dimensional drawings of these benchmark functions. Moreover, the cost functions along with the dimensions, ranges, and minimum inputs related to the single exponential benchmark functions are shown in Table 1-3. This benchmark set covers three main groups: unimodal functions with a unique global best for testing the convergence speed and exploitation ability of the algorithms; multimodal functions with multiple local solutions and a global optimum for testing local optima avoidance and exploration capability of an algorithm; and finally multimodal functions with a fixed dimension. The proposed algorithm is coded in MATLAB R2020b programming software.
The MATLAB code will be released after the acceptance of the paper. In order to verify the success of a newly proposed computational intelligence algorithm, it is recommended that the performance of the new method be compared with the other algorithms that are widely accepted in the field. In this paper, the results and performance of the proposed GSO are compared with other well-established optimization algorithms such as the Gravitational Search VOLUME 10, 2022  Algorithm (GSA) [29], Sine-Cosine Algorithm (SCA) [12], Tunicate Swarm Algorithm (TSA) [54], and Grey Wolf Optimizer (GWO) [55]. These algorithms have proved their effectiveness and robustness when compared with other well-established methods like PSO, GA, FA, and so on [12], [29], [54], [55]. It should be noted that the performance and convergence of these metaheuristic methods completely depend on the internal parameters of the algorithms. GSO has a simple structure and needs only two main parameters, N (number of objects) and MaxIter (maximum number of iterations).
It is found through experiments that a lower value of N results in premature convergence and a higher value improves exploration but increases elapsed time significantly. The appropriate value of N is considered to be 30, and the maximum number of iterations is equal to 1000. In Table 4, the key parameters of the selected methods are presented.
These values have been determined using the referencebased parameter identification process according to those recommended by their authors in the original papers. The references for each method are presented in the third column of Table 4.
Because metaheuristic methods are stochastic, the results of a single run may be unreliable, and the algorithms may obtain better or worse solutions than the previous one. Therefore, statistical analysis should be applied to have a fair comparison and effective evaluation of the algorithms. Regarding this issue, for the selected algorithms,        Tables 1-3.  The results of Tables 5-7 show the best (minimum), worst (maximum), mean (average), median, and standard deviation (Std) of the solutions obtained from experiments using the selected optimization algorithms. The best results among the five algorithms are shown in bold. According to the results of these tables in the following subsections, the exploration, exploitation, convergence rate, and scalability of the new method are investigated using a comparative performance comparison of GSO against four selected algorithms.

A. EXPLOITATION CAPABILITY
Unimodal test functions can be considered to investigate the exploitation capability of an optimization algorithm [55], [58]. In this study, to evaluate the ability of GSO to exploit the promising regions, seven unimodal benchmark functions (F1 to F7) are solved, and the results are compared with four selected optimization methods in Table 5. The results of this table show that, for all unimodal functions except F6, GSO could provide a better solution. In addition, for four functions (F1-F4), GSO reached the global optima. It means that the new algorithm has a large potential search space compared with the other optimization algorithms. It should be noted that the good exploitative capability of GSO is due to the effective position updating strategy.

B. EXPLORATION VERIFICATION
In order to evaluate the capability of an optimization algorithm to effectively explore the search space, multimodal benchmark functions that have many local optima are usually considered [55], [58]. Based on the presented procedure, 16 multimodal functions (F8 to F23) are minimized. According to the results of Tables 6-7, it can be observed that the best and mean values reached by GSO for most of the functions (except F12, F13, F17-F19) are significantly better than the other methods. However, the results for F17-F19 functions are also comparable to the other algorithms. In addition, for F12 and F13, mean values of GSO are smaller than the robust GSA method, and the results are much better than those obtained by SCA, TSA, and GWO. From the standard deviation point of view, which indicates the stability of the algorithm, the results show that GSO is a more stable method when compared with the other techniques.  Table 3.
From the analysis, it can be concluded that GSO either outperforms the other algorithms or performs almost equivalently. The consistent performance of the new method for such a comprehensive suite of multimodal benchmark functions verifies its superior capabilities of exploration. VOLUME 10, 2022

C. CONVERGENCE CAPABILITY
An effective optimization algorithm should converge to a global optimum, not prematurely converge to some local optimum. The convergence progress curves of GSO for some benchmark test functions are compared with GSA, SCA, TSA, and GWO in Fig. 6. The curves are plotted against the number of iterations, which is in the hundreds.
The figure shows that GSO outperforms the other algorithms in most cases. The curves of test functions show that GSO is capable of exploring the search space extensively and identifying the most promising region in less iteration because of its position-updating mechanism and utilization of a random solution instead of worst object during the iterations. VOLUME 10, 2022

D. IMPACT OF HIGH-DIMENSIONALITY
In order to present the ability of GSO to solve very highdimensional problems effectively, the dimensions of all the scalable benchmark functions (F1-F13) have been increased to 100. The algorithms' parameters are the same as in Table 4. The results are compared with those obtained by the other methods in Table 8. The results show that all 100-dim functions become more difficult than their 30-dim counterparts.
As it can be seen from Table 8, the GSO outperforms the competitors for all high-dimensional functions, which indicates the scalability and efficiency of the proposed method in terms of the number of variables in the optimization problem.

E. STATISTICAL SIGNIFICANCE ANALYSIS
A non-parametric pairwise statistical analysis should be conducted in order to determine the statistical significance of the comparative results between two or more algorithms. As recommended by Derrac et al. [59], to assess meaningful comparison between the proposed and alternative methods, the nonparametric Wilcoxon's rank sum test is performed between the results. In this regard, utilizing the best results obtained from 30 runs of each method, a pair-wise comparison was conducted.
Wilcoxon's rank sum test returns p-value, the sum of positive ranks (R+) and the sum of negative ranks (R−) [60]. Table 9 presents the results of Wilcoxon's rank sum test of GSO when compared with other methods. The p-value indicates the minimum significance level for detecting differences. In this study, α = 0.05 is considered the level of significance If the p-value of the given algorithm is greater than 0.05, then there is no significant difference between the two compared methods. Such a result is indicated with ''N.A'' in the winner rows of Table 9. On the other hand, if the p-value is less than α, it definitively means that, in each pair-wise comparison, the better result obtained by the best algorithm is statistically significant and it was not gained by chance. In such cases, if the R+ is greater than the R-, GSO performs better than the alternative method; otherwise, GSO performs poorly and the alternative algorithm performs better [61].
According to the results of Wilcoxon's rank sum test in Table 9, the pairwise comparison between GSO and GSA reveals that in the optimization of 23 test functions, the new method has superior performance in 17 cases and has inferior performance in 5 cases. In addition, for F16, both methods are statistically equivalent. Similarly, in the other pairwise comparison, for the majority of the test suite, GSO provides better results. Therefore, the nonparametric statistical analysis proves that GSO generates significantly better solutions and, comparatively, has superior performance over the other algorithms.

V. CONCLUSION
This study develops a novel population-based Golden Search Optimizer based on some principles of metaheuristic algorithms. In the proposed GSO algorithm, during the search process, the objects (candidate solutions) interact with each other and improve their positions based on the best position obtained so far as the reference point. In summary, the main features of GSO are as follows: it has just two internal parameters; it is easy to code; and it is easy to apply. The performance of the new method is tested by utilizing several experiments. First, a set of various unimodal and multi-modal benchmark functions have been considered to investigate the exploration, exploitation, and convergence rate of the proposed algorithm. Moreover, the results were compared with four well-known and recently developed algorithms, including GSA, SCA, TSA, and GWO. As per the results and findings, it was observed and may be concluded that GSO is capable of finding the global solution for most of the unimodal, multi-modal, and composite benchmark functions and outperforming the other algorithms in a statistically significant manner. While, for most of the benchmark test functions, all the competitor methods rarely reach the global optimal solutions. In order to evaluate the capability of the new method for solving high-dimensional problems, 13 scalable benchmark functions are optimized. Optimization results of scalable test functions proved that the performance of GSO remains consistent even if the dimensions of the problems are increased to 100. Finally, to provide a meaningful comparison and valid judgment between the proposed and alternative algorithms, the nonparametric Wilcoxon Rank Sum test has been conducted. According to the statistical experiments, it is evident that GSO outperformed wellestablished optimization methods, and its superiority is statistically significant.