An Effective Adjustment to the Integration of Optimal Computing Budget Allocation for Particle Swarm Optimization in Stochastic Environments

Although particle swarm optimization (PSO) is a powerful evolutionary algorithm for solving nonlinear optimization problems in deterministic environments, many practical problems have some stochastic noise. The optimal computing budget allocation (OCBA) has been integrated into PSO in various ways to cope with this. The OCBA can mitigate the effect of noise on PSO by selecting the best solution under a limited evaluation budget. Recently, with the increasing complexity of PSO applications, the evaluation costs are also increasing rapidly, which has sparked the need for more efficient PSO in stochastic environments. This article proposes a simple yet effective adjustment to the integration of OCBA to further improve the efficiency of PSO. The proposed adjustment allows OCBA to expand its search space to find the global best position more correctly such that the entire swarm can move on a better direction under stochastic noise. The experimental results on various benchmarks demonstrate the improved performance of PSO by the proposed adjustment under a limited budget compared with the latest studies. In addition, the results regarding fighters’ evasion flight optimization emphasize the practical need for the proposed adjustment.


I. INTRODUCTION
Inspired by the social behavior of swarm, particle swarm optimization (PSO) [1] is a population-based evolutionary algorithm widely used for solving complex optimization problems [2] or learning artificial neural networks [3]. PSO finds a good solution through a swarm of particles that iteratively and randomly explore the search-space at their velocities. The distinctive feature of PSO is that each particle memorizes its personal best (Pbest) solution found so far and shares the memory with its neighborhood to find the global best (Gbest) solution efficiently. However, because PSO was originally designed to work in deterministic environments, it can be inefficient for many real-life optimization problems that typically involve using stochastic factors to capture the The associate editor coordinating the review of this manuscript and approving it for publication was Diego Oliva . uncertainty in the real-world [4]. Unlike in a deterministic environment, fitness evaluation for a solution in a stochastic environment yields a noisy value; thus, particles may fail to distinguish a better solution, which disorientates the particles' movement significantly. So far, several strategies for mitigating the effect of noise on PSO have been proposed, and these are mainly based on a single-evaluation approach or a resampling approach.
The single-evaluation approach evaluates the fitness of a solution only once but is aimed at reducing the effect of the evaluated noisy values on PSO through many iterations of PSO. Fernandez-Marquez and Arcos [5] proposed an evaporation factor to worsen the outdated Pbest solution as PSO iterates, thereby preventing particles from premature convergence due to noise. They further introduced a dynamic factor to improve the performance of the previous version [6]. Rada-Vilela et al. [7] reduced the noise effect by taking the average for the noisy fitness values of neighboring particles, as the neighborhood's solutions become similar as PSO iterates.
On the other hand, the resampling approach takes multiple independent identically distributed samples to increase the accuracy of fitness evaluation for a solution [8]. The larger the number of samples collected is, the more likely one is to be obtain precise fitness values; however, this process can be very time consuming. In addition, it may lower the performance of PSO by reducing the number of iterations of PSO under a limited computing budget. Thus, the resampling approach combined ranking and selection (R&S) methods, such as the indifference-zone (IZ) [9], the optimal computing budget allocation (OCBA) [10], and the uncertainty evaluation (UE) [11], into PSO for efficient sample allocations. These R&S methods intelligently allocate a limited number of samples to correctly select the best solution from a finite set of alternatives.
Mohamed et al. [12] combined IZ to find the best solution from particles' current solutions, whereas Bartz-Beielstein et al. [13] and Choi [14] applied OCBA and UE, respectively, for the same purpose. However, to find the best solution more correctly, Choi considered the previous Gbest solution for allocation. Pan et al. [15] integrated the hypothesis test as well as OCBA into PSO such that the Pbest solutions could be changed with statistical significance. Horng et al. [16] applied PSO combined with OCBA to a two-stage algorithm to solve a wafer probe testing problem in semiconductor manufacturing. Rada-Vilela et al. [17] used OCBA to estimate particles' Pbest solutions more accurately, rather than using the current solutions. Although these works mainly focused on finding only the Gbest solution, Zhang et al. [18] proposed a new OCBA-based method to correctly select the Pbest solutions, as well as the Gbest solution, which showed superior performance compared with previous works. Meanwhile, Rada-Vilela et al. [19] developed a hybrid method involving both single-evaluation and resampling approaches, and Taghiyeh and Xu [20] suggested a new approach using a set of statistically equivalent Gbest solutions.
Practical optimization problems to which PSO is applied typically have a stochastic simulation model as the objective function. This is because the simulation can accurately analyze modern complex industrial systems that cannot be described as a closed-form analytic model, with just a few assumptions [21]. Recently, with the increasing complexity of the systems, the cost per simulation run (i.e., single evaluation of fitness) is also increasing rapidly, which sparks the need for more efficient PSO in stochastic environments. As shown in the literature, OCBA has been applied to PSO in various ways, including the dedicated variants for PSO, due to its superior efficiency. This article proposes a simple but effective adjustment to the integration of OCBA to further improve the efficiency of PSO under stochastic noise. The proposed adjustment allows OCBA to select the Gbest solution more correctly, thereby mitigating the effect of noise and increasing the performance of PSO under a limited evaluation budget.
The experimental results for various benchmark problems demonstrate improved efficiency compared with the existing works, and a case study on a practical problem emphasizes the need for the proposed adjustment.
The remainder of this article is organized as follows: Section II briefly introduces PSO and OCBA. Section III proposes the effective adjustment, and Section IV exhibits the experimental results. Finally, a conclusion is given in Section V.

II. BACKGROUND
This section briefly introduces PSO in a stochastic environment. In addition, OCBA and its integration to PSO are described.

A. PARTICLE SWARM OPTIMIZATION
Due to its simple and superior performance, various variants of PSO have been developed for various requirements, which is summarized in [22] and [23]. However, in this article, a standard version of PSO [24] is employed to provide a baseline for clarifying the efficiency improvement of the proposed adjustment.
The basic concept of PSO is that a swarm of m particles with individual movements converges to the Gbest solution via sharing their information with, one another as bees gather honey. When it comes to describing the movement, each particle i in the D-dimensional search space has its position X l i = x l i1 , . . . , x l iD that encodes a solution to the problem, and velocity V l i = v l i1 , . . . , v l iD , where the superscript l represents the iteration number of PSO. In addition, each particle has a memory to store its Pbest position P l i = p l i1 , . . . , p l iD found so far. In each iteration of PSO, particles define the Gbest position G l = g l 1 , . . . , g l D found so far in the entire swarm by sharing the memory. Then, the velocity and position of each particle are updated based on its P l i and G l such that the entire swarm can converge to G l while maintaining the explorations of each particle on the search space. For each dimension d ∈ {1, . . . , D}, the standard PSO updates the velocity and position as follows: In (1), c 1 and c 2 are constants called acceleration coefficients to control the balance of the attraction between P l i and G l . ε 1 and ε 2 are independent random numbers uniformly distributed in [0,1] to add some randomness to the explorations. χ is the constrictive factor used to achieve the convergence of swarm [25]. The entire procedure of standard PSO for deterministic environments is summarized in Algorithm 1.
Meanwhile, as mentioned previously, the exact value of fitness f X l i cannot be obtained in stochastic environments, and the j th fitness evaluation for X l i yields only a noisy samplef j X l i . It is assumed thatf j X l i follows a normal distribution with unknown mean f X l i and variance σ 2 i (i.e.,f j X l i = f X l i + N 0, σ 2 i ). This normality assumption is reasonable in practical cases where f (·) is typically VOLUME 8, 2020 defined as a simulation model. This is because the simulation output is obtained from an average value or batch means, so the central limit theorem holds [10]. For reducing stochastic noise in the evaluation, the resampling approach takes sample meanf X l i = 1 n i · n i j=1f j X l i , where n i is the number of evaluations for X l i .

Algorithm 1 Particle Swarm Optimization for Deterministic
Environments generate m particles in the D-dimensional search space with an initial position X 1 i = x 1 i1 , . . . , x 1 iD and velocity iD randomly, where the particle index i = 1, . . . , m. set the personal best position P 0 i ← X 1 i for each particle i for l = 1 to l max do evaluate the fitness f X l i for each particle i update P l i for each particle i: i and X l+1 i for each particle i with (1) and (2) end returnG

B. OPTIMAL COMPUTING BUDGET ALLOCATION
Given k positions X 1 , . . . , X k , increasing n i equally for all positions is inefficient. In particular, when f (·) is a practical simulation model, it can be very time consuming. OCBA is aimed at finding the truly best position for which fitness is a minimum among f (X 1 ), . . . , f (X k ) by allocating simulation budget T (i.e., a limited number of fitness evaluations) to X 1 , . . . , X k efficiently. This objective can be defined as follows: That is, OCBA determines n * i , the optimal allocations of T for each position to maximize the probability of the correct selection of X b , P{CS}. Here, X b is the estimated best position based on the sample mean. n * i can be calculated as follows [10]: Here, the unknown variance σ 2 i can be approximated by the sample variance s 2 i in practice. These optimal allocation rules of OCBA asymptotically maximize a lower bound of P{CS} as T → ∞ (cf. [10] for the detailed mathematical derivations). Because n * i is calculated based onf (X i ) and s 2 i , OCBA uses a heuristic sequential update procedure as shown in Algorithm 2. Before the iteration is started, n 0 samples are collected for every X i to obtain the minimum data off (X i ) and s 2 i . Then, until T is exhausted, additional samples are allocated in a sequential manner such that n * i can be calculated based on more accurate data. (4) and (5)

Algorithm 2 Optimal Computing Budget Allocation Method
As mentioned before, previous studies integrated OCBA into PSO to mitigate the effect of noise by finding the best one among the particles' current positions (i.e., [13], [15], and [16]) or Pbest positions (i.e., [17]). The next section proposes a simple yet effective adjustment to the integration of OCBA to further improve the efficiency of PSO in stochastic environments.

III. PROPOSED ADJUSTMENT
As shown in (1), the movement of particles depends on their own Pbest position and Gbest position; thus, if every P l i and G l is selected correctly, the effect of noise can be eliminated (i.e., PSO in stochastic environments becomes the same as that in deterministic environments). For each particle, the correct selection of P l i is made when the inequality between . Given m particles, the number of correctly selected P l i , N {CSP} can be defined as follows: That is, every P l i is selected correctly when N {CSP}= m. After P l i is selected, the Gbest position is defined as the best one from the selected P l i as shown in Algorithm 1. Because f P l i is less than or equal to f P l−1 i and f X l i in deterministic environments, G l , which is the best among P l 1 , . . . , P l m , is the same as the best among X l 1 , . . . , X l m and P l−1 1 , . . . , P l−1 m as follows: Thus, in stochastic environments, the correct selection of G l is made when the selected G l based onf X l i andf P l−1 i is truly the best. The probability of the correct selection of G l , P{CSG} can be defined as follows: When both N {CSP} and P{CSG} are maximized to m and 1, respectively, the effect of noise in PSO can be eliminated. However, a huge number of fitness evaluations are required to achieve this, which leads to the inefficiency of PSO in stochastic environments. If increasing N {CSP} or P{CSG}, respectively, has a different influence on the performance of PSO, it may be more efficient to concentrate a limited evaluation budget on maximizing the dominant one of the two. To verify this strategy, numerical experiments for the nine benchmark problems have been conducted using the most basic resampling approach, PSO_ER, which equally distributes the number of fitness evaluations to X l 1 , . . . , X l m . A summary of the nine problems can be found in Table 2.
Here, the experimental environments were designed based on [18] because it showed the best performance among the previous studies integrated OCBA into PSO. Gaussian noise with the variance of 25 was added to each objective function to create a stochastic environment [18]. For identifying the influence of N {CSP} and P{CSG}, several variants that select P l i and G l in different ways have been suggested based on PSO_ER, which are summarized in Table 1.
As shown in Table 1, PSO_ER basically selects P l i and G l based on sample meanf X l i obtained via iterative fitness evaluations. However, the first variant selects both P l i and G l correctly based on f X l i regardless off X l i (i.e., it is the same as the PSO in deterministic environments). For checking the influence of N {CSP}, the second variant selects only P l i correctly based on f X l i , and G l is selected based on f P l i of the correctly selected P l i . On the other hand, the third variant selects P l i based onf X l i but selects G l correctly depending on (7), regardless of the selected P l i , to identify the influence of P{CSG}. Meanwhile, like the third variant, the fourth one also selects P l i based onf X l i . However, G l is defined as the best one of the selected P l i (i.e., the probability of the correct selection under the selected P l i , P{CS/P l ∀i } is 1); thus, the selected G l may be incorrect depending on the selected P l i (i.e., P{CSG} ≤ 1). All of the variants, along with the original version, have the same typical settings: m = 50, c 1 = c 2 = 2.05, and χ = 0.72984. The maximum number of iterations of PSO is set to 100, and the number of fitness evaluations in the lth iteration, T l is defined as 5, 000 + 100 (l − 1), where l = 1, . . . , 100. That is, each particle in the lth iteration will be given 100+2 (l − 1) fitness evaluations.
For the nine benchmark problems, the average value of f G l , the average value of N {CSP}, and P{CSG} are estimated via 1,000 independently repeated experiments for each variant. The results for Levy problem are shown in Fig. 1. In addition, Fig. 2 represents that the average value of f G 100 after 100 iterations of PSO is normalized between 0 and 1 based on the results of the first variant (i.e., without noise) and the original PSO_ER. As shown in Figs Table 1 for the nine problems normalized to between 0 and 1 based on the results of the first variant (without noise) and PSO_ER. Here, the error bars were not depicted due to their tiny values after repeating 1,000 times. P{CS/P l ∀i } could not achieve 1 of P{CSG}, it converged to a better solution than that of the second one. This tendency was similarly observed in the experimental results of the remaining eight benchmark problems, as shown in Fig. 2.
The reason for the greater influence of maximizing P{CSG} can be found in fact that unlike P l i , G l attracts all particles. That is, the correctly selected G l can lead the entire swarm in a better direction for convergence. In addition, as shown in Fig. 1(c), N {CSP} had a certain value based on the most basic equal resampling method, whereas P{CSG} was close to zero based on the second variant maximizing N {CSP}. In other words, it is expected that maximizing P{CSG} secures a certain value of N {CSP}, which helps to further reduce the effect of noise. To sum up, the performance of PSO in stochastic environments can be improved efficiently by concentrating a limited evaluation budget on finding the Gbest position correctly. Algorithm 3 (l, X ∀i , P ∀i , N ∀i , n 0 , , T ) Adjusted Optimal Computing Budget Allocation Method for PSO collect n 0 samples for each X i , i ∈ {1, . . . , m} set n 1 = · · · = n m ← n 0 and calculatef (X i ) and s 2 i for each i if l > 1; then load N i for eachP i to X m+i : (4) and (5) for each i ∈ {1, . . . , k}, where k i=1 n * i = T t+1 collect max n * i − n i , 0 samples for each X i set n i ← n i + max n * i − n i , 0 for each i updatef (X i ) and s 2 i for each i and set b ← argmin i∈{1,...,k}f For one to find the Gbest position correctly, it is necessary to consider not only X l 1 , . . . , X l m , but also P l−1 1 , . . . , P l−1 m as shown in (8). However, previous studies integrated OCBA to select the best position from only X l 1 , . . . , X l m (i.e., [13], [15], and [16]) or P l−1 1 , . . . , P l−1 m (i.e., [17]). The selected best position may help some with finding the correct Gbest position, but it has limitations when it comes to maximizing P{CSG} as shown in the results of the fourth variant. Thus, this article proposes a simple adjustment that allows OCBA integrated into PSO to select the Gbest position correctly Algorithm 2 (i.e., the original OCBA) can be applied as it is to solve the problem in (9), but for one to maximize the efficiency of PSO, some steps for reusing the previous evaluation data of P l−1 i need to be added. Because every X l i has never been evaluated, n 0 samples are collected for each X l i to obtain initial dataf (X i ) and s 2 i for the following iterative allocations. On the other hand, each P l−1 i already has the evaluation data obtained via OCBA in the previous iteration of PSO. Thus, the unnecessary waste of a budget can be prevented by replacing the initial data of P l−1 i with these data when applying the proposed adjustment. As shown in Fig. 3, X l 1 , . . . , X l m each receive five samples (i.e., n 0 = 5) through initial allocation, whereas P l−1 1 , . . . , P l−1 m skip the initial allocation and reuse the previously collected samples. In this way, 5m samples can be saved. In addition, because the number of previously collected samples is typically larger then n 0 , the precise initial data of P l−1 i increases the efficiency of the allocation rules of OCBA in (4) and (5).
Algorithm 3 represents the adjusted OCBA for efficiently solving the problem in (9) based on Algorithm 2, including the reuse step. It reuses the previous evaluation data of P l−1 i using lines 3-5 in Algorithm 3. Then, until given iD randomly, where the particle index i = 1, . . . , m. set P 0 i ← X 1 i for each particle i for l = 1 to l max do evaluatef X l i andf P l−1 i for each particle i using OCBA: i and X l+1 i for each particle i with (1) and (2) end return G budget T is exhausted, additional samples are allocated sequentially depending on the allocation rules of (4) and (5) to select the Gbest position correctly from X l 1 , . . . , X l m and P l−1 1 , . . . , P l−1 m . The gray area in Fig. 3 illustrates Algorithm 3, and the samples allocated via Algorithm 3 are indicated by a yellow and red bar. Note that P l−1 1 , . . . , P l−1 m do not have the yellow bar that represents the initial allocation of n 0 . Algorithm 4 shows the PSO with Algorithm 3. As shown in Fig. 3, depending on the fitness evaluation VOLUME 8, 2020 data of X l 1 , . . . , X l m and P l−1 1 , . . . , P l−1 m via Algorithm 3, Pbest and Gbest positions are selected, and particles' velocities and positions are updated. Here, the evaluation data of P l 1 , . . . , P l m (i.e., newly selected Pbest positions) are stored for reuse in Algorithm 3 at the next iteration of PSO.
Meanwhile, the computational complexity of the original OCBA (i.e., Algorithm 2) is calculated as O (k), were k is the number of positions in its search space [10]. Because the previous studies integrated OCBA to select the best position from only X l 1 , . . . , X l m or P l−1 1 , . . . , P l−1 m , their complexity of the integrated OCBA is O (m) (i.e., k = m). On the other hand, the proposed adjustment makes OCBA double its search space by considering both X l 1 , . . . , X l m and P l−1 1 , . . . , P l−1 m to find the Gbest position correctly. That is, k increases to 2m in Algorithm 3. Accordingly, the amount of computation increases, but it does not affect the complexity of Algorithm 3. From the perspective of the big-O notation, the complexity of Algorithm 3 is still O (m), as in the integrated OCBA of the previous studies. In addition, this increase in computation can be negligible compared with the fitness evaluation costs in practical situations where the objective function is usually a stochastic simulation model.

IV. EXPERIMENTS
This section exhibits the experimental results on both benchmark and practical problems to demonstrate the effectiveness of the proposed adjustment.

A. BENCHMARK PROBLEMS
Algorithm 4 with the proposed adjustment was compared with the recent existing works in the resampling approach: PSO_OCBA [18], PSO_UE [14], and PSO_HYL [16]. In addition, PSO_ER and its first variant in Table 1 (i.e., without noise) were used as a control group. PSO_HYL applied OCBA as is to correctly select the best position among the particles' current positions, whereas PSO_UE used UE for the same purpose. However, compared with PSO_HYL, PSO_UE considers G l−1 as well as X l 1 , . . . , X l m in the allocation to find the Gbest position more correctly. PSO_OCBA combined an OCBA-based variant that focuses on finding both Pbest and Gbest positions correctly. Because PSO_OCBA showed the best performance among the competitors, the experimental environments were designed based on [18]. Twenty-three benchmark problems were chosen from [18], [24], and [26], and Table 2 summarizes them. The objective functions in problems 1-6 are simple unimodal, whereas problems 7-10 in the shaded area have multimodal functions with few local minima. In the remaining problems 11-23, the objective functions are highly complex multimodal functions with many local minima. Gaussian noise with the variance of 25 was added to each objective function to create a stochastic environment. All competitors have the same typical settings of PSO: m = 50, c 1 = c 2 = 2.05, and χ = 0.72984. In addition, the parameters for the R&S methods were set as follows: n 0 = 10, = 100, and T l = 5, 000 + 100 (l − 1), where l = 1, . . . , l max . The average value of f G l , the average value of N {CSP}, and P{CSG} until reaching 100 iterations of PSO (i.e., l max = 100) were estimated via 1,000 independently repeated experiments for each competitor. The results for the four selected problems are shown in Fig. 4. Table 3 represents the average value of f G 100 and the standard error of each competitor for the 23 problems, and Fig. 5 normalizes these average values in Table 3 to between 0 and 1 for easy comparison.
The experimental results clearly indicate the effectiveness of the proposed adjustment. As shown in Table 3 and Fig. 5, Algorithm 4 yielded a good solution close to the solution of the ''without noise'' version compared with the competitors after 100 iterations of PSO. In addition, it converged to a good solution quickly as shown in Figs. 4(a)-(d). This is because it selects the Gbest position most correctly under the limited budget. In the case of PSO_OCBA, the budget is distributed to select not only the Gbest position but also the Pbest positions, which decreases P{CSG}. Because PSO_HYL and PSO_UE do not take into account P l−1 1 , . . . , P l−1 m , they cannot select the Gbest position correctly, like the fourth variant of PSO_ER in Table 1. On the other hand, Algorithm 4, VOLUME 8, 2020  Table 1 and the results of PSO_ER, where the error bars were not depicted due to their tiny values, as shown in Table 3.
which concentrates the budget on selecting the Gbest position correctly by applying the proposed adjustment, achieved a higher P{CSG} than the competitors did, although P{CSG} It is speculated that this is due to the characteristics of their objective functions, but more detailed analysis will be needed, which is left as future study.
If a given budget is large enough (e.g., T l → ∞), PSO_OCBA, which finds both Pbest and Gbest positions correctly, may outperform Algorithm 4. However, the proposed adjustment is effective in terms of efficiency to improve the performance of PSO under a limited evaluation budget. This efficiency is necessary for practical optimization problems of which the objective function is a stochastic simulation model. To demonstrate the necessity of the proposed adjustment, Algorithm 4 was applied to optimize the fighter's evasion flight.

B. PRACTICAL PROBLEM
The evasion flight of a fighter against an approaching missile consists of descending and soaring steps. When a fighter detects a missile approaching from 20,000 m away, the fighter descends in maintaining a constant angle, called the leading angle. Then, the missile also descends along with the fighter. When the distance between the fighter and missile is under a certain value, called the soaring distance, the fighter radically soars upward using the maximum G-factor. Because missiles are usually several times faster than fighters are, the missile cannot catch up with the fighter's movement and finally overshoots (i.e., the fighter survives). However, if the leading angle is too small, or if the soaring distance is too short or too long, the missile can continue to chase the fighter without overshooting and shoot it down.
Dominating the air is one of the most important factors in winning a modern battlefield, so it is necessary to determine the optimal values of the leading angle and the soaring distance to maximize a fighter's survival rate. Leading angle l ranges from 0 to 90 degrees, and soaring distance s ranges from 1,000 m to 15,000 m. Given X = [l, s], a combination of the two variables, the survival rate of the fighter can be evaluated using the six degrees of freedom (6 DoF) aircraft simulation model [27], denoted by f (X). Fig. 6 illustrates the evasion flight of a fighter simulated by the model. Because the model has many random variables for describing uncertainty in the complex real world, it returns the fighter's survival:f (X) = 1 or destruction:f (X) = 0 (i.e., f (X) follows Bernoulli distribution) per simulation run of X. That is, the survival rate for X is defined as the expected value of f (X), E [f (X)]. To sum up, this optimization problem can be defined as follows: To solve the problem of (4), Algorithm 4 applied with the proposed adjustment was used. In addition, the other competitors in the benchmark problems were applied together to check the improved efficiency of PSO via the proposed adjustment. All parameters except for T l were set the same as those in the benchmarks, and T l was set to 800 + 200 (l − 1). Unlike previous benchmark problems, the 6 DoF aircraft simulation model has high evaluation cost; thus, each competitor was applied only 10 times independently to solve the problem of (4), and 10 solutions were obtained per each. For each obtained solution G 100 , E f G 100 was precisely estimated with many independently replicated simulations. Table 4 represents the average value and the standard error of 10 precisely estimated E f G 100 s for each competitor.  Among 10 solutions of Algorithm 4, the representative solution of which the E f G 100 is closest to the average value (i.e., 0.7831) in Table 4    PSO_OCBA, PSO_UE, and PSO_ER, and about 6.1% higher than the solution of PSO_ER as shown in Table 4. Considering that it is crucial to increase the fighter's survival rate even if the increment is just 1%, this is a huge improvement. If a greater evaluation budget is given (i.e., T l increases), the other competitors may also find a better solution like Algorithm 4. However, as shown in Fig. 6, the cost per run of the 6 DoF simulation model is expensive and timeconsuming, which may lead to prohibitively high computational costs for the optimization.
The improved efficiency of Algorithm 4 is attributed to the fact that Algorithm 3 with the proposed adjustment can select the Gbest position correctly. Fig. 7 represents the precisely estimated values of E [f (X)] and σ [f (X)] with many independently replicated simulations of the 6 DoF model for finely discretized X within the search space. As shown in Fig. 7, although the absolute value of σ [f (X)] is small, it is relatively large compared with the small difference of E [f (X)] between neighbors, which acts as a large noise effect. Based on the results in Fig. 7, the average value of E f G l , the average value of N {CSP}, and P{CSG} until reaching 100 iterations of PSO were estimated through 100 independently repeated experiments for each competitor. As shown in the results in Fig. 8, Algorithm 4 achieved a higher P{CSG} than the competitors; the correctly selected Gbest position could mitigate the large noise effect and lead the entire swarm to a better solution.
Like this practical problem, the proposed adjustment is effective for solving simulation-based optimization problems that have large amounts of stochastic noise and high evaluation costs using PSO. In particular, simulation models, such as the digital twins, which are in the spotlight in the Fourth Industrial Revolution era, are continuously updated to maintain synchronization with the corresponding real systems [28]. In light of this, optimal control settings should be found quickly based on the updated models to maximize the performance of the real systems. In this case, the proposed adjustment is essential for the use of PSO.

V. CONCLUSION
This article proposed a simple yet effective adjustment to the integration of OCBA to further improve the efficiency of PSO in stochastic environments. The proposed adjustment causes OCBA to correctly select the Gbest position that has a significant influence on the performance of PSO compared with the correctly found Pbest positions. Although the previous studies select the best one from only the particles' current positions or their previous Pbest positions using OCBA, the proposed adjustment extends the search space of OCBA to consider both the current positions and the previous Pbest positions. Thereby, OCBA can correctly select the Gbest position that is defined as the best position among the extended space. In addition, the proposed adjustment prevents OCBA from wasting the evaluation budget by reusing the existing evaluation data of the previous Pbest positions. The results of comparative experiments with the latest studies on various benchmark problems demonstrated the improved efficiency of PSO due to the proposed adjustment. The PSO applied with the proposed adjustment could find the Gbest position correctly compared with the competitors, and this exact Gbest position led the entire swarm in a better direction; thus, the adjusted PSO yielded a superior solution within a limited evaluation budget. In particular, the results of the evasion flight optimization problem demonstrated the need for the proposed adjustment when using PSO to solve practical optimization problems that typically have large amounts of stochastic noise and high evaluation costs. The proposed adjustment is expected to enable PSO to be used more effectively in the Fourth Industrial Revolution era, which requires fast and accurate optimization based on the digital twins.