Parameter Estimation of Lorenz Chaotic System Based on a Hybrid Jaya-Powell Algorithm

Parameter estimation of Lorenz chaotic system using a novel hybrid Jaya-Powell algorithm is proposed in this paper. Since the nonlinear dynamic system is complex with multi-dimension parameters, estimating parameters of the system can be considered as a muti-objective optimization task. The proposed Jaya-Powell algorithm combines the Jaya and Powell algorithm to search for the relatively global optimum and local optimum respectively, which offers a more accurate and effective estimation. The searching strategy of the proposed algorithm facilitates the balance of the exploration and exploitation in the optimization procedure. Due to no algorithm-speciﬁc parameters are required in the Jaya and Powell algorithm, the proposed Jaya-Powell can avoid deliberate ﬁne-tuning of corresponding parameters. To validate the accuracy and robustness of the proposed algorithm in parameter estimation, the simulation of Lorenz chaotic system and comparative experiments are conducted. Seven algorithms, including Jaya algorithm, Powell algorithm, Teaching-learning-based optimization (TLBO) algorithm, particle swarm optimization (PSO), genetic algo-rithm (GA), covariance matrix adaptation evolution strategy (CMA-ES), and cluster-chaotic-optimization algorithm (CCO), are considered as benchmarking algorithms in the comparison. The proposed hybrid Jaya-Powell algorithm outperforms seven benchmarking algorithms with the more accurate estimation and the relatively faster convergence. Based on the embedded system Raspberry pi 3, the proposed algorithm achieves the similar performance by comparing with the experiments conducted on the computer. The successful implementation via Raspberry pi 3 facilitates the application of the proposed algorithm in edge computing.


I. INTRODUCTION
As a prominent complex behavior in nonlinear dynamical systems, chaos has attracted much attention from researchers and been widely studied over the past three decades.The phenomenon of chaos has been applied in various fields, such The associate editor coordinating the review of this manuscript and approving it for publication was Mauro Gaggero .as image encryption [1], secure communication [2], financial market [3], and etc. Lorenz chaotic system [4], which was firstly proposed by the meteorologist Edward Lorenz, is considered as one of the representative chaotic systems.The major characteristics of Lorenz chaotic system is of two-fold, the extreme sensitivity to initial conditions and unstable periodic orbits.To control and synchronize Lorenz chaotic system, it is critical to capture the information of parameters due to the complexity of such system.However, determining parameters of Lorenz chaotic system is a challenging task in real applications.Thus, parameter estimation is a meaningful task in the analysis of Lorenz chaotic system.
A few pioneer studies related to parameter estimation of chaotic systems via the synchronization based methods have been reported in the literature.Parlitz et al. [5] developed a computer model to minimize the average synchronization error for parameter estimation of an unknown chaotic system.Maybhate and Amritkar [6] combined the synchronization and an adaptive control method to estimate parameters under a noisy environment.Hyun et al. [7] proposed an adaptive fuzzy observer for the synchronization of chaotic systems.Wang and Ge [8] discussed an adaptive backstepping method to synchronize uncertain chaotic systems, which was also reported by Yu and Zhang [9].Yu et al. [10] introduced a backstepping synchronization method based on the equivalent transfer function for chaotic systems.Although the synchronization based methods are capable to estimate parameters of chaotic systems, their real implementations might be lack of feasibility and bounded to specific models.
To achieve the feasibility in parameter estimation of chaotic systems, evolutionary computation based methods have been applied in this field and address parameter estimation as an optimization problem.Numerous studies based on classical evolutionary computation algorithms have been developed to estimate the parameter of uncertain chaotic systems.Tao et al. [11] applied genetic algorithm (GA) into parameter estimation of chaotic time series.He et al. [12] developed particle swarm optimization (PSO) to estimate parameters of chaos systems and achieved more accurate solutions by comparing with GA.Tang and Guan [13] introduced PSO to address the issue of parameter estimation in a time-delay chaotic system.Sun et al. [14] proposed a drift particle swarm optimization (DPSO), which is an improved algorithm based PSO, to estimate parameters of chaotic systems.Peng et al. [15] considered parameter estimation of Lorenz chaotic system via the differential evolution algorithm (DE) and discussed the influence of the population size on the optimization performance.Xiang-Tao and Ming-Hao [16] proposed a Cuckoo search algorithm with the orthogonal learning to estimate parameters.Zhang et al. [17] identified the unknown parameters of the chaotic system based on teaching-learning-based optimization (TLBO).Reported evolutionary computation algorithms [11]- [17] proved the feasibility of parameter estimation for chaotic systems.However, the aforementioned evolutionary computation algorithms are susceptible to being trapped into local optima and suffering from the premature convergence.
Based on classical evolutionary computation algorithms, hybrid methods which combine two different evolutionary algorithms have been developed to obtain better estimation results.Wang and Xu [18] proposed a hybrid biogeographybased optimization(BBO) algorithm to recognize unknown parameters.Gu et al. [19] introduced a hybrid algorithm named as HABCDE to estimate model parameters, which combined the differential evolution algorithm (DE) and the artificial bee colony algorithm (ABC).Lazzús et al. [20] developed a hybrid PSO-ACO algorithm for parameter estimation of chaotic systems.Gálvez et al. [21] combined the characteristics of clustering and the randomness of chaotic sequence, named as cluster-chaotic-optimization, to solve the optimization problem in chaotic systems.The developed hybrid algorithms [18]- [20] have the ability to achieve the satisfactory estimation performance, which relies on finetuned algorithm-specific parameters.
This research proposes a hybrid algorithm named as Jaya-Powell algorithm, which combines Jaya algorithm [22] and Powell algorithm [23] to balance the exploration and exploitation in parameter estimation of Lorenz chaotic system.Firstly, Jaya algorithm, a population-based method proposed by Rao [22] free of algorithm-specific parameters, is applied to optimize parameters of Lorenz chaotic system.The main idea of Jaya algorithm is offering the probability for candidate solutions to move close to the best solution and away from the worst solution.Based on the update strategy, Jaya algorithm is capable to explore more unknown spaces to obtain useful information and avoid to fall into the local optimum.As an adaptive algorithm, Jaya only needs basic parameters including the population size and the number of maximum iterations.Next, Powell algorithm [23] is employed to exploit the local information based on the approximatively global optimum obtained by Jaya algorithm.Finally, the optimal solution of parameter estimation for Lorenz chaotic system can be attained.To validate the performance of the proposed Jaya-Powell algorithm, seven algorithms are considered as the benchmarking algorithms in the comparative experiments, including Jaya, Powell, TLBO, PSO, GA, CMA-ES, CCO.Based on simulations, the proposed algorithm outperforms seven benchmarking algorithms on the accuracy and robustness of parameter estimation for Lorenz chaotic system.
The remaining parts of this paper are organized as follows.In Section II, the problem formulation is introduced.Next, the proposed hybrid algorithm, Jaya-Powell algorithm, is described in Section III.The benchmarking algorithms employed in comparative experiments are illustrated in Section IV.Section V discussed simulation results based on different algorithms.Finally, Section VI makes the conclusion.

II. PROBLEM FORMULATION
The principle of unknown parameter estimation for chaotic systems is described in this section.

A. THE GENERIC CHAOTIC SYSTEM
A n-dimensional generic chaotic system is considered as following (1): where X = (x 1 , x 2 , . . ., x n ) T ∈ R n represents the state vector of the original system, X 0 denotes the initial state, and ϕ 0 = (ϕ 0 1 , ϕ 0 2 , ..., ϕ 0 n ) represents a set of original systematic parameters.
Assuming that the structure of the generic chaotic system (1) is known, then the corresponding estimated system can be described as (2): where Y = (y 1 , y 2 , . . ., y n ) T ∈ R n denotes the state vector of the estimated system, φ = ( φ1 , φ2 , . . ., φn ) represents the set of estimated parameters.Estimating unknown parameters of the generic chaotic system can be formulated as an optimization problem, which targets on searching for a set of optimal parameters.The obtained optimal parameters can minimize the gap of behaviors between the original system and the estimated system.To attain the estimated parameters φ1 , . . ., φn , minimizing the loss function ( 3) is required.The optimization principle of parameter estimation for the generic chaotic system is depicted in Fig. 1 minJ where state vectors of the original system and the estimated system respectively, which are observed at time k.K is the total number of state vectors used in the estimation.
It is difficult to estimate parameters in the generic chaotic system due to its unstable dynamic orbits.Meanwhile, the behavior of the generic chaotic system is extremely sensitive to the initial state.Classical optimization algorithms are easy to fall into the local optimum, which cannot guarantees a satisfactory solution.

III. JAYA-POWELL ALGORITHM
In this section, a novel hybrid algorithm named as Jaya-Powell is proposed.The Jaya-Powell algorithm combines the adaptive Jaya algorithm and Powell algorithm to balance the exploration and exploitation, which offers the high probability to search for the global optimum.In the optimization procedure, Jaya algorithm [22] is firstly applied to find a relatively good solution and narrow down the searching space for further searching.Next, Powell algorithm A. JAYA ALGORITHM Jaya algorithm is an adaptive optimization algorithm, which facilitates candidate solutions to get closer to the current optimum and move away from the worst solution over iterations.The major characteristics of Jaya algorithm is that it is free of algorithm-specific parameters, such as the mutation probability and crossover probability of GA as well as social and cognitive parameters of PSO.The algorithm-specific parameters can greatly influence the optimization performance and need to be well fine-tuned.In Jaya algorithm, only the population size and the number of iterations need to be set in advance.
The update strategy of Jaya algorithm is illustrated as ( 5 The optimization procedure of Jaya algorithm is summarized as follows: Step 1: Set the population size, the range of parameters, and the maximum number of iterations.
Step 2: Initialize the population randomly with the specific range of parameters.
Step 3: Identify the best and worst solutions in the current solution space.
Step 4: Update the current solution based on the best and worst solutions according to eq. ( 5).
Step 5: Evaluate the value of the objective function for each updated solution.Then the updated solution will be accepted or abandoned according to eq. ( 6).
Step 6: If the maximum iteration is satisfied, the optimization procedure will be stopped.Otherwise, the procedure jumps to Step 3.

B. POWELL METHOD
Powell algorithm, targeting on searching for the optimum in multi-dimension solution space, is considered as a local optimization algorithm without algorithm-specific parameters.
It is a searching method employing the conjugate direction to speed up the convergence rate during the optimization.Meanwhile, Powell algorithm utilizes a bi-dimensional searching strategy instead of a hopping probe step to minimize the objective funtion, which can be applied to the nondifferentiable optimization problem.Let X 0 be the initial state vector and the objective function is denoted by f .Initialize a set of directions u i as the standard base vectors, u i = e i , i = 1, 2, .., n.The procedure of Powell method is summarized as follows: Step 1: Set the start point P 0 = X i and tolerance error e.
Step 2: Find the value of γ = γ i to minimize f (P i−1 +γ i ×u i ) and set P i = P i−1 + γ i × u i for i = 1, 2, .., n,.
Step 4: Set u n = P n − P 0 .
Step 6: Find the value of γ = γ min to minimize f (P 0 + γ min * u n ) and set X i = P 0 + γ min × u n .
Step 7: Repeat Step 1 to Step 5 until e is satisfied.
The initial vectors of Powell algorithm are replaced by a set of new conjugate vectors completely after n iterations.During the iterations, u n = P n − P 0 become linearly dependent at each stage and might contain incorrect information.The enhanced Powell method, which replaces one of the direction vectors selectively based on an obtained new direction vector, is proposed to guarantee better conjugate properties of the new group of vectors.The procedure of Powell method is illustrated in Algorithm 2.

Algorithm 2 Powell Algorithm
Input: Initial solution: X 0 ; Tolerance error: e; Objective function: f ; Output: Solution: X best ; repeat Set P 0 = X 0 ; for k := 1 to n do for i := 1 to n do Initialize the set of direction u i = e i ; Find a minimizer γ i ; Calculate f Record the subscript r; Set the u r as the maximum decrease; Calculate Set u = P n − P k ; Find the minimizer γ min ; Calculate f (P 0 + γ min × u); Update the search direction: [u 1 , . . ., u r−1 , u r+1 , . . ., u n , u]; Set X k+1 = P n + γ min × u;

IV. BENCHMARKING ALGORITHMS
To verify the performance of the proposed Jaya-Powell algorithm on parameter estimation for Lorenz chaotic system, seven benchmarking algorithms, including Jaya algorithm, Powell algorithm, teaching-learning-based optimization VOLUME 8, 2020 algorithm (TLBO), particle swarm optimization (PSO), generic algorithm (GA), covariance matrix adaptation evolution strategy (CMA-ES), cluster-chaotic-optimization algorithm (CCO), are employed in comparative experiments.Brief introductions of seven benchmarking algorithms are described in this section.

A. PSO
The major characteristics of PSO is to find the optima through the collaboration and information sharing among individuals.A group of particles in PSO move around the searching space based on the historical information of their own best locations and the best location of the whole swarm.The velocity of a particle is updated after one iteration via (7): where v i (t) is the velocity of the ith particle at time t.c 1 and c 2 are acceleration coefficients of the personal best p best and the global best position g best respectively.ω denotes the scaling factor of the particle velocity.x i (t) is the current position of the ith particle at time t.The position of a particle will be updated at time t + 1 based on (8): B. TLBO TLBO algorithm consists of two components, the teacher phase and the learner phase.As a global optimization algorithm, the best solution denotes the teacher and the remaining solutions are the learners at each iteration.TBLO algorithm does not require other algorithm-specific parameters except the population size and the number of iterations.

1) TEACHER PHASE
The parameter of a student will be updated according to the difference between the average performance and the best performance via ( 9) and ( 10) where M j,i is the mean value of jth subject after ith iterations.r i is a random number within the range of [0,1].X j,kbest,i is the best learner of jth subject.T F is the teaching factor and the X j,k,i is the updated solution.

2) LEARNER PHASE
Next, two students are selected randomly among the group of students to learn from the teacher, which offers the probability of enhancing the performance of students.

C. GA
GA is inspired by the law of evolution in nature.The optimization procedure of GA is summarized as follows: Step

D. CMA-ES
CMA-ES is considered as a state-of-the-art evolutionary strategy for the derivative-free global optimization.Firstly, individuals are initialized randomly and sampled according to the multivariate normal distribution.Next, selected individuals are considered as the parents and re-ordered based on the fitness function.Then the re-ordered individuals are updated via the covariance matrix of the current distribution.The iteration will be stopped once the optimum has been found.

E. CCO
The optimization strategy in CCO combines the characteristics of the clustering and the randomness of chaotic sequences.Based on CCO, the population is firstly divided into different clusters in each generation.Next, individuals in each cluster are updated via two operators named as intracluster and extra-cluster.Chaotic sequence are applied in such two operators to generate random numbers for searching better solutions.The procedure will be repeated to find the best solution.

V. SIMULATION AND COMPARITIVE EXPERIMENTS A. EXPERIMENT ENVIRONMENT
Simulation experiments are conducted on a computer having a single i5-6500 CPU and 4G memory.Meanwhile, the performance of the proposed algorithm has also been validated on Raspberry pi 3, which has the advantages of small size and portability.As the embedded device, Raspberry pi 3 is powerful for solving the ordinary problem and can be used in many conditions such as non-personal computer.It is a single-board computer developed by Raspberry Pi foundation which focuses on promoting the basic of technology on the computer.Based on Raspberry pi 3, the proposed algorithm is capable to be applied in edge computing.

B. SIMULATION
To demonstrate the superiority of the proposed algorithm, the original Lorenz system evolves freely from a random initial state firstly.A state vector X 0 = (0., 1., 1.05) is set as the starting point to conduct the behavior.300 successive state vectors, which are defined as X 1 , X 2 , . . ., X 300 and Y 1 , Y 2 , . . ., Y 300 , are selected both from original and estimated systems to calculate the fitness value.The step size of chaotic behaviors t is set to 0.01.The value of the objective function J is calculated the function (11): Parameter settings of the proposed algorithm and seven benchmarking algorithms are described as follows.To the same restrictive conditions for different algorithms, the maximum number of iterations, the relative error of the stopping criterion in the convergence, and the population size are set as 100, e = 10 −4 , 100 respectively.The searching ranges of three parameters are set as 9.0 ≤ σ ≤ 11.0, 20.0 ≤ ρ ≤ 30.0, 2.0 ≤ β ≤ 3.0 separately.Based on aforementioned parameters, there is no other algorithm-specific parameters to set for Jaya-Powell algorithm, Jaya algorithm, Powell algorithm, TLBO, and CCO.The algorithm-specific parameters of PSO are set as c 1 = c 2 = 2.05 and ω = 0.729 [24].In GA, the length of the chromosome L, the crossover rate P r , and the mutation rate P m are set as 3, 0.8, and 0.1 [12].The σ = 0.5 is set in CMA-ES, which is initialized randomly.In addition, each algorithm are conducted via 20 simulation experiments under the same condition to obtain more universal results.

C. RESULTS AND DISCUSSION
To validate the optimization performance of the proposed Jaya-Powell algorithm, two types of comparative experiments are employed.In the first type of comparison, all algorithms are conducted via a single experiments to roughly illustrate the corresponding convergence.As shown in Fig. 2, algorithms except Powell and CCO are capable to converge to a relatively satisfactory solution.Meanwhile, the proposed Jaya-Powell algorithm reaches the highest degree of accuracy among all algorithms, which is around e −20 .
In the second type of comparison, all algorithms are conducted to for parameter estimation of σ , ρ, and β over 20 independent experiments separately.The repeated experiments make it easier to spot anomalies and offer the more reliable performance of different algorithms.The optimization performance via the proposed Jaya-Powell algorithm and seven benchmarking algorithms, including the best, average, and worst performance over 20 repeated experiments, are summarized in Table 1.According to Table 1, it is clear that the proposed Jaya-Powell algorithm outperforms seven benchmarking algorithms on the best, average, and worst performance.The average performance of Jaya-Powell algorithm demonstrates its robustness of parameter estimation for Lorenz chaotic system, which also validates its independence of the random initialization.However, other benchmarking algorithms, such as Powell, GA, CMA-ES and CCO, achieves a poor level of robustness due to the overdependence on well-selected initializations.If the initialization of Lorenz chaotic system is away from the original solution or close to local optima, it is challenging for these benchmarking algorithms to search for satisfactory solutions.Fig. 3 describes the relative estimation error of parameters and the value of fitness function for different algorithms over 20 experiments.As depicted in Fig. 3, the strong randomness of initializations results in much negative effects on Powell algorithm while less impacts on Jaya algorithm.Thus, if a relatively better solution is obtained by Jaya algorithm, Powell algorithm can exploit the nearby space based on the obtained solution to achieve a more accurate solution.To illustrate the overall performance of the proposed algorithm and seven benchmarking algorithms, the root-meansquare error (RMSE) of the fitness function and average running time are utilized for the 20 repeated experiments.Based on Table 2, the proposed algorithm attains the lowest RMSE to demonstrate its robustness on parameter estimation.There is no significant difference of the average running time among different algorithm except CCO.Thus, the proposed Jaya-Powell algorithm achieves the best overall performance on parameter estimation of Lorenz chaotic system.

VI. CONCLUSION
In this paper, a novel hybrid algorithm combining Jaya algorithm and the Powell method named Jaya-Powell was proposed for parameter estimation of Lorenz chaotic system.The proposed hybrid algorithm focused on efficiently searching for a satisfactory solution via balancing the exploration and exploitation, which avoided to fall into the local optimum.The two types of comparative experiments demonstrated the superiority of the proposed algorithm on parameter estimation of Lorenz chaotic system by comparing with seven benchmarking algorithms.Due to the characteristics of nonspecific-parameters, it took less time for Jaya-Powell algorithm to converge to the optimum based on the guarantee of the estimation accuracy.
The numerical experiments was also conducted on the embedded environment.We conducted the parameter estimation via the proposed Jaya-Powell method on the Raspberry Pi 3. It demonstrated the feasibility of parameter estimation of Lorenz chaotic system on the embedded platform, which can be applied into edge computing.Both experiments on computer and Raspberry Pi 3 have validated the outperformance of the Jaya-Powell algorithm.
The future work will extend the parameter estimation of chaotic systems through the GPU-based implementation to achieve the high-speed performance.

FIGURE 1 .
FIGURE 1.The optimization principle of parameter estimation for the generic chaotic system.
[23] is employed to search for the best local solution based on the reduced searching space obtained from Jaya algorithm.The rapid convergence of Jaya algorithm and Powell algorithm ensures the searching speed during the optimization.The main optimization procedure of the proposed Jaya-Powell algorthim is described in Algorithm 1. Algorithm 1 Jaya-Powell Algorithm Input: The population size: N , Iteration: M , Tolerance: e, Model parameters of the ith population at the jth iteration: O i,j Output: Solution: O * best for i := 1 to N do Initialize O i,1 ; end Set j = 1; repeat Update solutions O i,j based on Jaya algorithm; Get the best solution O best ; Set e = J (O best ); Set m = m + 1. until termination criterion e or j(j <= M ) satisfied; Update O best via Powll algorithm to O * best ; Return O * best .
): O m,p,n = O m,p,n + r 1,p,n (O m,best,n − |O m,p,n |) −r 2,p,n (O m,worst,n − |O m,p,n |) (5) where O m,p,n is the value of the mth variable for the pth candidate at nth iteration.The O m,best,n and O m,worst,n are the values of the best and worst candidates in the searching space for the mth variable at the nth iteration.r 1,m,n and r 2,m,n are two random numbers for the mth variable at the nth iteration in the range [0, 1].O m,p,n is the updated solution.Based on the objective function J , the updated solution will be accepted and passed to the next iteration based on the rule (6).

1 :
Initialize the number and the length of chromosomes Step 2: Calculate the fitness value of individuals and select two individuals named as parents for next iteration.Step 3: Recombine two parents to generate children for the next iteration (crossover).Step 4: Mutate parents from children randomly.Step 5: Calculate the fitness value of each individual in the population and record the best solution.Step 6: Repeat Step 2 to Step 5 until the result is satisfied.

FIGURE 2 .
FIGURE 2. The convergence of different algorithms over a single experiment.

FIGURE 3 .
FIGURE 3. The relative estimation error of parameters and the value of the fitness function for different algorithms over 20 experiments: (a) σ ; (b) ρ; (c) β; (d) the value of the fitness function.
LI ZHUANG received the B.Eng. degree in measurement and control technology and instrument from the Huazhong University of Science and Technology, Wuhan, China, in 2016.She is currently pursuing the Ph.D. degree with the School of Data Science, City University of Hong Kong.Her research interests focus on computer vision, machine learning, computational intelligence, and related applications in intelligent transportation, renewable energy, and the industry.LONGPENG CAO received the B.S. degree in computer science and technology from the University of Science and Technology Beijing, in 2017, where he is currently pursuing the M.Phil.degree in computer science and technology.YONG WU received the master's degree from Southwest University, China, in 2003.He is currently a Senior Agronomist with the National Agricultural Technology Extension and Service Center, Beijing, China.His research interests focus on the soil water engineering domain.YONGHONG ZHONG received the master's degree from Northwest Agricultural and Forestry University, China.She is currently a Senior Agronomist with the National Agricultural Technology Extension and Service Center, Beijing, China.Her research interest focuses on the soil water engineering domain.LILI ZHANGZHONG received the Ph.D. degree in agricultural water-soil engineering from China Agricultural University, China, in 2016.She is currently a Senior Engineer with the National Research Center of Intelligent Equipment for Agriculture, Beijing, China.WENGANG ZHENG received the Ph.D. degree in computational mechanics from the Dalian University of Technology, China, in 2002.He is currently a Professor with the National Research Center of Intelligent Equipment for Agriculture, Beijing, China.His research interest focuses on the agricultural engineering domain.LONG WANG (Member, IEEE) received the M.Sc.degree (Hons.) in computer science from University College London, London, U.K. in 2014, and the Ph.D. degree in systems engineering and engineering management from the City University of Hong Kong, Hong Kong, in 2017.He is currently an Associate Professor with the Department of Computer Science and Technology, University of Science and Technology Beijing, Beijing, China.His research interests include machine learning, computational intelligence, computer vision, and their industrial applications.He was a recipient of the Hong Kong Ph.D. Fellowship, in 2014.He serves as an Associate Editor of IEEE ACCESS and Canadian Journal of Electrical and Computer Engineering, as well as an Academic Editor of PLOS ONE.He is also a Lead Guest Editor of data science-related special issues on Intelligent Automation & Soft Computing and Water.20522 VOLUME 8, 2020

TABLE 1 .
Statistical results of parameter estimation for Lorenz system based on different algorithms.

TABLE 2 .
The overall performance of different algorithms.