An Improved Sparrow Search Algorithm for Optimizing Support Vector Machines

Support Vector Machine (SVM) is often used in regression and classification problems. However, SVM needs to find proper kernel function to solve high-dimensional problems. We propose an improved Sparrow Search Algorithm Support Vector Machine (ISSA-SVM) algorithm to optimize the SVM kernel parameters. First, the problem of slow convergence due to the lack of ergodicity and poor diversity of the initial population is effectively overcome by using Sine chaotic map. Second, adaptive dynamic weight factors are induced not only to balance the global and local search capabilities, but also accelerate the convergence speed of the sparrow search algorithm. The simulation results of 11 benchmark test functions show that ISSA has faster convergence, more accurate search capability, and easier to jump out of local extremes than the SSA, Gray Wolf Optimization (GWO) and Whale Optimization Algorithm (WOA). That indicates ISSA has better convergence, better robustness, and stronger competitiveness. The experimental results on the coal gangue dataset show that the classification accuracy of ISSA-SVM algorithm is improved by 7.09% and 4.25% compared with SVM and SSA-SVM, respectively. Meanwhile, the classification time for a single image frame of ISSA-SVM algorithm is reduced by 20.15% and 13.74% compared with SVM and SSA-SVM, respectively.


I. INTRODUCTION
Support Vector Machine (SVM) [1], as a classic machine learning algorithm, is often used in regression and classification problems. When compared with other classifiers such as random forest and Bayes, it offers significant benefits in tackling problems of limited sample size, non-linearity, and high dimension while maintaining decent accuracy and efficiency [2]. However, SVM needs to find proper kernel function to solve high-dimensional problems. Therefore, the selection of kernel parameters seriously affects the performance of SVM. Manual selection of parameters will lead to poor accuracy and low efficiency of the classification algorithm. The swarm intelligence optimization algorithm can solve this problem. Many scholars have used the swarm intelligence optimization algorithm, such as the Gray Wolf Optimization algorithm The associate editor coordinating the review of this manuscript and approving it for publication was Jon Atli Benediktsson .
(GWO) [3], Whale Optimization Algorithm (WOA) [4] to optimize the SVM kernel parameters. However, the search accuracy of the GWO is easily affected by the head wolf and the search speed is low. WOA is easily influenced by the population's search range and cannot escape from local optima. As a result, new optimization algorithms must be developed to assure great accuracy and efficiency.
Sparrow Search Algorithm (SSA) [5] is a new form of intelligent optimization algorithm created by Chinese academic Xue Jiankai in 2020. The algorithm has a few initial parameters, and the position of the discoverer is constantly changing due to the influence of the early warning value, which can increase the search speed. Furthermore, the method includes a unique warning mechanism that permits the program to escape from the local optimum.
Yuanbo et al. [6] used SSA-SVM for state evaluation of relay protection devices. They showed that the accuracy of SSA-SVM in evaluating the health state of protection devices Nevertheless, SSA-SVM has certain drawbacks. First, it has insufficient variety of the starting population which leads to poor convergence ability in subsequent iterations [7]. Second, it has inadequate search accuracy due to the fast convergence speed. An improved sparrow search optimization SVM method (ISSA-SVM) is proposed in this paper to solve the drawbacks of SSA-SVM. First, an enhanced sine chaotic map is presented to increase the search range and assure the variety of the beginning population of SSA to solve the problem of insufficient diversity of the initial population. Second, an adaptive dynamic weight factor is adopted to balance the capability between global search and local search of the SSA-SVM. The ISSA is tested in the 11 benchmark test functions. Furthermore, the ISSA-SVM is used to classify the coal and gangue.

II. IMPROVED SPARROW SEARCH ALGORITHM A. THE PRINCIPLE OF SPARROW SEARCH ALGORITHM
There are two kinds of sparrows in the SSA, they are producer, scrounger. In addition, some sparrows are required to reconnaissance and early warning. The SSA need to update position of the producer, the position of the scrounger and the mechanism for reconnaissance and early warning. In SSA, the producer's own fitness is good, and it has a broad search range to guide the sparrow population to search. In order to obtain better fitness, the scroungers will either follow the producers to forage or go to different locations to gain additional energy. Some newcomers, though, will keep an eye on the discoverers and steal their food. When sparrow populations realize they are in danger of being preyed on, they immediately engage in anti-predation mechanisms.
By simulating the collective predation behavior of the sparrow population, SSA seeks the optimal solution in the solution space. Assuming that the sparrow population X consists of n sparrows, and each sparrow has m dimensions, we have The fitness value F X of all sparrows is expressed as: where f (·) represents the fitness function value.
The update position of the producer is shown in formula (3). Generally, the producer occupies 10%∼20% of the sparrow population: X t i,j is the current position information of the ith sparrow in the jth dimension, and t is the number of iterations. α is a random integer in the range (0,1), iter max is the maximum number of iterations, Q is a random number with a normal distribution, and L is a 1 * m matrix with all members equal to 1. R 2 ∈ [0,1] and S T ∈ [0.5,1] represent the warning value and the safety value, respectively. When R 2 <S T , it means that the foraging environment of the population is safe, no predators are found, and extensive search can be carried out. When R 2 ≥ S T occurs, it indicates that a sparrow has discovered predators and that the population must relocate to avoid predation.
The update of the scrounger's position is shown in formula (4). Except for the producer, all of the other sparrows in the population are scroungers.: In the formula (4), X worst represents the global worst position, X t+1 p represents the best position occupied by the current producer, A + = A T (AA T ) −1 , A is a 1×m matrix, the elements of it are randomly assigned 1 or −1. When i > n/2, it means that the ith sparrow with lower fitness is very hungry and needs to fly to other places to forage alone to obtain more energy. When i ≤ n/2, it means that the ith scrounger sparrow flies near the optimal location to forage.
The position update of sparrows that are aware of the danger is shown in formula (5), which generally occupies 10%∼20% of the total sparrow population: where X t best is the current global optimal position. β is the step size control parameter obeying the normal distribution of (0,1), K is a random number between [-1,1], ε is a very tiny constant to avoid the denominator from becoming zero. f i is the fitness value of the sparrow individual, and f w and f g are the current global worst fitness values and global best fitness values, respectively. When f i >f g , it means that the sparrow is on the edge of the population and is easy to be caught, so it needs to move to the global optimal position; when f i =f g , it means that the sparrow in the center is aware of the danger and should move to other sparrows to reduce the risk of predation.

B. IMPROVED SPARROW SEARCH ALGORITHM
The initial population of the SSA algorithm is not uniformly distributed in the search space, so it has low ergodicity, less population variety and slow speed of convergence. Furthermore, the e −x exponential function in equation (3) shows that the algorithm has slightly faster decline speed and smaller disturbance range at the start of the iteration, resulting in insufficient search. However, the decline speed of the algorithm is slow and the disturbance range is too large in the middle and later stages. It leads to slow convergence and easy to fall into local excellent. Therefore, this paper improves SSA algorithm from the following two aspects -optimizing the initial population and finding the optimal strategy.

1) OPTIMIZING INITIAL POPULATION USING IMPROVED SINE CHAOS MAP
Because of its unpredictability, ergodicity, and regularity, chaotic mapping is frequently employed by academics to tackle the problem of unequal distribution of starting populations. The Sine map is obtained by deforming the sine trigonometric function, but the map is not uniformly distributed in the phase plane, and the distribution probability varies greatly. Reference [8] proposed an improved sine chaotic map, the system model is as follows: The initial values are a 0 and b 0 , which are both random numbers in the range [0, 1]; the control parameter µ is any real number. When µ is a real number with absolute value greater than 1000, the chaotic system is uniformly distributed in the phase plane. y i+1 is the chaotic sequence value generated by iteration, mod represents modulo operation. The distribution of chaotic sequences generated by the chaotic map is shown in Figure 1.

2) INTRODUCING DYNAMIC ADAPTIVE WEIGHT TO IMPROVE THE OPTIMIZATION STRATEGY
The adaptive dynamic weight ω may effectively control the algorithm's balance between global and local search, allowing the system to adjust its search capacity with the number of iterations. In the early stages of the search, a wider range of search is required, thus a bigger weight must be assigned to ensure that the sparrow population has a higher large-scale search ability. Dynamic adaptive dynamic weights are included into the producer position update formula (3) in this research to increase the algorithm's search performance. The expression of ω is shown in the formula (7): where iter max is the maximum number of iterations of the algorithm, α and β are both random numbers in [0,1]. Figure 2 is the function image of ω.
The update formula of the producer position after adding ω is shown in the formula (8): In the formula (8), f t ib is the optimal fitness in the current iteration, and α is a random number in [0,1].

C. IMPROVED SPARROW SEARCH ALGORITHM STEPS
The steps of the improved sine chaos mapping and adaptive dynamic weights-based sparrow search algorithm (ISSA) are as follows: Step 1. Initialize parameters. The parameters include the population size n, the proportion of producers P D , the proportion of reconnaissance and early warning sparrows S D , the early warning value R 2 , the dimension m of the solving problem, the upper bounds ub, lower bounds lb, and the maximum number of iterations iter max .
Step 2. Use the improved sine chaotic map sequence in formula (6) to initialize the population, generate sparrow population X: n×m, and map them into the solution space of the original problem.
Step 3. Calculate and sort the fitness value f (·) of the sparrow, select the optimal fitness value f g and the worst fitness value f w , and record the corresponding position information X best and X worst .
Step 4. Select the producers. Only sparrows with better fitness value are selected as the producers according to the set ratio P D , and all the other sparrows are scroungers. The positions of producers and scroungers are updated according to the improved formula (8) and formula (4), respectively.
Step 5. Select S D sparrows randomly from the sparrow population for reconnaissance and early warning, and update their positions based on the formula (5).
Step 6. If the method has achieved the maximum number of iterations iter max or the solution accuracy, the loop is terminated and the optimization result is printed; conversely, return to step 3.

D. TESTING AND ANALYSIS OF THE IMPROVED SPARROW SEARCH ALGORITHM 1) PARAMETER SETTINGS
An optimization test is performed to validate the validity and practicality of ISSA. The SSA, WOA, and GWO were used for comparison. The maximum number of iterations of these four algorithms are set to 500 and the population size is 30 for the fairness. The other parameters of each algorithm are displayed in Table 1.

2) TEST FUNCTION
11 different types of benchmark functions were chosen to test the GWO, WOA, SSA and ISSA algorithms, respectively. There are four single-peak test functions (F 1 ∼ F 4 ), four multi-peak test functions (F 5 ∼ F 8 ) and three fixeddimension multimodal test function (F 9 ∼ F 11 ) in the 11 benchmark functions. Table 2 gives the basic information of the test functions.

3) EXPERIMENTAL RESULTS AND ANALYSIS
The simulation tests are carried out using MATLAB R2018b version, and the stability and convergence of the algorithm are checked using the test functions presented in Table 2. To eliminate bias induced by a single run, we run each method 30 times individually to determine the optimal value, average value, and standard deviation of the test function. The average value shows convergence accuracy of the algorithm, while the standard deviation reflects the robustness and stability of the algorithm. Table 3 displays the test results.
Because the unimodal test functions (F 1 ∼ F 4 ) have just one global optimum solution and no local solutions, they can better represent the local development capability of algorithms. According to the test of the unimodal functions in the Table 3, the optimization performance of ISSA is significantly superior to that of other algorithms, and its stability is robust. The original SSA performs better for the functions with the optimum point at 0. However, the mean and standard deviation show that the stability is insufficient. ISSA has stronger local optimization ability and stability than SSA, GWO and WOA.
The multimodal test functions (F 5 ∼ F 8 ) can easily prevent the algorithm from the global optimal solution because there are a large number of local optimal solutions. So, they can better reflect the ability to explore and jump out of the local optimal solution. From the experimental results of F 5 , F 7 , and F 8 , it is clear that ISSA outperforms other algorithms in terms of stability. Although all algorithms have found the global optimal solution when searching in F 6 , the GWO and WOA algorithms are incapable of breaking out of the local optima from the point view of mean and standard deviation.
The global search and local development capabilities of the algorithm may be tested using the fixed dimension functions (F 9 ∼ F 11 ). The experiments show that ISSA performs better than other algorithms for F 9 and F 10 . For F 11 , the best value of ISSA is the optimal value, and the mean value and the standard deviation is the second-best value among GWO, WOA, SSA and ISSA. Figure 3 depicts the convergence curves of the four algorithms on the 11 tested functions to better demonstrate the convergence performance of ISSA. The convergence curve shows that ISSA can discover the optimal value in the least amount of time, and the convergence speed of all test functions is faster than that of GWO and WOA. Although the convergence speed of ISSA on the 10th test function is comparable to that of GWO, its convergence accuracy is greater seen from the Table 3. ISSA converges faster and stay on the abscissa axis on the remaining 10 test functions, which demonstrates that ISSA is easily leap out of the local optimal solution and has a better global optimization search capability. ISSA shows strong competitiveness in terms of convergence, robustness, and stability.
To summarize, ISSA offers good convergence ability, high search accuracy, robustness and stability. ISSA outperforms the other three algorithms in terms of competitiveness, optimization capability, and development potential.

III. IMPROVED SPARROW SEARCH ALGORITHM OPTIMIZES SUPPORT VECTOR MACHINE CLASSIFICATION ALGORITHM
ISSA is used to optimize SVM parameters, and the combination of penalty factor C and kernel parameter sigma is optimized to avoid the blindness of manual parameter selection, resulting in greater classification accuracy for the ISSA-SVM classification method.
A. IMPROVED SPARROW SEARCH ALGORITHM OPTIMIZES SUPPORT VECTOR MACHINE CLASSIFICATION ALGORITHM SVM outperforms other approaches because it is founded on the Vapnik-Chervonenkis Dimension theory and the Structural Risk Minimization (SRM) concept, rather than Empirical Risk Minimization (ERM) [9]. In the case of linear separability, SVM is derived from the best classification surface. It aims to find the optimal hyperplane between samples to maximize the classification interval. It has a strong generalization ability to deal with small samples, high dimensions and nonlinear problems.
When SVM deals with nonlinear problems, the classification hyperplane equation is: The decision function is: Therefore, solving the optimal classification hyperplane problem can be expressed as: In the formula (11), ξ i ≥ 0, i = 1, · · ·, l, C is the penalty factor which affects the accuracy of classification. The larger C is, the greater the penalty of SVM for classification errors. This problem is solved by the Lagrange multiplier method, resulting in a dual optimization problem: where K x i , x j = φ (x i ) · φ x j is the kernel function, using the radial basis function as the SVM kernel function, sigma>0 is the kernel parameter. Therefore, the decision function and parameter b are respectively: In this case, N represents the number of standard support vectors, E(J ) represents the standard support vector set, and E represents the support vector set. In summary, how to choose the penalty factor C and the kernel parameter sigma has a significant impact on SVM classification accuracy. The ISSA-SVM classification technique is given in this paper, in which the radial basis function is chosen as the kernel function of SVM and ISSA is utilized to optimize the combination of penalty factor C and kernel parameter sigma of SVM. Figure 4 is the flowchart of the algorithm. We apply the ISSA-SVM to classify the coal and gangue. There are 230 images in the coal gangue datasets, including 121 images of coal and 109 images of gangue. Figure 5 shows some sample images. The size of all images is changed in batches to 256 * 256, and the dataset is randomly split into training set and test set approximately in 7:3 ratio. The training set contains 84 coals and 76 gangues, and the test set contains 37 coals and 33 gangues.

C. EXPERIMENTAL TESTING AND ANALYSIS
This paper compares the ISSA-SVM with the original SVM and SSA-SVM under the same conditions. Table 4 shows the time-consuming and the classification accuracy of the SVM, SSA-SVM and ISSA-SVM, respectively. The accuracy is calculated by formula (15).
In the formula (15), Accuracy is the classification accuracy, TP represents actual coal and judged as coal, TN represents actual gangue and judged as gangue, FP represents actual gangue but judged as coal, FN represents actual coal but judged as gangue. Table 4 shows that the classification accuracy of both SSA-SVM and ISSA-SVM are greater than 90%, and the classification accuracy is greatly improved compared with the traditional SVM. Compared with traditional SVM, the classification accuracy of SSA-SVM and ISSA-SVM is improved by 14.14% and 17.69%, respectively. The ISSA-SVM is more accurate than SSA-SVM. From the perspective of singleframe image time, ISSA-SVM is 20.15% shorter than SVM, and 13.74% shorter than SSA-SVM. So, the ISSA-SVM is faster than SSA-SVM and SVM.

IV. CONCLUSION
The ISSA-SVM classification algorithm is proposed in this paper. The test function experiment demonstrates that the ISSA-SVM can increase the global optimization ability, prevent the algorithm from slipping into a local optimum, and improve the convergence ability and convergence accuracy. Then, the ISSA-SVM algorithm is used to the classification of coal and gangue. Experiments show that the classification accuracy of the ISSA-SVM is much better than that of the SVM and SSA-SVM. Meanwhile, the speed of classification in ISSA-SVM is faster than that of other two algorithms. VOLUME 11, 2023