An improved LS-SVM based on SSOR-PCG

Support vector machines (SVM) have been applied successfully to pattern recognition and function estimations in the past decade. Least square support vector machine (LS-SVM) is proposed based on the standard SVM, which trains the SVM by solving a set of linear equations instead of quadratic programming (in the standard SVM). However, for large sample problems, the LS-SVM still has to suffer from the low efficiency or could not obtain the optimal solutions when the coefficient matrix (derived from the concrete learning problem) is ill-posed. To overcome these faults, we present an improved LS-SVM training method based on the symmetric successive over-relaxation preprocessing conjugate gradient (SSOR-PCG), prove that the symmetric positive definite of the corresponding coefficient matrix can be guaranteed by adjusting of the penalty coefficient. We analyze theoretically the superiority in the convergence of the proposed method over the standard LS-SVM. The theoretical work in this paper would benefit for the designs of training algorithms for LS-SVM. Numerical experiments are presented, which demonstrate that the proposed algorithm could speed up 10 times compared with the standard LS-SVM training algorithm, under the similar precisions.