On the Some New Preconditioned Generalized AOR Methods for Solving Weighted Linear Least Squares Problems

Recently, in the paper [Z.G. Huang, L.G. Wang, Z. Xu, J.J. Cui, Some new preconditioned generalized AOR methods for solving weighted linear least squares problems, Computational and Applied Mathematics, 37(2018) 415–438.], Huang et al, by using the generalized accelerated over-relaxation (GAOR) methods, proposed some new preconditioners for solving weighted linear least squares problems and discuss their comparison results. In this paper, we present a new model of GAOR methods to solve the weighted linear least squares problems. We prove that the new model is superior to the existing mentioned methods. Numerical examples are also reported to confirm our theoretical analysis.


I. INTRODUCTION
Let us consider the weighted linear least squares problem (WLLSP): where, W is the variance-covariance matrix; (see [1]- [4] and references therein). Least squares (LS) method for resolving linear equation systems and WLLSP as a generalization of the LS arises in many practical applications including linear programming, convex quadratic programming, linear regression, geodetic symmetrical transformations, electrical networks, boundary value problems, analysis of large-scale structures and the inequality constrained least squares problems [4]- [8]. For solving this problem, we can solve a linear system as follows: where, is a nonsingular matrix with B = (b ij ) p×p , C = (c ij ) q×q , L = (l ij ) q×p , U = (u ij ) p×q , q = n − p, and I is an identity matrix. The basic stationary iterative method to solve the system of linear equations Ax = d, with any splitting, A=M-N that The associate editor coordinating the review of this manuscript and approving it for publication was Heng Zhang. det(M ) = 0 is as follows: where, M −1 N is called the iteration matrix. This iterative scheme converges to the unique solution if and only if ρ(M −1 N ) < 1, where ρ(A) is the spectral radius of A. Based on the model of Eq. (3), there are some popular iterative methods; see [9]- [18] and references therein. However, if we solve Eq. (2) by the model of Eq. (3), we need the inverses of I − B and I − C, which is the main drawback of these methods.
To solve Eq. (2), Yuan and Jin [3] proposed the generalized AOR (GAOR) method for solving the WLLSP. Consider the following splitting for the matrix H : where, D = diag(H ), and C L and C u are strictly lower and upper triangular matrices of H , respectively. Now, to solve the Eq. (2) using the GAOR method, the matrix H is split as; .
Then, for w = 0, the GAOR method is as follows: where, I 1 and I 2 are identity matrices of orders (p) and (n-p) respectively, and: Recently, several scholars established numerous models to solve WLLSP [19]- [26]. Very recently, Huang et al. [26], to decrease the spectral radius of µ w,r , presented some models of the preconditioned GAOR method. These authors, using the splitting of Eq. (4), presented two preconditioning models as follow: Consider the preconditioner form of Eq. (2) as: where K i is a q × p matrix and the form of K 1 is as follows: Also, for K2: If q< p, then: If q = p, then: If q > p, then: And, Then, the preconditioned matrix with the preconditioner (7) can be decomposed by the following splittinḡ where, Furthermore, its iteration matrix defined as: Also, the second class of preconditioner is; where Then, the preconditioned matrix with preconditioner (15) can be decomposed by the following splitting: where, The iteration matrix of Eq. (20) also defined as: In this paper, we propose some new splittings of H andH . Moreover, we prove that the new splittings compare with the splittings of (4), (13) and (20) work better.

II. PREREQUISITE
We start with some basic notation and preliminary results which we allude to later. Definition 1 ([9], [10]):

Lemma 4 ([27], [28]): Let A, B are Z-matrix and A is an
Lemma 5 [9]: [10]: Let H be a Z-matrix. Then H is an M-matrix if and only if there is a positive vector x such that Hx > 0.

III. MAIN RESULT
Consider the following splittings; 1, 2) are diagonal, strictly lower and strictly upper triangular parts of B and C, respectively. Furthermore, and, Then, to solve the linear system (1), we consider the following splitting So, the iteration matrix of GAOR method with the splitting of Eq. (22) is: In the following we will compare our splitting with the splitting of Eq.(4). Theorem 7: Consider the matrix H in Eq. (2) and let µ (1) w,r and µ (2) w,r be the iteration matrices of the GAOR method by splittings of (4) and (22), respectively. If C ≤ 0, U ≤ 0, B i ≥ 0; (i = 1, 2) and ρ(µ (1) w,r ) < 1, then for 0 < w ≤ 1 and 0 ≤ r < 1 we have; ρ(µ (2) w,r ) ≤ ρ(µ (1) w,r ). Proof: From Eq. (6) and Definition 2, we can see that µ (1) w,r is M-splitting of H . Then, from the Lemma 3 H is M-matrix. Also, from (6), (23) and Definition 2, these splittings are regular. 33198 VOLUME 8, 2020 Now, consider µ (1) w,r = M −1 1 N 1 , and µ (2) w,r = M −1 2 N 2 . Then; Therefore; Since H is M-matrix, from Lemma 4 M 1 , M 2 are M-matrices. Therefore, M −1 2 ≥ M −1 1 . And by Lemma 5 the proof is completed. Similarly, we consider the following model for the preconditioned matrixH . We only consider the first kind of preconditioners; another class can be analogously verified. By Eq. (13): Now, consider the following splittings; where D i , −L i , −U i (i = 1, 2) are diagonal, strictly lower and strictly upper triangular parts of (I 1 − W i )B and K i U − C, respectively. Furthermore, And, Then, we have the following splitting ofH : Then iteration matrix of preconditioned GAOR method with the splitting of (26) is Theorem 8: Letμ w,r andμ w,r be the iteration matrices of the preconditioned GAOR method by splittings of (13) and (26), respectively. If the conditions of Theorem 7 are satisfied, then we have, ρ(μ w,r ) ≤ ρ(μ w,r ) < 1.
Proof: We know that H is M-matrix. Then by Lemma 6 it is easy to see thatH is M-matrix too. Therefore, similar to the proving process of Theorem 7, we conclude that these splittings are regular. Moreover, let Then; And by Lemma 6 the proof is completed.

IV. NUMERICAL EXAMPLE
In this section we test the effectiveness of two mentioned splitting methods. The experimental results were obtained using an Intel Celeron with a 2.8GHz 32-bit processor and 1GB RAM memory running Windows 7. The computational platform used was the MATLAB environment. y (0) is zero vector and the f was selected such that the exact solution of the system (2) is y T = (1, 2, 3, . . . , n) T ∈ R n . The stopping criterion is ε < 10 −6 . The number of iterations (denoted by IT) and spectral radius and CPU time are reported in the following tables in order to show the efficiency of our splitting methods.
In Table 2, we show the results of the corresponding iterative schemes with the following preconditioner (µ i = γ i = 1) and the splittings of (13) and (26): In Table 3, we show the results of the corresponding iterative methods with the following preconditioner (α i = β i = τ i = σ i = 1) and the splittings of (20) and (26): The results also show that the scalability of our algorithms is suitable. For example, from Table 3, we can see that when n=10, the iteration steps of Splitting (26) ≈ 0.83 × Splitting (18), but, when n is increased to 50, the iteration steps of Splitting (26) ≈ 0.73 × Splitting (18). From tables 1-3 we can see that, the iterative methods with our splitting, without extra cost per iteration step, perform much better than existing methods for solving weighted linear least squares problems. Furthermore, an iterative method with the splitting (22) performs much better than preconditioned methods with the splittings (13) and (20).

V. CONCLUSION
In this paper, we modified the solution procedures to solve the weighted linear least squares problem and improved the convergence rates of iterative methods. From the numerical results and theoretical analysis, we can conclude that the performance of the modified iterative method is much better in comparison with the existing methods.
M. FALLAH is currently an Associate Professor with the Islamic Azad University of Central Tehran Branch, Iran. He has 34 years of experience in scientific research in diverse fields of industrial engineering and computer science, which span multiple branches of numerical linear algebra, symbolic computation, nonlinear optimization, and others. He has published a large number of technical articles, research monographs and textbooks in international refereed and ranked journals, conference proceedings, and book chapters. S. A. EDALATPANAH received the Ph.D. degree in applied mathematics from the University of Guilan, Rasht, Iran. He is currently an Assistant Professor with the Ayandegan Institute of Higher Education, Iran. He is also an Academic Member of Guilan University and Islamic Azad University of Iran. His fields of interest are numerical linear algebra, and soft computing and optimization. He has published over 100 journal and conference proceedings papers in the above research areas. He serves on the editorial boards of several international journals. He is the Editor-in-Chief of the International Journal of Research in Industrial Engineering. VOLUME 8, 2020