Inertial Derivative-Free Projection Method for Nonlinear Monotone Operator Equations With Convex Constraints

In this paper, we propose an inertial derivative-free projection method for solving convex constrained nonlinear monotone operator equations (CNME). The method incorporates the inertial step with an existing method called derivative-free projection (DFPI) method for solving CNME. The reason is to improve the convergence speed of DFPI as it has been shown and reported in several works that indeed the inertial step can speed up convergence. The global convergence of the proposed method is proved under some mild assumptions. Finally, numerical results reported clearly show that the proposed method is more efficient than the DFPI.


I. INTRODUCTION
Consider the problem of finding y ∈ E such that where T : R n → R n is a monotone and Lipschitz continuous operator and E is a nonempty, closed and convex subset of R n . This problem has recently received remarkable attention as it arises in a number of applicable problems. For example, in constrained neural networks [1], nonlinear compressed sensing [2], [3], phase retrieval [4], [5], power flow equations [6], economic and chemical equilibrium problems [7], [8], non-negative matrix factorisation [9], [10], forecasting of financial market, portfolio selection models, price returns [11]- [13] and many more. As such, recently several derivative-free methods such as the conjugate gradient (CG) method have been proposed for solving problem (1). Given The associate editor coordinating the review of this manuscript and approving it for publication was Gokhan Apaydin . an initial point y 0 , the conjugate gradient method computes the next iterate as: where α k > 0 is a step size and d k is called the CG direction of search defined as The parameter β k is called the CG parameter. For more on derivative-free methods for solving (1), interested readers can refer to [14]- [35] and references therein.
Recently, several researchers are interested in how to improve the speed of convergence of existing iterative algorithms. One of the approach in this regard is the inertial extrapolation method where a new step called the inertial step is added to the existing step(s) of an iterative method. It has been shown that the inertial step enhance the speed of the existing methods such as methods for solving fixed VOLUME 9, 2021 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ point problems, variational inequality problems, equilibrium problems, split feasibility problems, and so on. By choosing two starting points y −1 and y 0 , the inertial term is defined as where {θ k } ∞ k=1 is a sequence satisfying certain condition. Inertial extrapolation method has been employed successfully in improving the convergence of the sequence generated by various algorithms. However, to the best of our knowledge, there is no theoretical proof to justify that, indeed, all one can find is numerical justification using some examples. However, the choice of the parameter θ k has an effect on the speed of convergence. For more on iterative methods with inertial extrapolation, the reader is referred to [36]- [41] and references therein.
Inspired by the inertial methods [36]- [41] and the derivative-free projection method proposed by Sun and Liu [17] which is an extension of the work of Cheng [42], we propose an inertial derivative-free projection method for finding solutions to problem (1). The method is based on the work of Sun and Liu [17], where the inertial term is incorporated in order speed up its convergence. The remaining part of this paper is organized as follows: the next section gives some preliminaries and the proposed algorithm, convergence results is provided in the third section, Numerical results in the fourth section and lastly the conclusion.
Notation. Unless otherwise stated, the symbol · stands for Euclidean norm on R n .

II. PROPOSED ALGORITHM
Definition 2.1: Let R n be an Euclidean space and T : R n → R n be a mapping. Then T is (ii) L-Lipschitz continuous, if there exists L > 0 such that

Definition 2.2:
Let E ⊂ R n be closed and convex, the projection of y ∈ R n onto E denoted by P E (y), is defined as

Lemma 2.3 ([43]):
Let E ⊂ R n be nonempty closed and convex. Then the following inequality hold: Lemma 2.4 ( [44]): Let y, x ∈ R n . Then the following equality hold: Lemma 2.5 ( [45]): Let {y k } and {x k } be sequences of nonnegative real number satisfying the following relation  = P E (y * − µu) for some u = T (y * ) and µ > 0. We make use of the following assumptions.
Assumption 1: (a) The feasible set E is a nonempty closed and convex subset of the Euclidean space R n . (b) T : R n → R n is monotone and L-Lipschitz continuous. (c) The solution set SOL(T, E) of (1) is nonempty. Assumption 2: Let {θ k } be a sequence of nonnegative real numbers satisfying the conditions: Based on the Sun and Liu [17] derivative-free projection method for monotone nonlinear equation with convex constraints called DFPI, we present an inertial derivative-free projection method for finding solutions to problem (1).
satisfying Assumption 2 and select the parameters: where, where (S.6) Set k = k + 1, and go back to (S.1). Remark 2.8: Let d k be generated by (2)-(3) in Algorithm 2.7. Then (4) is well-defined. That is, for all k ≥ 0, there exists a non negative integer i satisfying (4).

Lemma 3.1: The line search condition
Proof: Suppose there is k 0 ≥ 0 for which (4) is not true for any non-negative integer i, i.e., Using Assumption 1 (b) and allowing i → ∞, we have that On the other hand, from (6), which contradicts (7). Hence, (4) is well defined.
Moreover, the sequence {y k } and {x k } are bounded and Proof: By the monotonicity of the mapping T , we have By Lemma 2.3 (iii), (5), (9) and (10), it holds that for any y * ∈ SOL(T, E), From equation (11), we can deduce that Because ∞ k=1 θ k y k − y k−1 < ∞, then by Lemma 2.5, the limit of {y k − y * } exists and hence it is bounded. This implies that for all k, there exist M 0 > 0 such that y k −y * ≤ M 0 . Therefore, for all k we can deduce that and where M 1 = M 0 + y * and M = 2M 1 .
Using the above relations, we can have Since H is Lipschitz continuous, we have Also, using (14) and the monotonicity of T , This together with (9) and (10) Then, we have Hence the sequence {x k } is bounded since {v k } is bounded. Moreover as T is continuous and {x k } is bounded, then Combining (15) with (11), we have Thus, we have Adding (17) for k = 0, 1, 2, . . . and the fact that {T (x k )} is bounded, we have Now, let S k = k n=0 y n − y * 2 − y n+1 − y * 2 , then S k = k n=0 y 0 − y * 2 − y k+1 − y * 2 . As limit of { y k − y * } exists from (12) with limit say L 1 , then Using (18) together with the above inequalities, we conclude that lim k→∞ v k − x k = 0. Proof: Also, Using (8) and (20), we have Thus, from (8) and (20), we have lim k→∞ y k+1 − y k = 0. Therefore, ≤ y k+1 − y k + θ k y k − y k−1 .
Using (23) and Assumption 2, the desired equation is obtained.
Theorem 3.5: Let {y k } be a sequence generated via Algorithm 2.7. Using Assumption 1 and 2, then {y k } converge to an element of SOL(T, E).
Proof: We know that the sequence {y k } is bounded from (13). This implies that there exists a subsequence {y k j } of {y k } such that {y k j } converge to some pointȳ. Also, we have that v k j − y k j = θ k j y k j − y k j −1 → 0, as j → ∞. (24) Claim:ȳ ∈ SOL(T, E). Suppose on the contrary thatȳ / ∈ SOL(T, E). Then from (19) and (24), we have that Without loss of generality, if γ k j → γ * and T (x k j ) → T (x * ). Then since T is continuous, we have T (x * ) = T (ȳ). Therefore, from (25) It then follows from Lemma 2.6 that y * ∈ SOL(T, E), which is a contradiction. Hence, our claim holds. Substituting y * withȳ in (12), it is easy to see that lim  by Lemma 2.5. Sinceȳ is an accumulation point of {y k }, we obtain that {y k } converges toȳ.

IV. NUMERICAL EXAMPLES
By comparing the proposed inertial algorithm (Iner. DFPI) to the DFPI algorithm in [17], we show the numerical efficiency and computational advantage of the proposed inertial algorithm (Iner. DFPI) in this section. The MATLAB implementation of the algorithms was executed on a Windows 10 computer with Intel(R) Core(TM) i7 processor with 8.0GB of RAM and CPU of 2.30GHz using MATLAB R2019b software. The numerical experiment made use of the following test problems to measure the efficiency and robustness of the proposed inertial algorithm (Iner. DFPI).
The above listed problems are solved with dimensions n = 1000, 5000, 10, 000, 50, 000 and 100, 000. The parameters    For the compared method (DFPI), its parameters were set as reported in [17]. All iterative procedure terminate when T (v k ) < 10 −6 is fulfilled. If this condition is not satisfied after 1000 iterations, failure is declared. VOLUME 9, 2021   To illustrate in detail the efficiency and robustness of Iner. DFPI, we start by performing some numerical experiments with different coefficients of the parameter β k and the results are reported in Table 2 and 3. It can be observed from the  tables that the coefficient 0.01 is a good choice. In addition, we performed another numerical experiments with different sequences {θ k } and the results are reported in Table 4 and 5. It can be observed from the tables that the sequence θ k = 1 (2k+5) 2 is a good choice. We further employ the performance   profile proposed by Dolan and Morè in [50] in order to summarize Table 6-15. The profile is defined as follows:  where T P is the test set, |T P | is the number of problems in the test set T P , Q is the set of optimization solvers, and t p,q is the number of iterations (or the number of the function evaluations, or the CPU time (in seconds)) for t p ∈ T P and   Tables 2-11. It can be observed from the figures that Iner. DFPI algorithm performs better with a higher percentage win of at least 90% in all the three metrics, i.e., number of iterations, the number of function evaluations and the CPU time. As a consequence, we can conclude that Iner. DFPI algorithm is an efficient solver. It is worth mentioning that the good numerical performance of the Iner. DFPI algorithm is as a result of the inertial term v k , suitable control parameters such as ρ, σ and the sequence {θ k }.
A detailed report of our numerical experiments is reported in Table 6-15 in the appendix section. The abbreviations on the tables can be read as follows: n: denotes the dimension of the problem SP: denotes the starting points NOI: denotes the number of iterations In this paper, we suggested an inertial derivative-free method for solving nonlinear monotone operator equation. Based on the DFPI method, an inertial term was added to it in order to speed up its convergence. We used some mild assumptions to establish the global convergence of the proposed inertial method. To support the theoretical results, we perform some numerical experiments on some benchmark test problems with the proposed method and the DFPI. The results indicate that the proposed inertial method is faster than DFPI.

ACKNOWLEDGMENT
The author Auwal Bala Abubakar would like to thank the Postdoctoral Fellowship from King Mongkut's University of Technology Thonburi (KMUTT), Thailand. He also acknowledge with thanks, the Department of Mathematics and Applied Mathematics at the Sefako Makgatho Health Sciences University.