On the Convergence Rate of Three-Step Gradient Methods

In this paper, we develop a three-step gradient method for unconstrained optimization of convex functions with Lipschitz-continuous gradients. Given bounds on the Hessian of the objective function, we derive the sufficient and necessary conditions for its convergence. Furthermore, we establish the connection between the convergence rate and the algorithm parameters. Based on this connection, we present the algorithm parameters that guarantee the fastest convergence. The results show that the three-step gradient method will degenerate to the two-step gradient method when it reaches the fastest convergence.