Optimization Algorithms with Superlinear Convergence Rate

By converting optimization problems into optimal control problems, where the updated size of each iteration is the control input, and the control objective is to design the current control input to minimize the sum of the original objective function and the updated size for the future time instant, the optimization algorithm is proposed and updates almost along the optimal state trajectory. Intuitively, it converges rapidly and stably. We concentrate on stringently analyzing its convergence and superlinear convergence rate. It is noteworthy that this superlinear convergence rate exhibits nearly quadratic behavior. To bypass the inverse manipulation, we also provide the modified versions of the algorithm. Numerical experiments support the effectiveness of the proposed algorithm and its variants.