Semistability of Steepest Descent with Momentum for Quadratic Functions

Two steepest-descent algorithms with momentum for quadratic functions are considered. For a given learning rate, the sufficient and necessary conditions for the semistability of the steepest-descent algorithms with momentum are established. Moreover, the optimal momentum factors that generally make for the fastest convergence are obtained.