The reduction of gradient noise in gradient-based algorithm by using variable step-size technique

This paper proposes a different approach to control step-size adaptation. The objective is to ensure large step-size when the algorithm is far from the optimum with step-size decreasing as we approach the optimum. The proposed algorithm achieves this objective by employing the output signal energy to control step-size update. In the early stage of adaptation the estimate of the output signal energy is large, resulting in a large step-size. As we approach the optimal, the output signal energy approaches zero, resulting in a small step-size. This provides the fast convergence due to large initial step-size while ensuring low gradient noise due to the small final step-size, resulting in low variance of the filter coefficient.