The LMS algorithm with momentum updating

Several modifications of the well-known least-mean-square (LMS) algorithm have been proposed for improved operation. The authors analyze one such recent innovation that corresponds to the ordinary LMS algorithm with an additional momentum term, parameterized by the factor alpha . The analysis of convergence in the mean yields some novel behavior insofar that it leads to complex eigenvalues of the transition matrix for the mean weight vector. The convergence in the mean-square analysis demonstrates that instability will occur as alpha tends closer to 1, a result not predicted by the analysis of convergence in the mean.<>