Convergence analysis of adaptive filtering algorithms with singular data covariance matrix

The paper provides a rigorous analysis of the behavior of adaptive filtering algorithms when the covariance matrix of the filter input is singular. The analysis is done in the context of adaptive plant identification. The considered algorithms are LMS, RLS, sign (SA), and signed regressor (SRA) algorithms. Both the signal and weight behavior of the algorithms are considered. The signal behavior is evaluated in terms of the moments of the excess output error of the filter. The weight behavior is evaluated in terms of the moments of the filter weight misalignment vector. It is found that the RLS and SRA diverge when the input covariance matrix is singular. The steady-state signal behavior of the LMS and SA can be made arbitrarily fine by using sufficiently small step sizes of the algorithms. Indeed, the long-term average of the mean square excess error of the LMS is proportional to the algorithm step size. The long-term average of the mean absolute excess error of the SA is proportional to the square root of the algorithm step size. On the other hand, the steady-state weight behavior of both the LMS and SA have biases that depend on the weight initialization. The analytical results of the paper are supported by simulations.