Parameter drift in LMS adaptive filters

We examine general conditions under which the LMS adaptive filter generates unbounded parameter estimates when driven by bounded sequences. This unexpected parameter divergence, or drift, is related to the inadequacy of excitation in the input sequence and is characterized by slow (i.e., nonexponential) escape of the parameter estimate vector to infinity in spite of all other signals (inputs, outputs, prediction errors) remaining bounded or even decaying to zero. The analysis proceeds by showing that, in a general adaptive filtering setting, the sequence of regressors (information vectors) provides a natural decomposition of the parameter estimate space into subspaces, each corresponding to a characteristic class of filter excitation. This subspace decomposition is applied to the LMS adaptive filter, yielding direct links between filter behavior and modes of excitation. In particular, stability and instability of the parameter estimates in the presence of bounded disturbances and prediction errors is obtained for various classes of excitation. This behavior is examined in detail for the first-and second-order cases, and is sufficient to characterize the behavior of higher order adaptive filters. The instability (drift) results are due to modes of "decaying" excitation, and bounds on the rates of drift are derived. This drift mechanism is inherent in the algorithm and is not due to numerical implementation problems or violation of small step-size conditions. Examples are presented where drift may occur in restricted complexity filtering and in the related stochastic gradient algorithm. An analysis of leakage in terms of input excitation reveals a tradeoff in performance between parameter and prediction errors. A modified form of leakage, using the subspace decomposition, is suggested to remove this difficulty.