A lower bound on filtering error with application to phase demodulation

A new lower bound on nonlinear filtering mean square error (MSE) based on rate distortion theory is derived for message and observation models described by state space equations. Unlike previous contributions, the present bound is general in applicability: it can be used in beth the continuous and discrete time cases; it is applicable during the transient (e.g., acquisition) as well as steady state phases; vector-valued processes of arbitrary dimension can be treated; stationary as well as nonstationary processes can be considered; and essentially no restrictions are placed on the nonlinearities. Two easily implemented approaches to the evaluation of the new lower bound are given: one is analytical in nature while the other uses Monte Carlo simulation techniques. The theory is extended to filtering with distortion measures other then MSE. The MSE lower bound is applied to a phase demodulation problem and compared to other lower bounds which are based on rate distortion theory and the Cram\acute{e}r-Rao inequality. Results for this problem show the new lower bound to be tighter than the others in the nonlinear, high noise-to-signal ratio region of receiver operation.