Information and distortion in reduced-order filter design

The relation and practical relevance of information theory to the filtering problem has long been an open question. The design, evaluation, and comparison of (suboptimal) reduced-order filters by information methods is considered. First, the differences and similarities between the information theory problem and the filtering problem are delineated. Then, based on these considerations, a formulation that {\em realistically} imbeds the reduced-order filter problem in an information-theoretic framework is presented. This formulation includes a "constrained" version of the rate-distortion function. The Shannon lower bound is used both to derive formulas for (achievable) rose lower bounds for suboptimal filters and to prove that for thc reduced-order filter problem the given formulation specifies a useful relation between information and distortion in filtering. Theorems addressed to reduced-order filter design, evaluation, and comparison based on information are given. A two step design procedure is outlined which results in a decoupling of thc search in filter parameter space, and hence in computational savings.