Sequential structure and parameter-adaptive pattern recognition--I: Supervised learning

Bayes optimal sequential structure and parameter-adaptive pattern-recognition systems for continuous data are derived. Both off-line (or prior to actual operation) and on-line (while in operation) supervised learning is considered. The concept of structure adaptation is introduced and both structure as well as parameter-adaptive optimal pattern-recognition systems are obtained. Specifically, for the class of supervised-learning pattern-recognition problems with Gaussian process models and linear dynamics, the adaptive pattern-recognition systems are shown to be decomposable ("partition theorem") into a linear nonadaptive part consisting of recursive matched Kalman filters, a nonlinear part--a set of probability computers--that incorporates the adaptive nature of the system, and finally a part of the correlator-estimator (Kailath) form. Extensions of the above results to the
M
-ary hypotheses case where
M \geq 2
are given.