On the foundations of system identification

Summary form only given. One philosophically coherent position is to view system (or process) identification as the search for a theory (or model) in a given class that minimizes a (loss) function of (1) the cumulative prediction errors incurred using a particular model and (2) a measure of the complexity of the model (such as the McMillan degree of a linear predictor). The resulting identification method is referred to as a minimum-predictor-error (MPE) method. An alternative starting point taken in the minimum-description-length (MDL) theory due to Rissanen is to view a process or predictor model as an encoding device and to choose the model (in a given class) that minimizes the total number of bits needed to describe (1) the model plus (2) the number of bits required to describe the observations when encoded using the model. An extension of this idea is contained in Rissanen's stochastic complexity (SC) measure of a process. The author has related the MPE, MDL, SC and classical maximum-likelihood approaches to system identification.<
>