A new algorithm for the recursive identification of stochastic systems using an automaton with slowly growing memory

Consider a siscrete time system or process with i.i.d. outputs, X1,X2,... The distribution of Xi's is characterized by some unknown parameter, θ which is an element of θ, a subset of Z+, the positive integers. We propose to identify the parameter θ with an automaton which is realized by the following algorithm: The Xi's are observed sequentially. At instant i, a current best estimate (CBE) of θεθis generated, based on Xi alone. The best estimate to date (BED) of θ, at any instant, will be the particular element of θ which was previously the CBE of θ for the greatest number of consecutive instants. In the implementation of the algorithm, the state of te automaton will consist of the following four elements: the best estimate to date; the maximum number of consecutive times that this BED was the CBE of θ; the current best estimate of θ based on the latest value of Xi (= candidate for BED); and the number of consecutive times to date that this value has been the CBE of θ. It is shown that the BED converges strongly to the correct parameter. The following three specific applications are discussed: 1) Identification of the most probable value of a single sample from a set of discrete i.i.d. random variables. 2) Identification of the parameter of an observed sequence of i.i.d. Bernoulli random variables. 3) Identification of the parameters of a moving average process. (Notice that the observations of the system output in this case are not i.i.d.). The rate of convergence of the algorithm is shown to exceed that of other proposed recursive algorithms [1] for parameter identification. Furthermore, the amount of memory required is shown to grow as log log i, a rate significantly slower than that required by other algorithms. It is also shown that for a very large class of systems, convergence of parameter estimates is impossible without acess to an unbounded memory.