An unsupervised, stochastic learning model for syntactic pattern recognition using the discrete Kalman filter scheme

The authors propose a model for syntactic pattern recognition (SPR) in real-time. Their goal is to construct a self-organizing learning network that can process grammars of different types. The fundamental building block of the network is a stochastic unit which estimates the most probable symbol (primitive) in a given input sequence. The unit dynamics are governed by the scalar case of the discrete Kalman filter (DKF) algorithm. The unit consists of two parameters, "alpha" and "gamma". "Alpha" is the current symbol stored in the unit and "gamma" determines the degree of confidence in prediction. The learning algorithm is quite simple: the current symbol in the input sequence is compared with the symbol stored in the unit. If they both are same, then the confidence level for the stored symbol is increased but if the two symbols differ, the new symbol is selected probabilistically (generate a random number from a uniform distribution and accept the new symbol if the random number is greater than the confidence level). A single-layered network of such stochastic units (syntactic neurons) is constructed to build a SPR system. The DKF parameters attribute interesting properties to the unit (and to the network). For example, the stochastic update rule "implicitly generates a sigmoidal relationship" between the probabilities of occurrence of symbols (primitives) in the input and output sequences of the unit.