Structured neural networks for Markovian processes

A multi-layer perceptron (MLP) containing fixed structured regions consisting of delay lines and feedback units is stable under error-backpropagation. It is proposed that learning with a structured network succeeds where a fully connected, layered network fails. An example is presented: the input to the network is a time varying signal; when a hidden Markov model is used as input, the network learns to output the hidden state probability; performance reaches the theoretical (Baum-Welch forward pass) limit.<
>