Simple learning algorithm for recurrent networks to realize short-term memories

A simple supervised learning algorithm for recurrent neural networks is proposed. It needs only O(n/sup 2/) memories and O(n/sup 2/) calculations, where n is the number of neurons, by limiting the problems to a delayed recognition (short-term memory) problem. Since O(n/sup 2/) is the same as the order of the number of connections in the neural network, it is suitable for implementation. This learning algorithm is similar to the conventional static backpropagation learning. Connection weights are modified by the products of the propagated error signal and some variables that hold the information about the past pre-synaptic neuron output.