Stochastic weight update for recurrent networks

Stochastic weight update is a variant of error back-propagation algorithm for learning of artificial neural networks. It allows for efficient topology-independent implementation of backpropagation through time for recurrent networks. In stochastic weight update scenario, constant number of weights and neurons is randomly selected and updated. This is in contrast to the classical ordered update, where all weights/neurons are always updated. In this paper we will study performance of stochastic weight update on recurrent neural networks using concept of feedforward network with added recurrent neurons.