Backpropagation through time with fixed memory size requirements

A generalization of the backpropagation through time (BPTT) algorithm is presented, which, under reasonable assumptions, can lead to fixed memory size requirements. The idea is to model BPTT as the storage of activations and errors in tapped delay lines, and then generalize the tap delay line to a gamma memory. Since the depth of the gamma filter is T=K//spl mu/, where K is the filter order and /spl mu/ a scalar that can be varied between 0>