Phased backpropagation: a hybrid of BPTT and temporal BP

We present a synthesis of backpropagation through time (BPTT) and temporal backpropagation (TB) that permits the efficiency of TB in dealing with delay lines to be combined with the generality of BPTT with respect to arbitrary discrete-time network structures. We express our formulation in terms of an ordered network, subsuming less general network architectures such as time-delay networks and recurrent multilayer perceptrons. The resulting formulation can also be combined with derivative adaptive critics, thereby providing a further unification of techniques for training dynamic networks.