Pipelined neural tree learning by error forward-propagation

We propose a new parallel implementation of the neural tree feed-forward network architecture that supports efficient evaluation and learning regardless of the number of layers. The neurons of each layer operate in parallel and the layers are the elements of a pipeline that computes the output evaluation vectors for a sequence of input pattern vectors at a rate of one per time step. During the learning phase the desired outputs are presented as additional inputs and the pipeline computes in feed-forward manner the gradients of the errors with respect to the neuron evaluations. Thus it is possible to run different gradient descent learning algorithms on the pipeline with a performance comparable to the evaluation algorithm.