Backpropagation without weight transport

In backpropagation, connection weights are used to both compute node activations and error gradient for hidden units. Grossberg (1987) has argued that the dual use of the same synaptic connections (weight transport) constitutes a bidirectional flow of information through synapses, which is biologically implausable. In this paper we formally and empirically demonstrate the feasibility of an architecture equivalent to backpropagation, but without the assumption of weight transport. Through coordinated training with weight decay, a reciprocal layer of weights evolves into a copy of the forward connections and acts as the conduit for backward flowing corrective information. Examination of the networks trained with dual weights suggests that functional synchronization, and not weight synchronization, is crucial to the operation of backpropagation methods.<
>