Dynamics of error backpropagation learning with pruning in the weight space

Structural learning as proposed by Ishikawa [1994] has had success in reducing unimportant weights in an initially oversized multilayer feedforward neural network during training. This technique employs a forgetting constant /spl epsiv/ which determines the rate of weight decay during learning, A formalization of this method is obtained by considering the weights as the states of a dynamic system. Analysis of the system eigenvalues and simulation indicate that /spl epsiv/ controls removal of fixed points corresponding to the weights of oversized networks. Thus considering error backpropagation as a dynamic system has enabled analysis of structural learning.