An asymptotic singular value decomposition analysis of nonlinear multilayer neural networks

The nonlinear multilayer neural network architecture is analyzed as a set of coupled Moore-Penrose pseudo-inverse equations, which arise from an asymptotic approximation to each unit's output equation. A linear analysis of these equations is performed through singular value decomposition (SVD) of the input matrix and a matrix of the desired hidden values. This analysis exactly determines the values of the weights and the optimal number of hidden units needed for a given set of training patterns. The outputs of the resulting neural network asymptotically approach the desired output values for each input pattern. This analysis provides an approach to determining the computational complexity of constructing nonlinear feedforward neural networks. A simple example of the XOR problem illustrates the results of the analysis.<>