Constructive and robust combination of perceptrons

We propose a new strategy for a constructive training of feedforward neural networks to classify linearly nonseparable patterns. The algorithm results in a configuration of the first layer of the network, which is able to give a faithful internal representation of the input patterns. The weights of the network are obtained by the CadaTron algorithm introduced, which is able to separate clusters of data in a robust way. Iteratively, further neurons are added to the neural net in order to decrease the training error. Unnecessary neurons are removed, so this algorithm leads to a network with low complexity and excellent generalization properties. The results of this work are based on the classification of handwritten characters.