Back propagation fails to separate where perceptrons succeed

It is widely believed that the back-propagation algorithm in neural networks, for tasks such as pattern classification, overcomes the limitations of the perceptron. The authors construct several counterexamples to this belief. They also construct linearly separable examples which have a unique minimum which fails to separate two families of vectors, and a simple example with four two-dimensional vectors in a single-layer network showing local minima with a large basin of attraction. Thus, back-propagation is guaranteed to fail in the first example, and likely to fail in the second example. It is shown that even multilayered (hidden-layer) networks can also fail in this way to classify linearly separable problems. Since the authors' examples are all linearly separable, the perceptron would correctly classify them. The results disprove the presumption, made in recent years, that, barring local minima, back-propagation will find the best set of weights for a given problem.<
>