Ambiguous binary representation in multilayer neural networks

We develop a side direction process to assist the back propagation learning algorithm to resolve the premature saturation problem. To build this side process, we explore the idea of unfaithful representation which has been introduced in the tiling algorithm. The algorithm may grow to an unpredictably large network for a given pattern set. This unfaithful representation is equivalent to the ambiguous binary representation. Binary numbers are used to represent the output binary vectors of hidden layers. More training patterns of different classes map to the same binary number, more patterns are misclassified. Besides, the presence of ambiguous binary representations is also an important pointer of when and where we should add a new hidden neuron to the multilayer perceptron. In this work, we explain the happening of ambiguous binary representation and develop a method to alleviate it. Using this method, both the number of ambiguous binary representations and the backpropagation learning time are drastically reduced.