An error perturbation for learning and detection of local minima in binary 3-layered neural networks

In binary multilayer neural networks with a backpropagation algorithm, achievement of quick and stable convergence in binary space is a major issue for a wide range of applications. We propose a learning technique in which tenacious local minima can be evaded by using a perturbation of the unit output errors in an output layer in polarity and magnitude. Simulation results showed that a binary 3-layered neural network can converge very rapidly in binary space with insensitivity to a set of initial weights, providing high generalization ability. It is also pointed out that tenacious local minima can be detected by monitoring a minimum magnitude of the unit output errors for the erroneous binary outputs, and that the overtraining concerning to generalization performance for test inputs is roughly estimated by monitoring the minimum and maximum magnitudes of the unit output errors for the correct binary outputs.