Making a multilayered perceptron network say - "don't know" when it should

The response of a multilayered perceptron (MLP) network on points far away from the training data is generally not reliable. Ideally a network should not respond to a data point which lies far away from the boundary of its training data. We propose a new training scheme for MLPs as classifiers, which ensures this. Our scheme trains subnets for each class. Each subnet can decide whether a data point belongs to a certain class or not. Training each subnet requires data from the class which the subnet represents along with some points outside the boundary of that class. For this purpose we propose an easy but approximate method to generate points outside the boundary of a pattern class. The trained subnets are then merged to solve the multi-class classification problem. We demonstrate that an MLP trained by our method does not respond to points which lie outside the boundary of its training sample. Also, our network can deal with overlapped classes in a better manner.