Assessing the confidence of classification in artificial neural networks

Artificial neural networks (ANNs) have become very popular for data analysis over the past decade. In particular, feedforward neural network classifiers have, it may be argued, become so popular because they can estimate a posteriori probabilities directly by forming a mapping function from the data space to a probability space. If, however, we are to exploit the undoubted utility of ANNs in safety-critical environments then classification performance in itself is not enough. One of the key requirements of any statistical analysis system is to assess its own confidence in a decision. In the field of medical diagnostics, this requirement is paramount. Part of the problem for any Bayesian classifier is the fact that the posteriors, by definition, sum to unity. This means that a classification is made into one of a closed set of classes. If 'rogue' data appears then, even if it fails to conform to the statistics of 'genuine' data, it will be classified with apparent confidence into one of the output classes. We must, therefore, monitor the confidence in any classification decision. It is possible to further extend the sophistication of error and confidence estimates for ANNs by incorporation of more complex training and inference (using a full Bayesian methodology, for example). This paper looks at some of the issues involved in estimating errors and confidence limits in feedforward networks, and results are presented on an example of muscle tremor classification.