Feedforward neural networks for Bayes-optimal classification: investigations into the influence of the composition of the training set on the cost function

Under idealized assumptions (infinitely large training sets, ideal training algorithms that avoid local minima and sufficient neural network (NN) structures) trained NNs realize Bayes-optimal classifiers (BOCs) with identical costs as long as the training set is representative. Training sets with relative class frequencies different from the a priori class probabilities implement nonidentical costs, but result in an identical receiver-operator-characteristic (ROC). However under real-world conditions, the equivalence of NNs trained with different learn sets does not hold. Some effects of limited sample size and insufficient network structures are analyzed. For a simulated example the performances of NNs trained with different learn sets are compared. The results make clear that one has to find a trade-off between the exhaustive use of available information and the risk of getting stuck in local minima in order to choose optimal learn sets.