Improved generalization using robust cost functions

The authors present several strategies for improving the overall generalization obtained by the normal backpropagation algorithm when there are errors such as noise and/or irrelevant inputs in the training set. When the training set is noisy and small, certain inputs or patterns are wrong. Backpropagation and least squares can generate bad curves because they attempt to find curves that fit both the patterns that have errors in them and those that do not. These curves are usually far from being representative of the true population. The idea of generalization is to find the curve that best fits the true underlying population and not the training set. These strategies include using several robust cost functions that eliminate the effect the errors have over the training process.<
>