Adaptive regularization of neural networks using conjugate gradient

Andersen et al. (1997) and Larsen et al. (1996, 1997) suggested a regularization scheme which iteratively adapts regularization parameters by minimizing validation error using simple gradient descent. In this contribution we present an improved algorithm based on the conjugate gradient technique. Numerical experiments with feedforward neural networks successfully demonstrate improved generalization ability and lower computational cost.