Designing regularizers by minimizing generalization errors

To improve generalization ability, a regularizer is frequently used. An approach proposed here is to regard the estimate of model parameters as a function of those without a regularizer. By minimizing the calculated generalization error, the optimal function parameters and model parameters can be obtained. In the paper linear regression is adopted to carry out theoretical computation of generalization error. Asymptotic characteristics are also analyzed. It also contributes to the discovery of a new regularizer.