Generalization ability of universal learning network by using second order derivatives

In this paper, it is studied how the generalization ability of modeling of the dynamic systems can be improved by taking advantages of the second order derivatives of the criterion function with respect to the external inputs. The proposed method is based on the regularization theory proposed by Poggio and Givosi (1990), but a main distinctive point in this paper is that extension to dynamic systems from static systems has been taken into account and actual second order derivatives of the universal learning network have been used to train the parameters of the networks. The second order derivatives term of the criterion function may minimize the deviation caused by the external input changes. Simulation results show that the method is useful for improving the generalization ability of identifying nonlinear dynamic systems using neural networks.