Improving the extrapolation capability of neural networks

Neural networks can be used as an effective system identification tool in that they can model the vast majority of nonlinear systems to any arbitrary degree of accuracy. However, a fundamental disadvantage of neural networks is their inability to incorporate effectively first-principles models' information into their training so that their predictive capability is improved. This study proposes to use information obtained from a first principles model to impart a sense of extrapolation capability to the neural network model. This is accomplished by modifying the objective function to include an additional term that is the difference between the time rate of change of the error between the best first principles model estimate of the process and the neural network prediction. The performance of a feedforward neural network model that uses this modified objective function is demonstrated on a chaotic process and compared to the conventional feedforward network trained on the usual objective function.