Parameter estimation of ARX/NARX model: a neural network based method

In this paper, we consider the parameter-estimation problem of time domain models in the form of ARX or NARX. Instead of Least-Square based methods and the Maximum Likelihood method, the parameters of the time domain model will be estimated from test data using feed-forward neural networks. In the applications of neural networks, commonly used activation functions for function approximation are nonlinear transfer functions (e.g., Log-sigmoid function, Hyperbolic tangent function and polynomial function) as well as linear transfer function. In our study, we investigate the equivalence of linear/nonlinear Autoregressive exogenous (ARX/NARX) models with feed-forward neural network models, in which the activation function of hidden neuron is one of these most frequently used functions. The coefficients of ARX/NARX models are given by neural network weight values for different activation functions. We show that for pure linear systems, ARX models can be obtained from feed-forward neural networks with commonly used hidden neuron activation functions such as log-sigmoid function, hyperbolic tangent function, polynomial function, exponential function as well as pure linear function. For nonlinear systems, ARX and NARX models can be obtained from neural networks with polynomial or exponential hidden neuron activation functions to describe the linear contribution and nonlinear contribution respectively.