Influence of Noise on Information Theoretic Causality Measures for System Identification

Parameter estimation and model order reduction (MOR) are important steps in the development of engineering models for many real-world systems. While a number of parameter estimation and MOR methods exist for linear systems, the problem is considerably more challenging for nonlinear systems. Many current algorithms applied to nonlinear systems are susceptible to convergence to local minima or overfitting of measurement data, which can lead to problems with poor model fidelity with respect to both open-loop dynamics and response to control inputs. Recently, the authors introduced a method that leverages information theoretic causality measures to identify the parametric structure of a model and remove model components which are extraneous. This algorithm is based on a particular type of conditional entropy, called causation entropy, which reveals the critical state transition functions in a nonlinear model through formation of a so-called causation entropy matrix (CEM). Previous work demonstrated that the CEM can be used to reduce the order of a nonlinear model in several practical parameter estimation problems. However, the scope of this prior work was limited and did not consider the effects of noise on system measurements. This paper provides a study of the effects of measurement noise on the accuracy of the CEM method and the efficacy of the resulting parameter estimation process. The paper also explores the significance of the numerical magnitude of the causation entropy values contained in the CEM, which leads to a deeper understanding of how the CEM can be used in model order reduction for nonlinear systems more generally. Simulation examples are provided to demonstrate these trends in the form of a coupled linear harmonic oscillator and a nonlinear pendulum on a cart.