Robust Variable Projection Algorithm for the Identification of Separable Nonlinear Models

Robust nonlinear regression frequently arises in data analysis that is affected by outliers in various application fields such as system identification, signal processing, and machine learning. However, it is still quite challenge to design an efficient algorithm for such problems due to the nonlinearity and nonsmoothness. Previous researches usually ignore the underlying structure presenting in the such nonlinear regression models, where the variables can be partitioned into a linear part and a nonlinear part. Inspired by the high efficiency of variable projection algorithm for solving separable nonlinear least squares problems, in this article, we develop a robust variable projection (RoVP) method for the parameter estimation of separable nonlinear regression problem with
$L_{1}$
norm loss. The proposed algorithm eliminates the linear parameters by solving a linear programming subproblem, resulting in a reduced problem that only involves nonlinear parameters. More importantly, we derive the Jacobian matrix of the reduced objective function, which tackles the coupling between the linear parameters and nonlinear parameters. Furthermore, we observed an intriguing phenomenon in the landscape of the original separable nonlinear objective function, where some narrow valleys frequently exist. The RoVP strategy can effectively diminish the likelihood of the algorithm getting stuck in these valleys and accelerate its convergence. Numerical experiments confirm the effectiveness and robustness of the proposed algorithm.