Leveraging Proximal Optimization for Differentiating Optimal Control Solvers

Over the past few years, differentiable optimization has gained interest within machine learning, control, and robotics communities. It consists in computing the derivatives of the solutions of a given optimization problem which can then be used in learning algorithms. Until now, dedicated approaches have been proposed to compute the derivatives of various optimization problems (LPs, QPs, SOCPs, etc.). However, these approaches assume in general well-conditioned problems, limiting de facto their application to general optimal control problems (OCPs) widely used in robotics. In this work, we focus on the differentiation of solutions to such problems. We notably introduce a differentiable proximal formulation of equality-constrained LQR problems that accurately solves rank-deficient problems. This allows us to compute accurate gradients even in the case of problems that do not satisfy the standard linear independence constraint qualification (LICQ). Because any optimal control problem can be cast as an equality-constrained LQR problem in the vicinity of the optimal solution, we show that our robust LQR derivative computation can be exploited to obtain the derivatives of general optimal control problems. We demonstrate the effectiveness of our approach in dynamics learning and parameter identification experiments in both linear and nonlinear optimal control problems.