Centralized and Decentralized Implicit Differentiation

Exploiting the underlying structure of optimization programs has unlocked computational gains and, in some cases, enabled distributed solution methods. One should expect that this structure can be similarly exploited for other important tasks, such as differentiation. Differentiating through optimization programs has in fact attracted much interest and led to numerous applications, from optimization-based layers in machine learning models to complex design problems formulated as bilevel programs. However, existing differentiation methodologies rarely leverage problem structure. In this work, we propose an efficient framework for computing gradients of constraint-coupled optimization problems. Gradients are decomposed into local components—computed in a distributed manner—and a coupling term—handled by a central coordinator. This centralized framework achieves significant computational gains, especially for large systems, and we provide sufficient conditions to ensure its validity. Building on this, we develop a decentralized algorithm—eliminating the need for a central coordinator—by leveraging the exponential decay of sensitivities in graph-structured problems. Finally, we demonstrate how the decentralized scheme allows for accurate and efficient estimation of important emissions metrics on large dynamic power system models.