On the Interpretable Adversarial Sensitivity of Iterative Optimizers

Adversarial examples are an emerging threat of machine learning (ML) models, allowing adversaries to substantially deteriorate performance by introducing seemingly unnoticeable perturbations. These attacks are typically considered to be an ML risk, often associated with the black-box operation and sensitivity to features learned from data of deep neural networkss (DNNs), and are rarely viewed as a threat to classic non-learned decision rules, such as iterative optimizers. In this work we explore the sensitivity to adversarial examples of iterative optimizers, building upon recent advances in treating these methods as ML models. We identify that many iterative optimizers share the properties of end-to-end differentiability and existence of impactful small perturbations, that make them amenable to adversarial attacks. The interpretablity of iterative optimizers allows to associate adversarial examples with modifications to the traversed loss surface that notably affect the location of the sought minima. We visualize this effect and demonstrate the vulnerability of iterative optimizers for compressed sensing and hybrid beamforming tasks, showing that different optimizers tackling the same optimization formulation vary in their adversarial sensitivity.

1. Introduction

The unprecedented success of machine learning (ML), and particularly deep learning, gives rise to new risks and threats. A notable emerging threats is adversarial examples [1]. This attack allows an adversary to design minor, seemingly unnoticeable perturbations, which when corrupting an input to an ML model have a notable effect on its output. The last decade has witnessed an ongoing arms-race between new sophisticated adversarial attacks and the development of countermeasures aiming to mitigate sensitivity of ML models [2, 3].