A Le Cam Type Bound for Adversarial Learning and Applications

Robustness of machine learning methods is essential for modern practical applications. Given the arms race between attack and defense mechanisms, it is essential to understand the fundamental limits of any conceivable learning method used in an adversarial setting. In this work, we focus on the problem of learning from noise-injected data, where the existing literature falls short by either assuming a specific adversary model or by over-specifying the learning problem. We shed light on the information-theoretic limits of adversarial learning without assuming a particular adversary. Specifically, we derive a general Le Cam type bound for learning from noise-injected data. Finally, we apply our general bounds to a canonical set of non-trivial learning problems and provide examples of common types of noise-injected data.