Notes on the Performance Evaluation of Adversarial Examples

In recent years, the sensitivity of state-of-art ma-chine learning methods to small changes in the data has been under scrutiny due to concerns raised by their behavior on even slightly modified instances. As a response, in order to better understand their behavior and confront potential attacks, the field of adversarial examples generation has emerged, proposing methods of producing modified inputs versions that reveal model's vulnerabilities. We suggest two new metrics that can help in identifying parameter values in generating examples that lead to the most substantial change in the model's behavior and analyze their efficacy.