ARBiBench: Benchmarking and Analyzing Adversarial Robustness of Binarized Convolutional Neural Networks

Binarized convolutional neural networks (BCNNs), which restrict the weights and activations of the model to +1 or −1, provide notable reductions in memory requirements and enhanced model inference speed during deployment. Current research on BCNNs primarily revolves around addressing the performance degradation resulting from binarization. However, the investigation of the effects of extreme discretization on the robustness of BCNNs has been largely overlooked, despite its critical relevance to real-world applications. To this end, we propose ARBiBench, a comprehensive benchmark for evaluating the adversarial robustness of BCNNs in the image classification task. The key contributions of ARBiBench include: 1) systematically evaluating the robustness of seven influential BCNN methods across various architectures; 2) rigorous validation of diverse adversarial attack methods; and 3) novel empirical findings showing that BCNNs exhibit weaker robustness than full-precision networks on small datasets but surprisingly stronger robustness on large-scale datasets. Leveraging Information Bottleneck theory, we further demonstrate how data scale and model capacity collectively determine BCNNs’ adversarial robustness. These findings not only challenge conventional assumptions about BCNN security, but also provide new insights for developing robust yet efficient neural network architectures.