Robustness Benchmarking of Convolutional and Transformer Architectures for Image Classification

Amidst the burgeoning landscape of thousands of deep convolutional neural networks (CNNs), selecting a robust architecture for image classification poses a formidable challenge. The prime reason is the broad vulnerability of these DNNs to image corruption. The literature still does not understand which network is sensitive to which kind of corruption and to what extent. Our study rigorously analyzes DNNs across the classification spectrum, from pure CNNs (without attention layers) to state-of-the-art Vision Transformers (ViTs). To reach a robust conclusion, we have performed extensive experiments using 18 DNNs, 5 diverse datasets, and 15 corruption types of varying severity. Our analysis uncovers insightful and surprising findings concerning the robustness of different DNNs across corruption types. For example, it is observed that the ViT trained on CIFAR10 is found to be highly robust in handling noise corruptions, even of considerable severity (say, severity 4), but is found vulnerable to environmental corruptions of the same severity. At the same time, while the performance of pure CNNs is lower than that of ViT, they can handle environmental factors better than noise corruption. Another interesting observation showcased that these corruptions can even fool one of the popular network explainability algorithms, Grad-CAM. The heat map highlights the same region of interest on the noisy image, similar to clean images, but the class label differs. Shedding light on the vulnerabilities of deep learning models and revealing their vulnerability against particular corruption ensures the deployment of a correct network in the real world that deals with specific corruption frequently.