A Black-Box Targeted Misclassification Attack on Edge AI with Adversarial Examples Generated from RAW Image Sensor Data

Deep learning has witnessed pervasive deployment on edge devices over the past decade, especially for computer vision applications. However, its vulnerability to adversarial attacks, where visually imperceptible patterns cause machine learning models to malfunction, has raised significant security concerns. The connection between the image sensor and the application processors is typically not encrypted nor signed for data integrity verification, leaving the data link exposed to tampering threats. Previous works have demonstrated how these threats can be exploited. However, these methods typically inject attack patterns into the RAW image data without considering the effect of the image signal processing (ISP) pipeline, which can undesirably weaken the adversarial effects. Insofar such attacks have not succeeded in more powerful targeted misclassification fraud attacks where a selected target can be misclassified into the attacker's intended output. In this work, we propose a novel RAW image domain black-box attack that incorporates a differentiable ISP to train a knowledge-distilled substitute classifier to generate adaptive adversarial perturbations that survive the ISP. We show that such an attack is feasible by attacking the edge implementations of ResNet18 and MobileNetV2 with adversarial examples generated from their knowledge-distilled models by applying differentiable ISP on RAW formatted GTSRB test images captured by a Raspberry Pi camera. Our results demonstrate that its attack success rate surpasses previous direct mapping techniques by 10.37% and 13.07%, respectively for ResNet18 and MobileNetV2 in untargeted misclassification attack tasks with greater stealthiness when the adversarial examples are displayed on an LCD monitor for comparison. More importantly, it can achieve the targeted misclassification at an attack success rate of 95.09 % and 51.98 % respectively, which is currently impossible with existing camera-link attack methods. The results are reproducible with our code, which can be downloaded from https://github.com/BOWEN-HU/diff-isp.