Meta-Distillation: A Novel Adversarial Distillation Network for Adversarial Robustness

Adversarial attacks undermine neural network reliability and security. We introduce Meta-Distillation, a novel framework to improve student robustness in knowledge distillation using teacher-generated adversarial distributions. On CIFAR10, we achieve a record 56.53% adversarial accuracy, and on MNIST, 95.47%. Additionally, on the medical Alzheimer’s Disease dataset, our method yields 88.26% adversarial accuracy, surpassing prior benchmarks. Dynamic thresholding in teacher-student architectures enhances distillation, ensuring efficient, robust networks against adversarial threats. This has significant implications for secure applications in medical diagnostics, addressing critical vulnerabilities in neural networks.