On-Device Generative AI Model Robustness Enhancement via Quantization-Aware Normalization

Generative AI models have shown remarkable performance, but their considerable size leads to notable computational and memory overhead. Quantization is essential to deploy models on edge devices. This paper presents a novel quantization that leverages always existing normalization layers to enhance performance in terms of accuracy. We aims to minimize quantization loss in input and output feature maps by utilizing normalization layers. Our research provides valuable insights into improving efficiency and accuracy of Generative AI models, and we demonstrate additional performance gains can be achieved by plug-in to existing methods.