Exploring Quantization Approaches for Optimized Training and Inference for Edge AI Applications

Artificial Intelligence (AI) has significantly transformed a wide range of fields, revolutionizing diverse applications. In this AI-driven era, tasks across multiple domains are being optimized for enhanced efficiency and performance. For deployment in real-world scenarios, edge devices play a critical role due to several compelling advantages. However, edge devices have constraints in terms of computational power, battery life and available resources to implement AI models. Model quantization, reduces the precision of model parameters (weights and activations), is one such technique that results in smaller model sizes and faster inference times, without significantly compromising performance. In this study, we explore a detailed comparative analysis of several existing quantization methods, including post-training quantization (PTQ), quantization-aware training (QAT), within the context of edge AI applications. We evaluate the effectiveness of these techniques using key performance metrics such as accuracy, model compression, inference latency, and energy efficiency. We have explored quantization workflows of two popular frameworks TensorFlow and PyTorch, for generating compatible models for edge devices such as microcontrollers and GPUs, and conduct a feasibility analysis of FPGA-based deployment.