Efficient Deployment of Quantized Vision Transformers: A Survey

Vision Transformers have demonstrated significant potential across various tasks in the evolving landscape of computer vision. However, deploying these models on resource-constrained devices remains a challenge, particularly in optimizing the trade-offs between accuracy, latency, and model size. This survey investigates four prominent Vision Transformer models CrossViT, DualVit, ResT, and LocalViT and two advanced quantization techniques, AdaLog and Bi-ViT. It also explores the unique architectural challenges of each ViT model and evaluates the effectiveness of combining these models with post-training quantization techniques. Specific challenges addressed include the asymmetric distribution of activations post-GELU and the unbalanced nature of softmax outputs. Through detailed comparative analysis, considering top- 1 and top- 5 accuracy, latency, and model size, this study identifies DualViT combined with AdaLog as the most optimal solution. This combination maintained $\mathbf{8 3. 1 2 \%}$ Top-1 accuracy, with a reduction of model size by 83.4%, dropping from 94.04 MB to $\mathbf{1 5. 6} \mathrm{MB}$. This approach demonstrated the best performance across multiple metrics, offering insights into efficient deployment strategies for ViTs in real-world applications.