Accelerated Dynamic Quantization and Semantic Pruning for Large Language Models on Edge Communication Devices

Large language models have demonstrated strong performance in natural language processing tasks, but their high computational and memory demands hinder deployment on edge devices. To address this, we propose a lightweight inference acceleration method combining three techniques: dynamic activation quantization, semantic-aware pruning, and input-adaptive pruning scheduling. The method dynamically adjusts quantization parameters based on per-layer activation distributions to mitigate accuracy loss from outliers. It further incorporates word embeddings and syntactic features to assess token importance, enabling fine-grained, semantics-driven token pruning. Pruning strength is adaptively adjusted based on input length, improving adaptability across scenarios. Experiments show that our method achieves up to $2.8 \times$ inference speedup without compromising model performance, outperforming standalone weight quantization or attention pruning.