Joint Semantic Communication and Output Length Prediction for Efficient Large Language Model Inference

Recently, the large language models (LLMs) inference services of mobile devices are experiencing explosive growth, which has brought significant communication and computational pressure to the edge networks. To address these challenges, in this paper, we propose an efficient LLM inference framework over semantic communication. Specifically, we propose a lightweight joint semantic communication and output prediction model, which can achieve parameter sharing in the semantic extraction part. The proposed model not only ensures the performance of semantic communication but also improves the prediction accuracy. An adaptive scheduling algorithm is then developed to maximize the inference performance. Finally, extensive simulations demonstrate the effectiveness of the proposed mechanism with the popular ChatGLM3, and the LMSY dataset.