Split Knowledge Distillation for Large Models in IoT: Challenges and Solutions

Large models have been widely adopted across various domains, demonstrating remarkable capabilities in natural language processing, computer vision, and decision-making tasks. As Internet of Things (IoT) systems generate vast amounts of real-time data, deploying large models in these systems can enhance intelligent automation, real-time data analysis, and human-device interaction, unlocking new possibilities for smart environments. In this article, we first explore the applications of large models in IoT. We then introduce the key challenges in deploying these models, including energy efficiency, inference latency, and device heterogeneity, along with potential solutions. To address energy efficiency, we propose a split knowledge distillation framework that integrates knowledge distillation and split learning to efficiently compress large models for IoT deployment while preserving data privacy. Finally, we conduct a case study to evaluate the feasibility and performance of the proposed framework.