Privacy-Aware Split Federated Learning for LLM Fine-Tuning over Internet of Things

The proliferation of Internet of Things (IoT)-generated distributed personal data enables user-specific large language model (LLM) adaptation at the edge. The split federated learning (SFL) facilitates collaborative learning and reduces memory footprint by model splitting, which necessitates the transmission of intermediate activations, rendering it susceptible to reconstruction attacks and privacy breaches. In this paper, we present a privacy-aware SFL scheme addressing the accuracy-efficiency-privacy trilemma in LLM fine-tuning over heterogeneous IoT devices. Particularly, we develop a privacy quantification metric based on Fisher information to assess layer-wise privacy risks in smashed data transmission. Guided by this metric, we establish an analytical model that captures the intricate relationships between privacy leakage, fine-tuning convergence time, and device energy consumption. To optimize these three aspects, we formulate a multi-objective mixed-integer programming problem. Then, an -constraint-based block coordinate descent (BCD) algorithm is proposed to jointly determine the optimal LLM split layer, transmit power, and bandwidth allocation for IoT devices under their memory and network constraints. Extensive simulation results demonstrate the proposed scheme’s effectiveness in achieving 24% faster convergence, 40% lower energy consumption, and 7% reduced privacy leakage compared to baseline approaches, while maintaining competitive model accuracy.