Privacy and Fairness-Guaranteed Federated Preference Optimization for Large Language Models in Internet of Medical Things

Recently, the integration of Large Language Models (LLMs) with medical consumer electronics has provided more convenient healthcare services for doctors and patients, demonstrating significant potential. However, the high sensitivity of data and stringent privacy protection requirements in the Internet of Medical Things (IoMT) present substantial challenges to LLM policy optimization, which requires vast amounts of high-quality data. To address this, this paper presents an innovative approach that combines Federated Learning (FL) with Reinforcement Learning from Human Feedback (RLHF), aiming to optimize language generation strategies while ensuring the privacy of local medical preference data and fairness among participants. Firstly, we construct a FL framework capable of integrating preference data from multiple sources, enabling cross-domain collaborative updates and parameter sharing. Then, we propose a method for Direct Preference Optimization (DPO) in FL to reduce client-side training costs. To enhance the preference consistency and fairness of the global strategy across different clients, we introduce the local optimization with preference bias correction and global aggregation using performance-weighted schemes. Finally, we propose a Privacy and Fairness-Guaranteed Federated Preference Optimization (PFFPO) method specifically designed for IoMT. Experiments on a public medical dialogue dataset demonstrate that, compared with other FL methods, PFFPO achieves average improvements of 10.68% in preference alignment and 26.77% in fairness guarantee benchmarks, effectively balancing preference consistency and fairness.