MIPP-FL: Personalized Layer Privacy Protection Federated Learning Based on Mutual Information

Federated Learning (FL) has gained widespread attention because it doesn’t require users to share their private data. However, since sensitive information can still be inferred from the uploaded models by users, Differential Privacy (DP) is often employed to safeguard these submitted models. Existing work mainly focused on uniformly allocating privacy budget to each layer of submitted models, which can lead to performance degradation, such as slower convergence rates and reduced generalization capabilities of the global model. To address this issue, this paper proposes a novel Differential Privacy Federated Learning framework, called Personalized Layer Privacy Protection Federated Learning Based on Mutual Information (MIPP-FL). In MIPP-FL, we first calculate the entropies of each layer’s weights of local models to establish a distribution of weights for each training round. Based on the entropies, the mutual information between current and last training rounds can be further obtained. Finally, for a given local model, we dynamically allocate the privacy budget to each layer by considering the calculated mutual information. Theoretically, we have demonstrated that the proposed MIPP-FL framework ensures strict privacy guarantee. Moreover, extensive experiments have shown that our proposed method can improve accuracy and achieve a faster convergence rate than the existing method that allocated the same privacy budgets uniformly to all layers of a local model.