Towards Privacy-preserving Personalized Federated Relation Classification

Relation classification plays a crucial role in detecting semantic relations between annotated entities within text data, serving as a fundamental tool for knowledge structurization. In recent years, federated learning has emerged as a promising approach for training relation classification models in decentralized settings. Existing methods have focused on developing a robust server model by decoupling model training at the server from direct access to client-side text data, while taking advantage of distributed data sources. However, a significant challenge arises from the heterogeneous nature of client texts, characterized by diverse and skewed distributions of relations, which has limited the practicality of current approaches. In response to this challenge, this study introduces the concept of personalized federated relation classification, aiming to tailor strong client models to adapt to their individual data distributions. To further address the issues stemming from heterogeneous texts, a novel framework, referred to as ${\sf pFedRC}$, is proposed with several optimized designs. This framework incorporates a knowledge fusion method that leverages a relation-wise weighting mechanism, and a feature augmentation approach utilizing prototypes to adaptively enhance the representations of instances associated with long-tail relations. Although federated learning can safely ensure private data unexposed, it is important to recognize that, from an information theory standpoint, there still exists a possibility for a curious server to deduce private information by analyzing the shared knowledge uploaded by clients. To enhance the privacy guarantees of the personalized federated relation classification system, this work integrates client-level differential privacy mechanism into the federated training process. According to our theoretical analysis, ${\sf pFedRC}$ with client-level differential privacy can realize rigorous privacy guarantees. Experimental evaluations demonstrate the superiority of the proposed ${\sf pFedRC}$ framework over competing baselines in various settings, illustrating that the tailored techniques effectively mitigate the challenges posed by heterogeneous text data while preserving privacy guarantees. This research contributes to the advancement of learning privacy-preserving personalized relation classification models via taking advantages of data from multiple sources.