Federated Learning for Character Prediction for Text Generation

Modern mobile devices have access to enormous amounts of user data including text, images, speech, etc., which can be utilized to train high-performance learning models and enhance the user experience. However, accessing large amounts of data often raises concerns for user privacy and security. To address this, federated learning (FL) has emerged as a new machine learning approach that trains models on multiple decentralized edge devices (e. g. mobiles) or servers while protecting user privacy. In this paper, we present a distributed learning framework using a practical iterative average-based federated learning algorithm for the text generation task in Natural Language Processing (NLP). Our results show that text generation training under federated learning yields better performance than random guessing, demonstrating the feasibility of FL in language modeling. The study highlights the success of text generation techniques trained using federated learning, while emphasizing the importance of safeguarding user privacy and security.