A Design and Implementation of Bangla Next Word Predictor Based on Personalized Federated Learning Leveraging Model Agnostic Meta Learning and Semantic Analysis

Next word prediction for technological devices has become a contentious issue in predictive analytics. Word pre-diction requires a huge amount of data from a vast number of different users, which introduces issues like heterogeneous dataset management and privacy concerns, thus raising the importance of Federated Learning (FL) as it can learn from a large amount of sparsely distributed data while ensuring privacy. Further investigations on related studies have led us to contribute in this area based on the Bangla language due to the minimal appearance of this language in the domain of word prediction based on FL. In consideration of implementing our work, some of the prime issues have been noted where avoiding data poisoning attacks, prioritizing user preferences during the global update, and focusing on fast convergence to reach a high-quality model in a few training rounds have become unavoidable matters of contention. Besides this, generating initial models which can be easily adapted by the current or new users is also an important research problem. Based on these, we have proposed a structure that combines semantic analysis and personalized FL using Model-Agnostic Meta-Learning (MAML) to obtain the desired goals. Extensive evaluations have been done on Non-IID data for both FL and personalized FL with MAML on the next word prediction model, which justifiably achieved 83.05% and 86.23% test accuracy respectively on the server model after ten communication rounds across ten clients. Apart from that, the effective results of semantic approaches show an impressive outcome as intended. Thus, the limitations and the challenges of FL on next word prediction conceptually allowed us to use a variety of algorithms to turn them into advanced ones for further usage.