Transformer and Graph Neural Network-Based Federated Learning for Cellular Traffic Prediction With Sustainability Analysis

Mobile network traffic prediction is critical for efficient network management in 5G and future 6G systems. Federated Learning (FL) enables multiple base stations (BSs) to collaboratively train prediction models without sharing raw data, preserving privacy. This paper presents a unified and extended study of FL for cellular traffic forecasting, integrating state-of-the-art deep learning models including recurrent networks, attention mechanisms, Transformers, and Graph Neural Networks (GNNs). We evaluate these models on real 4G LTE traffic datasets (including recent publicly available data from 2018–2023) and additional large-scale synthetic scenarios. Our results show that FL achieves prediction accuracy comparable to centralized training while avoiding data transmission, and that advanced architectures (e.g. Transformer and graph-based models) can improve forecasting accuracy by up to 5-10% over federated LSTM baselines. We introduce a comprehensive sustainability indicator to assess the trade-off between accuracy and energy efficiency of each approach. Through detailed experiments, we quantify the energy consumption and communication overhead of training and inference for each model. The Transformer-based FL model yields the highest accuracy but incurs increased training energy, whereas simpler models are more energy-efficient. We provide visualizations of the accuracy-energy trade-offs and discuss how attention and spatial graph modeling influence performance. The study concludes that FL is a promising solution for privacy-preserving and sustainable mobile traffic prediction. We offer insights into model aggregation strategies, personalization, and the benefits of incorporating external (exogenous) data features. This work uniquely integrates advanced Transformer and GNN architectures within a federated framework, explicitly highlighting improvements over previous recurrent-based methods and emphasizing sustainability metrics critical for practical deployments. Moreover, our evaluation shows that the federated LSTM model achieved up to 10% lower sustainability score (S) compared to centralized approaches, quantifying practical gains for real-world deployments.