Secure Distributed Federated Learning for Cyberattacks Detection in B5G Open Radio Access Networks

The open radio access network (O-RAN) is designed to support the diverse wireless services for beyond 5th-generation (B5G) mobile networks. However, this also expands the potential attack surface, necessitating improved mechanisms for detecting cyberattacks. Advanced artificial intelligence (AI) algorithms, in conjunction with RAN intelligent controllers (RICs), can be utilized to identify threats such as distributed denial-of-service (DDoS) attacks. Nevertheless, AI introduces significant data privacy concern. To address these issues, secure federated learning (FL) can be leveraged to locally train cyberattack detection models and securely transmit the model data for aggregation, thus ensuring protection against eavesdropping. Moreover, peer-to-peer (P2P) FL can be used to avoid the single point of failure inherent in centralized FL. However, securing P2P FL with encryption/decryption or secure average computation (SAC) can result in high communication costs that do not scale well with the number of FL clients. In this paper, we propose a novel P2P FL strategy that ensures secure operation while significantly reducing communication costs. Specifically, we integrate client selection and transfer learning within the RIC-based P2P FL system to detect cyberattacks. Our experiments demonstrate the performance of our method across various scenarios with both balanced and unbalanced dataset distributions. We highlight its superiority in terms of accuracy, robustness, and cost compared to existing benchmarks. Furthermore, we extend our evaluation to a 5G O-RAN testbed, assessing the system’s efficiency, accuracy, and adaptability under real-time independent and non-independent and identically distributed (IID/non-IID) traffic conditions. This includes analyzing communication cost, execution time, model loss, and live traffic testing results for practical and real-time deployment.