Safe Multi-Agent Reinforcement Learning for Wireless Applications Against Adversarial Communications

Based on the network observations and learning parameters shared by the neighboring learning agents, multi-agent reinforcement learning (RL) has to enhance the performance over adversarial communications, in which spoofing attackers send fake learning messages to fool the learning agent and thus degrade the performance of wireless applications. In this paper, we propose a safe multi-agent RL algorithm for wireless applications against adversarial communications, in which each learning agent chooses the cooperative agents to share the learning information and authenticates the received learning messages before integrating them into the RL state formulation and the learning parameter update. The communication policy distribution for the cooperative agent selection is formulated based on the long-term discounted reward and the sharing reputation for each neighboring agent, which is updated based on the authentication results to indicate the probability as a spoofing attacker. Neural networks are designed to estimate the long-term discounted reward and the sharing reputation for the learning agent with sufficient computational resources in large-scale wireless networks to enhance the agent selection security. As a case study, our proposed algorithm is implemented in the unmanned aerial vehicle swarm anti-jamming video transmission against spoofing attackers that send fake received jamming power as well as Q-values and neural network weights in the anti-jamming transmission policy learning. Both simulation and experimental results are provided to verify the performance gain over the benchmark.